Assessing Animal Behaviour with the Help of Image Annotation

The number of use cases for AI is almost limitless. Machine learning has led to AI models that can save people in natural disasters, perform surgery or improve the performance of top-class athletes. One interesting emerging computer vision application is helping pet owners and farmers to interpret animal behaviour and emotions.

As with most AI use cases this technology is supported by conscientious video and image annotation. This important process involves human annotators adding labels and contextual information to images and video in order to create AI training data.

This blog will look at the need for animal behaviour interpretation and identify specific applications being developed in this area. We will also show how the annotation provider Keymakr is providing high quality support for key annotation types.

Communication problems

Humans communicate emotions and physical sensations through a diverse range of facial expressions. Animals possess far fewer facial muscles and display their feelings in subtle ways that are hard for humans to interpret. Many animals also hide feelings of pain or discomfort due to the evolutionary need to project strength and health to potential predators. Failure to accurately identify the true emotional state of animals can lead to mistakes in treatment and healthcare, and can affect animal welfare overall.

Applications for household pets

The facial expressions of cats are notoriously difficult to interpret. Cats are known to mask pain in most cases making it difficult for first time cat owners to know when their pet is injured or distressed. However, there are subtle cues that cats give that can be interpreted by experts and, increasingly, AI systems. Applications are becoming available that can assess a series of body language markers including: ear and whisker position, muzzle tension and head position.

Phone based apps, powered by machine learning algorithms, can interpret photos of cats and advise owners about their general wellness. If multiple markers of discomfort are present, apps can also suggest healthcare and wellbeing options. This AI use case can be a useful resource for cat owners and veterinarians.

Livestock management use cases

Identifying illness or injury in livestock animals can enable early intervention and treatment that both promotes animal welfare and protects agricultural profits. As with household pets this can be a challenge, due to the tendency of herd animals to hide impairments. Computer vision based AI models can be leveraged to analyse animal movements and identify minute changes in gait or behaviour for signs of poor health. Automating this kind of livestock monitoring frees up time for farm workers and enables 24/7 surveillance of general herd well-being.

Essential annotation types

The AI applications above are in part supported by image and video annotations. Annotators apply specific techniques to photos and video frames in order to create training data for animal behavior interpretation AIs:

• Semantic segmentation: This annotation type involves assigning each pixel in an image to a specific class. Annotators use annotation platforms to outline key parts of the image. In the case of pet behaviour apps semantic segmentation would mean using a polygon tool to outline a cat in a training image and assigning it a label. Semantic segmentation creates more detail in training data, for more functional models.
• Key point annotation: This type is defined by locating key features in images. For animals this would mean using points to identify eyes, nose, mouth and ears. This technique is essential for emotion recognition applications.
• Skeletal annotation: By adding lines to images, annotators can locate the skeletal structure of animals in training images. This annotation type enables better visual representations of animal movement.

Contact a team member to book your personalized demo today.