An In-Depth Look at Video Annotation

Today’s cutting edge computer vision based AI models are required to operate within an increasingly complex, dynamic and fast changing world. In order to reliably function within moving, real-time environments AI systems require training with high-quality annotated video data.

To create this data annotators use digital platforms to apply a number of annotation techniques to videos, taking each frame as an individual image. These annotated videos are then fed into machine learning models which in turn allow AI powered camera systems to identify objects, people, and conditions and respond to them accordingly, reacting to live video inputs.

For these models to be successful it is essential that developers have access to precisely annotated video, avoiding errors that could significantly impair the function of an end model. Affordable, scalable, and accurate video annotation is a significant challenge for any AI company. In response to this, professional annotation providers, like Keymakr, are providing industry leaders with annotation services that support their innovation.

This blog will begin by laying out the full suite of video annotation techniques and detailing how they are usually deployed. We go on to look at the restaurant industry, which is seeing a number of emerging AI use cases, enabled by video annotation. We will then focus on the unique challenges of video annotation, before going on to show how professional annotation providers can help overcome these issues.

Video annotation techniques

Video annotation techniques are designed to capture different types of information at different levels of detail. Developers can tailor video annotation to the needs of their projects by specifying a blend of annotation methods. Annotators can then apply these methods to create video training data:

• Bounding box annotation: Bounding box annotation is a fast and cost effective method of tracking objects from frame to frame in videos. Annotators cover target objects with a box, and label them with the relevant information (car, truck, bus etc).

• Polygon annotation: When a video contains irregularly shaped objects, annotators can deploy polygon annotation (small, interconnected lines) to capture granular detail.

• Semantic segmentation: The technique splits individual frames into object classes. For example, every vehicle will be identified with one colour, whilst the road and the sky also have assigned colours.

• Instance segmentation: Instance segmentation adds further detail by recording each instance of a particular object. In the previous example each car would have its own coloured pixels attached to it.

• Skeletal annotation: To track body movements from frame to frame annotators use skeletal annotation. Lines are drawn on the body and intersect at points of articulation.

• Key points annotation: This allows important features to be pinpointed in each frame.

• Lane annotation: This technique involves drawing lines to define the shape of linear objects, such as roads and bridges. It is particularly useful for video data for autonomous vehicle AI.

Emerging use cases: the restaurant industry

In fast paced industries techniques like the ones above are being used to train AI systems that could meaningfully increase efficiency and improve customer experience. The restaurant and hospitality sector is beginning to incorporate some the advances in computer vision to secure the future of dining in a changing world:

• Performance metrics: Computer vision systems can provide restaurant operators with a wealth of analytics and operational insight. AI powered cameras can track the movements of servers and customers, ensuring that new customers are greeted in a timely manner to improve the dining experience. Training video annotation allows AI models to recognize individuals and record their movements across thousands of frames.
• Early warnings: AI backed monitoring cameras can capture a wealth of information, and warn managers of potential issues. This could mean excessive wait times, untidy areas, or customers leaving before being seated. Line annotation of video data trains AI models to understand the restaurant space, whilst bounding boxes can locate areas of concern.
• Training opportunities: Restaurant chains value the opportunity to train their staff with real world, real time examples of best practice in their actual restaurants. AI systems can provide this high level perspective by creating performance metrics for entire restaurant chains. Computer vision technology, supported by high quality video annotation, is the only way to gather this level of detailed information.

The burden of video annotation

The promising use cases detailed above are just a fraction of the potential possibilities enabled by AI powered video systems. However, video annotation remains a unique challenge for tech companies, large or small. At 30 frames per second even a 2 minute piece of footage contains 3600 individual frames, each of which has to be painstakingly annotated. The multiplicative effect of video data can result in a daunting workload for fledgling data annotation operations.

The time consuming of video annotation can also lead to issues with scalability. In-house annotation teams are often small, and designed to fill one specific need. Video data needs, however, tend to change over time. As a result in-house teams can be overloaded with work at times and under worked at others. As companies grow and data needs change these kinds of inefficiencies can begin to affect the bottom line.

Video annotation can also be challenging in terms of cost. Developing the capacity to feed data hungry algorithms can be a significant financial burden, particularly for startups. The cost of hiring staff, providing office space and purchasing annotation tools can be significant.

Finally quality control can be hard to guarantee when video annotation is being carried out by a distributed workforce (in the case of crowdsourcing). Managers may have to communicate the specific demands of a project across different time zones and cultural contexts.

Experienced video annotation providers

As has been shown, in order to understand dynamic environments as they change in real time AI models need to be trained with precise video data. However, creating this data can prove to be a substantial distraction for growing AI companies.

Keymakr is a professional annotation service that relieves the burden of large scale video annotation from AI innovators. Keymakr’s annotation platform boasts unique project management and workflow capabilities, ensuring that video annotation is completed on time.

Our experienced annotation teams can respond quickly to changes in data needs and troubleshooting, whilst our quality control procedures guarantee precise data.

Contact a team member to book your personalized demo today.