How Image and Video Annotation Supports Autonomous Vehicles

The autonomous vehicles industry is experiencing enormous growth, management consultants McKinsey & Company expect widespread adoption of robotic taxis by 2030, necessitating a tremendous growth in vehicle production. But enormous technical challenges still remain.

News website Vox reports that a variety of outstanding issues are holding up the development of autonomous vehicles, from problems navigating bad weather to concerns over cyber security and hacking. Whilst these problems are significant the industry is innovating and iterating around them.

One arena in which this is happening is in data annotation. Accurate, scalable image and video annotation, provided by professional annotation services, can form part of the solution for many development roadblocks. Keymakr has been collaborating with a leading autonomous vehicle AI company to overcome challenges through smart data labeling.

Defining the Objectives

The client had three objectives for their machine learning model that required bespoke dataset creation. By outlining their aims they were able to collaborate with Keymakr as a service provider and optimise the annotation process. The objectives were as follows:

• Monitoring In-Car Behaviour: Safety and user experience can be greatly improved by implementing AI cabin monitoring. The client was looking to train its model to detect human facial expressions and track the passengers movement. In-car behaviour monitoring by AI could prevent accidents by alerting drivers when they are falling asleep, or are not paying full attention to the road.

• Vehicle Recognition in the Day and Night: One of the primary objectives of autonomous vehicles is, of course, identifying other vehicles and navigating around them. It is essential that this recognition process work one hundred percent of the time, to ensure absolute safety. This challenge is further complicated by low light conditions. Night driving can seriously compromise computer vision models that have not been exposed to the reality of night driving.

• Autonomous Vehicle Computer Vision: Autonomous vehicles also require a holistic understanding of their environment. This means being able to distinguish between the road, the sidewalk, and the sky. This is particularly difficult in complicated and busy urban environments where AI systems are being bombarded with a huge amount of sensory data.

Keymakr Facilitates Problem Solving

The client chose to partner with Keymakr to fulfill the annotated training data needs of this complex project. Professional annotation services, like Keymakr, are able to leverage experience from many other annotation projects to provide solutions for specific development challenges:

• To support the training of in-car monitoring Keymakr provided the client with 500 hours of annotated in-car video footage of various individuals driving. Keymakr’s skilled annotators applied skeletal annotations to the upper bodies of drivers and passengers to track the movement of body parts through each frame. They also used key point annotation to identify facial features throughout the videos. This data was then fed into the in-car monitoring AI helping it to learn how to interpret human behaviour on the road.
• Keymakr’s in-house team of annotators labeled many hours worth of traffic videos for vehicle recognition. Each vehicle was tracked through each frame, localized by its number plate. Each label also contained other information, such as: the carmaker, model, color, and so on. This process was carried out for daytime and nighttime footage, so as to create a dataset that accurately reflects real world light conditions.
• In order to support the client’s autonomous vehicle computer vision project Keymakr utilised bespoke annotation tools to create a dataset of 20,000 segmented images of American and European streets. Using semantic segmentation annotation techniques annotators divide images, pixel from pixel, into defined classes of object: car, road, sign, tree. These meticulously labeled images of busy city traffic are an essential part of the development of competent autonomous vehicle AI.

Keymakr’s teams of skilled annotators and experienced managers can provide annotation services at a competitive price. Contact a team member to book your personalized demo today.