The entire workflow was built around a flexible, iterative model using small batches, which were regularly delivered by Chef Robotics as the
requirements of their CV models evolved. The Keymakr team worked directly within the Keylabs platform, which allowed the pipeline to be quickly
adapted for new task types and enabled separate workflows for different subprojects.
At the initial stage, the projects focused on annotating containers and inserting trays using oriented bounding boxes. Operators defined the object
boundaries and the precise angle of the object inside the container. Over time, the pipeline became more complex, expanding into segmentation
tasks for ingredients, sauces, and objects that the robotic system needed to detect, grasp, or recognize during operation.
In many cases, the boundaries of the insert trays visually blended into the container s surface, further complicated by motion blur from the robotic arm
and the specifics of the installed camera system. To accurately annotate transparent-on-transparent objects, operators used multiple zoom levels
and continually compared object shapes against Chef Robotic’s reference materials.
One of the key solutions was the development of a custom pre-annotation approach for inserting trays inside containers. Before annotation began,
the system automatically placed bounding boxes with predefined sizes and proportions, while experts manually repositioned and rotated them to the
correct angle and adjusted them based on the actual placement of the insert tray in the image.
This workflow also changed the approach to partially occluded objects. Initially, annotation was performed only on the visible portion of the object,
but later operators began "reconstructing" the hidden boundaries behind obstructions. This was necessary to keep all bounding boxes consistent in
size and proportions so the model would not lose understanding of identical insert tray types due to partial occlusions or deformation of the visible
object area.
The visibility threshold also evolved during the project. In many situations, determining whether an object should still be annotated remained
subjective and required additional QA verification. Despite the data's complexity and high variability, the project maintained an average annotation
accuracy of 98.65%.
