Meeting date: 01-06-2023 (10h30)
This week I started by addressing the issues related to dataset labels. I made the correspondence file between COCO and BDD100k, but I may have to do it also for datasets like cityscapes depending on the models to be used.
In the receiver node, the colors are all assigned automatically: In object detection and semantic segmentation, each class has a distinct color. Instance segmentation has two modes, where one assigns a range of colors per class and one color from that range to the instance, and the other assigns random colors to each instance, regardless of its class.
The receiver's ability to handle different types of segmentation was more challenging, for organizing the masks by labels without greatly hindering performance and for being able to add or remove labels from the yaml file.
Using two models and two receivers is now working as before:
But now, two models outputting to a single receiver also works:
The window name displays the different models separated by plus signs
For instance segmentation, the color of each mask is not always the same since the index isn’t also the same frame after frame. This could be solved using a tracker, but I honestly have not looked into how to apply it to this infrastructure so as not to deviate from the objectives.
For the “synchronization”, I copied the previous receiver and added a tolerance parameter. If the inference result arrives with a certain delay, it is discarded.
In the following examples I gave a tolerance of approximately 1 frame of delay and it’s possible to see that the instance segmentation does not always keep up with the object detection.