Traffic Light Recognition for multi-camera system

[Mingyu1991]

We are working on designing the multi-camera traffic light recognition system to cover longer detection range and larger fov,
also to merge V2X/V2I into the system.
The current system is as follows:

The two output topic are:

/perception/traffic_light_recognition/rois
/perception/traffic_light_recognition/traffic_signals

Considering the perfect synchronization of different cameras is nearly impossible,
we propose to process cameras in different namespace and merge the result with a "Merger" node.
As show below:

The output changes to:

/perception/traffic_light_recognition/camx/rois
/perception/traffic_light_recognition/traffic_signals

@yukkysaito and @mkuri agreed with this design.
Please let me know your thoughts.
Thank you in advance.

[xmfcx]

Hello, thanks for the proposal!

What are done in the post-process and Merger blocks?

[miursh]

@Mingyu1991 Could you show the updated diagram?
Also, if you can, could you write the role of each node?

[Mingyu1991]

@xmfcx
Hello, Sorry for late reply.
Please kindly check the follow figure for updated diagram.

Let me briefly introduce the new nodes and functions in the diagram:

1. post process in Classifier
   We plan to implement status machine in this part to prevent from some false detection.
   For example, if the traffic light from last frame is green and detected as red, it would be processed as yellow.
   The detailed algorithm is not decided yet
2. traffic_light_merger
   This node is designed to receive and fusion the results of different cameras on the same traffic light.
3. traffic_signal_arbitrator(name might be updated in following implementation)
   This node is to receive and fusion the traffic lights of the same group in the cross.

[xmfcx]

Thanks for detailing the proposal!

I've used ChatGPT for finding a name for these 2 nodes with following prompt:

In Autoware, we are creating following 2 nodes for classification of the traffic lights:

Node 1: It combines the classification results from multiple camera classification pipelines and outputs a final classification result for the traffic light.

Node 2: It combines classification results that belong to the same traffic light group and outputs a single classification result for the given group that covers the same lanes.

How should we name these 2 nodes?

And from the responses:

Node 1:

• Multi-camera Traffic Light Classifier
• Camera Fusion Traffic Light Classifier
• Multi-View Traffic Light Detector
• Vision-Based Traffic Light Aggregator
• Multi-Pipeline Traffic Signal Recognizer
• Image Ensemble Traffic Light Classifier

Node 2:

• Traffic Light Group Classifier
• Lane-Level Traffic Light Group Classifier
• Roadway-Specific Traffic Signal Classifier
• Grouped Traffic Light Classification Node
• Intersection-Based Traffic Light Detector
• Aggregate Traffic Signal Classifier

My favorite node names are:

• Node 1: Multi-camera Traffic Light Classifier
• Node 2: Traffic Light Group Classifier

What do you think about the naming choices?

cc: @yukkysaito @Mingyu1991

Edit: I will update my choices to emphasize fusion over classification:

• Node 1: Multi-camera Traffic Light Fusion
• Node 2: Traffic Light Group Fusion

[Mingyu1991]

I like the idea of fusion.
Considering the traffic light nodes names are all like:
traffic_light_map_based_detector, traffic_light_fine_detector, traffic_light_classifier,
can we find some word like fusionner (it's Franch, although)?

[Mingyu1991]

Sorry for late update.
Let me share my discussion result with @xmfcx:

1. The proposed interface is OK.
2. The fusion algorithm should consider truncation, the credibility of detection.

I have submitted the PR in
autowarefoundation/autoware.universe#3442
So let me close this comment.