Performance analysis of a multi-container Autoware deployment

[oguzkaganozt]

Motivation

Now that the Open AD Kit containers are being used by different parties on different demos/activities etc. Testing multi-container behaviour and conducting a performance analysis can lead the way through the best possible aproach on how to upsteam multi-container deployment to the Autoware.

Test Suite 1: Reaction analyzer tool + AWSIM
Test Suite 2: Ros Trace Tools

Possible Approach

Phase 1
Utilizing Reaction Analyzer Tool for measuring end-to-end latency of the system with 2 containers (planning & perception) by using pre-recorded rosbags and map.

Phase 2
Replicating TUM's take on modular Autoware's setup on a smaller scale by using 7 modular containers with the help of Reaction Analyzer Tool for measuring end-to-end latency of the system

Definition of done

Performance analysis report comparing bare-metal, Phase-1 and Phase-2 deployment results in a summarized format.

[oguzkaganozt]

planning_control-2024-08-26-16-35-52-reaction-results.csv
perception_planning-2024-08-26-16-41-39-reaction-results.csv

[tejalbarnwal]

Hi @oguzkaganozt , could you please list some observations you made from the CSV files above?

[tejalbarnwal]

Also, reaction analyser tools help you to have latency checks; how do you plan to have throughput checks ?
Additionally, do you know of any measurement tools for checking networking overhead caused by multi-container docker-based deployment?

[takam5f2]

Test Suite 2: Ros Trace Tools

There may be another candidate tool; CARET. I think that ROS2 Tracing is difficult to apply a large scale application like Autoware.
https://tier4.github.io/caret_doc/main/