Non-deterministic sources in CARLA 0.9.11 during evaluation

[aremanap]

Hi there!

When I evaluate an agent several times on the same route using the leaderboard evaluation script, I notice some non-determinism and variance in the results (DS, RC, IM). What could be the sources of this variation?

I assume there is some sensor noise that contributes to these variations. Non-deterministic driving physics could also cause variations in trajectory and perceived environment.

Are there any other sources of randomness and how could their influence be mitigated?

Thanks in advance for your help!