Introduction

In my previous blog deterministic scenario simulation, I detailed why we define our simulator deterministic and what is required for the simulation to be deterministic. I also touched briefly on how we ensure traffic agents are calculated and behaving in a deterministic way. In this blog, I would like to discuss specifically traffic agents. To complement the previous post, an experiment was created where we run the same scenario multiple times, collect the telemetry data from it and then use this data to analyse the deterministic behaviour of the traffic agents in AVSandbox.  

The Experiment

A simple scenario where ego drives through 2 junctions is used in this test. For each junction we added a single traffic agent. In the first junction, the traffic agent is far from the vehicle and is free to travel. In the second junction, the traffic vehicle has to slow down to be able to react in case the Ego does not stop at the junction. This forces a drastic change in speed profile for that traffic agent causing further disruption to the traffic agent behaviour. In the video below you can see the scenario in action.

As part of the experiment, we then run this scenario several times and collect the telemetry data from each simulation. Once all simulations are completed, we can then visualise this data into our data analysis tools.  

Analysing the telemetry data

The main idea is to confirm that all vehicles behave in the same deterministic way for every simulation. A simple way to confirm this is by checking that all vehicles have the same position (XY) in each timestep of the simulation for all simulations. That way, if there are any changes in acceleration, heading, braking profile these changes would also be reflected in the vehicle’s position.  Image 1 (below) shows the plot containing the XY position of the Ego vehicle and 2 traffic agents going through the junctions during each simulation run.

Image 1. XY plot of Ego and Traffic in 10 simulations

As you can see, even though we plotted the data for 10 simulations, there are only 3 curves visible in the plot – 1 for each vehicle. That happens because all XY positions are the same for each simulation time step. Therefore, all positions are overlapping each other in the plot. This confirms all the vehicles behave the same way for every simulation run. In case any vehicle had a different behaviour, you would see more curves than the number of vehicles in the scenario. The same plotting demonstration can be done for any variable in the telemetry data. For instance, when plotting the velocity profile for the same 3 vehicles, once again we can see only 3 curves are visible (see Image 2).

Image 2. Velocity plot of Ego and Traffic agents in 10 simulations

Note that even though one of the traffic agents had to decelerate to prevent a possible collision, around simulation time 12.5 seconds, the traffic agent still behaved in a deterministic fashion. As a final demonstration of the data collected, using our built-in data analysis tools, it is possible to play both XY position and Velocity plots over simulation time and observe the values at each timestep of the entire simulation. In the video below, you can follow the markers, which demonstrates the value of each vehicle at a specific simulation time. As in the previous plots, even though we are plotting the data for 10 simulations, you can only see 3 markers as the data from each simulation perfectly overlapped on the top of each other.

 

Conclusion

In this blog, we have analysed the data generated by several simulation runs and compared them to confirm the traffic agents behave in a deterministic way. These will generate the same data and behave in the same way regardless of how many times a scenario will run. Although, internally we analyse the determinism of the simulation in a much more rigorous way, I believe the simple plots showing vehicles’ positions and velocity profiles can help to visualise, in an easier way, that the vehicles behave in the same way over multiple simulations.

As mentioned in deterministic scenario simulation, it is extremely important for traffic agents to be deterministic as this allows you to put an AV controller through the same scenario as many times as necessary without having to deal with traffic having different behaviours. That way the only changes will be in the AV Controller, making it much easier to validate changes into your controller.  

Written by: Felipe Ferrary – Senior Software Engineer & Saptarshi Mukherjee – Simulation Engineer

Please get in touch if you have any questions or have got a topic in mind that you would like us to write about. You can submit your questions / topics via: Tech Blog Questions / Topic Suggestion