The Sandbox knowledge hub discusses many of the crucial issues affecting the development, engineering, use and regulation of Autonomous Vehicles.
Scroll to explore
Understanding Operational Design Domain to Create Informed Safety in Autonomous Vehicles Deployment
One of the biggest challenges for autonomous vehicles (AV) is how to argue their safety case. The first step towards a state of informed safety is establishing the capability of an automated driving system (ADS) by defining its operational design domain (ODD).
What is an ODD?
“ODD is an operating condition under which a given driving automation system or feature thereof is specifically designed to function, including, but not limited to, environmental, geographical, and time of day restrictions, and/or the requisite presence or absence of certain traffic or roadway characteristics.” SAE J3016
The operational design domain (ODD) can be used to restrict where the automated driving system (ADS) is valid and thus confine the scope of the safety case, as well as the verification. In order to ensure this, use cases are needed to provide a strategy for a collection of operating conditions (OCs) and make sure that it is still within the scope of the ODD. The ODD defines the functional boundary of the system and modelling it with the structure below makes it modular and generalizable across potential use cases.
To complete the safety case, there is a need to ensure that the ADS will not exit its ODD. The ODD taxonomy defined in PAS 1883 helps specify and implement minimum safety requirements using sets of ODD attributes. PAS 1883 has three high level attributes as shown in the figure below.
Scenery which is everything that is geo-stationary
Environment which is everything that is atmospheric
Dynamic Elements which are all the movable objects
In a recent project, we created an ODD for an autonomous baggage tug where the ODD was created from the point-of-view of the airport operator and is defined based on the airside operating environment. It is important to realize that the ODD does not include the use case of the vehicle, rather, the ODD provides the boundary conditions within which the vehicle should be able to operate. Therefore, within this project, we also outlined the use case for the autonomous baggage tug.
To be able to test if an autonomous vehicle (AV) is capable of operating safely within its ODD, we first need to be able to define the set of scenarios that we can simulate. Currently this is a manual process for an engineer to analyse the ODD and use their knowledge of the vehicle use case and intended operation to define a set of scenarios such as an overtake manoeuvre or right turn at a junction. The scenarios on their own though are not enough to enable adaptive testing of the AV; we need additional information. We have started to define a framework that bridges this gap between the ODD and what is required to define the scenarios and enable adaptive testing.
One element of this framework is called Scenario Factor Analysis which is the process of deciding the most useful test case to simulate based on a combination of the risk, frequency and test coverage of test parameters individually, and as part of a scenario. The risk and frequency of each parameter is determined manually and is related to the ODD.
How to determine value
The probability and severity of failing requirements. How much damage could be caused to people and property
Calculated by an engineer using risk analysis
How often a discrete parameter is chosen or the distribution of a continuous parameter
Using the parameters values from previous test cases.
The likelihood of a test case occurring when the AV is in operation
Calculated by an engineer using statistical analysis of the ODD
Figure 2. Scenario Factor Description Table
The coverage factor is intended to ensure the adaptive testing covers the whole ODD without leaving big gaps in testing. The fewer tests that can be completed, the more important this factor is and the less chance the testing has to explore high risk or high frequency test cases. If there is time to conduct more testing, we can select test cases that can dive deeper in to the worst case scenarios and evaluate more parameter variations. The exact formula for creating new test cases is especially important.
Many test parameters are dependent on other parameters, for instance the rate of rainfall is irrelevant if there are no clouds in the sky. This can be used to reduce the number of test cases required for concise testing. As the number of tests increases, the probability of a failure in the AV is inversely correlated to the number of tests conducted. When using test automation to prove performance, human performance can be used as a benchmark.
Proposed Test Process
While test cases are often created using the ODD, there is currently no standard procedure for digitizing the ODD or using the ODD to create test cases automatically. With a digital ODD, adaptive testing can use information about the operating domain to generate the optimal next test case. Depicted in figure 3, the goal of this step is to identify short, common, risky and untested scenarios from within the vehicle ODD, such as a vehicle overtake manoeuvre or right turn at a traffic light. These are the primary tests of concern and passing them will result in obtaining the highest confidence in the AV in the shortest amount of time.
Figure 3. Overview of our test process
The process and adaptive test manager have been implemented as a proof-of-concept within our autonomous vehicle simulator and further exploration of our framework and findings will be disseminated in future blog posts.
Written by Amina Hamoud – Project Engineer
Please get in touch if you have any questions or have got a topic in mind that you would like us to write about. You can submit your questions / topics via: Tech Blog Questions / Topic Suggestion
The AVSandbox Knowledge Hub
Discover more about what makes AVSandbox unique. Explore our AVSandbox knowledge hub and find out about the issues, challenges and exciting developments that are behind the growth of the market for autonomous vehicles and advanced driver assistance systems.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.