Disclaimer

This document is the copyrighted property of ASAM e.V.

Any use is limited to the scope described in the license terms (https://www.asam.net/license). In alteration to the regular license terms, ASAM allows unrestricted distribution of this standard. §2 (1) of ASAM’s regular license terms is therefore substituted by the following clause: "The licensor grants everyone a basic, non-exclusive and unlimited license to use the standard ASAM OpenSCENARIO".

Foreword

This document is the summary of the work of the OpenSCENARIO 2.0 Concept Project. It summarizes the work and efforts of about 90 contributors. This document should serve as the starting point to an OpenSCENARIO 2.0 Development project, aimed at delivering an OpenSCENARIO 2.0 standard.

The OpenSCENARIO Concept Project (P2019-02), was initiated in parallel to the transfer project of the OpenSCENARIO standard to the ASAM domain. The goal of this transfer project (P2019-01) was a revision of OpenSCENARIO to ensure adherence to the ASAM standard format, leading to a major version release of v1.0.

The purpose of the Concept Project is thus to address features and adaptations to be implemented in future versions of the OpenSCENARIO standard. The initial scope of the project was defined in a kick-off in January 2019, where several core topics were identified:

  1. Interface to Models & Topology

  2. Parameter & Constraint Handling

  3. Measurements, Grading & Success

  4. Scenario Creation Methods

  5. Maneuver Description

  6. Glossary & Notation

  7. Global Architecture

Disclaimer

The concept document contains references to M-SDL (reference manual), a scenario description language made available to the public domain. It is used for syntactual examples where these may help in the understanding of the concepts contained herein. The language does not reflect on the final format of the standard, nor on any possible implementations of software tools.

1. Introduction

1.1. Overview

With the evolution of autonomous vehicles, the need for more sophisticated testing and validation of ADAS and autonomous driving system is increasing exponentially. It is predicted that simulation-based testing will replace the unachievable goal of test driving billions of kilometers for validation purposes. Different companies have already published their use of thousands of simulation-based tests for the validation of their autonomous systems (see e.g. [1], [2]).

OpenSCENARIO 1.0 is a low-level and concrete specification format, primarily designed to be read by simulation tools. Users, who create maneuver descriptions and tests, need a higher level of abstraction and potentially different ways to express this than in the current detailed XML-format of OpenSCENARIO 1.0.

There are multiple ideas how to express maneuver descriptions on a higher level of abstraction, with interfaces to general-purpose programming languages, textual stylized natural-language scenario descriptions, and the development of a domain-specific language (DSL) being among the expressed ideas.

A central aspect of consideration is the ability for domain experts to directly author and review scenarios without specific tooling, while retaining the ability to have a fully formalized description amenable to precise interpretation and execution by machines as well as humans.

Based on the considerations of the concept group, OpenSCENARIO 2.0 is proposed to be founded on the concept of a domain-specific language (see Section 2.3 and Section 5), that should support all levels of scenario description, from the very abstract to the very concrete in a suitable way.

The OpenSCENARIO 2.0 concepts take proven features and capabilities of OpenSCENARIO 1.0, like its event-based scenario execution model, and place them in a more general and expressive language framework, to serve as the foundation for both incremental improvements and more revolutionary enhancements.

In comparison to OpenSCENARIO 1.0, a more detailed set of actions and attributes for the relevant simulation models shall be defined to allow for a more comprehensive scenario description and to improve exchangeability. This is addressed by the introduction of a domain model (see Section 4).

Beyond such improvements, the proposed domain-specific language supports the creation of more complex scenarios through flexible composition and parametrization of component scenarios in ways that allow realistic, emergent behavior to occur. This is expected to enable massive testing and verification of new, complex hardware and software systems, and their interaction with the complex environment of driving.

The concept group believes that the proposed concepts and direction will enable OpenSCENARIO 2.0 to supply a step-function in its ability to describe full, complex simulations and/or tests, addressing all the required elements, and utilizing existing building blocks (like OpenDRIVE or OpenCRG).

In order to leverage investments in OpenSCENARIO 1.0, a clear migration path and compatibility concept is part of the overall OpenSCENARIO 2.0 concepts, as described in Section 1.3.1.

1.2. Intended Audience

The OpenSCENARIO concept document is intended to serve as the foundation of the development of the OpenSCENARIO 2.0 standard (the development project to follow the concept project). It is expected to serve as a reference and foundation for other standardization entities in the world, such as those standards intending to address test scenarios for autonomous vehicles (e.g. NP 34502). The concept document will also provide interested parties the opportunity to align their development activities with that of the standard.

1.3. Relations to Other Standards

1.3.1. Backward Compatibility to Earlier Releases

OpenSCENARIO 2.0 is intended to be a full superset of the features of OpenSCENARIO 1.0. This means that a migration path for OpenSCENARIO 1.x scenarios to OpenSCENARIO 2.0 will be included in the release version of OpenSCENARIO 2.0.

Migration of scenarios might require conversion to 2.0 syntax and semantics. The run time behavior of any scenario converted from the latest OpenSCENARIO 1.x to OpenSCENARIO 2.0 shall be the same.

A conversion of OpenSCENARIO 2.0 scenarios to OpenSCENARIO 1.x will in general not be possible, since OpenSCENARIO 2.0 is intended to be a true superset of OpenSCENARIO 1.x. However conversion of a subset of OpenSCENARIO 2.0 that maps to the feature set of OpenSCENARIO 1.x will be possible.

The standard development projects for the 1.x and the 2.0 tracks shall ensure an up-to-date ruleset for a migration path from 1.x to 2.0.

1.3.2. References to Other Standards

At the time of writing OpenSCENARIO is the only proposed standard to describe maneuvers and testing scenarios. The standard is designed to integrate with the requirements of other safety standards being developed in parallel. These include:

ISO

21448 (SOTIF), 34501 (Terms and Terminology for test scenarios), 34502 (Engineering framework for test scenarios), 34503 (ODD taxonomy for test scenarios), 34504 (Test scenario cartograms)

UL

UL4600

SAE

J3164 (draft)

2. Scope

2.1. Goals

OpenSCENARIO 2.0 should address the basic requirements and features stated in the project proposal. These are repeated here for clarity and their detailed description can be found in the concept project proposal document for OpenSCENARIO 2.0.

Table 1. Features
Feature Type

F001: Maneuver model

Change

F008: High level maneuver descriptions

New

F003: Traffic Model

New

F007: Parameter stochastics

New

F002: Driver Model

New

F004: Environmental Condition Model

New

F009: Replay of Recorded Scenarios

New

F010: Automatic parameter calculation

New

F005: Infrastructure Event Model

New

F006: Vehicle dynamics model

Change

F011: Additional metadata for parameters

New

F012: Language Constructs for Localization

New

Proposal workshop and revisions can be found in the workshop summary set of slides.

Table 2. Issue Descriptions
ID Title/Description

R001

Avoid Different Ways to Model

R002

Define Elements as 'Mandatory' Only When Absolutely Needed

R003

Maintain Independence and Open Linking Between Standards.

R004

Define Three Levels of Control for Ego Vehicles.

R005

Allow Tool-Vendor Specific Extensions.

R006

Allow Definition of Feature Subsets

R007

Define Semantics to Enable Reproducibility and Single Interpretation. (Workshop phrasing was: Well Defined Semantics Requirement )

R008

Allow both Open-loop and Closed-loop Simulation by the Same Maneuver Descriptions. (Workshop phrasing: Maneuver Description Shall be Suitable for Open-loop and Closed-loop Simulation)

R009

Define Parameter Boundaries

R010

Synchronize Maneuvers and Events

R011a

Allow Definition of Success Criteria for Individual Maneuvers, and for Full Scenarios and Tests - DUT criteria

R011b

Allow Definition of Success Criteria for Individual Maneuvers, and for Full Scenarios and Tests - non-DUT criteria

R012

Allow Textual Editing of the Format. (Workshop phrasing was: Suitability for textual editing)

2.2. Non-Goals

OpenSCENARIO 2.0 concept and standard are not expected to specify:

  1. A specific software architecture and tools aimed at using the format.

  2. Assumption on the testing platform and simulation tools being used.

  3. Method to store and categorize scenarios.

  4. Methods to specify Operational Design Domains (ODD).

  5. OpenSCENARIO is not enforcing any standard API to simulators and testing platform, although such a standard might be desirable in future.

2.3. Methodology

2.3.1. Declarative Language

The foundational concept of OpenSCENARIO 2.0 is to establish a domain specific language of a declarative nature. A declarative language describes what should happen on scenario execution (including the required parameterization/variation), rather than how to do it. A declarative language can also have a dual interpretation, i.e. provide a single scenario description which can be used to describe both how to make it and how to monitor that it indeed happened. This is important if we want to condition some operation on the fact that some other scenario has happened before, without having to describe in detail how to cause that scenario.

Basic methodology concepts developed by the concept project are stated in this section. These should be used to guide the development of the standard. Chapters 4, 7 and 8 will expand and supply more details on these concepts, as well as examples. In cases where examples are supplied, they are supplied using the foundation of the M-SDL language (reference manual).

2.3.2. DSL concepts

The Domain Specific Language is based on some foundational concepts introduced here. Detailed terminology and definitions can be found in chapter 3:

  • SCENARIO - A scenario describes the behavior of actors over time

    • Scenarios are defined via a scenario definition and invoked (activated, called) via a scenario invocation

  • ACTOR - Actors are the "players" (acting entities) in a scenario (e.g. vehicles, people, …​)

    • A scenario can define the behavior of a single actor or orchestrate the behavior of multiple actors

    • An actor can also represent a group of other actors, like traffic and car_group

  • ACTION - Actions (e.g. drive) define the basic capabilities of actors

  • PHASE - Phases are periods of time within a scenario corresponding to sub-scenario invocations. Phases can be atomic or composite.

    • An atomic phase is a phase consisting of an invocation of an action performed by a certain actor enriched by modifiers e.g. car1 drives with 40 to 50 mph on a two-lane road for 10 to 20 seconds.

    • A composite phase is a phase consisting of an invocation of a temporal composition operator such as serial or parallel.

    • A scenario thus consists of a potentially-hierarchical composition of phases, which together specify the behavior of one or more actors. Note that the invocation of a whole scenario is also a phase.

  • TEMPORAL COMPOSITION OPERATORS - Temporal composition operators_ (e.g. serial, parallel) can be used used to compose (atomic) phases or scenarios

  • CONSTRAINT/MODIFIER - Constraints/modifiers modify (influence) behavior

    • Constraints are a type of modifier specific to parameter values

    • Modifiers in general have a wider scope and can determine dynamics, paths, …​

    • Modifiers can in particular be used to specify in more detail how actors perform actions

  • Scenario definitions have parameters with types

    • Parameter types can be time or velocity or even car (i.e. a reference to an actor)

    • Parameter values are set when scenarios are invoked

    • A parameter’s final value will be set according to the type and the constraints applied.

  • COVERAGE - Coverage definitions define values to be collected for coverage analysis

2.3.3. DSL conceptual building blocks

OpenSCENARIO has a top level file; additional files can be imported into it, and additional type definitions and type extension may exist there:

  • type definition - defines a new scenario, actor, struct, …​

  • type extension - extends an existing type (e.g. actor) adding fields, constraints, …​

Scenarios contain:

  • parameter field - defines something which you can change/influence in the invocation:

    s: speed
    v1: car
  • variable field - defines a location to update during computations (e.g. to hold KPIs)

    !ttcat_end: time = sample(get_time_to_collision(), @end)
  • modifier/constraint - modify (influence) scenario behavior:

    keep(v1.color != v2.color)
    speed([10..15]kph, faster_than: v1)
    set_map("my_map.xodr")
  • coverage definition - defines what and when to write to the coverage database

    cover(overtake_duration, unit: ms, range: [0..3000], every: 100)
  • event - triggers (happens) when some condition happens

    event too_close is rise(distance_between(car1, car2) < 10m)
  • method definition - defines name and parameters of a method written in another language

    def distance_between(car1: car, car2: car): distance is external "python cars.dist"
  • behavior definition - says what the scenario does

    do overtake(v2: dut.car)

All of these (excluding behavior definitions ) can also appear as fields in actors and structs.

2.3.4. Composition

Composition is a key concept of the DSL: it enables to build, instantiate, call and re-use scenarios from existing scenarios or building blocks. Composition exists at all levels and on all building blocks; the DSL supplies the following abilities:

  • Instantiate actions of actors to atomic phases by possibly using modifiers

  • Compose phases via temporal composition operators like serial, parallel, if, repeat, …​

  • Define a new scenario using a name, parameters and a composition of other scenarios and phases

Example how to compose a new scenario by calling existing scenarios:

Static
scenario overtake_serial:
    car_a: car
    car_b: car
    do serial:
       overtake(v1: car_a, v2: dut.car)
       overtake(v1: car_b, v2: dut.car)

2.3.5. Constraints randomization and coverage

The DSL introduced two additional concepts that are included in the format and are intended to ease the writing of testing and simulating scenarios. The first is usage of constraints and modifiers and their semantic, the second to specify coverage ranges.

Constraints are used to:

  • Narrow down parameter values

  • Connect values of multiple parameters

Randomization semantics: generating a test (a concrete scenario) for any parameter not assigned a value. An OpenSCENARIO 2.0 scenario description allows for specification of parameters and probability distributions. The details on how to specify these will be developed in the standard development project.

  • The scenario/test provides guidance to the implementation with probability distributions

  • By default it will use a uniform distribution, but it is possible to request other distribution functions

  • Modifiers can be used to influence the scenario’s dynamics

    • Like speed, position, where on the map it should happen, …​

  • It is possible to specify constraints from very abstract to concrete descriptions

Some examples:

keep(v1.color == green)                   # A single value
keep(v1.color in [green, blue])           # A choice of two colors
keep(v1.color != v2.color)                # They can’t have the same color
keep(v1.color == red or v2.color == red)  # One of them must be red
keep(t in [7..12.5]s)                     # By default will use a flat distribution
keep(soft t == normal(10s, 2s)            # Will use a normal distribution with
                                          # a mean of 10 seconds and std deviation of 2

Modifiers can be used to influence the scenario dynamics. For instance, the modifiers below describe v3 relative to the other vehicles:

v3 modifiers
v3.drive(p) with:
  lane(right_of: v1)
  speed([7..15]kph, faster_than: v1)
  position([20..70]m, ahead_of: v1)
  position([10..30]m, ahead_of: v2)
  lane(same_as: v2)
  lateral([10..25]cm, left_of: v2)

Coverage:

  • Coverage is used to define verification goals and analyze what we did so far

  • Coverage statements are used to specify relevant ranges for which coverage data should be reported by the simulator/testing platform

  • Coverage can be used to collect values for multi-run analysis

Examples:

scenario traffic.overtake:
    v1: car
    v2: car
    side: av_left_right      # Is the overtake from the left or the right
    do parallel ...            # Actual behavior
      cover(side)
      cover(min_ttc(v1, v2), name: min_time_to_collision,
            unit: millisecond, range: [0..3000], every: 100)
      cover([side, min_time_to_collision]) # Cross coverage
    ...

2.3.6. Extensibility and portability

A key concept of the DSL is the ability to extend entities and the testing platform independence/portability between platform and simulators. It implies that the language allows to:

  • Add new simulator/project specific attributes to existing objects

  • Extend predefined scenarios

  • Add new model specific scenarios and modifiers

Example:

Suppose a specific simulator X supports a new weight attribute for cars, the file below will add it to every car (and every scenario using cars):

 extend car:
    weight: real
    keep(weight in [500..4000]kg)

2.3.7. Putting it all together

The following two examples of the same overtake scenario show how the concepts of composition, abstraction, modifiers and constraints can all be used in either a concrete or abstract manner.

Static
Figure 1. Resolution of a solution space with abstract and concrete scenarios
Concrete overtake scenario
scenario overtake_concrete:
  v1: car with(category: sedan, color: black)
  p: path
  path_explicit(p,[point("15",30m),
                point("95",1.5m), ...])

  do parallel():
    dut.car.drive(p) with:
      lane(2)
      speed(50kph)
    serial:
        A: v1.drive(p, duration:3s) with:
          speed(70kph)
          lane(2, at: start)
          lane(1, at: end)
          position(15m, behind: dut.car, at: start)
          position(1m, ahead_of:dut.car, at:end)
        B: v1.drive(p, duration: 4s) with:
          position(5m, ahead_of: dut.car, at: end)
        C: v1.drive(p, duration: 3s) with:
          speed(80kph)
          lane(2, at: end)
          position(10m, ahead_of: dut.car, at: end)
Abstract overtake scenario
scenario overtake:
  v1: car # the first car
  v2: car # the second car
  p: path


  do parallel(duration: [3..20]s):
    v2.drive(p)
    serial:
      A: v1.drive(p) with:
        lane(same_as: v2, at: start)
        lane(left_of: v2, at: end)
        position([10..20]m, behind: v2, at: start)
      B: v1.drive(p)
      C: v1.drive(p) with:
        lane(same_as: v2, at: end)
        position([5..10]m, ahead_of: v2, at: end)

3. Use Cases

As OpenSCENARIO 2.0 has the potential to cover a very wide range of use cases, the scope and functionality of the DSL to be developed will be driven by exemplified use cases. This ensures that

  1. the scope of the project is clear - both to the project participants and to anyone not directly involved in the initial development

  2. the requirements for the DSL can be easily analysed and extended based on further use cases in the future.

For simplicity of language the following user stories are phrased from an end-user perspective, who will be interacting with the tools and processes required to define, implement and consume results from test scenarios. Tool, language and data model implementation details are omitted on purpose.

3.1. User stories for OpenSCENARIO 2.0

A) SHARE

  1. As an AV/ADAS developer company, I can share with other companies the scenarios I built to test my technology.

  2. As an AV/ADAS developer company, I can search, review and reuse scenarios built by other companies.

  3. As a test engineer working for an AV/ADAS development company, I can build and run tests as similarly as possible to tests other developers at other companies are running.

  4. As a test engineer, I can build and run tests as similarly as possible on different execution platforms.

  5. As a researcher developing new technology, I can reutilize industry and open source scenarios to advance my research.

B) CERTIFY & ANALYSE

  1. As an auditor/regulator, I can understand how AV/ADAS developers are testing their products.

  2. As an auditor/regulator, I can compare the outcome of different execution platforms when they have the same OpenSCENARIO input.

  3. As a safety consultant, I can recommend specific scenarios and related conditions (parameters) to my clients to test their products.

  4. As a member of the public, I can learn more details about how AV/ADAS products are tested by AV/ADAS developers.

  5. As government agency, I can understand what parts of the Operational Domain are verified by an AV/ADAS developer through each scenario.

C) DEVELOP

  1. As a tool developer, I can reutilize constructs, artifacts and libraries to create tools compatible with other tool vendors in industry.

  2. As a service provider, I can integrate tools from multiple tool vendors to provide an integrated solution to test AV/ADAS scenarios.

  3. As a system engineer working for an AV/ADAS developer company, I can fully trace which hardware and software in the AV/ADAS stack is verified by which tests.

  4. As a software developer, I can process scenario information from different software/hardware releases and produce comparison charts to provide trend and gap analysis.

  5. As an existing tool provider or consumer, I can migrate information from previous versions of OpenSCENARIO into OpenSCENARIO 2.0.

  6. As a system engineer working for an AV/ADAS developer, I can decompose high level use cases in a standardised way.

D) CREATE

  1. As a content developer, I can use OpenSCENARIO 2.0 to create test scenarios that I can supply to my customers who use a OpenSCENARIO 2.0 compliant toolchain.

  2. As a test engineer, I can transform abstract test descriptions into OpenSCENARIO 2.0 (e.g. NCAP tests, UNECE regulations, any requirement list, …​).

  3. As a development project lead, I can write scenarios on an abstract level to discuss the functional behavior with the stakeholders.

  4. As a development project lead, I can create scenarios on an abstract level to document the functional behavior for legal reasons.

  5. As a stakeholder, I can create natural language scenarios without having any knowledge about technical details.

E) SOTIF-BASED RISK CONSIDERATION

  1. As a SOTIF safety engineer and/or V&V engineer, AV developer, scenario creator, I can use OpenSCENARIO 2.0 to discover scenarios that are going to uncover safety hazards. This refer to SOTIF and safety hazards that can be present even if the system is functioning as designed, without a malfunction.

  2. As a SOTIF safety engineer and/or V&V engineer, AV developer, scenario creator, I can use OpenSCENARIO 2.0 to create scenarios that are going to produce emergent behavior of the DUT to discover unknown unknows. OpenSCENARIO 2.0 shall enable to demonstrate that minimum residual risk is attained by the DUT. This is because SOTIF focuses on ensuring the absence of unreasonable risk due to hazards resulting from insufficiencies in the intended functionality or from reasonably foreseeable misuse.

F) DRIVING MISSION-BASED SCENARIOS

  1. As an end-to-end V&V engineer, I can use OpenSCENARIO 2.0 to enable specification of a driving mission through inclusion of multiple maneuvers in a sequence or in parallel for both DUT and any other traffic agents.

  2. As an end-to-end V&V engineer, I can use OpenSCENARIO 2.0 to enable accomplishing a select driving mission with an indication of whether the mission has been accomplished, what are the mission KPIs and how they are computed, and whether the unambiguous goals of the mission have been attained.

G) TRAFFIC MODEL INCLUSION

  1. As a traffic model developer, an ADS developer, or end-to-end V&V engineer, I can use OpenSCENARIO 2.0 to enable inclusion of multiple traffic models and AI-based traffic agents in the scenarios and evaluators. Also, OpenSCENARIO 2.0 shall enable inclusion of mechanisms to extract scenarios from the traffic models.

    OpenSCENARIO 2.0 shall enable inclusion of ground truth for comparison of expected behavior of traffic agents in addition to the DUT car.
  2. As a test engineer, I can transform from high level scenario description to low level scenario descriptions and vice versa.

H) EXECUTE

  1. As a test engineer, I can execute different OpenSCENARIO 2.0 files in an automated way with my OpenSCENARIO 2.0 compliant toolchain; no file modifications are needed.

  2. As a test engineer, I can execute the same OpenSCENARIO 2.0 files on different OpenSCENARIO 2.0 compliant toolchains.

  3. As a test engineer, I can convert abstract scenarios into tests.

I) DESCRIBE OBSERVATIONS

  1. A simulation tool can describe randomly executed simulation runs. If the Simulation was run with stochastic metrics, the user wants to have a concrete description of what happened in an OpenSCENARIO2.0 Format.

  2. A traffic observer can describe what occured in real world with OpenSCENARIO2.0

  3. A test engineer on a test track can describe with OpenSCENARIO2.0 what specific scenario he has observed on a test track. In such a way tolerances and deviations between test description and test execution will become obvious.

3.2. Use Cases for Scenarios

AV use cases are influenced by a variety of factors, such as ODDs, map features, traffic model, weather model, and others. The scenarios assigned to the use cases can be parametrized and typically run in multiple modes of operations. These modes include:

  • On track test

  • On road tests

  • In sim tests

  • In replay (i.e. resimulation) tests

The intent of OpenScenario 2.0 is to cover use cases at varying levels of autonomy. Hence, they should represent an adequate level of complexity including maneuvers and ODD features that are not accounted for otherwise. An example is an AV driving on a complex road with road features such as roundabouts, bus stops, highway ramps, entries and exits or in dense traffic where various maneuvers like cut-ins, lane reductions and cross traffic with other agents on the road.

The following list of use cases for scenarios attempt to cover the majority of the currently considered scope of features of OpenSCENARIO. The list adheres to the following format:

Summary

Title

Related user story(s)

Which user stories are related (Section 3.1)?

Covered abstraction levels

Of the multiple abstraction levels, which are relevant?

Description

Detailed break down of the example use case

Example scenario
  • Function description (not part of the scenario):

    • Customer function level:

      • System behavior level:

  • Scenario description:

    • Abstract/concrete description:

  • Test description (not part of the scenario):

    • Precondition:

    • Test description:

Additional information

…​

3.2.1. Use Case Example 1: Evaluation/validation/checks

Example for demonstrating autonomous vehicle functionality. A simple example for a generic use case in Level 2 driving domain comprised of a scenario and an evaluator.

Summary

Evaluator checks/validation for scenario descriptions

Applicable user story(s)

D3

Covered abstraction levels

Mainly abstract, concrete details

Description

Scenario of an ego car driving on a road between the lanes. The evaluator checks whether the ego car is able to keep driving between the lanes. Expected outcome: Car must drive segments of oval map with LK feature enabled at all times and without any failures.

Example scenario
  • Abstract/concrete description: Ego car starts driving with 40 km/hour speed at a given location in oval map.

  • Test description: Enable Lane Keeping Assistance (LK) at 10 seconds. Let the car drive in oval map for another 120 seconds.

Additional information

The specification of an initial design for 'Lane Keep Assistance' is provided below. The table represents the flow of events required in the scenario and the corresponding flow of events in the evaluator.

usecase 1 lk
Scenario Sequence Evaluator

EgoStartPosition

Pre:: Detect Ego position (centered in Ego lane)

EgoStartSpeed

Pre:: Detect Ego start speed (e.g. 40km/h)

EgoStartLK

Pre:: Detect Ego engages LK (e.g. after 10s of driving)

SetWatchdog

Pre:: Start timer for maneuver (e.g. 120s duration)

EgoSensorOK

Pre:: Assert Ego sensors are functional and clean

ActivateEgoLK

Post:: Assert Ego LK working and engaged

AssistEgoWithLK

Post:: Assert that the distance between Ego and center of the lane is less than a threshold (e.g. 1 meter deviation indicates fail)

3.2.2. Use Case Example 2: Regulations

Summary

Regulations are one source for verification and validation of vehicles. Regulations contain requirements that shall be fulfilled by vehicles and corresponding test descriptions. The information about maneuvers, parameters, test conditions etc. is only given in plain natural language. With OpenSCENARIO 2.0 one can create corresponding scenarios. Those scenarios can be used for test execution, sharing and analyses. Details of UN R 79 as one example can be found in the following section. As example the lane change procedure suppression in a critical situation is given.

Applicable user story(s)

A*, B*, C*, D*, E*

Covered abstraction levels

Mainly abstract, partly concrete

Description

Lane change procedure suppression in a critical situation; UN Regulation No. 79 for Automatically Commanded Steering Function (ACSF) of Category C [UN R79]

"ACSF of Category C" implies a function, which is initiated/activated by the driver and which can perform a single lateral maneuver (e.g. lane change) when commanded by the driver.

Requirements are given as follows (as described in section 5.6.4.6.8 of the regulation):

  • The lane change procedure shall be suppressed automatically by the system when the system detects a critical situation before the lane change maneuver has started (5.6.4.7)

    • A situation is deemed to be critical when, at the time a lane change maneuver starts, an approaching vehicle in the target lane would have to deccelerate at a higher level than 3m/s, 0.4 seconds after the lane change maneuver has started, to ensure the distance between the two vehicles is never less than that which the lane change vehicle travels in 1 second.

Example scenario
  • Abstract/concrete description: The test vehicle shall be driven in a lane of a straight test track, which has at least two lanes in the same direction of travel, with road markings on each side of the lanes (as in section 3.5.4 of UNR79).

    1. The vehicle speed shall be: Vsmin + 10km/h.

    2. The ACSF of Category C shall be activated (standby mode) and another vehicle shall approach from the rear in order to enable the system (5.6.4.8.3).

    3. The approaching vehicle shall then pass the vehicle under test entirely.

    4. A Lane Change Procedure shall then be initiated by the driver.

    5. The test shall be repeated for the following condition, which shall occur before the lane change maneuver has started:

      • The lane change maneuver has not commenced within 5.0 seconds following the initiation of the lane change procedure. (e.g. another vehicle is driving in the adjacent lane in a critical situation as described in paragraph 5.6.4.7.).

    6. The requirements of the test are fulfilled if the lane change procedure is suppressed, for each of the test cases above (3.5.4.2)

  • Test description: Enable Lane Keeping Assistance (LK) at 10 seconds. Let the car drive in oval map for another 120 seconds.

Additional information

A Lane Change Procedure in the case of ACSF of Category C starts when the direction indicator lamps are activated by a deliberate action of the driver and ends when the direction indicator lamps are deactivated. It comprises the following operations:

  • Activation of the direction indicator lamps by a deliberate action of the driver

  • Lateral movement of the vehicle towards the lane boundary

  • Lane Change Maneuver

  • Resumption of the lane keeping function

  • Deactivation of direction indicator lamps.

A "Lane Change Maneuver" is part of the Lane Change Procedure and:

  • Starts when the outside edge of the tire tread of the vehicle’s front wheel closest to the lane markings touches the inside edge of the lane marking to which the vehicle is being maneuverd,

  • Ends when the rear wheels of the vehicle have fully crossed the lane marking.

3.2.3. Use Case Example 3: Electronic stability control (ESC) testing

Summary

Sine with dwell test sequence, including slowly increasing steer as preparation maneuver.

Applicable user story(s)

A4, A5, B3, D2, E1, E2

Covered abstraction levels

Concrete level (ISO specifies concrete values for the shown scenario)

Description

As a preparation for the actual sine with dwell test series, the vehicle is subjected to two series of runs of the slowly increasing steer test, using a constant vehicle speed of 78-82 kph and a steering pattern that increases by 13.5 deg/s until a lateral acceleration of 0.5 g is obtained. Each test series includes three repetitions. One series uses counter clockwise steering, and the other series uses clockwise steering. The goal is to determine a reference steering wheel angle A, which represents the steering wheel angle leading to a steady-state lateral acceleration of 0.3 g for the test vehicle.

As a preparation for the actual sine with dwell test series, the vehicle is subjected to two series of runs of the slowly increasing steer test, using a constant vehicle speed of 80 km/h and a steering pattern that increases by 13.5 deg/s until a lateral acceleration of 0.5 g is obtained. Each test series includes three repetitions. One series uses counter clockwise steering, and the other series uses clockwise steering. The goal is to determine a reference steering wheel angle A, which represents the steering wheel angle leading to a steady-state lateral acceleration of 0.3 g for the test vehicle.

During the actual sine with dwell test, the vehicle is steered using a steering pattern of a sine wave at a frequency of 0.7 Hz with a delay of 500 ms beginning at the second peak amplitude, see Figure below. Several tests are performed in a series in which the amplitude of the steering pattern is increased with each test. The series is performed twice, where one series starts with counter clockwise steering, and the other series start with clockwise steering. The sine with dwell test series starts with a specific fraction of the reference angle A, and is increased from run to run by half the reference angle. The calculation for the maximum steering angle amplitude to be applied is also specified by the standard.

Example scenario

N/A

Additional information

The main purpose of ISO 19365 is to provide a repeatable and discriminatory method for comparing simulation results to measured test data from a physical vehicle for sine with dwell tests, which are typically used to evaluate the performance of an electronic stability control (ESC) system, see "Passenger cars - Validation of vehicle dynamic simulation - Sine with dwell stability control testing (ISO 19365:2016)".

The comparison is made for the purpose of validating the simulation tool for this type of test when applied to variants of the tested vehicle. It is applicable to passenger cars as defined in ISO 3833. The sine with dwell test method is based on the test method specified in regulations USA FMVSS 126 and UN/ECE Regulation No. 13-H.

3.2.4. Use Case Example 4: Stakeholder Discussion

Summary

Scenario usage as a basis for stakeholder discussion.

Applicable user story(s)

A1, A2, C6, D3, D4, D5

Covered abstraction levels

Mainly abstract, probably concrete details

Description

The stakeholders in the context of a driving function development project provide information about the project setup, neighbor systems, sensors, actors and software/hardware constraints. To define the driving function, the project lead needs to discuss the function goals and behavior with the stakeholders. For this, a short and easily comprehensible scenario description, which is understandable without detailed technical knowledge, should be available. A corresponding two or three dimensional visualization can help.

Example scenario

Function description (not part of the scenario):

  • Customer function level: My driving function (emergency brake assist) has to perform an emergency braking, if either an object in the trajectory appears suddenly and the driver has no chance to react, or if the driver fails to react in a normal braking situation and the distance to the object falls below a threshold

  • System behavior level: If the distance to an object in the trajectory falls below threshold C1 and the speed is above V1, inform the user visually. If the user does not react within X seconds, perform an emergency brake with -15 m/s2.

    Scenario description:

  • Abstract description: Highway with at least one lane, a car follows one lane, an object is in front of the car. The distance is Y meters.

  • Logical and concrete descriptions: extensions of the abstract description, e.g. by defining lane width, car size, etc.

Test description (not part of the scenario):

  • Precondition: Apply the scenario described above

  • System-under-test: Implementation of the developed driving function

  • Test description:

    • If dist_obj < C1 && v > V1, check if message is shown to the user.

      • Start timer t.

    • If t > X, check if brake == true && a < -15 m/s2.

      • If true, test = pass, otherwise test = fail.

3.2.5. Use Case Example 5: Proving ground testing

Summary

Scenarios for proving ground testing.

Applicable user story(s)

A4, D1, D2, E1, E2, E3

Covered abstraction levels

Concrete

Description

A scenario description is required to enable the tester to test a scenario on the proving ground.

Example scenario

Scenario description: Cut-In on highway. Only option for VUT is to emergency brake.

  • Scenario takes place on a 2-lane highway. VUT is following another vehicle (TSV#3) with Highway-Pilot (L3) active. VUT and lead-vehicle are on the left lane. An additional Vehicle (TSV#2) is right next to VUT on the right lane. It stays besides the VUT with an offset of 4m from front bumper to front bumper. A slow vehicle (TSV#1) is changing lane from right to left and cuts in between TSV#3 and VUT. The VUT has only the option to emergency brake and not to change lane.

  • Stationary Conditions:

    • Speed of TSV#3 = 80km/h

    • Speed of TSV#1 = 20km/h

    • Lane Change of TSV#2: Length = 20m; Offset = 3.7m to left; shape = sinusoidal

  • Triggers:

    • TSV#1 which does the lane change: LC has to be completed when distance of VUT to TSV#1 is 20m

  • Control:

    • TSV#2 has to follow the position of VUT on the right lane with an offset of 4m from front bumper to front bumper

usecase 4

Additional Information

Birds-eye view perspective with logged data (Youtube)

3.2.6. Use Case example 6: NHTSA Scenario Descriptions

Summary

Scenarios from NHTSA test description formats should be fully coverable by an OpenSCENARIO 2.0 description

Applicable user story(s)

A1, A4, B1, B4, D2, D3, E3

Covered abstraction levels

Concrete

Description

The DUT is travelling straight in the second rightmost lane at 40 mph (lane speed). DUT is approaching a signalized intersection with a red signal state while a pedestrian is crossing at a speed of 1.2 m/s.

usecase 6

Example scenario
  • Meta-Data:

    • Map/Template: PHX

    • N Scottsdale Rd & E Thomas Rd

    • GPS: 33.4802924, -111.9259426

    • Density: Low ( 2 of each )

    • Vehicle: 1 Static / 1 Dynamic

    • Bicycle: 1 Static / 1 Dynamic

    • Pedestrian: 1 Static / 1 Dynamic

    • Lane Count: 6

    • Actor: Pedestrian

    • Scenario Total: 1

  • Baseline Scenario:

    • Expected SDV Speed: 40 mph (17.88 m/s)

    • SDV Start Position:

      • Right Travel Lane (second lane from the right)

      • 60 m before the intersection on N Scottsdale Rd.

      • Heading northbound

    • End Position: 50 m after the intersection

    • Actor Starting Position:

      • 0 m at the crosswalk on N Scottsdale

      • Front Right Near side of the intersection

    • End Position:

      • 2 m on the other side of the intersection

      • Crossing N Scottsdale

      • Moving from Front Right Near to Front Left Near, ~35 m

    • Actor Speed: 1.2 m/s

    • Trigger Points:

      • Pedestrian OverlapAction Trigger_BP

      • 40 m before the intersection

    • Trigger Variation Extent: N/A

  • Expected Termination Condition(s):

    • No Progress Timeout

  • Metrics:

    • Simulation_standard_test

Additional information
  • Composite/OD Tags

  • Infrastructure

  • Composite Tags

3.2.7. Use Case Example 7: Replay Observed Traffic Situation

Summary

A traffic camera observes certain traffic situations. This measured data will be transformed into OpenSCENARIO2.0. At the end a OpenSCENARIO2.0 file will contain the concrete scenario description. This example shall be representative for all measured data, no matter if recorded by a drone, from within a car, during test track runs or simulation runs or real world traffic.

Applicable user story(s)

I1, I2, I3

Covered abstraction levels

Concrete

Description

A highway section is observed from the bird perspective. In one period of observation a cut-in maneuver is observed and this traffic situation shall be converted into a concrete scenario description.

Example scenario

Following section of highway is observed.

usecase 7

Data from following measurements channels is available:

Name Description Unit

frame

The current frame.

[-]

id

The track’s id.

[-]

x

The x position of the upper left corner of the vehicle’s bounding box.

[m]

y

The y position of the upper left corner of the vehicle’s bounding box.

[m]

width

The width of the bounding box of the vehicle.

[m]

height

The height of the bounding box of the vehicle.

[m]

xVelocity

The longitudinal velocity in the image coordinate system.

[m/s]

yVelocity

The lateral velocity in the image coordinate system.

[m/s]

xAcceleration

The longitudinal acceleration in the image coordinate system.

[m/s]

yAcceleration

The lateral acceleration in the image coordinate system

[m/s]

frontSightDistance

The distance to the end of the recorded highway section in driving direction from the vehicle’s center.

[m]

backSightDistance

The distance to the end of the recorded highway section in the opposite driving direction from the vehicle’s center.

[m]

dhw

The Distance Headway. This value is set to 0, if no preceding vehicle exists.

[m]

thw

The Time Headway. This value is set to 0, if no preceding vehicle exists.

[s]

ttc

The Time-to-Collision. This value is set to 0, if no preceding vehicle or valid TTC exists.

[s]

precedingXVelocity

The longitudinal velocity of the preceding in the image coordinate system. This value is set to 0, if no preceding vehicle exists.

[-]

precedingId

The id of the preceding vehicle in the same lane. This value is set to 0, if no preceding vehicle exists.

[-]

followingId

The id of the following vehicle in the same lane. This value is set to 0, if no following vehicle exists.

[-]

leftPrecedingId

The id of the preceding vehicle on the adjacent lane on the left in the direction of travel. This value is set to 0, if no such a vehicle exists.

[-]

leftAlongsideId

The id of the adjacent vehicle on the adjacent lane on the left in the direction of travel. In order for a vehicle to be adjacent and not e.g. preceding, the vehicles must overlap in the longitudinal direction. This value is set to 0, if no such a vehicle exists.

[-]

leftFollowingId

The id of the following vehicle on the adjacent lane on the left in the direction of travel. This value is set to 0, if no such a vehicle exists.

[-]

rightPrecedingId

The id of the preceding vehicle on the adjacent lane on the right in the direction of travel. This value is set to 0, if no such a vehicle exists.

[-]

rightAlsongsideId

The id of the adjacent vehicle on the adjacent lane on the right in the direction of travel. In order for a vehicle to be adjacent and not e.g. preceding, the vehicles must overlap in the longitudinal direction. This value is set to 0, if no such a vehicle exists.

[-]

rightFollowingId

The id of the following vehicle on the adjacent lane on the right in the direction of travel. This value is set to 0, if no such a vehicle exists.

[-]

laneId

The IDs start at 1 and are assigned in ascending order. Since the Lane ids are derived from the positions of the lane markings, the first and last ids typically do not describe any useable lanes. For details, see the definition of the coordinate system.

[-]

A compiler transforms the measured data into a scenario file. This example comes from the highD Dataset. Other data sources might have slightly different measurement procedures. This is just one example for the use case and should be representative for all measured data that needs to be transformed into OpenSCENARIO2.0.

3.2.8. Use Case Example 8: Scenario definition for entering a roundabout

Summary

A left turn at a roundabout should be described as a scenario

Applicable user story(s)

D1

Covered abstraction levels

Concrete

Description