Scenario-Based Testing of Cyber-Physical Systems

Description

In recent years, the development of automated and autonomously operating cyber-physical systems such as cars or drones is on the rise. Testing the safe behavior of these complex systems in all relevant situations is a challenge that is not easily solved.
Due to the high amount of needed test samples, the verification and validation of cyber-physical systems only with real-world testing is infeasible. To complement these efforts, we propose the use of scenario-based testing. In this method, the cyber-physical system is tested in different scenario types, which describe situations in which we aim to test the system's behavior. For cars, examples of these scenario types are "Lane Change" or "Cut-In". Each of these scenario types includes parameters to describe its various aspects, e.g., starting position of the system under test. When generating test cases for these scenario types, concrete values are assigned to all parameters of the considered scenario type.
 

Since it is not feasible to test the cyber-physical system in all possible scenario instances of one scenario type, we need to select "good" test cases that challenge the system under test. A test case is a "good" test case if it can reveal potential faults in the system. When generating these "good" test cases, different optimization techniques such as search-based techniques can be used.

Publications

  • Kolb, Nicola; Schmidt, Tabea; Fröhlich, Joachim; Pretschner, Alexander: When to Apply Scenario-Based Testing to a New System Domain? 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), IEEE, 2023 mehr… BibTeX Volltext ( DOI )
  • Kolb, Nicola; Hauer, Florian; Golagha, Mojdeh; Pretschner, Alexander: Data-Driven Assessment of Parameterized Scenarios for Autonomous Vehicles. In: Lecture Notes in Computer Science. Springer International Publishing, 2022 mehr… BibTeX Volltext ( DOI )
  • Kolb, Nicola; Jordan, Claudius; Huber, Florian; Pretschner, Alexander: Automatic Evaluation of Automatically Derived Semantic Scenario Instance Descriptions. 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), IEEE, 2022 mehr… BibTeX Volltext ( DOI )
  • Schmidt, Tabea; Pretschner, Alexander: StellaUAV: A Tool for Testing the Safe Behavior of UAVs with Scenario-Based Testing. IEEE International Symposium on Software Reliability Engineering (ISSRE) , IEEE, 2022, 37-48 mehr… BibTeX Volltext ( DOI ) Volltext (mediaTUM)
  • Pretschner, Alexander; Hauer, Florian; Schmidt, Tabea: Tests für automatisierte und autonome Fahrsysteme. Informatik Spektrum, 2021 mehr… BibTeX Volltext ( DOI )
  • Kolb, Nicola; Hauer, Florian; Pretschner, Alexander: Fitness Function Templates for Testing Automated and Autonomous Driving Systems in Intersection Scenarios. 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), IEEE, 2021 mehr… BibTeX Volltext ( DOI )
  • Schmidt, Tabea; Hauer, Florian; Pretschner, Alexander: Understanding Safety for Unmanned Aerial Vehicles in Urban Environments. 2021 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2021 mehr… BibTeX Volltext ( DOI )
  • Florian Hauer, Bernd Holzmüller: A sound approach to scenario-based testing as the basis for safety argumentations. Automotive Testing Technology Magazine Special Issue, 2020 mehr… BibTeX
  • Calo, Alessandro; Arcaini, Paolo; Ali, Shaukat; Hauer, Florian; Ishikawa, Fuyuki: Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems. 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), IEEE, 2020 mehr… BibTeX Volltext ( DOI )
  • Calò, Alessandro; Arcaini, Paolo; Ali, Shaukat; Hauer, Florian; Ishikawa, Fuyuki: Simultaneously searching and solving multiple avoidable collisions for testing autonomous driving systems. Proceedings of the 2020 Genetic and Evolutionary Computation Conference, ACM, 2020 mehr… BibTeX Volltext ( DOI )
  • Florian Hauer, Ilias Gerostathopoulos, Tabea Schmidt, Alexander Pretschner: Clustering Traffic Scenarios Using Mental Models as Little as Possible. 2020 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2020 mehr… BibTeX Volltext ( DOI ) Volltext (mediaTUM)
  • Florian Hauer, Alexander Pretschner, Bernd Holzmüller: Re-Using Concrete Test Scenarios Generally Is a Bad Idea. 2020 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2020 mehr… BibTeX Volltext ( DOI ) Volltext (mediaTUM)
  • Florian Hauer, Bernd Holzmüller: How Many Test Scenarios Do We Need for Testing Automated and Autonomous Driving Systems? TÜV / TUM Fachtagung "Automatisiertes Fahren" 2019 mehr… BibTeX Volltext (mediaTUM)
  • Florian Hauer, Tabea Schmidt, Bernd Holzmuller, Alexander Pretschner: Did We Test All Scenarios for Automated and Autonomous Driving Systems? 2019 IEEE Intelligent Transportation Systems Conference (ITSC) , IEEE , 2019 mehr… BibTeX Volltext ( DOI ) Volltext (mediaTUM)
  • Florian Hauer, Alexander Pretschner, Bernd Holzmüller: Fitness Functions for Testing Automated and Autonomous Driving Systems. 38th International Conference on Computer Safety, Reliability and Security, Springer International Publishing , 2019 mehr… BibTeX Volltext ( DOI ) Volltext (mediaTUM)
  • Florian Hauer, Bernd Holzmüller: Szenario-Optimierung für die Absicherung von automatisierten und autonomen Fahrsystemen – PrePrint für den Tagungsband der FKFS AutoTest Fachtagung 2018. Chair of Software and Systems Engineering, 2018, mehr… BibTeX Volltext (mediaTUM)