Write a Blog >>
ICST 2022
Mon 4 - Fri 8 April 2022
Wed 6 Apr 2022 10:00 - 10:15 at Margaret Hamilton - ICST Regression Testing Chair(s): Rachel Tzoref-Brill

Simulation-based testing is a popular approach for testing autonomous driving systems (ADS), in which different types of scenario are designed to test the ADS under different driving conditions. Given a specific test goal, a generation approach (e.g., search-based testing) is usually employed to find a scenario covering such goal; for example, it can find a scenario in which the autonomous vehicle collides. The generated scenarios may contain some elements that are irrelevant for the achievement of the test goal; if this is the case for a scenario that exposes a failure, for ADS engineers it is difficult to identify the root cause, as the ADS interacts with several traffic participants and it is not clear which of these are essential to trigger the failure. This problem emerged during the collaboration with our industry partner, for which, in the past, we proposed different test generation approaches for their ADS path planner, but these may produce test scenarios that are not minimal. To tackle this problem, in this paper, we propose an approach that, given an ADS test scenario, simplifies it by removing all the traffic participants that are not needed. As output, the approach provides a scenario that still covers the test goal as the initial scenario, but contains the minimum number of traffic participants. The approach consists in iteratively generating simplified scenarios by removing some traffic participants and determining, by observing the test execution, which can be actually removed and which must be kept in the scenario. Three policies are investigated to remove traffic participants whose classification is not know: single policy, binary policy, and adaptive policy. Experiments have been conducted on several scenarios generated for the path planner. Results show that the binary policy is the one that usually can find the minimal scenario with the minimum number of simplification attempts, but, in particular cases, the adaptive policy is better.

Wed 6 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

09:45 - 11:00
ICST Regression TestingResearch Papers / Journal-First Papers / Industry at Margaret Hamilton
Chair(s): Rachel Tzoref-Brill IBM Research
09:45
15m
Talk
TESRAC: A Framework for Test Suite Reduction Assessment at Scale
Research Papers
João Becho LaSiGE & FCUL, University of Lisbon, Frederico Cerveira CISUC & DEI, University of Coimbra, João Leitão NOVA LINCS & FCT, NOVA University of Lisbon, Rui André Oliveira NOVA LINCS & FCT, NOVA University of Lisbon
10:00
15m
Talk
Less is More: Simplification of Test Scenarios for Autonomous Driving System Testing
Industry
Paolo Arcaini National Institute of Informatics , Xiaoyi Zhang National Institute of Informatics, Japan, Fuyuki Ishikawa National Institute of Informatics
10:15
15m
Talk
An effective formulation of the multi-criteria test suite minimization problem
Journal-First Papers
Okan Orsan Ozener Özyeğin University, Hasan Sozer Ozyegin University
Link to publication DOI
10:30
15m
Talk
Dynamic test prioritization of product lines: An application on configurable simulation models
Journal-First Papers
Urtzi Markiegi Mondragon Unibertsitatea, Aitor Arrieta Mondragon University, Leire Etxeberria Mondragon Unibertsitatea, Goiuria Sagardui Mondragon University
Link to publication DOI
10:45
15m
Live Q&A
Discussion and Q&A
Research Papers