Write a Blog >>
ICST 2022
Mon 4 - Fri 8 April 2022
Wed 6 Apr 2022 17:00 - 17:15 at Margaret Hamilton - ICST Empirical Study Chair(s): Mohammad Reza Mousavi

Test flakiness forms a major testing concern. Flaky tests manifest non-deterministic outcomes that cripple continuous integration and lead developers to investigate false alerts. Industrial reports indicate that on a large scale, the accrual of flaky tests breaks the trust in test suites and entails significant computational cost. To alleviate this, practitioners are constrained to identify flaky tests and investigate their impact. To shed light on such mitigation mechanisms, we interview 14 practitioners and perform a grey literature review with the aim to identify (i) sources of flakiness within the testing ecosystem, (ii) impacts of flakiness, (iii) measures adopted by practitioners when addressing flakiness, and (iv) automation opportunities for these measures. Our analysis shows that, besides tests and code, flakiness stems from interactions between the system components, the testing infrastructure, and external factors. We also highlight the impact of flakiness on testing practices and product quality and show that the adoption of guidelines together with a stable infrastructure are key measures in mitigating the problem.

Wed 6 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

16:45 - 18:00
ICST Empirical StudyJournal-First Papers / Research Papers at Margaret Hamilton
Chair(s): Mohammad Reza Mousavi King's College London
16:45
15m
Talk
An experimental and practical study on the equivalent mutant connection: An evolutionary approach
Journal-First Papers
Pedro Delgado-Pérez Universidad de Cádiz, Francisco Chicano University of Malaga
Link to publication DOI
17:00
15m
Talk
A Qualitative Study on the Sources, Impacts, and Mitigation Strategies of Flaky Tests
Research Papers
Sarra Habchi University of Luxembourg, Luxembourg, Guillaume Haben University of Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg
Pre-print
17:15
15m
Talk
As Code Testing: Characterizing Test Quality in Open Source Ansible Development
Research Papers
Mohammed Mehedi Hasan Independent University, Akond Rahman Tennessee Tech University
Pre-print File Attached
17:30
15m
Talk
A Survey on How Test Flakiness Affects Developers and What Support They Need To Address It
Research Papers
Martin Gruber BMW Group, University of Passau, Gordon Fraser University of Passau
Pre-print
17:45
15m
Live Q&A
Discussion and Q&A
Research Papers