Write a Blog >>
ICST 2022
Mon 4 - Fri 8 April 2022
Tue 5 Apr 2022 14:00 - 14:15 at Margaret Hamilton - ICST Field / Production testing Chair(s): Annibale Panichella

Software systems may fail in production environments, causing system crashes, erroneous outputs, and overall system instability. Thoroughly testing software systems in development environments can reduce but not avoid failures in production environments, due to both the complexity of contemporary software applications that lead to a myriad of execution conditions impossible to sample exhaustively, and behaviors that emerge in production and cannot be predicted and exercised during development.

This paper presents an approach for testing software systems in production environments to explore novel execution scenarios and detect error states early that emerge only in production environments, before they propagate to system failures. We propose field-ready test cases that combine test data derived from the production environment to execute software components in the presence of yet unexplored execution scenarios, with general assertions to expose error states, before the system fails. Intuitively, the approach conceives the production environment as a testbed for opportunistically executing unit test cases that exploit objects that become available only in production as test data, to reveal error states before system failures.

The paper presents the results of a set of experiments with field-ready test cases that we conservatively produced for the JFreeChart and Apache Commons Lang libraries by adapting their original test suites. Our field-ready test suites execute 64% faults that the original test suites miss, and exposes 33% failures.

Tue 5 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:00 - 15:15
ICST Field / Production testingResearch Papers / Industry at Margaret Hamilton
Chair(s): Annibale Panichella Delft University of Technology
14:00
15m
Talk
Testing Software in Production Environments with Data from the Field
Research Papers
Luca Gazzola Università degli Studi di Milano-Bicocca, Leonardo Mariani University of Milano-Bicocca, Matteo Orru University of Milano Bicocca, Mauro Pezze USI Lugano; Schaffhausen Institute of Technology, Martin Tappler TU Graz; Silicon Austria Labs
14:15
15m
Talk
Automatic Error Classification and Root Cause Determination while Replaying Recorded Workload Data at SAP HANA
Industry
Neetha Jambigi University of Innsbruck, Thomas Bach SAP, Felix Schabernack SAP, Michael Felderer University of Innsbruck
14:30
15m
Talk
FAUSTA: Scaling Dynamic Analysis with Traffic Generation at WhatsApp
Industry
Ke Mao Meta, Timotej Kapus Facebook, Inc., Lambros Petrou Facebook, Inc., Ákos Hajdu Facebook, Matteo Marescotti Facebook, Inc., Andreas Löscher Facebook, Inc., Mark Harman Facebook, Inc., Dino Distefano Facebook London
14:45
15m
Talk
Smoke Testing of Cloud Systems
Research Papers
Cecilio Cannavacciuolo , Leonardo Mariani University of Milano-Bicocca
15:00
15m
Live Q&A
Discussion and Q&A
Research Papers