Write a Blog >>
ICST 2022
Mon 4 - Fri 8 April 2022
Tue 5 Apr 2022 14:30 - 14:45 at Margaret Hamilton - ICST Field / Production testing Chair(s): Annibale Panichella

We introduce FAUSTA, an algorithmic traffic generation platform that enables analysis and testing at scale. FAUSTA has been deployed at Facebook to analyze and test the WhatsApp platform infrastructure since September 2020, enabling WhatsApp developers to deploy reliable code changes to a code base of millions of lines of code, supporting over 2 billion users who rely on WhatsApp for their daily communications. FAUSTA covers expected and unexpected program behaviors in a privacy-safe controlled environment to support multiple use cases such as reliability testing, privacy analysis and performance regression detection. It currently supports three different algorithmic input generation strategies, each of which construct realistic backend server traffic that closely simulates production data, without replaying any real user data. FAUSTA has been deployed and closely integrated into the WhatsApp continuous integration process, catching bugs in development before they hit production. We report on the development and deployment of FAUSTA’s reliability use case between September 2020 and August 2021 during which period it has found 1,876 unique reliability issues, with a fix rate of 74%, indicating a high degree of true positive fault revelation. We also report on the distribution of fault types revealed by FAUSTA, and the correlation between coverage and faults found. Overall, we do find evidence that higher coverage is correlated with fault revelation.

Tue 5 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:00 - 15:15
ICST Field / Production testingResearch Papers / Industry at Margaret Hamilton
Chair(s): Annibale Panichella Delft University of Technology
14:00
15m
Talk
Testing Software in Production Environments with Data from the Field
Research Papers
Luca Gazzola Università degli Studi di Milano-Bicocca, Leonardo Mariani University of Milano-Bicocca, Matteo Orru University of Milano Bicocca, Mauro Pezze USI Lugano; Schaffhausen Institute of Technology, Martin Tappler TU Graz; Silicon Austria Labs
14:15
15m
Talk
Automatic Error Classification and Root Cause Determination while Replaying Recorded Workload Data at SAP HANA
Industry
Neetha Jambigi University of Innsbruck, Thomas Bach SAP, Felix Schabernack SAP, Michael Felderer University of Innsbruck
14:30
15m
Talk
FAUSTA: Scaling Dynamic Analysis with Traffic Generation at WhatsApp
Industry
Ke Mao Meta, Timotej Kapus Facebook, Inc., Lambros Petrou Facebook, Inc., Ákos Hajdu Facebook, Matteo Marescotti Facebook, Inc., Andreas Löscher Facebook, Inc., Mark Harman Facebook, Inc., Dino Distefano Facebook London
14:45
15m
Talk
Smoke Testing of Cloud Systems
Research Papers
Cecilio Cannavacciuolo , Leonardo Mariani University of Milano-Bicocca
15:00
15m
Live Q&A
Discussion and Q&A
Research Papers