Software systems may fail in production environments, causing system crashes, erroneous outputs, and overall system instability. Thoroughly testing software systems in development environments can reduce but not avoid failures in production environments, due to both the complexity of contemporary software applications that lead to a myriad of execution conditions impossible to sample exhaustively, and behaviors that emerge in production and cannot be predicted and exercised during development.

This paper presents an approach for testing software systems in production environments to explore novel execution scenarios and detect error states early that emerge only in production environments, before they propagate to system failures. We propose field-ready test cases that combine test data derived from the production environment to execute software components in the presence of yet unexplored execution scenarios, with general assertions to expose error states, before the system fails. Intuitively, the approach conceives the production environment as a testbed for opportunistically executing unit test cases that exploit objects that become available only in production as test data, to reveal error states before system failures.

The paper presents the results of a set of experiments with field-ready test cases that we conservatively produced for the JFreeChart and Apache Commons Lang libraries by adapting their original test suites. Our field-ready test suites execute 64% faults that the original test suites miss, and exposes 33% failures.