Automating Differential Testing with Overapproximate Symbolic Execution
Before releasing updated software, developers typically run regression test suites to verify that the code changes behave as intended and do not introduce unintended behaviors. Regression test suites, however, are often incomplete and can miss relevant software behaviors. To mitigate this issue, we introduce a technique for detecting potentially unintended behavioral changes. Given two versions of the software, our technique first finds functions that reach modified code in either version. It then performs under-constrained symbolic execution on both the original and revised programs, starting from the identified functions. If the execution reaches changed code, the technique compares corresponding program states across versions and logs differences. Because the so-generated raw log can contain a large number of redundant entries and a mixture of both intended and unintended differences, our technique clusters, orders, and ranks the logged differences to help developers identify feasible, unintended differences. We evaluated our technique by (1) applying it to a large set of known regressions, (2) comparing our results with those of a state-of-the-art approach, and (3) applying it to a set of refactored versions of a popular program, Redis. Overall, our results are promising. For the known regressions, our technique automatically identified more than half of the known regressions, generated a low number of false positives, ranked the true positives above the false positives in most cases, and reported more regressions than the state-of-the-art technique we used as a baseline. For the refactored Redis versions, our technique reported no regressions.
Thu 7 AprDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
16:45 - 18:00 | ICST Symbolic executionResearch Papers / Industry / Testing Tools / Journal-First Papers at Margaret Hamilton Chair(s): Arie van Deursen Delft University of Technology, Netherlands | ||
16:45 15mTalk | Translating EULYNX SysML models into Symbolic Transition Systems for Model-Based Testing of Railway signaling systems Industry Tobias Bachmann Universiteit van Amsterdam, Machiel van der Bijl Axini, Daan van der Meij ProRail, Djurre van der Wal University of Twente, Ana Oprescu University of Amsterdam | ||
17:00 15mTalk | Model checking C++ programs Journal-First Papers Felipe R. Monteiro Amazon, Mikhail R. Gadelha Igalia, Lucas C. Cordeiro University of Manchester, UK Link to publication DOI | ||
17:15 15mTalk | SIFT: A Tool for Property Directed Symbolic Execution of Multithreaded Software Testing Tools Tuba Yavuz University of Florida | ||
17:30 15mTalk | Automating Differential Testing with Overapproximate Symbolic Execution Research Papers | ||
17:45 15mLive Q&A | Discussion and Q&A Research Papers |