Write a Blog >>
ICST 2022
Mon 4 - Fri 8 April 2022
Wed 6 Apr 2022 15:30 - 15:45 at Margaret Hamilton - ICST AI II Chair(s): Donghwan Shin

Testing video games is an increasingly difficult task as traditional methods fail to scale with growing software systems. Manual testing is a very labor-intensive process, and therefore quickly becomes cost prohibitive. Using scripts for automated testing is affordable, however scripts are ineffective in non-deterministic environments, and knowing when to run each test is another problem altogether. Manual testing and writing scripts make up the current industry standard and methodology for game testing, but the writing is on the wall for this practice. The modern game’s complexity, scope, and player expectations are rapidly increasing where quality control is a big portion of the production cost and delivery risk. Reducing this risk and making production happen is a big challenge for the industry currently. To keep production costs realistic up-to and after release, we are focusing on preventive quality assurance tactics alongside testing and data analysis automation. We present SUPERNOVA (Selection of tests and Universal defect Prevention in External Repositories for Novel Objective Verification of software Anomalies), a system responsible for test selection and defect prevention while also functioning as an automation hub. By integrating data analysis functionality with machine and deep learning capability, SUPERNOVA assists quality assurance testers in finding bugs and developers in reducing defects, which improves stability during the production cycle and keeps testing costs under control. The direct impact of this has been observed to be a reduction in 55% or more testing hours for an undisclosed sports game title that has shipped, which was using these test selection optimizations. Furthermore, using risk scores generated by a semi-supervised machine learning model, we are able to detect with 71% precision and 77% recall the probability of a change-list being bug inducing, and provide a detailed breakdown of this inference to developers. These efforts improve workflow and reduce testing hours required on game titles in development.

Wed 6 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

15:15 - 16:30
ICST AI IITesting Tools / Research Papers / Industry at Margaret Hamilton
Chair(s): Donghwan Shin University of Luxembourg
15:15
15m
Talk
Learning Realistic Mutations: Bug Creation for Neural Bug Detectors
Research Papers
Cedric Richter Carl von Ossietzky Universität Oldenburg / University of Oldenburg, Heike Wehrheim Carl von Ossietzky Universität Oldenburg / University of Oldenburg
15:30
15m
Talk
SUPERNOVA: Automating Test Selection and Defect Prevention in AAA Video Games Using Risk Based Testing and Machine Learning
Industry
Alexander Senchenko Electronics Arts, Naomi Patterson Electronics Arts, Hamman Samuel Electronics Arts, Dan Ispir Electronics Arts
Pre-print
15:45
15m
Talk
RiverGame - a game testing tool using artificial intelligence
Testing Tools
Ciprian Paduraru University of Bucharest, Miruna Gabriela Paduraru University of Bucharest , Alin Stefanescu University of Bucharest
16:00
15m
Talk
Machine Learning Based Invariant Generation: A Framework and Reproducibility Study
Research Papers
Jan Haltermann University of Oldenburg, Heike Wehrheim Carl von Ossietzky Universität Oldenburg / University of Oldenburg
16:15
15m
Live Q&A
Discussion and Q&A
Research Papers