Deep learning testing is a new, recent area of software testing that is gaining increasing attention because of the widespread adoption of deep learning components to solve complex problems, such as image and natural language processing. This new area is developing its own language, methodology and toolset, often inheriting notions and approaches that proved to be successful for testing of traditional software. However, many such notions and approaches need to be revisited and reformulated to account for the specific features of deep learning components.

In this talk, I will consider the following questions associated with key, basic deep learning testing notions: What is a bug? What is a valid test input? Are Deep learning components deterministic? When is a fault detected? When is a mutant killed? What is a unique failure? I will give my answers to these questions, showing that adopting one definition or another has major implications on the conclusions we can draw about the effectiveness of novel testing techniques, or lack thereof.In this talk, I will consider the following questions associated with key, basic deep learning testing notions: What is a bug? What is a valid test input? Are Deep learning components deterministic? When is a fault detected? When is a mutant killed? What is a unique failure? I will give my answers to these questions, showing that adopting one definition or another has major implications on the conclusions we can draw about the effectiveness of novel testing techniques, or lack thereof.

Paolo Tonella is Full Professor at the Faculty of Informatics and at the Software Institute of Università della Svizzera Italiana (USI) in Lugano, Switzerland. He is Honorary Professor at University College London, UK and he is Affiliated Fellow of Fondazione Bruno Kessler, Trento, Italy, where he has been Head of Software Engineering until mid 2018. Paolo Tonella holds an ERC Advanced grant as Principal Investigator of the project PRECRIME. Paolo Tonella wrote over 150 peer reviewed conference papers and over 50 journal papers. His H-index (according to Google scholar) is 59. He is/was in the editorial board of the ACM Transactions on Software Engineering and Methodology, of the IEEE Transactions on Software Engineering, of Empirical Software Engineering, Springer, and of the Journal of Software: Evolution and Process, Wiley. Paolo Tonella teaches Information Modeling and Analysis at the Master in Data and Software Engineering.

Paolo Tonella has given foundational contributions to Software Engineering, in the area of code analysis and testing. His ICSE Most Influential Paper (MIP) award winning paper, Analysis and Testing of Web Applications, initiated a new stream of research devoted to the development of testing techniques for web applications. His comprehensive book Reverse Engineering of Object-Oriented Code laid the foundations for the reverse engineering of object-oriented systems. His ISSTA 2004 paper Evolutionary Testing of Classes is recognized as a milestone for the automated generation of object oriented test cases. One of the most widely used Java test case generators, EvoSuite, can be traced back to this seminal ISSTA paper and to the associated tool, eToc.