An Empirical Study of IR-based Bug Localization for Deep Learning-based Software
As the impact of deep-learning-based software (DLSW) increases, automatic debugging techniques for guaranteeing DLSW quality are becoming increasingly important. Information-retrieval-based bug localization (IRBL) techniques can aid in debugging by automatically localizing buggy entities (files and functions). The low-cost advantage of IRBL can alleviate the difficulty of identifying bug locations due to the complexity of DLSW. However, there are significant differences between DLSW and traditional software, and these differences lead to differences in search space and query quality for IRBL. That is, IRBL performance must be validated in DLSW.
We empirically validated IRBL performance for DLSW from the following four perspectives: 1) similarity model, 2) query generation, 3) ranking model for buggy file localization, and 4) ranking model for buggy function localization. Based on four research questions and a large-scale experiment using 2,365 bug reports from 136 DLSW projects, we confirmed the salient characteristics of DLSW from the perspective of IRBL and derived four recommendations for practical IRBL usage in DLSW from the empirical results. Regarding IRBL performance, we validated that IRBL performance with the combination of bug-related features outperformed that of using only file similarity by 15% and IRBL ranked buggy files and functions on average of 1.6th and 2.9th, respectively. Our study is valuable as a baseline for IRBL researchers and as a guideline for DLSW developers who wish to apply IRBL to ensure DLSW quality.
Thu 7 AprDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:00 - 11:00 | ICST Machine and Constraints LearningResearch Papers / Journal-First Papers at Margaret Hamilton Chair(s): Gunel Jahangirova USI Lugano | ||
10:00 15mTalk | Documentation-based functional constraint generation for library methods Journal-First Papers Renhe Jiang Nanjing University, Zhengzhao Chen Nanjing University, Yu Pei Hong Kong Polytechnic University, Minxue Pan Nanjing University, Tian Zhang Nanjing University, Xuandong Li Nanjing University Link to publication DOI | ||
10:15 15mTalk | Learning-based mutant reduction using fine-grained mutation operators Journal-First Papers Link to publication DOI | ||
10:30 15mTalk | An Empirical Study of IR-based Bug Localization for Deep Learning-based Software Research Papers Misoo Kim Sungkyunkwan University, Youngkyoung Kim Sungkyunkwan University, Eunseok Lee Sungkyunkwan University | ||
10:45 15mLive Q&A | Discussion and Q&A Research Papers |