Influence of Pure and Unit-Like Tests on SBFL Effectiveness: An Empirical Study

One of the most challenging and time-consuming aspects of debugging is identifying the exact location of the bug. We propose the concept of a Pure Unit Test (PUT), which, when fails, can unambiguously determine the location of the faulty method. Based on developer experience, we established three heuristics to evaluate the degree to which a test can be considered a unit test, if it cannot be considered as a PUT (we call these Unit-Like Tests, or ULTs). We examined how and when PUTs and ULTs affect Spectrum-Based Fault Localization efficiency. The results demonstrate that, for more complex systems, a higher proportion of unit tests in the relevant test cases can enhance the effectiveness of fault localization. When the number of PUTs is high enough, fault localization becomes trivial, in that case running SBFL is not necessary. Moreover, our findings indicate that different kinds of ULTs can have a large impact on the efficiency of fault localization, particularly for simpler bugs where they can quickly and effectively pinpoint the problem areas.

Keywords:     Fault localization, Spectrum-Based Fault Localization, unit testing, test levels.