Poster: Supporting JavaScript Experimentation with BugsJS

In our recent work, we proposed BugsJS, a benchmark of several hundred bugs from popular JavaScript serverside programs. In this abstract paper, we report the results of our initial evaluation in adopting BugsJS to support an experiment in fault localization. First, we describe how BugsJS facilitated accessing the information required to perform the experiment, namely, test case code, their outcomes, their associated code coverage and related bug information. Second, we illustrate how BugsJS can be improved to further enable easier application to fault localization research, for instance, by filtering out failing test cases that do not directly contribute to a bug.
We hope that our preliminary results will foster researchers in using BugsJS to enable highly-reproducible empirical studies and comparisons of JavaScript analysis and testing tools.

Keywords: JavaScript, bug database, real bugs, fault localization, benchmark, reproducibility, BugsJS.