Downloads provided by UsageCounts
Bug benchmarks are used in development and evaluation of debugging approaches. Quantitative performance comparison of different debugging approaches is only possible when they have been evaluated on the same dataset or benchmark. However, benchmarks are often specialized towards usage for certain debugging approaches in their contained data, metrics, and artifacts. Such benchmarks can not be easily used on debugging approaches outside their scope as such approach may rely on specific data such as bug reports or code metrics not included in the dataset. Furthermore, benchmarks vary in their size w.r.t. the number of subject programs and the size of the individual subject programs. For these reasons, we have performed a systematic literature review where we have identified 73 benchmarks that can be used to evaluate debugging approaches. We compare the different benchmarks with respect to their size and the provided information such as bug reports, contained test cases, and other code metrics. Furthermore, we have investigated how well the benchmarks realize the FAIR guiding principles. This comparison is intended to help researchers to quickly identify all suitable benchmarks for evaluating their specific debugging approaches. More information can be found in the publication: Thomas Hirsch and Birgit Hofer: "A Systematic Literature Review on Benchmarks for Evaluating Debugging Approaches", submitted to the Journal of Systems and Software, under review.
Fault localization, Debugging, Automatic repair, Benchmark
Fault localization, Debugging, Automatic repair, Benchmark
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
| views | 13 | |
| downloads | 4 |

Views provided by UsageCounts
Downloads provided by UsageCounts