
arXiv: 2208.05266
handle: 20.500.14243/417679
Empirical studies on formal methods and tools are rare. In this paper, we provide guidelines for such studies. We mention their main ingredients and then define nine different study strategies (usability testing, laboratory experiments with software and human subjects, case studies, qualitative studies, surveys, judgement studies, systematic literature reviews, and systematic mapping studies) and discuss for each of them their crucial characteristics, the difficulties of applying them to formal methods and tools, typical threats to validity, their maturity in formal methods, pointers to external guidelines, and pointers to studies in other fields. We conclude with a number of challenges for empirical formal methods.
FOS: Computer and information sciences, D.3.1, D.2.2, Formal Languages and Automata Theory (cs.FL), Formal methods, D.2.4, F.4, D.2; D.2.4; F.3.1; F.4; F.4.3; D.3.1; D.2.2, Computer Science - Formal Languages and Automata Theory, Guidelines, Software Engineering (cs.SE), Computer Science - Software Engineering, Empirical studies, F.4.3, D.2, F.3.1, formal methods; empirical studies; guidelines
FOS: Computer and information sciences, D.3.1, D.2.2, Formal Languages and Automata Theory (cs.FL), Formal methods, D.2.4, F.4, D.2; D.2.4; F.3.1; F.4; F.4.3; D.3.1; D.2.2, Computer Science - Formal Languages and Automata Theory, Guidelines, Software Engineering (cs.SE), Computer Science - Software Engineering, Empirical studies, F.4.3, D.2, F.3.1, formal methods; empirical studies; guidelines
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 10 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 10% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Top 10% |
