Towards a unified checklist for empirical research in software engineering: first proposal

Abstract

Background: Current checklists for empirical software engineering cover either experimental research or case study research but ignore the many commonalities that exist across all kinds of empirical research. Identifying these commonalities, and explaining why they exist, would enhance our understanding of empirical research in general and of the differences between experimental and case study research in particular. Aim: In this short paper we design a unified checklist for empirical research, that identify commonalities and differences between experimental and case study research. Method: We design the unified checklist as a specialization of the general engineering cycle, which itself is a special case of the rational choice cycle. The unified checklist is based on an analysis and integration of a number of existing checklists. Results: The current version of the checklist exhibits a shared structure of experimental and case study research. Conclusions: Although the checklist exhibits a shared underlying structure of empirical research, its limitations are that it ignores other research methods such as meta-research or surveys and that very little empirical validation of usability and utility of the checklist has been performed so far. We are currently performing one validation, and are planning additional ones. These validations will likely lead to improvements of the current proposal.