Experience-based guidelines for effective and efficient data extraction in systematic reviews in software engineering

Abstract

To systematically collect evidence and to structure a given area in software engineering (SE), Systematic Literature Reviews (SLR) and Systematic Mapping (SM) studies have become common. Data extraction is one of the main phases (activities) when conducting an SM or an SLR, whose objective is to extract required data from the primary studies and to accurately record the information researchers need to answer the questions of the SM/SLR study. Based on experience in a large number of SM/SLR studies, we and many other researchers have found the data extraction in SLRs to be time consuming and error-prone, thus raising the real need for heuristics and guidelines for effective and efficient data extraction in these studies, especially to be learnt by junior and young researchers. As a 'guideline' paper, this paper contributes a synthesized list of challenges usually faced during SLRs' data extraction phase and the corresponding solutions (guidelines). For our synthesis, we consider two data sources: (1) the pool of 16 SLR studies in which the authors have been involved in, as well as (2) a review of challenges and guidelines in the existing literature. Our experience in utilizing the presented guidelines in the near past have helped our junior colleagues to conduct data extractions more effectively and efficiently.