Abstract
Context : In any discipline, replications of empirical studies are necessary to consolidate the acquired knowledge. In Software Engineering, replications have been reported since the 1990s, although their number is still small. The difficulty in publishing, the lack of guidelines, and the unavailability of replication packages are pointed out by the community as some of the main causes. Objective : Understanding the current state of replications in Software Engineering studies by evaluating current trends and evolution during the last 6 years. Method : A Systematic Mapping Study including articles published in the 2013–2018 period that report at least one replication of an empirical study in Software Engineering. Results : 137 studies were selected and analysed, identifying: ${i}$ ) forums; ii) authors, co-authorships and institutions; iii) most cited studies; iv) research topics addressed; ${v}$ ) empirical methods used; vi) temporal distribution of publications; and vii) distribution of studies according to research topics and empirical methods. Conclusions : According to our results, the most relevant forums are the Empirical Software Engineering and Information and Software Technology journals, and the Empirical Software Engineering and Measurement conference. We observed that, as in previous reviews by other researchers, most of the studies were carried out by European institutions, especially Italian, Spanish, and German researchers and institutions. The studies attracting more citations were published mainly in journals and in the International Conference on Software Engineering . Testing, requirements, and software construction were the most frequent topics of replication studies, whereas the usual empirical method was the controlled experiment. On the other hand, we identified research gaps in areas such as software engineering process, software configuration management, and software engineering economics. When analysed together with previous reviews, there is a clear increasing trend in the number of published replications in the 2013–2018 period.
Highlights
Replication of empirical studies in Software Engineering (SE) is an essential activity for achieving greater validity and reliability in research results [2], [13]
SYSTEMATIC MAPPING STUDY RESULTS we address each of the research questions defined in Section III-A2 using graphs to visualize collected data and providing subsequent interpretations
WORK In this article, we have presented an Systematic Mapping Study (SMS) on replications of empirical studies in SE from 2013 to 2018
Summary
Replication of empirical studies in Software Engineering (SE) is an essential activity for achieving greater validity and reliability in research results [2], [13]. Most definitions consider replications to be repetitions of research procedures already performed in a so–called original or baseline studies [6]. As many other processes in SE, replications can be classified according to several criteria. Depending on whether or not they are carried out by the same experimenters as in the original study, they can be classified as. According to the degree to which the original experiment procedure is followed, they can be classified as exact and conceptual [15] or closed and differentiated [16]. Basili et al refer to strict replications when the original study is duplicated as accurately as possible [17], whereas Gómez et al classify them as literal, operational and conceptual depending on the changes carried out and their purpose [18]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
