The Use of Visual Text Mining to Support the Study Selection Activity in Systematic Literature Reviews: A Replication Study

Abstract

Background: Systematic literature reviews (SLRs)are an important component to identify and aggregate research evidence from different empirical studies. One of the activities associated with the SLR process is the selection of primary studies. The process used to select primary studies can be arduous, particularly when the researcher faces large volumes of primary studies. Aim: An experiment was conducted as a pilot test to compare the performance and effectiveness of graduate students in selecting primary studies manually and using visual text mining (VTM) techniques. This paper describes a replication study. Method: The same experimental design and materials of the previous experiment were used in the current experiment. Result: The previous experiment revealed that VTM techniques can speed up the selection of primary studies and increase the number of studies correctly included/excluded (effectiveness). The results of the replication confirmed that studies are more rapidly selected using VTM. We observed that the level of experience in researching has a direct relationship with the effectiveness. Conclusion: VTM techniques have proven valuable in the selection of primary studies.