Utilizing ASReview in screening primary studies for meta-research in SLA: A step-by-step tutorial

Abstract

Meta-research, including meta-analyses and systematic methodological reviews, has proven to be a useful tool for obtaining a comprehensive understanding of research questions by numerically summarizing data and methodological features in a given literature. As part of the review procedure, researchers select primary studies to be included in their analysis. However, this process is resource-intensive and prone to human error. In this tutorial, we introduce a practical application of artificial intelligence (AI), known as ASReview, that can facilitate the screening process. Using a simulated data set derived from a published meta-analysis, we offer step-by-step guidance on how to incorporate the tool into the screening process. We cover the essential steps, including the preparation of the data set, the import of the data set, the labeling of the study as relevant or irrelevant (for inclusion or not), as well as the saving of the results for the researcher's record and sharing for transparency in the spirit of open science. In addition, the tutorial addresses essential factors to consider in the AI-aided screening process, such as stopping rules. We acknowledge potential limitations of the tool and provide a couple of alternatives for interested readers. Our overall goal is to contribute to advancing and promoting meta-research in SLA by facilitating the screening process in the era of AI.