A Fully Integrated Systematic Review of Mixed Methods Design-Based Research

Abstract

Design-based research (DBR) is an educational research methodology that is commonly used in the fields of education, instructional technology, and learning sciences. When conducting DBR, researchers collaborate with practitioners (e.g., educators) and other stakeholders (e.g., parents, community members), often including the learners themselves, for the purpose of developing and evaluating innovative solutions to real-world problems within specific contexts, with a primary focus on improving practice and generating practical knowledge. DBR is particularly suited to mixed methods research. However, it is not clear the extent to which mixed methods research approaches are used in DBR studies, as opposed to monomethod research approaches that involve the sole use of qualitative research approaches or the sole use of quantitative research approaches. Therefore, in this study, what we refer to as a fully integrated systematic review of Scopus-indexed works from January 1, 1960 to May 31, 2022 was conducted to determine the prevalence of mixed methods DBR (MM-DBR) studies. This review yielded only 68 published works wherein the author explicitly declared their study as representing some form of a MMDBR study, with the majority of these MM-DBR studies being published within the last decade. Most notably, for all but 4 of these 68 studies, the level of integration occurred at the low end of the integration continuum, being characterized by mixed methods research designs wherein integration only occurred at the interpretation stage of the DBR process. More than two thirds of the authors (29.2%) neither explicitly specified nor described adequately their mixed methods research design. More than one half (i.e., 56.9%) of the MM-DBR studies were not grounded within the mixed methods research literature to any degree at all. Most notably, for all but four studies (i.e., 5.88%), the level of integration occurred at the low end of the integration continuum wherein integration only occurred at the interpretation stage of the MM-DBR process, representing only partial integration of the quantitative and qualitative research components/phases/cycles. As such, we call for more DBR researchers not only to consider using mixed methods research approaches but also to consider using full(er) integration approaches, as we move further into the fifth Industrial Revolution and beyond.