The Good, the Bad, and the Ugly in Mixed Methods Research and Design

Abstract

Mixed methods research and design (MMRD) deservedly continues its ascent in the social and related sciences. Its popularity has generated a critical mass of theoretical and empirical contributions, which has infiltrated and influenced many important research fields, particularly in education, health, and evaluation. Despite its wide recognition, also from government agencies and various national and private funding bodies, and despite its successes in academic and other types of research, many theorists, methodologists, and empirical researchers remain apprehensive, considering this approach insufficiently rigorous. They are often right. This text will identify some of which I consider the most notable weaknesses based on my observations as editor, reviewer, supervisor, project director, and consultant of various mixed methods–related projects and programs. It is an idiosyncratic list, reflecting tastes, interpretations, and positions not necessarily shared by some of my esteemed colleagues. It is hoped that this text stimulates debates that improve the rigor and scope of MMRD. The first generation of mixed methods researchers, from the end of the 19th century, conducted their mixing, blending, combining, or meshing of different data collection and analysis methods informally and unencumbered by subsequent notions relating to the incompatibility thesis and the paradigm war, due in part to the absence of conventions, methodological sophistication, and orthodoxy (Tashakkori & Teddlie, 1998). Important works from the 1990s, notably by Julia Brannen, Alan Bryman, Vicki L. Plano Clark, John W. Creswell, Abbas Tashakkori, Charles Teddlie, and others, form the core of the second generation. They provided mixed methods researchers with a vocabulary, taxonomy, and process description, which paved the way for its current success. Part of the secret of MMRD’s success is based on the adoption of the division of labor between qualitative and quantitative methods as formulated by critical theory-, postmodernity-, and hermeneutics-inspired texts a decade earlier. Until today, an idiosyncratic and unsatisfactory interpretation of philosophical pragmatism was the pacifier in the paradigm war. A closer inspection reveals theoretical and conceptual shortcomings of this sleight of hand, which translate into either logical and procedural inconsistencies or diplomatic disregard of the rules and regulations outlined in the research methods literature (Bergman, 2008). The following are the inconsistencies, challenges, or unresolved problems, loosely grouped into two interrelated families: conceptualization/theorization and design.