Navigating the validity tradeoffs of entrepreneurship research experiments: A systematic review and best-practice suggestions

Abstract

Experimental methods provide important advantages for advancing academic understanding of entrepreneurship. Yet, the complex and comingled relationships between some of entrepreneurship's key characteristics pose thorny methodological challenges to entrepreneurship researchers – notably to negotiate important tradeoffs between the ideals of external and construct validity. To facilitate the sound mobilization of experimental methods in entrepreneurship research, we present an overview of critical validity challenges plaguing entrepreneurship research experiments and assess the validation practices mobilized in 144 studies using such methods. Building on these findings, we develop a practical guide of actionable validation strategies to help experimenters navigate the above tradeoffs and conduct entrepreneurship research experiments that are realistic, theoretically meaningful, and that help establish the causal effects of their focal variables. By doing so, we contribute a set of pragmatic means to support the mobilization of experimental methods for advancing entrepreneurship research. Executive summaryExperimental methods hold important advantages for advancing the theoretical and practical understanding of entrepreneurial action. The power of research experiments lies in their ability to yield convincing evidence supporting the causal effects of the factors they investigate.Unfortunately, entrepreneurship research experiments are often criticized for offering pale copies of both the realities they attempt to model and the theoretical constructs they aim to study. The entrepreneurial process and its related activities count many intertwined characteristics – such as radical uncertainty, temporal dynamics, high personal stakes and other constraints – that can prove difficult to integrate in experimental studies. Moreover, these characteristics' comingled relationships raise important tensions with experimental methods' core principle of surgically focusing on the causal effects of a few manipulated variables. As a result, entrepreneurship research must overcome a number of validity challenges and tradeoffs in order to successfully leverage experimental methods and offer findings that convincingly support the causality of their theoretical developments.Compounding these difficulties, the guidance typically offered in research methods textbooks tends to focus on generic issues that are altogether separate from the specific challenges of conducting valid entrepreneurship research experiments. Because of this, many research manuscripts mobilizing experimental methods arrive at the review process with important shortcomings. This threatens the field's knowledge accumulation and typically calls for authors to refine their work and collect additional data.To help researchers face these challenges, we offer a three-part compendium focused on bolstering the validity of entrepreneurship research experiments. First, we complement the generic observations of research methods monographs by presenting an overview of the validity tradeoffs inherent to mobilizing experimental methods in entrepreneurship research – and of strategies for addressing these.Second, and to demonstrate that overcoming these challenges is not a trivial task, we conduct a structured literature review of the external and construct validation strategies deployed in a comprehensive corpus of 144 relevant articles published in both entrepreneurship-focused journals and broader generalist journals from the applied behavioral and social sciences. Among the most positive elements emerging from our analyses, we note upward trends in the mobilization of pilot tests, in the conduct of multiple experiments within single papers as well as in the use of parallel studies using different data collection methods. We also observe increased efforts to explain the practical relevance of experimental findings. To our surprise, however, our analyses uncover downward trends in the mobilization of pre-tests to examine the research material's external validity and representativeness, as well as the continued publication of studies that do not report empirical evidence regarding their focal manipulations' construct validity. This is concerning. Such practices undermine these experiments' abilities to yield convincing evidence in support of their theorized causal effects – and for what these imply for fostering entrepreneurial action.Third, and in light of the results obtained, we develop a practical guide of actionable strategies for navigating the validity tradeoffs of entrepreneurship research experiments. Grouping these strategies together allows us to systematize recommendations typically found across various methods monographs, thereby offering an overall scheme to help entrepreneurship researchers navigate the validity tradeoffs inherent to conducting realistic and theoretically meaningful experiments. To make our recommendations as actionable as possible, we develop an extensive step-by-step guide of design and assessment strategies relevant to entrepreneurship research experiments. The guide spans the entire research process – from conceiving an experimental study to collecting, analyzing and reporting the results. The guide also includes a number of relevant exemplars from many different studies we analyzed.By discussing these entrepreneurship-specific validity tradeoffs and combining that with an overview of strategies for addressing them, our study offers an entrepreneurship-centered synthesis that equips entrepreneurship researchers with the necessary tools for conducting experimental research that advances understanding of the causal effects supporting entrepreneurial action and its related phenomena.