The Role of External Validity in Theoretical Research

Abstract

Calder, Phillips, and Tybout (1982) have criticized my analysis of the role of external validity in theoretical consumer research (Lynch 1982). In particular, they disputed my argument that theoretical researchers must concern themselves with -the generalizability of their research findings. The issues raised by the debate are important because of the impact that the advice of Calder et al. could have, if accepted, upon theory development within our discipline. This paper will elaborate ways in which we can use evidence bearing on the external validity of theoretically predicted effects in an opportunistic fashion to hasten theoretical progress. In an earlier paper, Calder et al. (1981) argued from the standpoint of a falsificationist philosophy of science (Popper 1959) and quite correctly noted that theories can be rejected if their predictions can be shown to be false for any subjects, settings, and events within their domain. Research practices often thought to enhance external validity-e.g., the use of heterogeneous samples of respondents and of uncontrolled field settings-were deemed unnecessary and even undesirable in theoretical research, because they inflate error variance and make it more difficult to detect systematic violations of the predictions of one's theory. Calder et al. concluded that because these research methods compromise the effort to provide the most rigorous possible test of one's theory (by reducing statistical power), external validity is of minimal relevance to theoretical consumer research, at least as it pertains to individual experiments. My criticism of that thesis was that Calder et al. failed to distinguish the research methods that are commonly thought to increase external validity from the concept of external validity-i.e., the degree to which the effects of experimental manipulations are independent of the levels of supposedly irrelevant background factors (Campbell and Stanley 1966; Cook and Campbell 1979). This is not a mere semantic distinction. One of my major points was that these research practices (the use of heterogeneous, representative samples of respondents and of realistic, uncontrolled field settings) do not have the efficacy in increasing external validity that is commonly ascribed to them. If background factor x treatment interactions exist of which the researcher is unaware (as seems likely), these research practices can mask a substantial lack of external validity. External validity is highly relevant to theoretical consumer research. Evidence demonstrating that theoretically predicted effects lack external validity-in that they fail to generalize across various levels of background factors presumed to be theoretically irrelevant-would indicate that the theory lacked construct validity. Certain research methods that allow a partial assessment of external validity should thus be routinely adopted. Unlike the methods for increasing external validity that Calder et al. criticized, these methods compromise neither statistical conclusion validity nor construct validity. These methods are Cook and Campbell's (1979) model of deliberate sampling for heterogeneity and the selective approach of choosing some small number of background factors to be varied orthogonally to the treatments (Lynch 1982). Both of these methods ensure a high level of within-block homogeneityand hence, statistical power-while allowing some (incomplete) assessment of external validity. The inevitably imperfect nature of this assessment stems from the fact that the researcher cannot hope to anticipate and block upon all *John G. Lynch, Jr. is Assistant Professor of Marketing, College of Business Administration, University of Florida, Gainesville, FL 32611. The author is grateful to Bobby Calder for a very useful discussion of the ideas in this paper, and to Joel Cohen for helpful comments on an earlier draft.