Abstract
The experimental revolution in the social sciences is one of the most significant methodological shifts undergone by the field since the ‘quantitative revolution’ in the nineteenth century. One of the often valued features of social science experimentation is precisely the fact that there are (alleged) clear methodological rules regarding hypothesis testing that come from the methods of the natural sciences and from the methodology of RCTs in the biomedical sciences, and that allow for the adjudication among contentious causal claims. We examine critically this claim and argue that some current understandings of the practices that surround social science experimentation overestimate the degree to which experiments can actually fulfil this role as “objective” adjudicators, by neglecting the importance of shared background knowledge or assumptions and of consensus regarding the validity of the constructs involved in an experiment. We take issue with the way the distinction between internal and external validity is often used to comment on the inferential import of experiments, used both among practitioners and among philosophers of science. We describe the ways in which the more common (dichotomous) use of the internal/external distinction differs from Cook and Campbell’s original methodological project, in which construct validity and the four-fold validity typology were all important in assessing the inferential import of experiments. We argue that the current uses of the labels internal and external, as applied to experimental validity, help to encroach a simplistic view on the inferential import of experiments that, in turn, misrepresents their capacity to provide objective knowledge about the causal relations between variables.
Highlights
This article belongs to the topical collection "Objectivity in Social Research", edited by Julie Zahle and Petri Ylikoski
Because of the common tendency to think of the internal/ external validity pair as if it exhausted the inferential realm of experiments, we are all too accustomed to deal with the questions regarding constructs as questions that pertain to either internal or external validity, even though this is not so in the Campbellian classification
Though only recently Cartwright has noted that “external validity” is a somewhat confusing term, she has used it within a dyadic internal/external interpretation, and helped popularized it among philosophers. She has mainly used it to describe the limitations of randomized controlled trials (RCTs) for extrapolative purposes, as part of an important and necessary effort to appease some of the overly enthusiastic versions of Evidence Based Policy (EBP). This dyadic use of the internal/external validity distinction has too contributed to the neglect of construct validity, and to an overly simplistic view of the relationship between an experiment and its inferential import, by helping to obscure the fact that the same experimental intervention can be conceptualized as representing different constructs, and that often background assumptions is crucial in determining the inferences that we make from one and the same experiment
Summary
To systematize the way in which we convert a research question into a particular, concrete, design that we can test on the ground Within this framework, the distinction between internal and external validity was conceived by Campbell (1957) as part of an admirable and sustained methodological effort directed at understanding the pitfalls of research analysis and causal inference in the social sciences. To ignore construct validity is to ignore that, when we experiment, we do need to make inferences (that in turn, can be valid or invalid) regarding whether the independent and dependent variables in our experiment (say, a and b) correctly represent the higher order constructs that are the target of our research question (A and B) Note, and this is crucial, that these inferences are not (necessarily) causal, and they are, in any event, different from the causal inference that we aim to test (which will relate to the general question of whether As cause Bs). While the Campbellian project always considered the four types of validity, it is not an exaggeration to say that, whereas part of their validity project became universalized (the internal and external validity dyad), another part (statistical-conclusion and construct validity) got lost to many practitioners and experimental commentators, and to philosophers
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call