The theory-driven approach to validity

Abstract

Current approaches for dealing with validity tend to focus on one type of validity and play down the importance of others. For example, internal validity is treated by Campbell and Stanley as “sine qua non,” whereas, Cronbach judges external validity to be prime importance in research. However, these single validity oriented approaches may achieve one type of validity at the expense of other types of validity. As an alternative, we propose a theory-driven approach to validity. Our central argument is that a model or theory should be formulated in a program evaluation and the modeling process should include the identification of potential threats to validity in research. In this sense, the purpose of applying a technique such as randomization is to strengthen rather than to replace the model or theory. That is, program evaluation should be model or theory-oriented instead of research design oriented. Randomization can be used in any research if it is applicable, but we need rely less on randomization as a safeguard to internal validity if the threats in a research can be adequately identified and controlled. Since the structural relationships among the relevant variables and intervening processes are carefully modeled, the theory-driven approach avoids the pitfalls of black-box evaluation and provides better understanding of the causal mechanisms underlying the relationships between treatment and effects. Unlike randomized experiments which aim only at internal validity, a theory-driven approach provides a general framework to deal with internal, external, construct and statistical conclusion validity.