Temporal validity as meta-science

Abstract

The “credibility revolution” has forced quantitative social scientists to confront the limits of our methods for creating general knowledge. As a result, many practitioners aim to generate valid but local knowledge and then synthesize and apply that knowledge to predict what will happen in a target context. Positivist social science has until recently been hamstrung with other, more immediate threats to validity and inference, but I argue that recent advances in statistical approaches to the problem of external validity reveal limits of the current paradigm. This article and the term “temporal validity” illustrate the intrinsic limits of agnostic (i.e., assumption-free) external validity when the target setting is in the future. These limits, I argue, suggest a re-orientation of social science methodology. We should acknowledge that no research design, no empirical knowledge, is perfectible; instead, we should explicitly aim to increase the amount and quality of knowledge we produce. I believe it is useful to characterize this perspective as part of “Meta-Science,” an emerging paradigm within the social sciences. “Temporal validity” and the implied “knowledge decay” thus represent a meta-scientific intervention aimed at increasing the usefulness of the knowledge we produce. Among other structural reforms, I argue that the binary reality of academic scholarship (a paper is published or not) reifies the perfectibility of empirical knowledge and is thus an impediment to recognizing the continuous nature of all forms of scientific validity.