Validity Beyond Measurement: Why Psychometric Validity Is Insufficient for Valid Psychotherapy Research.

Femke L Truijens,Melissa M De Smet,Shana Cornelis,Reitske Meganck,Mattias Desmet

doi:10.3389/fpsyg.2019.00532

Abstract

In psychotherapy research, “validity” is canonically understood as the capacity of a test to measure what is purported to measure. However, we argue that this psychometric understanding of validity prohibits working researchers from considering the validity of their research. Psychotherapy researchers often use measures with a different epistemic goal than test developers intended, for example when a depression symptom measure is used to indicate “treatment success” (cf. outcome measurement for evidence-based treatment). However, the validity of a measure does not cover the validity of its use as operationalization of another target concept within a research procedure, nor the validity of its function toward an epistemic goal. In this paper, we discuss the importance of considering validity of the epistemic process beyond the validity of measures per se, based on an empirical case example from our psychotherapy study (“SCS”, Cornelis et al., 2017). We discuss why the psychometric understanding of validity is insufficient in covering epistemic validity, and we evaluate to what extent the available terminology regarding validity of research is sufficient for working researchers to accurately consider the validity of their overall epistemic process. As psychotherapy research is meant to offer a sound evidence-base for clinical practice, we argue that it is vital that psychotherapy researchers are able to discuss the validity of the epistemic choices made to serve the clinical goal.

Highlights

Any psychology scholar looking for information on the validity of a psychotherapeutic or clinical study knows where to find it: under “Measures” in the Methods section
We argued that the default psychometric understanding of “validity” in psychology is insufficient in capturing all the validity issues involved in the epistemic process of psychotherapy research
We used the example of the Beck Depression Inventory (BDI) to show that reliance on psychometric validity does not guarantee a valid psychotherapy research at large

Summary

Introduction

Any psychology scholar looking for information on the validity of a psychotherapeutic or clinical study knows where to find it: under “Measures” in the Methods section. In psychotherapy research it is common, and often formally required for publication, to use the IMRAD-format (IntroductionMethods-Results-and-Discussion) to report on empirical results, in which the use of validated. As psychotherapy research is applied research with the clear goal of understanding and improving clinical practice, the validity of the entire research process is vital for epistemic, clinical and societal reasons. We aim to address psychotherapy researchers and use concrete clinical research data to discuss why it is insufficient for valid psychotherapy research to limit the understanding of validity to instruments.

Objectives

Conclusion