Errors in treatment outcome monitoring: Implications for real-world psychotherapy.

Abstract

In the last several years, considerable progress has been made in treatment outcome monitoring (TOM) in psychotherapy. Numerous instruments have been developed to assist practicing psychotherapists in assessing the impact of their services, and several clinical tools have been developed to directly improve the quality of services provided. As a field, we have begun to view these outcome monitoring and feedback systems with increasing confidence as evidence accrues to support their efficacy. In this paper we examine the types of errors that may occur in making inferences in TOM, in particular the determination of whether change in psychological symptoms is occurring, has occurred, or not. We examine this as any other empirical question, using the classic hypothesis-testing framework to describe two types of errors in decision-making. In particular, we discuss the strengths and vulnerabilities of two prominent assessment strategies in TOM (general and multidimensional measurement) in order to minimise inferential errors and maximize the effectiveness of outcome monitoring efforts. Finally, we provide a few examples of new developments that make use of multidimensional measurement to minimise Type II errors to improve outcomes. Keywords: treatment outcome monitoring, psychotherapy, measurement, the Treatment Outcome Package (TOP) Although many mental health professionals can, when called upon, identify what tiiey consider their own specific areas of expertise or specialisation, most treat clients with a variety of diagnoses, comorbidity, and personalities. Similarly, any largescale organisation for mental healdi care, be it a clinic, hospital, or a managed care company, will necessarily need to provide treatments for a broad range of clinical problems: given enough time or a large enough client population, virtually all diagnoses listed in the major diagnostic systems will be represented in a caseload. In addition, practicing psychotherapists and counselors frequently do not specialise in only one level of severity of clientele. Psychotherapists may have some relatively less severe clients, and tiiey may have other clients whose clinical severity may require longer treatments, more intense intervention, and/or more frequent appointments. The result is tremendous heterogeneity among clients on a given therapist's caseload. This heterogeneity in patient presentations - defined at a minimum by the type of symptoms present as well as their level of severity - is an inescapable part of everyday psychotherapeutic practice. In addition to having direct impacts on die assessment, case formulation, and treatment planning for each client, this heterogeneity may also have important implications on a set of tasks in which an increasing number of clinicians (as well as large-scale management organisations) are invested: treatment outcome monitoring (TOM). In the last several years, several instruments have been developed and validated to assess the impact of psychotherapy in day-to-day practice. The implementation of these instruments (and associated clinical tools and interpretive aides) has largely been driven by empirical evidence supporting their clinical usefulness. However, heterogeneity of clients may influence the accuracy of inferences about clients and therapists that can be made from this type of outcome tracking. The goal of this article is to describe some potential inferential errors of two different types of self-report measures used for treatment outcome monitoring - general and multidimensional. As such instruments are likely to be used frequently in day-to-day practice, the article also discusses some clinical implications of these potential errors. Statistical Decision Making The most common approach to statistical decision making will be familiar to most readers. In this approach, the general goal is to decide whether a given set of data support or reject a particular null hypothesis. …