Reliability, factor structure, and validity of an observer-rated alliance scale with youth.

Abstract

Measurement limitations lessen the conclusions about the role of the alliance in youth psychosocial treatment. This article examined the score reliability, factor structure, and validity of the 9-item Therapy Process Observational Coding System for Child Psychotherapy-Alliance scale (TPOCS-A). The sample, 51 youth aged 7-15 years (Mage = 10.36 years, SD = 1.90; 86.3% White; 60.8% male, 39.2% female), met diagnostic criteria for a principal anxiety disorder and received cognitive-behavioral therapy. Treatment sessions (N = 463) were coded by independent coders using the TPOCS-A along with observational measures of treatment adherence and therapist competence. Youth and therapists also completed self-report alliance measures at the end of each session. Reliability estimates, ICC(2,2), at the item level indicated a mean interrater reliability of .68 (SD = .10) and a mean coder stability of .64 (SD = .11). An exploratory factor analysis identified a one-factor solution with five items. TPOCS-A scores evidenced convergent validity with the therapist and adolescent reports of alliance but did not converge with the child-report alliance measure. TPOCS-A scores evidenced discriminant validity when compared with scores on adherence and competence measures. The reliability and validity profile of the 9- and 5-item versions of the TPOCS-A were similar. Overall, findings support the reliability and validity of the TPOCS-A scores, but questions remain about how best to assess the alliance with children. (PsycInfo Database Record (c) 2021 APA, all rights reserved).