Predictive validity of evidence-based practices in supported employment: a systematic review and meta-analysis

Abstract

Purpose The purpose of this paper is to explore the predictive validity of two measures of fidelity to the individual placement and support (IPS) approach to supported employment. Design/methodology/approach A systematic review and meta-analysis was conducted of IPS programs. In total, 30 studies provided information characterizing 69 cohorts and 8,392 participants. Predictive validity was assessed by a precision and negative prediction analysis and by multivariate analysis of deviance. Findings Fidelity scores on the IPS-15 scale of 60 or less accurately predicted poor outcomes, defined as 43 percent or less of participants commencing employment, in 100 percent of cohorts. Among cohorts with IPS-15 fidelity scores of 61-75, 63 percent attained good employment outcomes defined as 44 percent or more commencing employment. A similar pattern emerged from the precision analysis of the smaller sample of IPS-25 cohorts. Multivariate analysis of deviance for studies using the IPS-15 scale examined six cohort characteristics. Following adjustment for fidelity score, only fidelity score (χ2=15.31, df=1, p<0.001) and author group (χ2=35.01, df=17, p=0.01) representing an aspect of cohort heterogeneity, remained associated with commencing employment. Research limitations/implications This study provides evidence of moderate, yet important, predictive validity of the IPS-15 scale across diverse international and research contexts. The smaller sample of IPS-25 studies limited the analysis that could be conducted. Practical implications Program implementation leaders are encouraged to first focus on attaining good fidelity, then supplement fidelity monitoring with tracking the percentage of new clients who obtain a competitive job employment over a pre-defined period of time. Originality/value The evidence indicates that good fidelity may be necessary but not sufficient for good competitive employment outcomes.