The generalizability of brief ratings of psychosocial functioning

Abstract

The multivariate generalizability of the Role Functioning Scale (RFS), serving as a replacement for the Social Adjustment Scale global ratings, was investigated. Severely mentally disturbed clients of a public mental health bureau, who were participating in one of two clinical trials that compared the cost-effectiveness of case management or substance abuse services, were interviewed and rated by psychiatric technicians, operating as four-person teams. The generalizability of single subscales was notably less than the generalizability of the combination of four subscales. Still larger differences in generalizability were noted between proposed future applications of the RFS that might employ one, as opposed to more than one, interviewer. Variations in the power to detect differences in service effectiveness were related to the levels of generalizability. Applications of this, or any, brief rating instrument should benefit from estimating the generalizability of the ratings for the proposed studies and employing estimates of generalizability to adjust sample size requirements.