The validity and utility of the California Family Risk Assessment under practice conditions in the field: A prospective study

Abstract

Objective Analysis of the validity and implementation of a child maltreatment actuarial risk assessment model, the California Family Risk Assessment (CFRA). Questions addressed (1) Is there evidence of the validity of the CFRA under field operating conditions? (2) Do actuarial risk assessment results influence child welfare workers’ service delivery decisions? (3) How frequently are CFRA risk scores overridden by child welfare workers? (4) Is there any difference in the predictive validity of CFRA risk assessments and clinical risk assessments by child welfare workers? Method The study analyzes 7,685 child abuse/neglect reports originating in 5 California counties followed prospectively for 2 years to identify further substantiated child abuse/neglect. Measures of model calibration and discrimination were used to assess CFRA validity and compare its accuracy with the accuracy of clinical predictions made by child welfare workers. The extent of use of an override feature of the CFRA and child welfare worker reliance on CFRA risk scores for making service decisions were analyzed. Results Imperfect but better-than-chance predictive validity was found for the CFRA on a range of measures in a large temporal validation sample ( n = 6,543). For 114 cases where both CFRA risk assessments and child welfare worker clinical risk assessments were available, the CFRA exhibited evidence of imperfect but better-than-chance predictive validity, while child welfare worker risk assessments were found to be invalid. Child welfare workers overrode CFRA risk assessments in only 114 (1.5%) of 7,685 cases and provided in-home services in statistically significantly larger proportions of higher- versus lower-risk cases, consistent with heavy reliance on the CFRA. Conclusions/practice implications Until research identifies actuarial models exhibiting superior predictive validity when applied in every-day practice, the CFRA is, and will be a valuable tool for assessing risk in order to make in-home service-provision decisions.