The national child protection alert system in New Zealand: A prospective multi-centre study of inter-rater agreement

Abstract

BackgroundThe New Zealand health system assigns a unique identifier to every user of health services. The National Child Protection Alert System enables an alert to be placed against this identifier, making child protection information available to health providers throughout the country. Local health-based multi-disciplinary teams receive requests for an alert and decide whether an alert should be placed, but the proportion of requests accepted for an alert varies widely across New Zealand. ObjectiveTo examine whether eliminating variation in information presented to multi-disciplinary teams eliminates variation in the proportion accepted for an alert. Participants and settingHealth professionals working in multi-disciplinary child protection teams in publicly funded healthcare services across New Zealand. MethodsAll multi-disciplinary teams were asked to consider a set of 25 standardized child protection cases following their usual practice. The proportion accepted for an alert was compared by agreement analysis. ResultsEighteen of 20 teams completed the study. The proportion accepted for an alert varied from 48 to 96% (median 88%, inter-quartile range 73–89%). The proportion where the local team agreed with the majority decision varied from 64 to 96% (median 88%, inter-quartile range 81–96%). The kappa coefficient for inter-rater agreement varied from 0.25 (95% CI 0.00–0.76) to 0.83 (95% CI 0.52 to 1.00). The overall kappa coefficient was 0.37. ConclusionsEliminating variation in information provided does not eliminate variation in the proportion accepted for an alert. More research is needed into factors influencing alert decisions in the New Zealand National Child Protection Alert System.