Automated Identification of Domestic Violence in Written Child Welfare Records: Leveraging Text Mining and Machine Learning to Enhance Social Work Research and Evaluation

Abstract

Objective: Child welfare agencies often lack information about the front-end service needs of the families they serve. Thus, the current study tests the feasibility of text mining and machine learning procedures for identifying problems related to domestic violence documented in child welfare investigation summaries. Method: We labeled child welfare investigation summaries (N = 1,402) for the presence or absence of an active domestic violence service need. Labeled documents were then used to develop text mining and machine learning models and test their accuracy and reliability. Results: Machine learning models achieved greater than 90% accuracy when compared with human coders. Fleiss kappa estimates of coding reliability between the top-performing model and human reviewers exceeded .80, indicating that our model could support human reviewers to complete this coding task. Conclusion: Results provide strong evidence that text mining and machine learning procedures can be a cost-effective solution for extracting meaningful insights from text data. Although unsuitable for case-level predictive analytics, insights derived from these procedures can be particularly useful for investigating the prevalence, temporal trends, and geographic distribution of domestic violence-related needs in the child welfare system. These methods could substantially enhance the use of text data in social work research and evaluation.