Using Big Data in Policy Evaluations: the Troubled Families Programme

Abstract

BackgroundThe Ministry for Housing, Communities and Local Government have carried out one of the biggest data linkage exercises in government in order to evaluate the impact of the Troubled Families Programme. Linking individual and family level data across multiple administrative datasets has proven to be both innovative and cost-effective, enabling us to place children in the broader context of their family and household circumstances for our analysis.
 ObjectivesTo use administrative datasets to measure children’s service use outcomes, for both programme and comparison individuals and families, to assess the impact of the Troubled Families Programme on outcomes for ‘children needing help’.
 MethodsThe comparison group provide a counterfactual, used to derive a robust assessment of the programme’s impact on children’s outcomes: in this case child safeguarding. Linked datasets means we can control for both individual and family level characteristics, such as parental employment, benefits, school attendance, children and adult offending and the circumstances of siblings. We have used propensity score matching to control for all covariates impacting on both treatment and outcome status.
 FindingsPreliminary findings show a statistically significant reduction in the number of ‘children in need’ in the 6-12 month period after intervention start compared to the matched comparison group, and a reduction in the number of ‘looked after children’. There was an increase in the number of children with a ‘child protection plan’, but this was not statistically significantly different to the comparison group.
 ConclusionsWhile access to such a wide range of individual and family characteristics is a key methodological advantage to evaluation, challenges include: missing data, time lags in the datasets and complex variable definitions. We have worked and continueto work with other government departments to overcome these. Whilst still in its earliest stages preliminary results for the programme’s impact are encouraging.