Fidelity of implementation of the Strengthening Families Programme 10-14 UK in Wales UK: a mixed-method process evaluation within a randomised controlled trial

Abstract

Background: A current pragmatic randomised controlled trial of the Strengthening Families Programme (SFP) is evaluating impacts on family functioning, alcohol use and other outcomes. This presentation describes a mixed-method process evaluation which explored how SFP was delivered by local and statutory bodies in seven areas of Wales. Methods: Interviews and self-report data from SFP staff; observation of SFP sessions; and routine data from each area were used to assess the extent to which SFP was implemented with fidelity. Descriptive analyses of quantitative data were conducted with appropriate tests of significance and reliability. A thematic framework was developed for coding and analysis of qualitative data. Variation within and between quantitative and qualitative results was discussed leading to alternative hypotheses which guided further analysis to reveal underlying implementation processes. Overall findings were explained using Negotiated Order Theory. Preliminary Results: 56 seven-week programmes were delivered to 330 families. Fidelity to the programme manual was generally high: 67% of staff reports indicated 80% or more of SFP activities were delivered with high fidelity. Between areas, the proportion of high-fidelity activities varied from 55% to 77%. This was partly explained by the extent of pre-session preparation for SFP facilitators. Fidelity to implementation guidelines also varied. Standards for staffing numbers and consistency were attained in 97% of programmes; this high rate was assisted by support from SFP co-ordinators’ colleagues at times when the intended process of recruiting staff from other agencies proved unworkable. Conclusions: While SFP content was delivered with reasonably high fidelity overall, analysis has identified some implementation processes which were not entirely as intended. Mixed-methods analysis performed a crucial function in elucidating how deviation from specified processes could sometimes be useful (in maintaining staffing standards) and sometimes less beneficial (in delivering content with fidelity). A theoretical approach offered insights into interpersonal and organisational interactions underlying SFP implementation which may be useful for other pragmatic trials.