Abstract
BackgroundAlthough theory-driven evaluations should have empirical components, few evaluations of public health interventions quantitatively test the causal model made explicit in the theory of change (ToC). In the context of a shared sanitation trial (MapSan) in Maputo, Mozambique, we report findings of a quantitative process evaluation assessing intervention implementation, participant response and impacts on hypothesised intermediary outcomes on the pathway to trial health outcomes. We examine the utility of path analysis in testing intervention theory using process indicators from the intervention’s ToC.MethodsProcess data were collected through a cross-sectional survey of intervention and control compounds of the MapSan trial > 24-months post-intervention, sampling adult residents and compound leaders. Indicators of implementation fidelity (dose received, reach) and participant response (participant behaviours, intermediary outcomes) were compared between trial arms. The intervention’s ToC (formalised post-intervention) was converted to an initial structural model with multiple alternative pathways. Path analysis was conducted through linear structural equation modelling (SEM) and generalised SEM (probit model), using a model trimming process and grouped analysis to identify parsimonious models that explained variation in outcomes, incorporating demographics of respondents and compounds.ResultsAmong study compounds, the MapSan intervention was implemented with high fidelity, with a strong participant response in intervention compounds: improvements were made to intermediary outcomes related to sanitation ‘quality’ – latrine cleanliness, maintenance and privacy – but not to handwashing (presence of soap / soap residue). These outcomes varied by intervention type: single-cabin latrines or multiple-cabin blocks (designed for > 20 users). Path analysis suggested that changes in intermediary outcomes were likely driven by direct effects of intervention facilities, with little contribution from hygiene promotion activities nor core elements expected to mediate change: a compound sanitation committee and maintenance fund. A distinct structural model for two compound size subgroups (≤ 20 members vs. > 20 members) explained differences by intervention type, and other contextual factors influenced specific model parameters.ConclusionsWhile process evaluation found that the MapSan intervention achieved sufficient fidelity and participant response, the path analysis approach applied to test the ToC added to understanding of possible ‘mechanisms of change’, and has value in disentangling complex intervention pathways.Trial registrationMapSan trial registration: NCT02362932 Feb-13-2015.
Highlights
Theory-driven evaluations should have empirical components, few evaluations of public health interventions quantitatively test the causal model made explicit in the theory of change (ToC)
While process evaluation found that the Maputo Sanitation (MapSan) intervention achieved sufficient fidelity and participant response, the path analysis approach applied to test the ToC added to understanding of possible ‘mechanisms of change’, and has value in disentangling complex intervention pathways
ToC is recommended by official bodies concerned with clinical and complex intervention trials [2, 13], a systematic review of the uses of ToC in public health intervention evaluations found that very few have used the approach alongside trials of health impact, explored the influence of context on outcomes, or quantitatively tested the causal model [9]
Summary
Theory-driven evaluations should have empirical components, few evaluations of public health interventions quantitatively test the causal model made explicit in the theory of change (ToC). Public health interventions are either implicit or explicit embodiments of theory, involving the expectation that intervening will alleviate a problem via a set of intermediary outcomes and relying on a set of assumptions about how participants respond to programme activities to produce outcomes [4] These ‘mechanisms of change’ are highly sensitive to context [4]; and recent approaches to evaluation – such as the ‘realist’ approach [5] – highlight the need for evaluation approaches that assess how intervention effects differ by contextual factors such as social group or location to understand ‘what works, for whom and in what circumstances’ [6]. ToC is recommended by official bodies concerned with clinical and complex intervention trials [2, 13], a systematic review of the uses of ToC in public health intervention evaluations found that very few have used the approach alongside trials of health impact, explored the influence of context on outcomes, or quantitatively tested the causal model [9]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
