Abstract
BackgroundTheory-based evaluation (TBE) approaches are heralded as supporting formative evaluation by facilitating increased use of evaluative findings to guide programme improvement. It is essential that learning from programme implementation is better used to improve delivery and to inform other initiatives, if interventions are to be as effective as they have the potential to be. Nonetheless, few studies describe formative feedback methods, or report direct instrumental use of findings resulting from TBE. This paper uses the case of Scotland’s, National Health Service, early years’, oral health improvement initiative (Childsmile) to describe the use of TBE as a framework for providing feedback on delivery to programme staff and to assess its impact on programmatic action.MethodsIn-depth, semi-structured interviews and focus groups with key stakeholders explored perceived deviations between the Childsmile programme 'as delivered’ and its Programme Theory (PT). The data was thematically analysed using constant comparative methods. Findings were shared with key programme stakeholders and discussions around likely impact and necessary actions were facilitated by the authors. Documentary review and ongoing observations of programme meetings were undertaken to assess the extent to which learning was acted upon.ResultsOn the whole, the activities documented in Childsmile’s PT were implemented as intended. This paper purposefully focuses on those activities where variation in delivery was evident. Differences resulted from the stage of roll-out reached and the flexibility given to individual NHS boards to tailor local implementation. Some adaptations were thought to have diverged from the central features of Childsmile’s PT, to the extent that there was a risk to achieving outcomes. The methods employed prompted national service improvement action, and proposals for local action by individual NHS boards to address this.ConclusionsThe TBE approach provided a platform, to direct attention to areas of risk within a national health initiative, and to agree which intervention components were 'core’ to its hypothesised success. The study demonstrates that PT can be used as a 'translational tool’ to facilitate instrumental use of evaluative findings to optimise implementation within a complex health improvement programme.
Highlights
Theory-based evaluation (TBE) approaches are heralded as supporting formative evaluation by facilitating increased use of evaluative findings to guide programme improvement
This paper provides a much needed example outside of the American context, demonstrating that despite challenges suggested within the evaluation literature, a national TBE, implemented in Scotland, with appropriate resources, can achieve sufficient stakeholder consensus to develop an adequate representation of programme complexity, and maintain a level of ongoing stakeholder engagement, facilitative of programme improvement
This paper addresses recent calls for case study examples describing the use of TBE approaches
Summary
Theory-based evaluation (TBE) approaches are heralded as supporting formative evaluation by facilitating increased use of evaluative findings to guide programme improvement. This paper uses the case of Scotland’s, National Health Service, early years’, oral health improvement initiative (Childsmile) to describe the use of TBE as a framework for providing feedback on delivery to programme staff and to assess its impact on programmatic action. Theory-based evaluation ‘Programme theory’ (PT) describes the process through which an intervention is hypothesised to impact on outcomes and the conditions under which this occurs; that is, it sets out how an intervention is expected to bring about change in a particular context [1,2]. Theory-based evaluation (TBE) encourages stakeholders to focus on PT by examining activities used to effect change and their linkages to intended outcomes [1,3]. Without delineating and testing PT or assessing implementation fidelity, it is impossible to distinguish between ‘implementation failure’ and ‘theory failure’: the enduring problem of the ‘black box’ [2,7]
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
