Abstract
BackgroundComplex interventions such as self-management courses are difficult to evaluate due to the many interacting components. The way complex interventions are delivered can influence the effect they have for patients, and can impact the interpretation of outcomes of clinical trials. Implementation fidelity evaluates whether complex interventions are delivered according to protocol. Such assessments have been used for one-to-one psychological interventions; however, the science is still developing for group interventions.MethodsWe developed and tested an instrument to measure implementation fidelity of a two-day self-management course for people with epilepsy, SMILE(UK). Using audio recordings, we looked at adherence and competence of course facilitators. Adherence was assessed by checklists. Competence was measured by scoring group interaction, an overall impression score and facilitator “didacticism”. To measure “didacticism”, we developed a novel way to calculate facilitator speech using computer software. Using this new instrument, implementation fidelity of SMILE(UK) was assessed on three modules of the course, for 28% of all courses delivered.ResultsUsing the instrument for adherence, scores from two independent raters showed substantial agreement with weighted Kappa of 0.67 and high percent agreement of 81.2%. For didacticism, the results from both raters were highly correlated with an intraclass coefficient of 0.97 (p < 0.0001). We found that the courses were delivered with a good level of adherence (> 50% of scored items received the maximum of 2 points) and high competence. Groups were interactive (mean score: 1.9–2.0 out of 2) and the overall impression was on average assessed as “good”. Didacticism varied from 42% to 93% of total module time and was not associated with the other competence scores.ConclusionThe instrument devised to measure implementation fidelity was reproducible and easy to use. The courses for the SMILE(UK) study were delivered with a good level of adherence to protocol while not compromising facilitator competence.Trial registrationISRCTN57937389.
Highlights
Complex interventions such as self-management courses are difficult to evaluate due to the many interacting components
We found no associations between didacticism and adherence (β = 0.23 [95% CI: -0.27 – 0.87]), or between didacticism and overall impression scores (β = 0.081 [95% CI: -0.52 – 0.68])
We developed an instrument for intervention fidelity measuring both delivery of core items and the way they were presented to the group
Summary
Complex interventions such as self-management courses are difficult to evaluate due to the many interacting components. Implementation fidelity evaluates whether complex interventions are delivered according to protocol Such assessments have been used for one-to-one psychological interventions; the science is still developing for group interventions. Evaluating the effectiveness of complex interventions, such as a group self-management course, can be challenging They include multiple, often interacting components and present a host of problems for researchers trying to evaluate their impact upon patient outcomes. Such interventions typically follow specific therapeutic models, protocols and manuals, one important task is to document the extent and way in which the intervention was implemented, not least because this can be a mediator of study outcomes [1]. Information about the study design [5, 6] and enactment of skills as determined by a process evaluation [7] can be found elsewhere
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
