Realist Evaluation Methodology Research Articles

Contexts and complexities: a realist evaluation of integrated care system leadership.

This paper presents a realist evaluation of leadership within an integrated care system (ICS) in England. This paper aims to examine which aspects of leadership are effective, for whom, how and under what circumstances. Realist evaluation methodology was used, adopting prior realist review findings as the theoretical framework to refine explanations of how and why leadership within an ICS is effective. Between January and November 2023, 23 interviews with ICS leaders took place, alongside 7 meeting observations and documentary analysis. The Realist And Metanarrative Evidence Syntheses: Evolving Standards (RAMESES) guidance informed the study design, conduct and reporting. The findings highlight two overarching infrastructural contexts influencing leadership in ICSs: the impact of the post-COVID-19 pandemic legacy and the differences between health and social care regulatory and financial environments. Findings demonstrate that ICS leaders identified a strong sense of purpose as crucial for guiding decisions and creating a psychologically safe environment for open, honest discussions, fostering calculated risk-taking. Whilst a shared vision directed priority setting, financial pressures led to siloed thinking. Leadership visibility was linked to workforce morale, with supportive leadership boosting morale amidst evolving ICS landscapes and confidence in data-driven decisions supported prevention activities. However, financial constraints hindered responsiveness and innovation in addressing health inequalities. By examining ICS leadership post-COVID-19 pandemic and amidst varying regulatory and financial environments, this study contributes to the emerging literature on systems leadership and offers practical guidance for leaders navigating the complexities of integrated care.

(081) Enhancing Treatment for Patients with Medically Unresolved Persistent Sexual Dysfunction: What can be Learnt from Mixed Methodology Evaluation-Research?

Abstract Introduction Introduction: In adult healthcare delivery worldwide, medically unresolved persistent physical symptoms (uPPS) are of significant concern (Bermingham et al., 2010; Sumathipala, 2007), with patients often requiring an enhanced therapeutic approach (Nimnuan et al., 2001; Dwamena, et al., 2009). When routine medical care is not achieving desired patient outcomes for Unresolved Persistent Sexual Symptoms (uPSS) (erectile dysfunction (ED), rapid ejaculation (RE), genital pain and diminished sexual desire), the evidence for the best practice is limited. More research is needed in this area and whilst RCTs remain the gold standard in evidence-based medicine, there are limitations. At present there is no international consensus on the use of mixed method research parameters in studies involving patients with uPPS. Sexual Medicine teams are ideally placed to develop an enhanced approach to their own uPSS sufferers. We share our experience of an evaluation-research process and report its use, adaptations, and important findings. Objective Objectives: To present our experience of Realist Evaluation methodology in developing a cross-modality consultation model: Engaging the adult patient with uPSS. Methods Methods: Following NHS ethical approval using RAMESES standards (Wong et al., 2013), we conducted mixed method data comparisons by 3 stages: 1. A wide scoping of the literature, 1990–2014, 2. A Realist Literature Synthesis (2001–2014) of efficacy or effectiveness of Short-term Psychodynamic Psychotherapy (STPP), compared through common themes of engagement with Cognitive Behavioural Therapy (CBT) for uPPS. 3. Demographics, referral sources, length of history and clinical intervention outcomes were extracted from an established NHS psychosexual service. The comparative literature findings were used to develop a common-factor thematic analysis of 24 full transcripts from four adult prospective consecutive cases. Validated measures of wellbeing and symptoms severity with goal attainment measures supported further. Results Main Results: A synthesis of the literature from stages 1. and 2., supported by clinical experience, led to a succinct definition for the majority of adult uPPS sufferers within which the uPSS cohort is embraced. A Realist equation, ‘Context plus Mechanism = Outcomes’, developed across the case data showed the simple means of discovering the unique precipitating, predisposing, and perpetuating factors of the individual’s uPSS (an approach potentially transferrable to other uPPS). Further, the extraction of common factors perpetuating the uPSS over time plus an accessible set of collaborative principles of engagement for disabling these, substantiated the consultation model. Of the measures, personal goal attainment improvements over time in therapy were the most aligned to the transcript analyses showing improved subjective outcomes and personal satisfaction. Conclusions Conclusions: The literature synthesis revealed the paucity of high-quality mixed method research into uPPS talking therapy interventions and none for heterogenous uPSS. This was the first use of an enhanced evaluation-research methodology in this context. Due to the data emerging from the embedded cases it was possible to construct the foundational cross-modality consultation model. At the time of the study, the lack of available clinical service comparator limits the generalisation of our findings to uPPS. Our experience shows from the systematically developed outcome data that realist evaluation can be used appropriately, requiring further research. Disclosure No.

Developing and evaluating Compassionate Workplace Programs to promote health and wellbeing around serious illness, dying and loss in the workplace (EU-CoWork): a transdisciplinary, cross-national research project

Background: Most employees will experience serious illness, caregiving, dying and loss (End-of-Life (EoL) experiences) at multiple points throughout their working lives. These experiences impact affected employees but also their colleagues in terms of health and wellbeing, and the workplace as a whole in terms of workplace safety, productivity and labour relations. The impact of EoL experiences on employees means that workplaces are called to play a more active role in providing support for EoL experiences. Aim: To describe how the EU-CoWork (2024–2028) project addresses its main aims to (1) create Compassionate Workplace cultures, practices and policies and improve health and wellbeing for employees dealing with EoL experiences in different national work contexts in Europe; (2) describe and evaluate the process of co-creation and implementation of Compassionate Workplace Programs (CWPs) and how these influence the programs’ outcomes. Design: EU-CoWork employs a facilitated and co-creative Developmental Evaluation approach to the development of 12 tailored CWPs across four European countries (Belgium, Austria, Sweden and Greece). Methods: To evaluate the outcomes and processes leading to these outcomes, a mixed-methods Realist Evaluation methodology is applied, formulating and testing Context-Mechanism-Outcomes configurations and combining longitudinal quantitative and qualitative data collections. Results: EU-CoWork will generate evidence to support an expanded model of occupational health and safety risk factors sensitive to the specific challenges related to employees’ EoL experiences. In doing so, several challenges will have to be navigated: involving employees with EoL experiences while avoiding overburdening them, avoiding tokenistic engagement, managing power differentials, balancing the need for scientific rigour with the flexibility required in co-creation, reconciling different epistemologies and disciplinary traditions and organisational resistance to change. Conclusion: There are potential long-lasting broader societal impacts through the stimulation of open discourse on EoL topics, the reconciliation of work and care, and changes in gendered work and care patterns.

Vignettes to Support Theory Refinement: Methodological Insights From a Realist Evaluation

Realist evaluation methodology aims to understand social programmes by revealing what works, for whom, in what circumstances, and how and why. Realist evaluation starts with generating initial programme theories (IPTs), which are subsequently tested and refined systematically using a multi-methods approach. This article describes a case study of the utilisation of vignettes, or short hypothetical stories, as part of the teacher-learner cycles recommended within realist evaluation. First, we explore the methodological alignment between vignettes and realist evaluation. We then present a specific case example of the application of vignettes as a data collection tool and discuss the potential advantages and the challenges of using vignettes within realist evaluation. Finally, we offer recommendations for researchers who wish to employ vignettes as a powerful instrument that can be used to better explain IPTs to participants and, in turn, enrich their participation in theory refinement within the realist evaluation framework.

User experience and analytics inform the development of an innovative telehealth curriculum: ROAD MAP (Recovery-Oriented Approach to Dementia through Meaningful Activity Participation)

Objective:To user-test a recovery-oriented, clinician-facilitated, web-based, self-management intervention accessible via mobile device: ROAD MAP. The initial programme theory underlying this research is that supporting the experiences of CHIME (connectedness, hope and optimism, identity, meaning in life and empowerment), in persons living with dementia (PLWD) will have health-promoting benefits for the PLWD.Method:The intervention will be delivered to 20 dyads of community-dwelling PLWDs and their carers via a five-week, online, recovery-based curriculum. Five convenience-sampled occupational therapists (OT) will be trained online in a 10-hour training programme to deliver the intervention. All data collection instruments are informed by realist evaluation (RE) methodology and enquire into initial programme theories (IPTs) used to develop the curriculum and the ROAD MAP digital tool. Data collection, between January and March 2023, will occur during both the OT training and the five-week pilot study. The OT facilitators will provide qualitative feedback on the ROAD MAP technology. Data will be collected via pre-post self-completion forms, semi-structured interviews, a focus group, and weekly guided reflective journal. PLWD’s self-reported, user experience will be live polled within the weekly one-hour long sessions of the pilot. Their digital usage analytics will be generated by the MyGuide platform on which the ROAD MAP intervention is built. These data will be manually screened for IPT relevance and used to test emerging programme theories.Results:All data will be collected by end of March 2023 and analysed by end of May 2023. NVivo will be used to generate refined programme theories according to RE methodology. This will provide evidence of plausible, causal context-mechanism-outcome configurations which may optimize the refined version of the ROAD MAP digital intervention, curriculum and facilitator training methods.Conclusion:This study will increase knowledge of a methodology for developing useable and acceptable recovery-oriented telehealth tools for PLWD. This intervention could directly enhance the education of health care professionals and improve the equitable delivery of dementia services.

A realist evaluation of the development of extended pharmacist roles and services in community pharmacies

BackgroundInternationally, community pharmacy models of care have been moving away from a focus on dispensing to extended, clinically-focused roles for pharmacists. ObjectivesTo identify how community pharmacy strategies were being implemented in Aotearoa New Zealand; how changes were expected to influence health and health system outcomes; what extended services were being delivered; the responses of pharmacists, other health professionals and consumers to these developments; and the contexts and mechanisms supporting the successful implementation of new community pharmacy services. MethodsA realist evaluation methodology was employed, to explore a complex policy intervention. Realist evaluation explores the contexts (C) within which initiatives are introduced and identifies the mechanisms (M) triggered by different contexts to produce outcomes (O). Realist evaluation processes iteratively develop, test, and refine CMO configurations. In this study, initial programme theories were developed through key government and professional policy documents, then refined through key informant interviews, a survey and interviews with pharmacists and intern (pre-registration) pharmacists, and finally, 10 case studies of diverse community pharmacies. ResultsFour intermediate health service outcomes were identified: development of extended community pharmacist services; consumers using extended community pharmacist services; more integrated, collaborative primary health care services; and a fit-for-purpose community pharmacy workforce. Enabling and constraining contexts are detailed for each outcome, along with the mechanisms that they trigger (or inhibit). ConclusionsThere are wide-ranging and disparate levers to support the further development of extended community pharmacy services. These include aligning funding with desired services, undergraduate educators and professional leaders setting expectations for the pharmacists’ role in practice, and the availability of sufficient funding and time for both specific extended service accreditation and broader postgraduate training. However, no simple “fix” can be universally applied internationally, nor even in pharmacies within a single jurisdiction, to facilitate service development.

Understanding the Mechanisms of Change in the Supportive and Respectful Maternity Care Intervention in Sindh, Pakistan: Provider Perspectives.

We conducted this qualitative investigation to explore the mechanisms of change in providing respectful care resulting from the supportive and respectful maternity care intervention (S-RMC) in Sindh, Pakistan. We applied the principles of realist evaluation methodology with a descriptive explanatory research design. We conducted in-depth interviews with 36 maternity care providers at secondary-level public health facilities where S-RMC was implemented for 6 months. The S-RMC broad components included capacity-building of maternity teams and systemic changes for improvements in governance and accountability within public health facilities. Data were analyzed using a deductive content analysis approach. We identified mechanisms of change, categorized by the S-RMC components: (1) S-RMC training: insight into women's feelings and rights, realization of the value that nonclinical staff can play, understanding of team coordination, orientation in psychosocial components of maternity care; (2) assessment of women's psychosocial vulnerabilities: identification of women's differential needs beyond routine care to provide woman-centered care; (3) psychosocial support: effective engagement with women and within maternity teams and the customization of woman- and companion-focused care; (4) care coordination: improved coordination among clinical and nonclinical staff to provide personalized care and psychosocial support and proper handover to ensure continuity of care; (5) assessment of quality of care: identification of service gaps from women's feedback; and (6) performance review and accountability: monthly performance review meetings to establish team member communication, systematic awareness of the maternity team's performance and challenges, and implementation of collective corrective actions. Our findings pointed to S-RMC working along multiple pathways-and concertedly with various health system components-to enable positive processes and behavioral change in maternity teams.

Exploring the Impact of a Digital Counseling Tool on Family Caregivers: An Initial Program Theory Based on Realist Methodology

Supporting family caregivers has been identified as a high-priority health goal worldwide. To reach this group, low-threshold, flexible, and accessible support services are needed that are individually adapted to the needs of the affected families. The newly developed digital counseling service “Alles Clara” intends to provide support for this group by connecting them with healthcare experts, allowing for professional and individual counseling at any time and place. To demonstrate the impact of Alles Clara and to be able to further develop and improve the service, it is to be evaluated. To uncover what works, how, under which conditions, and for whom, we drew on the realist evaluation methodology to devise an initial program theory. In this article, we describe the development of and present the initial program theory of Alles Clara. Internal program documents and guided individual interviews with stakeholders and counselors served as data sources to gradually develop a preliminary understanding of Alles Clara’s intended impact and modes of action in an iterative process. The initial program theory outlines central chains of events, mechanisms, and outcomes of the digital service. Family caregivers experience relief through expanding awareness, appreciation, trusting relationships, and reflection. For professionals, a new professional environment, a healthful culture, quality, and meaningfulness affect job satisfaction. Central to unfolding its mechanisms is that the digital counseling service enables an experience of “being seen” while “not being seen.” The results provide information about the impact of Alles Clara on both the target groups, identify possible areas for service improvement, and serve as a basis for evaluation and regular monitoring, especially, when it comes to the question of whether and how digital services can be supportive in the sense of a person-centred orientation of health care.

Feedback for Emergency Ambulance Staff: A National Review of Current Practice Informed by Realist Evaluation Methodology.

Research suggests that feedback in Emergency Medical Services (EMS) positively affects quality of care and professional development. However, the mechanisms by which feedback achieves its effects still need to be better understood across healthcare settings. This study aimed to understand how United Kingdom (UK) ambulance services provide feedback for EMS professionals and develop a programme theory of how feedback works within EMS, using a mixed-methods, realist evaluation framework. A national cross-sectional survey was conducted to identify feedback initiatives in UK ambulance services, followed by four in-depth case studies involving qualitative interviews and documentary analysis. We used qualitative content analysis and descriptive statistics to analyse survey responses from 40 prehospital feedback initiatives, alongside retroductive analysis of 17 interviews and six documents from case study sites. Feedback initiatives mainly provided individual patient outcome feedback through "pull" initiatives triggered by staff requests. Challenges related to information governance were identified. Our programme theory of feedback to EMS professionals encompassed context (healthcare professional and organisational characteristics), mechanisms (feedback and implementation characteristics, psychological reasoning) and outcomes (implementation, staff and service outcomes). This study suggests that most UK ambulance services use a range of feedback initiatives and provides 24 empirically based testable hypotheses for future research.

The Impact of Ex-Post Legislative Evaluations in Healthcare: A Mixed Methods Realist Evaluation Study Protocol for Conducting Case Studies.

Recent studies on the impact of ex-post legislative evaluations show that there are different types of impact and different factors that can influence it. These include the context of a legislative evaluation, research quality, and interactions between researchers and other actors within the evaluation process. However, thorough empirical research in this area is lacking. This warrants empirical research into the factors that influence the impact of ex-post legislative evaluations, so these insights can be used to increase the likelihood of ex-post legislative evaluations having an impact. In this protocol, we report on the realist evaluation methodology that will be used to evaluate the impact of three ex-post legislative evaluations in the Dutch healthcare sector. The mixed methods realist evaluation approach will facilitate this theory-driven, qualitative research. The study will consist of the following three steps: (1) Initial programme theory development, (2) theory validation, and (3) theory refinement. Knowledge from two scoping reviews conducted previously, and two subsequent expert meetings will form the basis for developing the initial programme theory. During this study, three case studies will be conducted, in which three individual ex-post legislative evaluations will be examined. Specificmethods for data collection will include: documentary review, observation, structured questionnaires and focus group discussions with purposefully identified key stakeholders. Using the framework approach, the data will be analysed thematically in a within-case analysis followed by a cross-case analysis. This protocol provides insight into how the study will be conducted. As this study uses multiple qualitative researchmethods to answer one question, this protocol supports refining data collection procedures. Careful consideration of the approach beforehand can minimise pitfalls, reduce publication bias and improve reproducibility. The protocol therefore specifies how the research question will be answered in detail, and this provides solid guidance for the research process.

The impact of three years application of the Health Care Team Challenge on inter-professional collaborative practice: A realist evaluation approach

Literature suggests that there are positive outcomes from Interprofessional Practice and Education (IPE) concerning changes in attitude, knowledge and skills, yet there is limited evidence about changes in professional behaviour and patient outcomes. The Singapore Ministry of Health highlighted the significance of IPE in 2010, and it was then incorporated into respective health professional programmes. To use realist evaluation methodology to explore and understand the impact of a three-year implementation of the Health Care Team Challenge (HCTC) underpinned by interprofessional practice and education principles upon health professionals’ practice at the National University Hospital (NUH) of Singapore. Ten semi-structured interviews were conducted to explore participants’ views of the HCTC. The findings resonated with other international studies, and in addition, the impact of local culture was discussed and compared with other Asian studies. Following realist evaluation, two working hypotheses evolved from the totality of the study. This study suggested a positive impact of the HCTC and indicated the need for interprofessional education to occur within post-graduate education and continuing professional development for health care professionals. This study offers a reference point for other teams keen to review and refine team functioning.

A Tale of Two Cities: Unpacking the Success and Failure of School Street Interventions in Two Canadian Cities.

One innovative strategy to support child-friendly cities is street-based interventions that provide safe, vehicle-free spaces for children to play and move about freely. School streets are one such innovation involving closing streets around elementary schools to vehicular traffic to improve children’s safety as they come and go from school while providing opportunities for children to play and socialize on the street. Launching these initiatives in communities dominated by automobiles is enormously challenging and little is known about why these interventions are successfully launched in some places but not others. As part of a larger research project called Levelling the Playing Fields, two School Street initiatives were planned for the 2021–2022 school year; one initiative was successfully launched in Kingston, ON, while the second initiative failed to launch in Montreal, QC. Using a critical realist evaluation methodology, this paper documents the contextual elements and key mechanisms that enabled and constrained the launch of these School Streets in these cities, through document analysis and key informant interviews. Our results suggest that municipal and school support for the initiative are both imperative to establishing legitimacy and collaborative governance, both of which were necessary for a successful launch.

Realist analysis of whether emergency departments with primary care services generate ‘provider-induced demand’

BackgroundIt is not known whether emergency departments (EDs) with primary care services influence demand for non-urgent care (‘provider-induced demand’). We proposed that distinct primary care services in EDs encourages primary care demand, whereas primary care integrated within EDs may be less likely to cause additional demand. We aimed to explore this and explain contexts (C), mechanisms (M) and outcomes (O) influencing demand.MethodsWe used realist evaluation methodology and observed ED service delivery. Twenty-four patients and 106 staff members (including Clinical Directors and General Practitioners) were interviewed at 13 EDs in England and Wales (240 hours of observations across 30 days). Field notes from observations and interviews were analysed by creating ‘CMO’ configurations to develop and refine theories relating to drivers of demand.ResultsEDs with distinct primary care services were perceived to attract demand for primary care because services were visible, known or enabled direct access to health care services. Other influencing factors included patients’ experiences of accessing primary care, community care capacity, service design and population characteristics.ConclusionsPatient, local-system and wider-system factors can contribute to additional demand at EDs that include primary care services. Our findings can inform service providers and policymakers in developing strategies to limit the effect of potential influences on additional demand when demand exceeds capacity.

At the heart of a diverse technology: Applying a realist evaluation methodology to a university live streaming programme


 
 
 Researching web-based lecture technologies in higher education is a complex undertaking (Morris, Swinnerton, & Coop, 2019), making experimental trials and summative evaluations impractical. The implications of existing literature in this area are therefore uncertain (Kay, 2012; O’Callaghan, Neumann, Jones, & Creed, 2017). In this paper we report on the use of an alternative research design, realistic evaluation (King, Dawson, Rothberg, & Batmaz, 2017; Pawson and Tilley, 1997), to gain insight into the impact of a university lecture live-streaming initiative. The initiative provided synchronous and asynchronous access to video of 129 weekly lectures using two different platforms. Surveys of 306 students and 49 staff, a focus group of five students, and an exploratory study of trace data of 359 students indicated that student access to the videos was lower and more idiosyncratic than expected. The ability to identify useful content, appropriate instructional methods, and participation by peers were reported to encourage students to become involved. Our results confirm the importance of specific contexts and forms of behaviour in encouraging beneficial lecture video use. We also show time of initial view to be a promising way to study students’ strategies for using lecture video content.
 
 

The Influencing Contexts and Potential Mechanisms Behind the Use of Web-Based Self-management Support Interventions: Realistic Evaluation.

BackgroundSelf-management can increase self-efficacy and quality of life and improve disease outcomes. Effective self-management may also help reduce the pressure on health care systems. However, patients need support in dealing with their disease and in developing skills to manage the consequences and changes associated with their condition. Web-based self-management support programs have helped patients with cardiovascular disease (CVD) and rheumatoid arthritis (RA), but program use has been low.ObjectiveThis study aimed to identify the patient, disease, and program characteristics that determine whether patients use web-based self-management support programs or not.MethodsA realistic evaluation methodology was used to provide a comprehensive overview of context (patient and disease characteristics), mechanism (program characteristics), and outcome (program use). Secondary data of adult patients with CVD (n=101) and those with RA (n=77) were included in the study. The relationship between context (sex, age, education, employment status, living situation, self-management [measured using Patient Activation Measure-13], quality of life [measured using RAND 36-item health survey], interaction efficacy [measured using the 5-item perceived efficacy in patient-physician interactions], diagnosis, physical comorbidity, and time since diagnosis) and outcome (program use) was analyzed using logistic regression analyses. The relationship between mechanism (program design, implementation strategies, and behavior change techniques [BCTs]) and outcome was analyzed through a qualitative interview study.ResultsThis study included 68 nonusers and 111 users of web-based self-management support programs, of which 56.4% (101/179) were diagnosed with CVD and 43.6% (78/179) with RA. Younger age and a lower level of education were associated with program use. An interaction effect was found between program use and diagnosis and 4 quality of life subscales (social functioning, physical role limitations, vitality, and bodily pain). Patients with CVD with higher self-management and quality of life scores were less likely to use the program, whereas patients with RA with higher self-management and quality of life scores were more likely to use the program. Interviews with 10 nonusers, 10 low users, and 18 high users were analyzed to provide insight into the relationship between mechanisms and outcome. Program use was encouraged by an easy-to-use, clear, and transparent design and by recommendations from professionals and email reminders. A total of 5 BCTs were identified as potential mechanisms to promote program use: tailored information, self-reporting behavior, delayed feedback, providing information on peer behavior, and modeling.ConclusionsThis realistic evaluation showed that certain patient, disease, and program characteristics (age, education, diagnosis, program design, type of reminder, and BCTs) are associated with the use of web-based self-management support programs. These results represent the first step in improving the tailoring of web-based self-management support programs. Future research on the interaction between patient and program characteristics should be conducted to improve the tailoring of participants to program components.

Open Mental Health: A Mental Health Eco System Developed in Collaboration With Communities

AimsSomerset drew upon the Community Mental Health (MH) Service Framework, and local experience and recognised that too often service user experience of mental health was beset with artificial barriers, gateways and eligibility with people sometimes needing to deteriorate before they got the vital support they needed. In addition, we recognised that the gap between secondary care and primary care was experienced by service users as a “cliff edge”. The radical redesign of mental offers in Somerset aimed to deliver an experience of “no wrong door” and where, via a partnership between health, social care and Voluntary, Community and Social Enterprise (VCSE), people's needs could be met; both in terms of specific mental health offers as well as tackling the wider determinants of mental ill health.MethodsOMH was launched early 2019, just before the pandemic, and so traditional project methodology did not always apply. Instead all partners focused on standing up the offer at pace in order to support the population within Somerset with the emotional and psychological consequences of the pandemic, and any pre existing challenges. The model was co-produced with Experts by Experience significantly contributing to the shape of the offers.The model developed comprised of two key facets: •A mental health offer, where the needs of individuals, families and carers are met by a range of health, social care and VCSE partners (including the wider determinants of mental well being such as finance / housing)•A mental health eco system developed in collaboration with communities (including underserved communities), where support is available throughout geographical communities and communities of identity.ResultsThe project has delivered both quantitively and qualitatively. Over 4100 additional appointments are offered to people in Somerset per month and reported outcomes and evaluation has been very positive. Work is underway to collate further Patient Related Outcome Measures. Following a Realist Evaluation methodology, the researcher in residence has highlighted the culture changes that are key to delivering the aspirations of the Community MH Service framework. These include; the narrative of “no wrong door”, an increased and open range of offers and interventions, blended staffing models across traditional organisational boundaries, partnership working, the role of lived experience and the ambition of addressing inequalities.ConclusionOMH, the product of Community MH Transformation in Somerset, is a radical and co produced redesign of MH services in partnership with the VCSE and Local Authority that has improved access and support to all people in Somerset.

How, when, and why do inter-organisational collaborations in healthcare work? A realist evaluation.

Inter-organisational collaborations (IOCs) in healthcare have been viewed as an effective approach to performance improvement. However, there remain gaps in our understanding of what helps IOCs function, as well as how and why contextual elements affect their implementation. A realist review of evidence drawing on 86 sources has sought to elicit and refine context-mechanism-outcome configurations (CMOCs) to understand and refine these phenomena, yet further understanding can be gained from interviewing those involved in developing IOCs. We used a realist evaluation methodology, adopting prior realist synthesis findings as a theoretical framework that we sought to refine. We drew on 32 interviews taking place between January 2020 and May 2021 with 29 stakeholders comprising IOC case studies, service users, as well as regulatory perspectives in England. Using a retroductive analysis approach, we aimed to test CMOCs against these data to explore whether previously identified mechanisms, CMOCs, and causal links between them were affirmed, refuted, or revised, and refine our explanations of how and why interorganisational collaborations are successful. Most of our prior CMOCs and their underlying mechanisms were supported in the interview findings with a diverse range of evidence. Leadership behaviours, including showing vulnerability and persuasiveness, acted to shape the core mechanisms of collaborative functioning. These included our prior mechanisms of trust, faith, and confidence, which were largely ratified with minor refinements. Action statements were formulated, translating theoretical findings into practical guidance. As the fifth stage in a larger project, our refined theory provides a comprehensive understanding of the causal chain leading to effective collaborative inter-organisational relationships. These findings and recommendations can support implementation of IOCs in the UK and elsewhere. Future research should translate these findings into further practical guidance for implementers, researchers, and policymakers.

Undergraduate medical teaching with remote consultations in general practice: a realist evaluation.

Supervisors historically educated students in primary care in face-to-face contexts; as a result of COVID-19, students now experience patient consultations predominantly remotely. There is a paucity of evidence regarding the facilitators and barriers to supervising students for excellent educational impact in the remote consultation environment. To understand the facilitators and barriers to educating medical students using remote consultations in primary care, and the consequences for students in terms of educational impact. A realist evaluation methodology was adopted to identify causal chains of contexts, mechanisms, and outcomes, describing how the teaching and learning functioned on a sample of medical students and GP tutors from two medical schools in London, UK. An initial programme theory was developed from the literature and a scoping exercise informed the data collection tools. Qualitative data were collected through online questionnaires (49 students, 19 tutors) and/or a semi-structured interview (eight students, two tutors). The data were coded to generate context-mechanisms-outcome configurations outlining how the teaching and learning operated. The results demonstrated a sequential style of supervision can positively impact student engagement and confidence, and highlighted a need to address student preparation for remote patient examinations. Students found passive observation of remote patient encounters disengaging, and, in addition, reported isolation that impacted negatively on their experiences and perceptions of primary care. Student and tutor experiences may improve through considering the supervision style adopted by tutors, and through interventions to reduce student isolation and disengagement when using remote patient consultations in primary care.

Future approaches to evaluating tourism in the developing world: Assessing realism in the Solomon Islands

Tourism is often used as a tool for economic growth and development. However, evaluating the effectiveness of tourism interventions to deliver intended benefits, and contribute to outcomes such as poverty alleviation, continue to be debated. This research examined the advantages of using the realist evaluation methodology when assessing tourism as an intervention in a developing country. An inductive research approach based on the Context, Mechanism and Outcome (C-M-O) configuration was used to capture stakeholders’ views about the benefits and beneficiaries of tourism interventions in the Solomon Islands. This research demonstrates how the realist evaluation methodology may be used to develop a holistic understanding of the effectiveness of tourism interventions by evaluating what works, for whom, how and in what circumstances and respects. This understanding has practical implications for development partners to inform decision-making that enhances the effectiveness of tourism interventions to deliver greater benefits to local communities.

Large-scale implementation of stroke early supported discharge: the WISE realist mixed-methods study

Background In England, the provision of early supported discharge is recommended as part of an evidence-based stroke care pathway. Objectives To investigate the effectiveness of early supported discharge services when implemented at scale in practice and to understand how the context within which these services operate influences their implementation and effectiveness. Design A mixed-methods study using a realist evaluation approach and two interlinking work packages was undertaken. Three programme theories were tested to investigate the adoption of evidence-based core components, differences in urban and rural settings, and communication processes. Setting and interventions Early supported discharge services across a large geographical area of England, covering the West and East Midlands, the East of England and the North of England. Participants Work package 1: historical prospective patient data from the Sentinel Stroke National Audit Programme collected by early supported discharge and hospital teams. Work package 2: NHS staff (n = 117) and patients (n = 30) from six purposely selected early supported discharge services. Data and main outcome Work package 1: a 17-item early supported discharge consensus score measured the adherence to evidence-based core components defined in an international consensus document. The effectiveness of early supported discharge was measured with process and patient outcomes and costs. Work package 2: semistructured interviews and focus groups with NHS staff and patients were undertaken to investigate the contextual determinants of early supported discharge effectiveness. Results A variety of early supported discharge service models had been adopted, as reflected by the variability in the early supported discharge consensus score. A one-unit increase in early supported discharge consensus score was significantly associated with a more responsive early supported discharge service and increased treatment intensity. There was no association with stroke survivor outcome. Patients who received early supported discharge in their stroke care pathway spent, on average, 1 day longer in hospital than those who did not receive early supported discharge. The most rural services had the highest service costs per patient. NHS staff identified core evidence-based components (e.g. eligibility criteria, co-ordinated multidisciplinary team and regular weekly multidisciplinary team meetings) as central to the effectiveness of early supported discharge. Mechanisms thought to streamline discharge and help teams to meet their responsiveness targets included having access to a social worker and the quality of communications and transitions across services. The role of rehabilitation assistants and an interdisciplinary approach were facilitators of delivering an intensive service. The rurality of early supported discharge services, especially when coupled with capacity issues and increased travel times to visit patients, could influence the intensity of rehabilitation provision and teams’ flexibility to adjust to patients’ needs. This required organising multidisciplinary teams and meetings around the local geography. Findings also highlighted the importance of good leadership and communication. Early supported discharge staff highlighted the need for collaborative and trusting relationships with patients and carers and stroke unit staff, as well as across the wider stroke care pathway. Limitations Work package 1: possible influence of unobserved variables and we were unable to determine the effect of early supported discharge on patient outcomes. Work package 2: the pragmatic approach led to ‘theoretical nuggets’ rather than an overarching higher-level theory. Conclusions The realist evaluation methodology allowed us to address the complexity of early supported discharge delivery in real-world settings. The findings highlighted the importance of context and contextual features and mechanisms that need to be either addressed or capitalised on to improve effectiveness. Trial registration Current Controlled Trials ISRCTN15568163. Funding This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 22. See the NIHR Journals Library website for further project information.