Principles Of Realist Evaluation Research Articles

Unlocking the mechanisms of change in the MAMAACT intervention to reduce ethnic disparity in stillbirth and newborns' health: integration of evaluation findings.

Ethnic disparities in stillbirth exist in Europe and suboptimal care due to miscommunication is one contributing cause. The MAMAACT intervention aimed to reduce ethnic disparity in stillbirth and newborns' health through improved management of pregnancy complications. The intervention encompassed training of antenatal care midwives in cultural competencies and intercultural communication combined with health education materials for the expecting parents about symptoms of pregnancy complications. The evaluation consisted of a qualitative in-depth implementation analysis and a process evaluation embedded in a cluster randomized trial including 19 of 20 maternity wards in Denmark. In this article, the findings from the different evaluation perspectives are integrated. The integration follows the principles of realist evaluation by analyzing to what extent the MAMAACT activities were generating mechanisms of change in interaction with the context. The integration analysis shows that the health education materials in the MAMAACT intervention contributed to heightened health literacy concerning pregnancy complications among pregnant women. Additionally, the training of midwives in cultural competency and intercultural communication raised awareness among midwives. Nonetheless, the exclusive emphasis on midwives and the inflexibility in care provision hindered them from changing their communication practices. To enhance the cultural competence in maternity care, it is essential to implement more comprehensive initiatives involving healthcare professionals in maternity care at all levels, from pregraduate to postgraduate. Adequate interpreter services and management support should also be ensured. Currently, the Danish antenatal care system faces challenges including inadequate information transfer between healthcare sectors, insufficient differentiation of care, and inflexibility in midwife scheduling. This results in a lack of responsiveness to the individual needs of women with immigrant backgrounds, potentially reproducing health inequities.

Realist Evaluation of Home-Based Palliative Care Programme for Non-Cancer Patients

Introduction: Disproportionately higher deaths attributed to non-cancer causes in Singapore highlight the need for palliative beyond those with cancers. The Violet Program (ViP) was developed to provide home-based palliative to non-cancer patients residing in the Eastern region of Singapore. VIP aims to (i) reduce unnecessary acute hospital utilization (ii) improve symptoms management (iii) reduce caregiver burden through caregiver support (iv) honour patient preferences better through integrated care while (v) keeping healthcare cost affordable. It is acknowledge that successful implementation and the long-term sustainability of ViP is contingent on its ability to effectively attract, grow and develop its manpower resources. However, identifying, attracting, developing, and retaining competent and engaged community-based palliative care workforce staff remains a major challenge. The aim of this study is to understand attributes of staff and contextual factors well suited for ViP and factors leading to sustainability of such a workforce
 Methods: This study was conducted in 2 phases according to realist evaluation principles. First, initial programme theories (IPT) related to sustainability of palliative care manpower were developed through a scoping review and semi-structured interviews with those who were involved in the conceptualization of ViP. IPTs were expressed in Context (C) Mechanism (M) and Outcome (O) configurations. Then, IPTs were tested using interviews with healthcare care workers (HCW) who were involved in ViP. Only CMO configurations supported by substantial evidence were elicited and refined. 
 Results: 36 HCW of varied roles were interviewed. At the HCW level, (i) familiarity with palliative care framework (ii) comfort in dealing with death (iii) passion for palliative care field (iv) exposure to palliative care during medical/nursing training and (iv) ability to self-care were found to be important attributes for ViP care team. At the institutional level, (continuous) training and provision of socio-emotional support to staff were found to contribute to retention of staff. Self-efficacy for the required tasks, empathy for affected patients/family, passion-driven motivation and dedication to go the extra mile and strong emotional resilience were found to be the key driving forces behind the sustainability of workforce for ViP for the longer term. 
 Implications: This study illustrated the applicability of using realist evaluation for assessment of context specific insights for the identification of suitable workforce and how to sustain them in the longer term. These insights can potentially be used as selection criteria for recruitment of staff for ViP and points to area for training of staff. 

Investigating the processes and influences involved in the transformational journeys of Registered Nurse Degree Apprentices: A realist informed qualitative study

AimThis study aimed to investigate students’ learning journeys across the duration of a new registered nurse degree apprenticeship programme and to develop an understanding of the contextual factors, mechanisms and outcomes involved. BackgroundRegistered nurses are the largest group of healthcare workers globally, but shortages exist. To encourage existing UK healthcare employees into nursing, national investment was made into Registered Nurse Degree Apprenticeships. In 2018 a UK health service organisation and a university collaboration led to development of a nursing degree apprenticeship programme. Research into these novel undergraduate programmes in nursing is lacking, with scarce evidence or understanding of processes and experiences involved in such programmes. Design and methodsAn exploratory qualitative design informed by realistic evaluation was employed. Three sequential semi-structured interviews were conducted with an entire cohort (n=8) across the 18-month programme (24 interviews). Focused interviews were also undertaken with practice assessors (n=8) involved in the apprentices’ journey. Initial thematic analysis was followed by application of Realist Evaluation principles and a temporal lens to move beyond qualitative description. ResultsAnalysis identified four interrelated temporal themes, each comprising contextual factors, mechanisms and outcomes acting to inhibit or facilitate transition across the apprenticeship journey: (1) Starting out (Latent Ambition, enhanced motivation and expectations): programme availability and conditions enabled enactment of ambitions to become qualified nurses, preconceived assumptions regarding roles influenced expectations. (2) Initial stages (Identity dissonance, transition to academia, becoming a student nurse) related to changing identity and re-conceptualisation of their role to student nurse, alongside transitioning into higher education. (3) Travelling through (Being an apprentice, social capital, self-efficacy and confidence) facilitated confident integration into placements, although over confidence could be risky. Academic struggles prompted some episodes of emotional dissonance. (4) Moving beyond to become registered nurses (Confidence, empowerment and loyalty in the transition to becoming a nurse) saw apprentices transform, with revised understandings of nursing, increased empowerment and self-efficacy. All expressed gratitude and loyalty for the apprenticeship opportunity, with all securing jobs in their employing organisation. ConclusionThis study is one of the first of its kind, providing detailed insight into processes experienced by students over the duration of a novel apprenticeship programme. Analysis identified several factors that facilitated and inhibited progress in participants' ‘learning journeys’, mapping context, mechanism, outcome configurations which came into play at various stages. These were influential in successful completion of the programme by all apprentices, resulting in the development of competent Registered Nurses.

Does citizen engagement improve development outcomes? A realist-informed systematic review of participation and accountability mechanisms

ABSTRACT Effective, accountable and transparent institutions, that engage in inclusive and participatory governance, are crucial for the sustainability of global development investments. However, there is a debate about whether effective approaches to improving governance processes operate from the bottom up (e.g. by enabling citizens to hold service providers accountable) or the top down (by enabling service providers to be held accountable by the State). This paper systematically reviews participation and accountability mechanisms in a range of sectors, drawing on principles of realist evaluation to develop and test middle-range theory using framework synthesis and statistical meta-analysis. We show that interventions promoting citizen engagement through participatory priority setting or accountability mechanisms are often effective in stimulating active citizen engagement in service delivery and realising improvements in access to services, where they facilitate direct engagement between service users and front-line service providers, such as in health care. However, citizen engagement interventions alone are not effective where services are accessed independently of service provider staff, for example road infrastructure. Interventions promoting participation by increasing citizens’ pressures on politicians to hold providers to account are also not usually able to influence service delivery.

Evidence-based policymaking: Lessons from the Chilean Substance Use Treatment Policy.

Evidence-based policymaking is a guiding paradigm of substance use treatment (SUT) policy, that seeks to prioritise scientific criteria over other concerns (e.g., economic, political) when addressing policy decisions. We provide a comprehensive analysis of the context and mechanisms that enable and constrain evidence to improve the Chilean SUT policy and draw some lessons that might be useful to other contexts, particularly low and middle-income countries. This study relied on an interpretive case study design based on the principles of realist evaluation. We included interviews (N≈17) with international, national, regional, and local policymakers and experts, as well as technical and clinical teams from private and public care SUT providers in Chile. Complex sets of institutional realities and notions of 'evidence' shared by actors - between other elements- guide the SUT policy decisions and shape the specific type of evidence considered relevant. Evidence is understood in Chile in narrow terms, and national non-experimental research is often overlooked. This limits the possibility of studying other research questions that could contribute to improving and informing SUT policy. In contexts where addiction research resources are limited, it appears necessary to re-frame the notion of "evidence", to consider relevant national non-experimental knowledge to strengthen SUT policy and achieve its goals. Indeed, this study is an example of how methodological approaches, such as case analysis, can provide a powerful heuristic alternative contribution to the local and global mental health debate.

An ounce of prevention is worth a pound of cure—the arts as a vehicle for knowledge translation and exchange (KTE) in public health during a pandemic: a realist-informed developmental evaluation research protocol

IntroductionThe COVID-19 crisis has led to the adoption of strict and coercive preventive measures. The implementation of these measures has generated negative impacts for many communities. The situation is especially...

Do information-based measures affect active travel, and if so, for whom, when and under what circumstances? Evidence from a longitudinal case-control study

Soft, information-based measures to encourage walking and cycling for travel are increasingly being recommended alongside infrastructure investments. Using principles of realist evaluation, we evaluate measures implemented as part of the European Physical Activity Through Sustainable Approaches (PASTA) study in Vienna (Austria), Örebro (Sweden), Rome (Italy), and Antwerp (Belgium) over a 3-year cohort study, and a further follow-up 2.5 years later in Vienna and Örebro. Increases in active modes of travel due to the interventions were most significant for walking, one year after the intervention, and for people in full-time employment. Increases in e-bike use were associated with changes in perceptions of cycling, while increases in walking were not associated with any changes in perceptions of walking. We find evidence supporting previous findings that information provision is unlikely to work as a standalone intervention in the longer run, but may be effective when combined with other policies. • Robust longer term evaluation of measures encouraging active travel in cities. • Soft measures helped increase walking and e-bicycling in all 4 cities 2014–2016. • Soft measures had the most long-term effect on e-bicycling, per the 2019 survey. • Public transit improvements can increase walking and may reduce driving.

Building resilience in oncology teams: Protocol for a realist evaluation of multiple cases.

BackgroundTeams caring for people living with cancer face many difficult clinical situations that are compounded by the pandemic and can have serious consequences on professional and personal life. This study aims to better understand how a multi-component intervention builds resilience in oncology teams. The intervention is based on a salutogenic approach, theories and empirical research on team resilience at work. This intervention research involves partnership between researchers and stakeholders in defining situations of adversity and solutions appropriate to context.MethodsThe principles of realist evaluation are used to develop context-mechanism-outcome configurations of a multi-component intervention developed by researchers and field partners concerned with the resilience of oncology teams. The multiple case study involves oncology teams in natural contexts in four healthcare establishments in Québec (Canada). Qualitative and quantitative methods are employed. Qualitative data from individual interviews, group interviews and observation are analyzed using thematic content analysis. Quantitative data are collected through validated questionnaires measuring team resilience at work and its effect on teaming processes and cost-effectiveness. Integration of these data enables the elucidation of associations between intervention, context, mechanism and outcome.DiscussionThe study will provide original data on contextual factors and mechanisms that promote team resilience in oncology settings. It suggests courses of action to better manage difficult situations that arise in a specialized care sector, minimize their negative effects and learn from them, during and after the waves of the pandemic. The mechanisms for problem resolution and arriving at realistic solutions to professional workforce and team effectiveness challenges can help improve practices in other settings.

“Putting Down and Letting Go”: An Exploration of a Community-Based Trauma-Oriented Retreat Program for Military Personnel, Veterans, and RCMP

(1) Background: Current military members, veterans, and Royal Canadian Mounted Police (RCMP) experience higher rates of posttraumatic stress disorder (PTSD) and moral injury (MI). Trauma-oriented retreats have been offered as a means of addressing these concerns. This article aims to explore the impact of a non-evidence-based trauma-oriented retreat for the above populations experiencing PTSD or MI; (2) Methods: This qualitative study, nested within the larger mixed-methods pre/post longitudinal follow-up study, examined the experiences of 124 military members, veterans, and RCMP who participated in the retreat. Data were collected from semi-structured interviews and first-hand observations of the organization. Analysis was conducted using thematic analysis while being informed by realist evaluation principles; (3) Results: The results showed that important contextual elements were related to participants being ready, having multiple comorbidities and using the program as a first or last resort. Effectual mechanisms included a home-like setting; immersion; credibility of facilitators; experiential learning; an holistic approach; letting go, and reconnecting to self. Outcomes included: re-finding self, symptom management, social connection, and hope for a meaningful life. The gendered analysis suggested less favorable results; (4) Conclusions: Care is warranted as the evidence-base and effectiveness of trauma-oriented retreats yet needs to be established prior to broad use.

A realist evaluation of a home-based end of life care service for children and families: what works, for whom, how, in what circumstances and why?

BackgroundChildren’s palliative and end of life care is underpinned internationally by a commitment to provide care and support in the family’s preferred place, which may include home, hospital or hospice. Limited evidence on models of best practice for the provision of children’s end of life care at home is available. This realist evaluation of a novel, home-based end of life care service explored what works for whom, how, in what circumstances and why.MethodsAdopting principles of realist evaluation, an initial programme theory (IPT) was developed from multiple data sources including a scoping review, service documentation review, audit of service data, and qualitative data gathered from stakeholder (n = 6) and family interviews (n = 10). Three families who had used the service were identified as case studies and interviews with professionals involved in their care (n = 20) were conducted to test the IPT. The findings informed the revised CMOs illustrating the contexts and mechanisms which underpin how and why the service works and for whom.ResultsCMO configurations were identified explaining how and why the service works for families across five core components: anticipatory approach to care planning and delivery, advance care planning, service responsiveness and flexibility, 24/7 nurse-led service with 24-h medical support, and partnership working. Key mechanisms include establishing trusting relationships, building skills and parent confidence to deliver care, early advance care planning discussions with clear documentation, providing a single point of contact for families, workforce planning and resources to deliver the service as intended, effective communication and leadership within and between providers of the service, and use of joint policies and procedures. Recommendations for future development and expansion of the service are also discussed.ConclusionsThe findings highlight core components making this service a success and areas of challenge which continue to be addressed as the service develops. With increasing demand for home-based end of life care for children these components provide a structure which can help to guide service development to meet the needs of these families in other regions to ensure that children and families receive good quality care in their place of choice.

Student perspectives on experiencing a Northern Ontario portage and wilderness immersion program at Laurentian University in Northern Ontario, Canada

ABSTRACT This study examined student experiences participating in a peer-led three-day wilderness canoe excursion in Northern Ontario, Canada. This program is a mandatory outdoor experience program (MOEP) for third-year students and is the final course in a progressive series of outdoor experiences offered at the beginning of the first, second, and third year. Guided by principles of realist evaluation, the written reflections from consenting students in the 2015 MOEP cohort were analyzed to identify the contexts and mechanisms associated with enjoyment or lack of enjoyment of their trip. These findings will help improve the MOEP program and have implications for similar intentionally led outdoor programs offered at other higher education institutions.

Why do strategies to strengthen primary health care succeed in some places and fail in others? Exploring local variation in the effectiveness of a community health worker managed digital health intervention in rural India

IntroductionDigital health interventions (DHIs) have huge potential as support modalities to identify and manage cardiovascular disease (CVD) risk in resource-constrained settings, but studies assessing them show modest effects. This study...

Opening up the black box of a Gateway to Medicine programme: a realist evaluation

ObjectivesA Gateway to Medicine programme, developed in partnership between a further and higher education setting and implemented to increase the socioeconomic diversity of medicine, was examined to identify precisely what...

Intervening in the cycle of poverty, poor housing and poor health: the role of housing providers in enhancing tenants\u2019 mental wellbeing

Poverty, poor housing and poor health are complexly interconnected in a cycle that has proven resistant to intervention by housing providers or policy makers. Research often focuses on the impacts of the physical housing defects, particularly upon rates of (physical) illness and disease. There has been comparatively little research into the ways in which housing services can underpin the generation of positive health and, especially, wellbeing. Drawing on qualitative data from 75 tenants in the social and private rented sectors, this paper describes the findings of a research project that tracked tenants’ experiences across their first year in a new tenancy in Greater Glasgow, Scotland. The project collected data on tenants’ perceptions of housing and housing service quality, financial coping and health and wellbeing, which was analysed using the principles of Realist Evaluation to elucidate impacts and causal pathways. Being able to establish a sense of home was key to tenants’ wellbeing. The home provided many tenants with a recuperative space in which to shelter from daily stressors and was a source of autonomy and social status. A sense of home was underpinned by aspects of the housing service, property quality and affordability which are potentially amenable to intervention by housing providers. These findings raise questions about the extent to which social housing providers and the private rental market in the UK are able to meet the needs of vulnerable tenants. They suggest that approaches to housing provision that go beyond providing a basic dwelling are needed to successfully intervene in the cycle of poverty, poor housing and poor health.

How do participatory methods shape policy? Applying a realist approach to the formulation of a new tuberculosis policy in Georgia

ObjectivesThis paper presents the iterative process of participatory multistakeholder engagement that informed the development of a new national tuberculosis (TB) policy in Georgia, and the lessons learnt.MethodsGuided by realist evaluation...

Improving oral health in nursing home residents: A process evaluation of a shared oral care intervention.

To evaluate the process of implementing an oral care intervention in nursing homes in a Danish municipality. Older people with aged natural dentition require preventive and curative oral health care. An intervention based on principles of situated learning was implemented to establish closer cooperation between dental and nursing staff in nursing homes, leading to improved oral hygiene in nursing home residents. An embedded multiple-case study combined with principles of realist evaluation unfolded in three phases: Formulation of initial programme theory, Testing and Refining the programme theory. The COREQ checklist is followed in reporting. Observations, six group interviews and 22 face-to-face interviews with dentists, dental practitioners, nursing home managers, care professionals and residents were conducted in three nursing homes (n=41). Three main outcomes of a programme theory were identified, relating to (a) residents, in the form of new oral care routines; (b) interdisciplinary working, in the form of professional pride in performing sufficient oral care; (c) organisational level changes, in the form of increased interdisciplinary knowledge sharing. The overarching supportive mechanisms were the creation of relationships between residents, dental practitioners and care professionals as well as nursing home management taking responsibility for structure, planning and knowledge sharing. The situated learning perspective supported residents and care professionals' competencies in performing sufficient oral care. The shared oral care intervention supports an individual and multidisciplinary assessment of nursing home residents' ability to self-care concerning oral care. Contextual factors, supportive and restraining mechanisms influence the intervention's success. Understanding the complexity within interdisciplinary cooperation in primary nursing and unravelling the necessary properties to enhance nursing home residents' oral health care are areas of improvement for care service in nursing homes.

Evaluation of an English language peer tutoring intervention

PurposeDesertion rates in Colombian universities remain unacceptably high. In the field of foreign languages, academic failure is particularly concerning since English language instruction is compulsory in most universities. To address the issue of poor student performance and high dropout rates, the University of Colombia has set up a peer tutoring scheme (PTS) for English as a foreign language (EFL) students in order to inform programme development. The paper aims to discuss this issue.Design/methodology/approachThe study was informed by realist evaluation principles. Qualitative in-depth interviews were conducted with eight PTS stakeholders supplemented by documentary analysis of the programme’s publicity material on the PTS website. The data were analysed using thematic analysis.FindingsFindings reveal discrepancies between the “espoused theory” about how the programme operates and the “theory-in-use”. In particular, according to stakeholders, the programme does not appear to be used by many of those EFL students who would benefit from it, which suggests that the programme is not as effective as it could be. Student and teacher contextual factors and mechanisms may explain the reasons for issues with programme effectiveness.Research limitations/implicationsFormative evaluations such as the current study can provide rich contextual information, but cannot be generalised to other settings. Also, this study does not explore the perspective of peer tutors and tutees, which means key variables may have been overlooked. Further research into the perspectives of tutors and tutees would therefore be needed to firm up these conclusions.Practical implicationsDue to the scarcity of literature into EFL peer tutoring interventions in higher education (HE), it is hoped that these findings will have relevance for similar contexts. The current evaluation highlights the influence of contextual factors such as willingness to ask for help, student motivation, student priorities, tutor credibility, teacher workload, timetabling and scheduling issues and involvement from teachers on the success of open-access peer tutoring programmes for EFL students in higher educational settings.Originality/valueAs far the researcher is aware, this is the first evaluation of an EFL peer tutoring programme in a private HE context in Colombia, and one of only a handful of studies into EFL peer tutoring programmes. The findings therefore have implications for those working in similar contexts.

Implicit Stigma Recognition and Management for Health Professionals.

Stigma against individuals with mental illness has disastrous consequences for patient outcomes. Better approaches to reducing stigma in health care professionals are required. Implicit stigma education is an emerging area of research that may inform the design and implementation of stigma reduction programs. In this "in brief report," the authors describe the evaluation of a novel implicit stigma reduction workshop for health professionals. The authors conducted a realist evaluation using a longitudinal multiple case study approach. Once a conceptual model was established, three case studies were conducted on physicians and nurses (n = 69) at an academic health sciences center. Within each case, pre- and post-attitudinal scales and qualitative data from semi-structured interviews were used. Consistent with realist evaluation principles, context-mechanism-outcome configuration patterns were analyzed. An implicit stigma recognition and management workshop produced statistically significant changes in participant attitudes in two out of three contexts. The qualitative evaluation described the perceptions of sustainable changes in perspective and practice. The degree to which individual participants learned with and worked among inter-professional teams influenced outcomes. Implicit stigma recognition and management is a useful educational strategy for reducing stigma among health professionals. Once stigma is recognized, curricular interventions may promote behavioral change by encouraging explicit alternative behaviors that are sustained through social reinforcement within inter-professional teams.

The Implementation of an Innovative Hydration Monitoring App in Care Home Settings: A Qualitative Study.

BackgroundIn response to marked concern regarding inadequate fluid intake recording in care homes, an innovative mobile hydration app was collaboratively developed. “Hydr8” aimed to facilitate accurate recording and communication of residents’ fluid intake and ultimately increase care quality and patient safety.ObjectiveThe aim of this study was to examine the implementation of Hydr8 in a sample of care homes in one area in England.MethodsThe principles of Realist Evaluation and Action research were drawn upon throughout the study. Overall, 5 care homes participated in this study, 3 interview-only sites and 2 case-study sites, where interviews and observations were conducted at 3 time-points. Furthermore, 28 staff members participated, including care staff, management, a registered nurse, and administrative staff.ResultsFindings suggest that Hydr8 benefits practice, enhancing the understanding of hydration and person-centered care and improving staff communication. However, technical glitches hindered the seamless embedding of Hydr8 into everyday practice, and enthusiasm for long-term use was dependent on the resolution of issues. In addition, Hydr8 heightened perceptions of personal accountability, and while managers viewed this as positive, some staff members were apprehensive. However, individuals were enthusiastic about the long-term use and potential of Hydr8.ConclusionsUtilizing the findings of this study to further develop and adapt Hydr8 indicates the long-term use of Hydr8 as promising. Although perceptions of Hydr8 were primarily positive, setbacks in its implementation and use created difficulties in normalizing the solution into everyday practice. This study highlights the need for education related to hydration practice and a change of infrastructure in care home settings to implement technical solutions and changes to care.

Midwifery continuity of carer: Developing a realist evaluation framework to evaluate the implementation of strategic change in Scotland

Midwifery continuity of carer (MCC) models result in better clinical outcomes for women and offer midwives a superior way of working when compared to other models of maternity care. Implementing a MCC model, a key recommendation of the Scottish Government Maternity and Neonatal Strategy Best Start, requires significant restructuring of maternity services and changes to midwives’ roles. Careful evaluation is therefore required to monitor and understand how the policy affects care providers and users. Realist evaluation is an appropriate methodology for evaluating programmes of change set within complex social organisations, such as health services, and can help to understand variations in outcomes and experiences. This paper presents the approach taken using the principles of realist evaluation to identify key programme theories, which then informed an evaluation framework and a midwives’ evaluation tool. The comprehensive survey-tool developed for midwives has the potential to be used more widely to evaluate comparable strategic change in this area.