Realist Evaluation Methodology Research Articles

Disability Data Collection in a Complex Humanitarian Organisation: Lessons from a Realist Evaluation.

In recent years, global attention to disability inclusion in humanitarian and development contexts, notably comprising disability inclusion within the Sustainable Development Goals, has significantly increased. As a result, UN agencies and programmes are increasingly seeking to understand and increase the extent to which persons with disabilities are accounted for and included in their efforts to provide life-saving assistance. To explore the effects and effectiveness of such measurement, this paper applies a complexity-informed, realist evaluation methodology to a case study of a single measurement intervention. This intervention, ‘A9’, was the first indicator designed to measure the number of persons with disabilities assisted annually by the United Nations World Food Programme (WFP). Realist logic of analysis combined with complexity theory was employed to generate context-mechanism-outcome configurations (CMOC’s) against which primary interviews and secondary data were analysed. We show that within the complexity of the WFP system, the roll-out of the A9 measurement intervention generated delayed, counter-intuitive and unanticipated effects. In turn, path dependency and emergent behaviours meant that the intervention mechanisms of yesterday were destined to become the implementation context of tomorrow. These findings challenge the current reliance on quantitative data within humanitarian-development disability inclusion efforts and contribute to our understanding of how data can best be leveraged to support inclusion in such contexts.

Strengthening Community Health Systems Through Novel eHealth Initiatives? Commencing a Realist Study of the Virtual Health Rooms in Rural Northern Sweden.

Background: Unlike the large body of research that has examined the ‘success’ or ‘failure’ of eHealth in terms of patient and provider perceptions or cost- and clinical effectiveness, the current study teases out ways through which a novel eHealth initiative in rural northern Sweden might result in more distal or systemic beneficial outcomes. More specifically, this paper aims to explore how and under what circumstances the so-called virtual health rooms (VHRs) are expected to improve access to person-centred care and strengthen community health systems, especially for elderly residents of rural areas. Methods: The first phase of the realist evaluation methodology was conducted, involving qualitative interviews with 8 key stakeholders working with eHealth, business development, digitalisation, and process management. Using thematic analysis and following an abductive-retroductive analytical process, an intervention-context-actor-mechanism-outcome(ICAMO) configuration was developed and elicited into an initial programme theory. Results: The findings indicate that a novel eHealth initiative, which provides reliable technologies in a customized facility that connects communities and providers, might improve access to person-centred care and strengthen community health systems for rural populations. This is theorized to occur if mechanisms acting at individual (such as knowledge, skills and trust) and collective (like a common vision and shared responsibilities) levels are triggered in contexts characterised by supportive societal transitions, sufficient organisational readiness and the harnessing of rural cohesiveness and creativity. Conclusion: The elicited initial programme theory describes and explains how a novel eHealth initiative in rural northern Sweden is presumed to operate and under what circumstances. Further testing, refinements and continued gradual building of theory following the realist evaluation methodology is now needed to ascertain if the ‘VHRs’ work as intended, for whom, in what conditions and why.

How and why does Local Area Coordination work for people in different contexts?

The increasing adoption of Local Area Coordination across the United Kingdom as a strengths-based approach to acting on inequalities which impact on individual health and well-being, and reducing reliance and avoidable use of health and social care services, has catalysed increasing calls for evidence to justify economic commitment. In a time of austerity where extreme pressure is on resources to prove short-term outputs, Pawson and Tilley’s realist evaluation methodology holds significant promise in asking critical questions of how and why programmes work. Ultimately, such philosophical standpoints facilitate opportunities to examine whether the sustainability of programmes are cost-effective for the system in the longer term. This article draws upon the findings of a realist evaluation of Local Area Coordination on the Isle of Wight and establishes how and why the programme works for people and local communities. A blend of realist approaches, Q-method and realist interviews were adopted within this study. The study’s sample was a cross section of 18 people who engaged with the Local Area Coordination programme across the Isle of Wight. The findings of the evaluation established that the Local Area Coordinators’ ability to facilitate a ‘golden triangle’ of listening, trust and time were factors which made Local Area Coordination work. It was also clear that Local Area Coordination worked for different people in different ways, demonstrated through the contextual differences between three subgroups who were categorised based on shared viewpoints, and presented through the holistic narratives and corroborating interview data.

The comprehensive framework for integrated healthcare implementation: a realist evaluation of positive parenting in rural primary care in the US

PurposeThis study aimed to develop a comprehensive framework that encompasses contextual determinants of integrated care implementation. The initial framework was developed based on literature and was refined based on contexts and mechanisms that facilitated or interfered with integrating a pilot parenting intervention, Behavior Checker (BC), into the routine service delivery in a rural primary care clinic in the USA.Design/methodology/approachThis study was a single organizational case study informed by the realist evaluation methodology. Data collection methods included key informant interviews, healthcare staff surveys, a focus group and direct observation of meetings. Data analysis focused on identifying the context-mechanism-outcome (CMO) frameworks, a heuristic used in a realist evaluation to map pathways of the interactions between program inputs and contextual conditions leading to intended and/or unintended outcomes.FindingsThe identified CMOs and antecedent contexts identified informed the process of revising the initial Comprehensive Framework for Integrated Healthcare Development and Implementation (CF-IHDI). The revised CF-IHDI contained six parent domains of outer setting, basic structures, people and value, intervention characteristics, organizational capabilities for change and key processes.Research limitations/implicationsThe CMOs and core characteristics of contextual conditions that activated facilitating mechanisms can inform future studies examining healthcare integration efforts.Practical implicationsThe CF-IHDI can guide primary care clinics in identifying factors and strategies to consider when integrating parenting or other psychosocial interventions into primary care routine service delivery.Originality/valueThe CF-IHDI developed in this study contributes to the knowledge of contexts and mechanisms that facilitate and interfere with integrated care implementation.

Impact of the ‘Learning from Excellence’ programme in NHS trusts: an exploratory study

Background/Aims The ‘Learning from Excellence’ (LfE) programme aims to provide a means to identify, appreciate, study and learn from episodes of excellence in frontline healthcare. The aim of this study was to explore the impact of LfE on organisational performance in NHS trusts in the United Kingdom (UK), how this impact is achieved and which contextual factors facilitate or hinder impact. Methods A survey and case studies, including individual interviews, were conducted. Data were analysed through qualitative content analysis, informed by realist evaluation methodology. Contextual factors, mechanisms and outcomes were coded, sub-themes were generated and a programme theory was developed. Results There were 47 respondents to the survey (27% response rate) and three case studies were performed. A total of 25 contextual factors were identified, of which 18 acted as a facilitator to uptake or as a barrier if absent. Additionally, 11 mechanisms and 9 outcomes were identified. A programme theory explaining how LfE can impact organisational performance in NHS trusts in the UK, and which contextual factors play a role before design, during design and during implementation, was developed. Conclusions LfE was welcomed by participants, but at the time of this study LfE had not been implemented for long enough for outcomes to be measured. Further research, particularly measuring outcomes and validating mechanisms, as well as exploring the learning based on reported events, is needed.

Can a case lead approach deliver the “craft and graft” of integration?

PurposeThe aim of this study was to explore the experiences and outcomes for adults with complex needs over time, within and between two teams that delivered integrated care across different Councils' services. The teams' approach to integration included two key features: a “case lead” way of working and the team itself operating as a single point of access (SPA) for residents in given neighbourhoods with high deprivation.Design/methodology/approachThe study was designed as evaluation research located in the realist tradition. Two teams acted as a case study to provide an in-depth understanding of how the case lead approach and SPA delivered the craft and graft of integrated working in the teams. Mixed methods of data collection included residents' ratings of their quality of life on five domains in an outcome measure over a six-month period. Residents and staff working in the teams also participated in semi-structured interviews to explore their respective experiences and receiving and delivering integrated care. The costs of care delivery incurred by residents were calculated based on their demands on public services in the year leading up to the teams' intervention and the projected costs for one year following this.FindingsThe relationship between team context, case leads' inputs and residents' outcomes was mediated through the managerial style in the integrated teams which enabled case leads to be creative and do things differently with residents. Case leads worked holistically to prevent residents being in crisis as well as giving practical help such as sorting debts and finances and supporting access to volunteering or further education. Residents rated their quality of life as significantly improved over a six-month period and significant savings in costs as result of the teams' support were projected.Originality/valueThe study used a multi-evaluation realistic evaluation methodology to explore the relationship between team context, case leads' inputs and residents' outcomes in terms that integrated services across different District and County Council Departments.

Successful Community Resettlement of Men with Learning Disabilities Who Have Completed a Hospital-Based Treatment for Sexual Offending

Abstract Treatment for sexual offending equips men with learning disabilities with tools required for pro-social community living. In the past, risk aversiveness prevented discharges from hospital, but fieldwork took place at the time of the Transforming Care Agenda, which sought to enable more people to return to their communities. This offered the opportunity to gain unique insights into community resettlement planning in cases that require ongoing risk management. Eleven case studies were examined through qualitative interviews with the men and professionals. A realist evaluation methodology was applied to examine how treatment outcomes manifested longer term. It was evidenced that treatment had equipped men with risk management tools, as well as encouraged them to develop realistic visions for their pro-social futures and that both outcomes come to fruition under conditions that allow positive risk taking. The welfareist and user-led nature of working towards pro-social community living makes this a useful toolkit for social work, whilst input from forensic health services was valued for skilling up the social care workforce. However, discharge practices continued to be influenced by contextual factors, including local availability of resources and personal attributes, such as men’s sexual preferences and levels of compliance and some men remained in hospital.

Overcoming the challenge of “fuzzy” assessment and feedback

PurposeThis paper explores the challenge of “fuzzy” assessment criteria and feedback with a view to aiding student learning. The paper untangles three guiding principles as mechanisms to enhance the effectiveness of assessment and feedback through overcoming the inherent challenges which stem from tacit judgement during assessment.Design/methodology/approachThe paper applies a realistic evaluation methodology, with a framework for assessment and feedback consisting of three principles – Means, Opportunity and Motivation (MOM). Through in-depth interviews with undergraduate and postgraduate management students the paper identifies how and when the means, opportunity and motivation principles impact student learning through assessment and the utility of the feedback received on their learning.FindingsThe findings in the paper illustrate that students do not always understand the feedback they receive on their learning because they do not fully understand the criteria to which it refers due to the tacit dimensions of assessment. The findings substantiate the proposition that effective assessment processes must ensure that students have the means, opportunity and motivation to use feedback and to understand the criteria, a central component of which is understanding tacit dimensions of assessment.Practical implicationsThe paper deciphers three practical implications for instructors related to (1) teaching, (2) course and program design and (3) the nature of the feedback instructors should provide.Originality/valueWhile prior scholarship has flagged the challenge of “fuzzy” assessment and feedback, this paper identifies when and how the means, opportunity and motivation principles are manifested in the process of making the tacit components of assessment codified and actionable, a critical process in developing expert learners.

Innovative recruitment and clinical orientation programme to manage NHS junior doctor shortfall: A district hospital experience

International medical graduates (IMGs) contribute significantly towards the NHS care provision. No standardised clinical orientation programme (COP) for IMGs new to the NHS exists. Our objective was to describe recruitment and retention strategies for junior doctors (JDs) in general medicine and develop a framework to anticipate outcomes of these interventions using the realist evaluation methodology. We performed quality improvement interventions of recruitment and COP for new entrant IMGs in our organisation employed between December 2017 and April 2019. Twenty-three IMGs were recruited, 96% successfully completed the COP with a mean contract duration of 13±5 months. From the academic year 2017/18 to 2018/19, mean JD post occupancy increased from 54±3 to 73±4 JDs (p<0.001) and JD locum spend fell by £1.9 million. Our structured COP provides a stable, trained and financially sustainable JD workforce. Application in broader NHS settings is recommended.

Collaboration between parents and SLTs produces optimal outcomes for children attending speech and language therapy: Gathering the evidence.

BackgroundCollaboration between parents and speech and language therapists (SLTs) is seen as a key element in family‐centred models. Collaboration can have positive impacts on parental and children's outcomes. However, collaborative practice has not been well described and researched in speech and language therapy for children and may not be easy to achieve. It is important that we gain a deeper understanding of collaborative practice with parents, how it can be achieved and how it can impact on outcomes. This understanding could support practitioners in daily practice with regard to achieving collaborative practice with parents in different contexts.AimsTo set a research agenda on collaborative practice between parents and SLTs in order to generate evidence regarding what works, how, for whom, in what circumstances and to what extent.Methods & ProceduresA realist evaluation approach was used to make explicit what collaborative practice with parents entails. The steps suggested by the RAMESES II project were used to draft a preliminary programme theory about collaborative practice between parents and SLTs. This process generates explicit hypotheses which form a potential research agenda.Discussion & ConclusionsA preliminary programme theory of collaborative practice with parents was drafted using a realist approach. Potential contextual factors (C), mechanisms (M) and outcomes (O) were presented which could be configured into causal mechanisms to help explain what works for whom in what circumstances. CMO configurations were drafted, based on the relevant literature, which serve as exemplars to illustrate how this methodology could be used. In order to debate, test and expand our hypothesized programme theory for collaborative practice with parents, further testing against a broader literature is required alongside research to explore the functionality of the configurations across contexts. This paper highlights the importance of further research on collaborative practice with parents and the potential value of realist evaluation methodology. What this paper adds Current policy in education, health and social care advocates for family‐centred care and collaborative practice with parents. Thereby, collaborative practice is the preferred practice for SLTs and parents. In this paper, we explore collaborative practice and use a realist evaluation approach to achieve the aim of setting a research agenda in this area. Researchers use realist evaluation, a methodology originally developed by Pawson and Tilley in the 1990s, to explore the causal link between interventions and outcomes, summarized as what works, how, for whom, in what circumstances and to what extent. Realist evaluation provides a framework to explore configurations between contexts (C), mechanisms (M) and outcomes (O). We used this methodology to take a first step at making explicit what collaborative practice is and how it might be achieved in different contexts. We did this by drafting a preliminary programme theory about collaborative practice, where we made explicit what context factors and mechanisms might influence outcomes in collaborative practice between parents and SLTs. Based on this programme theory, we argue for the need to develop a research agenda on collaborative practice with parents of children with speech, language and communication needs. The steps between a programme theory and a research agenda could entail exploring each CMO, or step in the programme theory, and evaluating it against the existing literature—both within and beyond speech and language therapy—to see how far it stands up. In this way, gaps could be identified that could be converted into research questions that would stimulate debate about a research agenda on collaborative practice. Understanding how collaborative practice can be achieved in different contexts could support SLTs to use mechanisms to optimise collaborative practice intentionally and tailor interventions to the specific needs of families, thereby enhancing collaborative practice between parents and SLTs.

Intentional rounding in hospital wards to improve regular interaction and engagement between nurses and patients: a realist evaluation

© Crown copyright 2013. Contains public sector information licensed under the Open Government Licence v3.0© Queen’s Printer and Controller of HMSO 2019. This work was produced by Harriset al.under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.20132019Crown copyrightQueen’s Printer and Controller of HMSOBackgroundThe government response to the care failures at the Mid Staffordshire NHS Foundation Trust led to the policy imperative of ‘regular interaction and engagement between nurses and patients’ (Francis R.Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry. London: The Stationery Office; 2013. © Crown copyright 2013. Contains public sector information licensed under the Open Government Licence v3.0) in the NHS. The pressure on nursing to act resulted in the introduction of the US model, known as ‘intentional rounding’, into nursing practice. This is a timed, planned intervention that sets out to address fundamental elements of nursing care by means of a regular bedside ward round.ObjectivesThe study aims were to examine what it is about intentional rounding in hospital wards that works, for whom and in what circumstances.DesignA multimethod study design was undertaken using realist evaluation methodology. The study was conducted in four phases: (1) theory development; (2) a national survey of all NHS acute trusts in England; (3) in-depth case studies of six wards, involving individual interviews, observations, retrieval of routinely collected ward outcome data and analysis of costs; and (4) synthesis of the study findings.SettingThe study was conducted in acute NHS trusts in England.ParticipantsA total of 108 acute NHS trusts participated in the survey. Seventeen senior managers, 33 front-line nurses, 28 non-nursing professionals, 34 patients and 28 carers participated in individual interviews. Thirty-nine members of nursing staff were shadowed during their delivery of intentional rounding and the direct care received by 28 patients was observed.Review methodsA realist synthesis was undertaken to identify eight context–mechanism–outcome configurations, which were tested and refined using evidence collected in subsequent research phases.ResultsThe national survey showed that 97% of NHS trusts had implemented intentional rounding in some way. Data synthesis from survey, observation and interview findings showed that only two of the original eight mechanisms were partially activated (consistency and comprehensiveness, and accountability). The evidence for two mechanisms was inconclusive (visibility of nurses and anticipation); there was minimal evidence for one mechanism (multidisciplinary teamwork and communication) and no evidence for the remaining three (allocated time to care, nurse–patient relationships and communication, and patient empowerment). A total of 240 intentional rounds were observed within 188 hours of care delivery observation. Although 86% of all intentional rounding interactions were observed to be documented, fidelity to the original intervention [i.e. the Studer Group protocol (Studer Group.Best Practices: Sacred Heart Hospital, Pensacola, Florida. Hourly Rounding Supplement. Gulf Breeze, FL: Studer Group; 2007)] was generally low.LimitationsIntentional rounding was often difficult for researchers to observe, as it was rarely delivered as a discrete activity but instead undertaken alongside other nursing activities. Furthermore, a lack of findings about the influence of intentional rounding on patient outcomes in the safety thermometer data limits inferences on how mechanisms link to clinical outcomes for patients.ConclusionsThe evidence from this study demonstrates that the effectiveness of intentional rounding, as currently implemented and adapted in England, is very weak and falls short of the theoretically informed mechanisms. There was ambivalence and concern expressed that intentional rounding oversimplifies nursing, privileges a transactional and prescriptive approach over relational nursing care, and prioritises accountability and risk management above individual responsive care.Future workIt is suggested that the insights and messages from this study inform a national conversation about whether or not intentional rounding is the optimum intervention to support the delivery of fundamental nursing care to patients, or if the time is right to shape alternative solutions.FundingThe National Institute for Health Research Health Services and Delivery Research programme.

Ameliorating Workplace Harassment among Direct Caregivers in Canada’s Healthcare System: A Theatre-Based Intervention

The article reports on a theatre-based intervention designed to address workplace harassment among direct caregivers in Canada. The study is part of a larger analytical project that relies on labour process theory and critical realist evaluation methodology to understand what interventions work, how, for whom, and under what circumstances. Using Theatre of the Oppressed techniques, the reported intervention addresses workplace harassment by challenging the normative codes governing social interactions in participants’ workplaces. The study’s analysis indicates that the intervention’s Theatre of the Oppressed activities energized the participating caregivers to imagine, enact and collectively assess new social interactions. The caregivers developed strategies to resist the oppressive relations of their employment and became competent contesters of dominant discourses circulating in their workplaces. The solidarity developed through the bodily sculptures and enacted scenarios elicited participants’ deliberative exchange about workplace harassment and awakened a collective will to carry their revelations back to their workplaces.

A realist evaluation of student use of a virtual reality smartphone application in undergraduate legal education

AbstractThis paper provides a realist evaluation of the Open Justice virtual reality (VR) smartphone app, designed to develop presentation skills for students on an undergraduate legal education module. This work addresses two research questions: what proportion of students engaged with the Open Justice app, and what worked for whom in what circumstances and why? Questionnaire and interviews were conducted and analysed using a realist evaluation approach. This approach is particularly pertinent to the evaluation of how emerging educational technologies are used, as it can point to the potential affordances of a technological intervention, in addition to assessing its current use. In keeping with the realist evaluation approach, data analysis was completed using the context–mechanism–outcome framework. The findings suggest that more than half of students did not engage with the app during their studies, but this was not surprising as it was a pilot project into using VR with our students and such innovations can be difficult to manage in distance education contexts. Those who did engage with it recognised the potential of immersive VR to contribute to legal skills development, but they found the application of the technology, in this context, to be of limited use. By utilising a realist evaluation framework, this study contributes to the emerging field of programme theory‐based evaluations of educational technology, which might form the basis of further research. Practitioner NotesWhat is already known about this topic Smartphone‐based virtual reality technology has developed exponentially and is now widely and cheaply available. Smartphone‐based virtual reality technology has the potential to develop professional skills, and thereby provide a useful addition to experiential learning pedagogies. This is of particular relevance to the development of professional legal skills in the context of clinical legal education. Realist evaluation methodology is increasingly common in health literature to evaluate the impact of education programmes. What this paper adds This is the first empirical study of the use of VR technology to develop professional skills in a distance learning setting. This is the first paper to use realist methodology to evaluate virtual reality technology in legal education. It provides a granular analysis of particular contexts and user responses that can facilitate or inhibit learner engagement with VR technology. Implications for practice and/or policy It provides an analysis of VR used to support teaching and learning, where the students are distance learners—this highlights issues and challenges to be overcome. It provides an evidence base to inform the development of, and further research into, smartphone‐based virtual reality for legal education settings. It provides a detailed example of how realist methodology can be utilised to evaluate educational technology in real‐world settings.

Pupils attending a shared placement between a school and alternative provision: Is a sense of school belonging the key to success?

AimThe use of alternative provision for pupils identified as having challenging behaviour and social, emotional and mental health difficulties is widespread across the UK and the majority of these young people do not access a school whilst attending alternative provision. However, many pupils receive provision through a shared placement between a mainstream school and an alternative provider, which means they actively attend both settings on a regular basis. Although shared placements occur frequently across the UK, there has been very little research focusing on this area. This study explored the perceptions and experiences of staff and pupils in relation to shared placements, with a particular aim of analysing the nature and importance of sense of belonging.MethodologyAdopting a realistic evaluation methodology (Pawson &amp; Tilley, 1997), semi-structured interviews were used to identify the outcomes of the shared placement arrangement as well as the important contextual conditions linked to sense of belonging. In addition, each pupil completed the Psychological Sense of School Membership Scale (Goodenow, 1993) to examine their sense of belonging to educational provision.FindingsThe findings suggest that for some pupils, shared placements led to greater engagement with mainstream education and improvements in behaviour, whilst for others, a shared placement led to further disengagement from the mainstream education system. Sense of school belonging was highlighted as a strong predictor of positive outcomes and this was linked to various school attitudes and practices.LimitationsThis study has limitations in that it is relatively small-scale and additional research would be useful to confirm and extend these findings.ConclusionA strong sense of school belonging is a good predictor of positive outcomes for children receiving education through a shared placement. As sense of belonging appears to be closely linked to school attitudes and practices, supporting schools to promote a sense of belonging is likely to be useful for this population.

Indigenous patient experiences of returning to country: a qualitative evaluation on the Country Health SA Dialysis bus

BackgroundRates of End-Stage Kidney Disease among Aboriginal and Torres Strait Islander (Indigenous) Australians in remote areas are disproportionately high; however, haemodialysis is not currently offered in most remote areas. People must therefore leave their ‘Country’ (with its traditions and supports) and relocate to metropolitan or regional centres, disrupting their kinship and the cultural ties that are important for their wellbeing. The South Australian Mobile Dialysis Truck is a service which visits remote communities for one to two week periods; allowing patients to have dialysis on ‘Country’, reuniting them with their friends and family, and providing a chance to take part in cultural activities. The aims of the study were to qualitatively evaluate the South Australian Mobile Dialysis Truck program, its impact on the health and wellbeing of Indigenous dialysis patients, and the facilitators and barriers to using the service.MethodsFace to face semi-structured interviews were conducted with 15 Indigenous dialysis patients and 10 nurses who had attended trips across nine dialysis units. Realist evaluation methodology and thematic analysis established patient and nursing experiences with the Mobile Dialysis Truck.ResultsThe consequences of leaving Country included grief and loss. Barriers to trip attendance included lower trip frequencies, ineffective trip advertisement, lack of appropriate or unavailable accommodation for staff and patients and poor patient health. Benefits of the service included the ability to fulfil cultural commitments, minimisation of medical retrievals from patients missing dialysis to return to remote areas, improved trust and relationships between patients and staff, and improved patient quality of life. The bus also provided a valuable cultural learning opportunity for staff. Facilitators to successful trips included support staff, clinical back-up and a co-ordinator role.ConclusionsThe Mobile Dialysis Truck was found to improve the social and emotional wellbeing of Indigenous patients who have had to relocate for dialysis, and build positive relationships and trust between metropolitan nurses and remote patients. The trust fostered improved engagement with associated health services. It also provided valuable cultural learning opportunities for nursing staff. This format of health service may improve cultural competencies with nursing staff who provide regular care for Indigenous patients.

Empowering communities: drawing on evidence to build successful community engagement initiatives

Background: Although community engagement is seen as key to achieving citizen-centred and collaborative healthcare systems, organisations and neighbourhood networks are still searching for effective ways to engage citizens in shaping the health and care landscape. Many Dutch regions are currently implementing and fostering community engagement initiatives; however, little is known about which engagement approaches are effective under which circumstances, and how to implement such initiatives successfully. This holds especially true for initiatives focusing on disadvantaged groups. The National Institute for Public Health and the Environment and the University of Tilburg, in co-design with stakeholders of six Dutch community engagement initiatives, are conducting a mixed-methods multiple case study evaluating six community engagement initiatives. The study is following the realist evaluation methodology to investigate which community engagement initiatives are effective in improving communities' health outcomes and the sustainability of healthcare systems. The University of Maastricht is investigating the dynamics between health and citizen participation on a neighbourhood level. As barriers and possibilities for participation are supported by material and spatial components of the environment and constructed in the lack of daily contacts between neighbours and professionals, the study conducts an in-depth ethnography of the life of inhabitants of three low-income neighbourhoods in Maastricht. Local experiences of non-participation are gathered and related to the social and material conditions of the neighbourhoods. Aims and objectives: The workshop's main objectives are to present two distinctive investigations into community engagement and to provide professionals with the analytical tools and the evidence-base to build their own successful initiatives that can help improve communities' health and wellbeing. Format: During the session, we will briefly present our investigations, focusing on our preliminary results. In an interactive group session, we will examine the common lessons learned, including the often divergent ideas and motivations that organisations and communities have regarding community engagement. Discussions about the key enablers and barriers for community engagement initiatives will inspire participants to reflect upon and implement their own initiatives. Target audience: Our target audience includes participants involved or interested in developing and implementing community engagement initiatives, in co-design with communities themselves. Participants will share their own thoughts and experiences during the session. Learnings/take away: Participants will learn that community engagement is expected to bring a wider range of services together that reflect citizens' more holistic views and needs. However, barriers including service fragmentation which hampers professionals' willingness to take on board citizens' more holistic views, will need to be addressed before this can be achieved. Participants will gain insight into the key enablers for overcoming these barriers, like addressing power imbalances between professionals and citizens and within communities. Participants will also learn of the supportive conditions for community engagement such as informal contacts and the availability of publicmeeting places. This will enable them to overcome barriers and build a vision for community engagement that is shared between professionals and citizens.

Improving Hospital Discharge Arrangements for People who are Homeless: The Role of Specialist Integrated Care

Background: In England, 70% of people who were homeless on admission to hospital were discharged back to the street without having their care and support needs addressed (Department of Health, 2013). Furthermore they experienced high hospital readmission rates and often resorted to inappropriate use of the Emergency Department. Following these data, Government funding was provided to enable hospitals to work with local partners to develop specialist integrated homeless hospital discharge schemes, including intermediate or step-up/step-down care. In this paper we report preliminary findings from a realist evaluation which explored the effectiveness of the different schemes established. Schemes took many different formats, and tended to be either uniprofessional, comprising housing workers, or multi-professional, comprising: GPs; occupational therapists; nurses; social workers; housing workers; and peer navigators. Methods: The study was carried out in England between September 2015 and February 2018. It was funded by the National Institute of Health Research (NIHR). It employs a realist evaluation methodology, comparing sites with specialist integrated care (n=4) to those with standard care (n=2). Realist evaluation is designed to explore the relationship between the local context, the mechanisms and resources deployed and the outcomes. The overall aim is to generate hypotheses about ‘what works for whom, in what circumstances and why’. Across the six study sites in-depth observational fieldwork was undertaken. This encompassed interviews with 60 practitioners and stakeholders, and 60 people who were homeless on admission to hospital. People who were homeless were interviewed shortly after discharge and again three months later. An economic evaluation and ‘data linkage’ (across a total of 20 sites) were also undertaken to interrogate the emerging realist hypotheses. Preliminary Results: Our emerging hypothesis is that there is no ‘magic bullet’ or single most effective solution. Findings so far indicate that improving hospital discharge arrangements for people who are homeless requires action on many different fronts to weave together a range of potentially effective mechanisms and resources. To illustrate this, we will present examples of different local configurations evolving over time, in rural and urban contexts, mapping the advantages and disadvantages of each. We will pinpoint a number of key attributes of effective systems (e.g. clear protocols for the discharge of people who are homeless, discharge co-ordination and intermediate care) and the factors that may explain why these are necessary and workable in some but not all locations. Throughout, we will tease out the concept of ‘specialist integrated care’ and whether this is indicative of improved quality or further fragmentation. Limitations and future research: The limitations of the research are that it is focused on England. We hope that during the conference we can engage with international colleagues with a view to scoping ideas for future comparative research in the field of integrated care transitions. Reference: 1. Department of Health. Homeless Hospital Discharge Fund. London: Department of Health. 2013

‘Improvement focused’ evaluation of place-based initiatives: An examination of three methodologies

Place-based initiatives offer a comprehensive, whole-of-community approach to solving complex problems. Impact evaluation of complex initiatives is challenging and alternative ‘improvement focused’ methodologies, such as developmental evaluation, action research and quality improvement, are being used. Limited understanding exists about how these methodologies work when used in place-based initiatives, which contexts they are individually best suited to and what they can achieve. This article examines the methodologies of developmental evaluation, action research and quality improvement when applied to the evaluation of place-based initiatives. The approach used a realist evaluation methodology, involving a 10-year literature review and three ‘instrumental’ case studies, which is described in detail in an accompanying paper (Heery, Naccarella &amp; McKenzie, 2018). Contextual factors, mechanisms and outcomes for the application of developmental evaluation, action research and quality improvement to place-based evaluation were identified so as to build a theory for each methodology. The three methodologies have similar mechanisms; they are cyclical and comprise planning, doing, studying and acting, and all result in continuous improvement of the initiatives, increasing collaboration and increasing evaluation capacity. A key difference between the methodologies is their perceived purpose. These findings can support practitioners, commissioners and users of evaluation working in the place-based arena in three ways: advocating for the adoption of ‘improvement focused’ evaluation; selecting the most appropriate ‘improvement focused’ methodology; and identifying and addressing the facilitating factors particular to the selected methodology to increase the effectiveness of the evaluation.

Realist evaluation of allied health management in Queensland: what works, in which contexts and why.

Objective Allied health structures and leadership positions vary throughout Australia and New Zealand in their design and implementation. It is not clear which organisational factors support allied health leaders and professionals to enhance clinical outcomes. The aim of this project was to identify key organisational contexts and corresponding mechanisms that influenced effective outcomes for allied health professionals. Methods A qualitative realist evaluation was chosen to describe key aspects of allied health organisational structures, identify positive outcomes and describe how context and processes are operationalised to influence outcomes for the allied health workforce and the populations they serve. Results A purposive sample of nine allied health leaders, five executives and 49 allied health professionals were interviewed individually and in focus groups, representing nine Queensland Health services. Marked differences exist in the title and focus of senior allied health leaders' roles. The use of a qualitative realist evaluation methodology enabled identification of the mechanisms that work to achieve effective and efficient outcomes, within specific contexts. Conclusions The initial middle range theory of allied health organisational structures in Queensland was supported and extended to better understand which contexts were important and which key mechanisms were activated to achieve effective outcomes. Executive allied health leadership roles enable allied health leaders to use their influence in organisational planning and decision-making to ensure allied health professionals deliver successful patient care services. Professional governance systems embed the management and support of the clinical workforce most efficiently within professional disciplines. With consistent data management systems, allied health professional staff can be integrated within clinical teams that provide high-quality care. Interprofessional learning opportunities can enhance collaborative teamwork and, when allied health professionals are supported to understand and use research, they can deliver positive patient and business outcomes for the health service. What is known about the topic? A collective allied health organisational structure encourages engagement of allied health professionals within healthcare organisations. Organisational structures commonly include management and leadership strategies and service delivery models. Allied health leaders in Queensland work across a range of senior management levels to ensure adequate resources for sufficient suitably skilled professional staff to meet patient needs. What does this paper add? Literature to date has described how allied health professionals operate within organisational structures. This paper examines key aspects of allied health management, governance and leadership, together with mechanisms that support allied health professionals to deliver effective clinical and business outcomes for their local community. What are the implications for practitioners? Health service executives and allied health leaders should consider supporting executive allied health leadership roles to influence strategic planning and decision-making, as well as to deliver outcomes that are important to the health service. When allied health leaders implement integrated professional and operational governance systems, executives described allied health professionals as influential in supporting team-based models of care that add value to the business and improve outcomes for patients. When allied health leaders use consistent data management, executives reinforced the benefit of aligning activity data with financial costs to monitor, recognise and reimburse appropriate clinical interventions for patients. When allied health leaders support allied health workforce capability through educational and research opportunities, clinicians can use research to inform their clinical practice.

“Improvement focused” evaluation of place-based initiatives: An approach to examining three methodologies

Place-based initiatives offer a comprehensive, whole-of-community approach to solving complex problems. Impact evaluation of complex initiatives is challenging and alternative “improvement focused” methodologies, such as developmental evaluation, action research, and quality improvement, are being used. Limited understanding exists about how these methodologies work when used in place-based initiatives, which contexts they are individually best suited to, and what they can achieve. This article outlines an approach to examining the application of developmental evaluation, action research, and quality improvement to the evaluation of place-based initiatives. The approach used a realist evaluation methodology and involved a literature review and three “instrumental” case studies. The approach described in this article can support others to design and undertake their own research to inform decision making around the most suitable methodologies in the emerging field of place-based initiative evaluation.