Developmental Evaluation Approach Research Articles

Improving COVID-19 Vaccine Uptake in Saskatchewan, Canada: A Developmental Evaluation Approach

Background The Developmental Evaluation of a COVID-19 vaccination program was an early response to assess a complex emergent mass vaccination program to support learning and adaptation. Objective The primary objective of a multi-disciplinary team of researcher-evaluators was to facilitate organizational learning among key stakeholders to improve decision-making and increase vaccine uptake in Saskatchewan, Canada. Methods Aligned with the Developmental Evaluation approach, data collection was rooted in adjustment and flexibility to meet the evolving needs of the vaccination program. Data were primarily collected using meeting observations and program documentation. As the program progressed, the data collection was adjusted, and two surveys were conducted targeting COVID-19 vaccine recipients and vaccine immunizers. Data were analyzed iteratively in consultation with stakeholders. Results Nine feedback reports were generated over a nine-month evaluation period. Seven reports highlighted meeting observation results that revealed the program issues, probable causes, and implications. The evolving issues ranged from vaccine shortage, delay, and supply fluctuation to inter-organizational miscommunication and vaccine hesitancy. Two reports were produced from survey findings to delve into the persistent issue of vaccine hesitancy. Conclusion Effective solutions to complex issues of Saskatchewan’s COVID-19 mass immunization require a systems approach based on new ways of thinking and collective decision-making.

Codesigning enhanced models of care for Northern Australian Aboriginal and Torres Strait Islander youth with type 2 diabetes: study protocol

IntroductionPremature onset of type 2 diabetes and excess mortality are critical issues internationally, particularly in Indigenous populations. There is an urgent need for developmentally appropriate and culturally safe models of...

Improving Children's Rehabilitation Service Quality and Experience Through Innovative Programming: A Developmental Evaluation of the Bright Ideas Program.

Evidence-informed practice (EIP) is a widely accepted approach to providing rehabilitation services, however there are barriers to implementation. Bright Ideas is an innovative program developed by KidsAbility's Rocket Discovery Centre, aiming enhance uptake of EIP through incubating ideas identified by KidsAbility staff linked to organizational priorities of enhancing service access and outcomes. To evaluate Bright Ideas by identifying perceived successes and challenges associated with the program with the aim of supporting clinical innovation and the adoption of evidence-informed practices in children's rehabilitation centres. A developmental evaluation approach was utilized to complete a real-time evaluation of Bright Ideas. Bright Ideas program materials were reviewed to describe the program. A focus group and interview were conducted with staff who led Bright Ideas projects (n = 7) to gain insight into their experiences. Additionally, interviews were conducted with the program coordinators (n = 2) to understand the evolution of the program. Data was analysed using qualitative content analysis. Successes that drive the Bright Ideas program and roadblocks it has encountered were identified across three dimensions: the personal level (involving Bright Ideas project leads), the programmatic level (focusing on the Bright Ideas program), and the organizational level (pertaining to KidsAbility as a whole). Key roadblocks included a lack of knowledge about the program organizationally, restricted time resources, competing organizational priorities, and an absence of infrastructure to scale ideas from Bright Ideas into the organization. Bright Ideas is an innovative program aiming to make impactful change by enabling staff to lead implementation-driven projects. Our findings highlight many successes of the program and use the Theoretical Domains Framework and Behaviour Change Wheel to make recommendations to support its sustained implementation at KidsAbility.

User experiences of an AI application for predicting risk of sexually transmitted infections.

Awareness of one's individual risk of sexually transmitted infections (STIs) and human immunodeficiency virus (HIV) is a necessary precursor to engagement with prevention strategies and sexual health care. Web-based sexual health applications may improve engagement in sexual health prevention and care by providing individualised and evidence-based sexual health information. The STARTOnline (Supporting Timely and Appropriate Review and Treatment Online) study sought the views of sexual health service users on three web-based sexual health applications to better understand their usefulness, acceptability and accessibility. This paper reports the views and experiences of users of one of the applications called MySTIRisk. MySTIRisk estimates the risk of three common STIs and HIV using data from attendees of a metropolitan sexual health service. This study used a bespoke qualitative design, informed by a developmental evaluation approach. Melbourne Sexual Health Centre clinic attendees' views were sought using semi-structured interviews conducted between October 2023 and January 2024 via videoconferencing, telephone and on site at the clinic. Data was analysed using qualitative data analysis methods. A diverse group of 14 participants described an ideal pathway to better sexual health outcomes that might be facilitated by use of the MySTIRisk application, particularly for individuals with limited sexual health knowledge, or affected by stigma and geographical barriers. This pathway was described as: 1) being concerned about my sexual health; 2) checking my STI risk easily and privately; 3) understanding and trusting the result; and 4) deciding how to look after my health. Factors that might influence this pathway were also described, including areas for improvement in accessibility and acceptability. These findings support the role of web-based sexual health applications in facilitating access to sexual health education and behavioural change and underscore the importance of codesign approaches in improving their uptake and impact.

Developing and evaluating Compassionate Workplace Programs to promote health and wellbeing around serious illness, dying and loss in the workplace (EU-CoWork): a transdisciplinary, cross-national research project

Background: Most employees will experience serious illness, caregiving, dying and loss (End-of-Life (EoL) experiences) at multiple points throughout their working lives. These experiences impact affected employees but also their colleagues in terms of health and wellbeing, and the workplace as a whole in terms of workplace safety, productivity and labour relations. The impact of EoL experiences on employees means that workplaces are called to play a more active role in providing support for EoL experiences. Aim: To describe how the EU-CoWork (2024–2028) project addresses its main aims to (1) create Compassionate Workplace cultures, practices and policies and improve health and wellbeing for employees dealing with EoL experiences in different national work contexts in Europe; (2) describe and evaluate the process of co-creation and implementation of Compassionate Workplace Programs (CWPs) and how these influence the programs’ outcomes. Design: EU-CoWork employs a facilitated and co-creative Developmental Evaluation approach to the development of 12 tailored CWPs across four European countries (Belgium, Austria, Sweden and Greece). Methods: To evaluate the outcomes and processes leading to these outcomes, a mixed-methods Realist Evaluation methodology is applied, formulating and testing Context-Mechanism-Outcomes configurations and combining longitudinal quantitative and qualitative data collections. Results: EU-CoWork will generate evidence to support an expanded model of occupational health and safety risk factors sensitive to the specific challenges related to employees’ EoL experiences. In doing so, several challenges will have to be navigated: involving employees with EoL experiences while avoiding overburdening them, avoiding tokenistic engagement, managing power differentials, balancing the need for scientific rigour with the flexibility required in co-creation, reconciling different epistemologies and disciplinary traditions and organisational resistance to change. Conclusion: There are potential long-lasting broader societal impacts through the stimulation of open discourse on EoL topics, the reconciliation of work and care, and changes in gendered work and care patterns.

Adapting a men’s behaviour change program to online delivery using a developmental evaluation approach

Men’s behaviour change programs constitute an important family violence intervention strategy. In 2020, the COVID-19 pandemic and associated restrictions brought increased risk of family violence and service demand. When face-to-face groups became impossible under restrictive public health orders, the Men’s Behaviour Change Program provided by Relationships Australia Victoria was delivered using videoconferencing technology. We describe a developmental evaluation approach to the adaptation of the Men’s Behaviour Change Program to online delivery. Developmental evaluation can inform innovation in complex environments and emphasises learning and adaptation at program development phase. We demonstrate the application and utility of this approach to the rapid adaptation of social services for online delivery. Quantitative and qualitative data were collected from participants, facilitators and external observers and subjected to regular collaborative review to inform practice development. The results point to a steep learning curve, during which facilitators and observers identified practices to support effective online delivery of the Program. The evaluation indicated that, following adaptations, Program objectives were achievable using this method of delivery and positive dynamics remained possible in online groups. This finding is consistent with the current No to Violence position that online men’s behaviour change programs have an important and ongoing role in the specialist family violence services sector.

EdTech in humanitarian contexts: whose evidence base?

This study reports on the design and development of a methodological toolbox prototype for evaluating EdTech deployed in the contexts of fragility and crisis. The project adopted a bottom-up approach: training EdTech users in participatory action research approaches was followed by a comprehensive mapping of problems in the Azraq refugee camp that might be addressed through the chosen EdTech installed in a local Makerspace. Students as researchers used a developmental evaluation approach to deepen their understanding of evaluation as a concept and as a process and proceeded to match the results of their Azraq camp problem-tree analysis with evaluation questions related to the EdTech tools available in the Makerspace. The study concludes with a proposed methodological toolbox prototype, a set of approaches and processes that include research capacity building in fragile contexts, and user-led evaluation that emphasizes the notion of evaluation as a learning process driven by those designed to benefit from EdTech in fragile contexts.

From the nexus vision to the NexusIPE™ learning model

ABSTRACT The Nexus vision of simultaneously transforming health professions education and healthcare delivery to achieve Triple (now Quadruple) Aim outcomes was first articulated in the 2012 proposal and funding of the National Center for Interprofessional Practice and Education (National Center). Over the past decade, the National Center has worked with over 70 sites implementing large scale and practice-based interprofessional practice and education (IPE) programs. Because what is needed to implement the Nexus to achieve Quadruple Aim outcomes was not well understood in 2012, the National Center took a social innovations and developmental evaluation approach. This iterative method led to the development of the National Center NexusIPE™ Learning Model that adapts the 3-P high-level stages (Presage, Process, and Product), proposed as a framework for IPE by Barr and colleagues. National Center collaborators’ lessons learned about the Nexus vision are highlighted in this issue and provide real-world examples of elements of the NexusIPETM Learning Model. Reflecting on ten years of experience, the National Center leaders recognize the need for Nexus transformation and the relevance of the NexusIPETM Learning Model today as education and health systems grapple with mounting workforce challenges. The model provides opportunities to address growing workforce shortages, provide equitable care that leads to health, and support the well-being of practice teams in the face of challenges such as the COVID-19 pandemic

136 Evaluating the ten-year progression of the Miami CTSI Mentored Translational Research Scholars (KL2) Program Awards

OBJECTIVES/GOALS: The goal of Miami CTSI’s Mentored Translational Research Scholars (KL2) Program is to identify outstanding early-stage investigators and prepare them to become the next generation of highly skilled independent researchers who translate fundamental knowledge and new technologies from the laboratory to the clinic and to the community. METHODS/STUDY POPULATION: Evaluation of the KL2 program focused on assessing progress in the following areas: 1) scholar productivity during and after graduating from the program; 2) continued engagement of KL2 program graduates in clinical and translational research; scholar productivity during and after graduating from the program; 3) outcomes such as establishing independent research careers, obtaining research funding, and establishing collaborations; and 4) research impact. We used a developmental evaluation approach and benefits-framework model to conduct program evaluation, incorporating both in a program-specific logic model. Bi-annual surveys were used to measure scholar feedback and progress, and utilization of CTSI services. RESULTS/ANTICIPATED RESULTS: Since 2013, the KL2 program has trained 21 scholars,17 of whom have completed training, and four are still in the program. Of the 21 scholars, 52% are female and 38% are underrepresented minorities (URM)–significantly higher than the 12% URM for overall CTSA KL2 scholars. Scholars have 176 total publications related to their KL2 projects and have received $52.4m in total research funding. In addition, 50% of the projects focused on health disparities. Overall, 48% of KL2 scholars have received large subsequent federal awards including three NIH K awards and six NIH R awards. Scholars actively utilized CTSI services during and after graduation. Programmatic enhancements such as adding institutional scholars and using scholar feedback to improve program resources were also implemented. DISCUSSION/SIGNIFICANCE: Miami CTSI’s KL2 program has demonstrated success in fulfilling its program goals. The process and outcome evaluation has provided a better understanding of program performance and progress and has demonstrated alignment with CTSI’s overall goals on addressing health disparities and its commitment to diversity and equity.

Creating opportunities for telehealth education, assessment, and care through Hotspotting (CO-TEACH): An interprofessional telehealth delivery curriculum and guide

Telehealth offers a healthcare solution when functional, social, or geographic barriers arise; however, many clinical education curricula are ill-equipped to prepare learners for effective, multidisciplinary telehealth consultations. The COVID-19 pandemic drastically expanded reliance on telehealth in clinical practice and highlighted the urgent need to educate students across health professions in telehealth delivery. To address this educational gap, student and faculty collaborators from Duke University and the University of North Carolina at Chapel Hill utilized a developmental evaluation approach to rapidly design and pilot a compact, five-session, virtual curriculum to train students from a broad range of health professions in telehealth delivery. Through focused readings, large group didactics, and small group discussion, the curriculum provided a general overview of telehealth, addressed specific issues in health equity and access in virtual care both locally and nationally, introduced skills for conducting telehealth visits, and employed a patient simulation for students to practice their newly learned skills. The curriculum was delivered virtually over eight weeks to 36 students from ten health professions programs (occupational therapy, social work, pharmacy, audiology, speech-language pathology, dentistry, medicine, nursing, physical therapy, and physician assistants) across two universities. End-of-course feedback surveys identified that students most valued the opportunity to work with learners from different professions, practical instruction for telehealth, and a standardized patient experience. On post-course surveys students reported high self-efficacy in their telehealth competencies and indicated very positive attitudes toward interprofessional education. Overall, the compact nature of this curriculum facilitates adaptability to other student-led groups.

Bridging the Gap Between Transformative Practices and Traditional Public Health Approaches.

The release of the 24/7 Wall Street publication in November 2018 labeling the Waterloo-Cedar Falls, Iowa, community as the worst city for Black Americans to live in the United States sparked conversations and calls to action across the community-from the kitchen table to the boardroom. It became clear that this was a critical moment for innovative leadership from public health. To bring together these individual efforts, and diverse perspectives, toward deeper understanding and system transformation, Black Hawk County Public Health, with support from Engaging Inquiry, adopted a participatory action approach of dynamic system mapping and systemic strategy design. Using participatory methodologies and tools for systems analysis, stakeholders developed a system map visualizing the patterns driving inequitable outcomes, but also the bright spots and resiliencies, identified and experienced across the community. This map, populated with local stories and data, represents a "theory of context," and offered a foundation of understanding and connectivity upon which high-impact opportunities for engagement could be identified and developed. By working across sectors and building on existing energy and resources present within the community, systemic strategies were designed to foster critical shifts toward a healthier community. A developmental evaluation approach is applied throughout the mapping process to maximize rapid learning and adaption of a complex challenge when stakeholder engagement and trust building is essential. Participatory systems mapping shows promise as an adaptive approach to allow public health departments to go beyond the traditional roles of public health practice, to collaborate with nontraditional partners, and implement strategies, initiatives, and/or policies that explicitly address inequity and the social determinants of health. Adaptive strategies provide opportunities for the local public health system to identify community health improvement strategies that align with Public Health 3.0.

Creative processes in co-designing a co-design hub: towards system change in health and social services in collaboration with structurally vulnerable populations

Background:Co-design is an approach to engaging stakeholders in health and social system change that is rapidly gaining traction, yet there are also questions about the extent to which there is meaningful engagement of structurally vulnerable communities and whether co-design leads to lasting system change. The McMaster University Co-Design Hub with Vulnerable Populations Hub (‘the Hub’) is a three-year interdisciplinary project with the goal of facilitating partnerships, advancing methods of co-design with vulnerable populations, and mobilising knowledge. Aims and objectives:A developmental evaluation approach inspired by experience-based co-design was used to co-produce a theory of change to understand how the co-design process could be used to creatively co-design a co-design hub with structurally vulnerable populations. Methods:Twelve community stakeholders with experience participating in a co-design project were invited to participate in two online visioning events to co-develop the goals, priorities, and objectives of the Hub. Qualitative data were analysed using a thematic content analysis approach. Findings:A theory of change framework was co-developed that outlines a future vision for the Hub and strategies to achieve this, and a visual graphic is presented. Discussion and conclusions:Through critical reflection on the work of the Hub, we focus on the co-creative methods that were applied when co-designing the Hub’s theory of change. Moreover, we illustrate how co-creative processes can be applied to embrace the complexity and vulnerability of all stakeholders and plan for system change with structurally vulnerable populations.

Documenting development of interdisciplinary collaboration among researchers by visualizing connections

AbstractIn 2015, a multidisciplinary group of academic researchers and extension personnel was awarded a US Department of Agriculture Coordinated Agricultural Project grant. The team’s goal was to understand human decision-making in the context of potential livestock disease outbreaks, investigating the impediments to increased adoption of biosecurity practices by producers from multiple perspectives. Counting publications and presentations was not deemed adequate evidence of growth of interdisciplinary collaboration. Instead, we took a developmental evaluation approach, collecting data to illuminate patterns of interconnectedness across disciplinary boundaries. These data, mapped annually on a baseline framework, reveal the team moving from disciplinary groups with disparate epistemologies and interests to a cohesive interdisciplinary team involved in many unexpected and emergent projects.

The Mitigating Toxic Stress study design: approaches to developmental evaluation of pediatric health care innovations addressing social determinants of health and toxic stress

BackgroundHealth care administrators must establish and promote effective partnerships with community agencies to address social determinants of health, including reducing exposure of infants and young children to chronic stress. Because infants’ experiences are inextricably tied to their caregivers, an important target for mitigating “toxic” stress exposure in early childhood is through reducing parents’ experiences of chronic stress in addition to protecting children from direct experiences of harm such as physical or sexual abuse. Conducting screening to identify when children are exposed to early life adversity is a first step; connecting families to needed support services is an essential component to addressing identified challenges. This paper presents the methodology of a three-year study of health care systems innovations designed to engage and support parents of infants to prevent and mitigate children’s toxic stress exposures.MethodsKey study features included: 1) multi-component study in five U.S. communities and nine pediatric health care clinics and the families they serve, 2) a developmental evaluation approach to describe how innovations are experienced over time at three levels—community systems, pediatric providers, and families, and 3) rapid cycle feedback conducted with communities, clinics and families to co-interpret data and findings. Data sources included: 1) focus groups and interviews with community stakeholders, clinic staff, and families, 2) electronic health record and Medicaid services data extracted to assess health care quality, utilization, and financial impact, and 3) clinic-recruitment of 908 parents of infants in a longitudinal survey. Results. The sample is briefly characterized based on responses to the enrollment phase of the parent survey.ConclusionsWe discuss the study design elements’ contribution to generating evidence needed by innovators, communities, and clinics to modify and sustain investments in these innovations to prevent or mitigate the effects of exposure to toxic stress on young children.

Use of Developmental Evaluation for Organizational Learning

AbstractIn this chapter, we draw on the data collected during our evaluations of six TAACCCT grants to describe the challenges, successes, and transformative effect of adopting a developmental evaluation approach for community college programs.

Abstract PO-041: A framework for transformed community-academic partnerships to reduce cancer disparities

Abstract To meaningfully impact cancer disparities, scientists and community need to work differently together. Community-based participatory research (CBPR) offers a strong first step but in practice often falls short of its full potential. The Community Cancer Science Network (CCSN) is an initiative of the Medical College of Wisconsin Cancer Center to address breast and lung cancer disparities statewide. This initiative began with a three-year study and design team process which identified a need for academic medicine and community to come together to address disparities in Wisconsin. A design team made up of basic and population science researchers, clinicians and community-based organization leaders representing communities impacted by disparities convened to gather data and examine trends to better understand root causes. The team identified several hurdles impeding authentic community-academic partnerships including: 1) isolated solutions putting the primary responsibility for change on individuals and communities; 2) significant issues of trust due to racism and other forms of oppression; 3) misunderstandings and misinformation about community priorities/needs and the intersections of science and quality of life; and 4) structures to support working together in authentic ways. Following this work, the initiative moved into the next phase to address these hurdles. In this phase, CCSN uses developmental evaluation to help shape new ways of working together to go beyond CBPR by explicitly addressing mistrust, shared understanding, power and privilege differentials and more. CCSN begins with creating change in academic medicine and places community members at the table as experts. We report on first year learning, including the development of our guiding framework, managing resistance to change, and putting in practice an anti- oppression, co-learning model to support innovation and engage community and researchers as equitable partners. Further, we discuss the challenges of using a developmental evaluation approach in an academic environment. These include: 1) the need for significant shifts within core leadership teams and funders to look inward at their commitment to change internally; 2) differences in beliefs and practice between evaluation and research around planning, credibility of evidence, and use of data and language; 3) multiple levels of leadership models and engagement structures starting at different places and with varying understandings and assumptions of the models and approaches we are trying to use; 4) moving forward within a set of much larger systems that are becoming ready and changing at different paces. Conclusion. Eradicating cancer disparities requires novel ways to facilitate authentic community and academic medicine collaborations. Developmental evaluation supports an iterative process to quickly identify challenges and successes promoting ongoing learning and the ability to pivot and adapt approaches. Citation Format: Laura Pinsoneault, Tobi Cawthra, Beth Brunner, Linda Conlon, Lyle Ignace, Judy Heubel, Sandra Millon-Underwood, Hallgeir Rui, Jose Salazar, Ellen Velie, Alonzo Walker, Melinda Stolley. A framework for transformed community-academic partnerships to reduce cancer disparities [abstract]. In: Proceedings of the AACR Virtual Conference: Thirteenth AACR Conference on the Science of Cancer Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; 2020 Oct 2-4. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2020;29(12 Suppl):Abstract nr PO-041.

Evaluation of Teacher Professional Development Program—A Developmental Evaluation Approach

The professional development of teachers is the main mechanism for improving classroom teaching and improving students’ learning outcomes. To have secondary school teachers improve their teaching by using the new teaching approach of project-based learning, this study engaged teachers to participate in Teachers’ Project-Based Learning Program (T-PjBLP) to learn new knowledge, pedagogy, and skills. During the program implementation, the study applied a developmental evaluation (DE) approach to evaluate teachers’ professional development and to assist them to develop an initiative program for students. In the DE approach, the role of the researcher was as a team member, shaped the course of development to help participants learn, and tracked the professional development process and results through a digital technology platform. The participants in this study consisted of 7 science teachers coming from two secondary schools. The major findings indicated that more than 95% of participants agreed that digital technology was a wonderful platform to provide them with an opportunity to share ideas, feedback, and experiences about what they learned from the program. Also, most of them provided positive feedback about their learning. To conclude, the program effectively changed teachers teaching beliefs from the teacher-centered approach to the learner-centered approach and used what they learned from the program to develop an initiative program for students. However, some limitations encountered in applying the DE approach to evaluate the program also were discussed in this study.

Using a developmental evaluation approach to create a supportive curriculum for first year students

Background and objectives: First year students experience a significant transition when entering nursing school. The purpose of this research was to explore first year nursing students’ experiences to enhance and innovate the undergraduate nursing program at a large public Canadian university.Methods: The Faculty of Nursing approached their curriculum redesign process utilizing a Developmental Evaluation (DE) framework. Nineteen first year students participated in semi structured interviews and focus groups where they discussed their personal experiences as well as the perceived strengths and weaknesses of the program. After thematic analysis of the data, recommendations were provided to the faculty administration to guide changes made to the new curriculum.Results: Students appreciated opportunities where they could apply their knowledge to real-world situations. Students also expressed many sources of stress, such as inconsistency within and between courses, differing expectations, content, instruction style, and evaluation. They also voiced that there was a lack of communication and support from the Faculty and identified issues with grading systems.Conclusions: The findings from this study highlighted the need to revise the nursing curriculum to provide more student support and foster a positive student-faculty relationship. The current structure of nursing programs has created competition among students, causing a greater focus on obtaining higher grades than on meaningful learning. Integrated learning with authentic experiences was best received by first year students and provided for a collaborative environment. Finally, the findings from this study highlight the opportunities created by utilizing a DE approach to evaluate and innovate nursing curricula.

Quality improvement at scale: evaluation of the drivers and barriers to adoption and sustainability of an intervention to reduce late referral in chronic kidney disease

This quality improvement project aimed to drive large scale and sustained change to reduce the burden of chronic kidney disease in the UK. The intervention is a software program that...

Assessing research impact: A framework and an evaluation of the Society of Family Planning Research Fund’s grantmaking (2007–2017)

The Society of Family Planning Research Fund (SFPRF) provides grants for research on abortion and contraception. In 2017, SFPRF conducted a retrospective evaluation of its investment in family planning research. Using a developmental evaluation approach, we created a framework for assessing research impact in family planning and applied it to an analysis of our grantmaking between 2007 and 2017. Our framework consists of 30 indicators of research impact, which span nine impact categories from building researchers’ capacity to influencing individuals, communities, and systems. Through application of this framework to our grantmaking, we learned that our grantmaking has helped build the research capacity of emerging and established family planning scholars and advance the field of family planning by supporting the creation of a robust scholarly evidence base. At the same time, we identified less evidence of impact on policy and practice. The results of this analysis directed SFPRF to move towards more focused funding opportunities, including longer-term and larger investments, and to prioritize partnerships between researchers and knowledge brokers.