Kirkpatrick's Training Evaluation Model Research Articles

Reimagining Faculty Development: A 5-Year Journey of Outcomes and Lessons Learned of a Physician Assistant Faculty Fellowship

Introduction Faculty sufficiency remains one of the major bottlenecks in health workforce development. There is a need for effective models for practicing clinicians to successfully transition to academic teaching roles. This article describes outcomes and lessons learned at the 5-year mark for a statewide faculty development workforce initiative for the physician assistant (PA) profession at the PA Leadership and Learning Academy (PALLA) in Maryland. Methods The PALLA PA faculty fellowship is a state-funded workforce initiative that consists of a 10-month training program that includes synchronous workshops, asynchronous learning modules, experiential teaching opportunities, and completion of an individual project in the realm of scholarship/service under longitudinal mentorship. Programmatic outcomes were longitudinally tracked using the Kirkpatrick model of training evaluation to evaluate the faculty training program at 4 levels of criteria: reaction (level 1), learning (level 2), behavior (level 3), and results (level 4). Key performance indicators for the fellowship include the ability for fellow graduates to provide educational support to Maryland PA programs, engage in, and promote scholarship in the PA profession and build an academic portfolio. Results To date, 4 cohorts of PAs (23 individuals) have completed the fellowship. Learner reaction (level 1) evaluated through surveys, and interviews reveal a positive fellowship experience, with fellows noting benefits of the longitudinal nature of the experience and individualized mentorship and fellows self-report high level of achievement of fellowship learning goals (level 2). Local and national PA review of fellow curricular and scholarly products has been positive (level 3). After graduation, 9 fellows are engaged in faculty roles, providing academic support to local PA programs; 11 fellows have had scholarly products accepted; all fellow graduates have generated an academic portfolio; and 2 fellows have provided new service to PA organizations (level 4). Discussion Four years of fellowship data demonstrate positive impact across Kirkpatrick 4 levels of training evaluation. Nearly, all participants have new contributions in teaching, scholarship, or service roles because of the fellowship. Key lessons learned were the benefits of the longitudinal nature of the experience for scaffolding of learning and ongoing, individualized mentorship.

Assessing the Impact of the Portfolio Workshop and the Use of the Rubric Matrix for Reflective Writing on Medical Faculty and Students.

Purpose The portfolio can be used as a crucial tool for self-reflection, which allows us not only to showcase achievements but also course correct on our personal and professional journey. However, there is a significant lack of awareness among medical professionals about portfolios.Arranging a workshop to impart this knowledge could be a potential mitigation approach. This study aims to assess the impact of workshops on portfolios on students' and faculty's knowledge. In addition, the study also analyzes the effect of using rubrics on reflective writing skills. Method A portfolio workshop was organized for the medical faculty and students in the Bundelkhand Government Medical College, Sagar, M.P. The Kirkpatrick model of training evaluation along with a rubric for the evaluation of reflective writing skills were used to measure the effectiveness of the workshop. Pre and post-tests for the workshop, pre and post-reflective writing skills, and workshop feedback were collected using questionnaires. The Shapiro-Wilk test and the Wilcoxon signed rank test were applied to the data collected. Results Out of 89 registrations for the workshop, only 81 people consented to the workshop and participated in the study. The total number of faculty was only 17 and the rest were students from all the phases. Both the Shapiro-Wilk test and the Wilcoxon signed rank test showed a significantly small p-value, stating that there was a significant positive impact on the knowledge, perception, and effectiveness of the workshop. Conclusion This study clearly outlines the positive impact of conducting a workshop on portfolios. A significant increase in participants' knowledge of portfolios is identified.Similarly, employing rubrics has a significant increase in the quality of reflective writing skills.

429 Sub-internship Academy: Implementation and Evaluation of Single-Day Resident-Driven Focused Sub-internship Preparation Course

INTRODUCTION: Strong performance in neurosurgical sub-internships requires familiarity with clinical and technical skills which may not otherwise be covered in standard medical school curricula. Thus, a need exists for immersive and comprehensive sub-internship preparation programs, which should be optimized based on student feedback. METHODS: Academy curriculum involved hands-on and discussion-based elements split into three stations: (1) suturing and general information, including a suturing and knot tying workshop and discussion of expectations of sub-interns; (2) introduction to the operating room and drilling, covering patient positioning, surgical technology, OR etiquette, and craniotomy basics; and (3) cases and clinical skills, reviewing clinical imaging, focused neurological exams, and medical note writing. An anonymous online post-survey was administered to collect data on participants’ perceptions of the utility of the academy. RESULTS: Twelve medical students participated in the inaugural neurosurgery sub-internship academy. Nine responded to a post-survey, which revealed following ratings: the overall program was rated as having maximal impact on sub-internship readiness and enthusiasm for neurosurgery by eight (88.9%) and seven (77.8%) respondents, respectively, and trends were comparable for individual curriculum components. A largely positive impact on mentorship access was also observed. CONCLUSIONS: Student perceptions of the relevance and utility of the sub-internship academy were positive, and the program fostered enthusiasm for neurosurgery and opportunities for mentorship. Evaluation of the program’s actual impact on preparedness and sub-internship performance as well as student retention in the field of neurosurgery will be performed in the future, in accordance with the Kirkpatrick model of training evaluation.

EFFECT OF TRAINING PROGRAMS ON TRAINEES’ LEARNING

Employees’ learning is the primary driver of organizational sustainability. To achieve sustainability, organizations are required to invest in training for their employees’ continuous learning. Evaluation of training programs is the heart of continuous improvement in employees’ learning. This study investigates the effect of training programs on employees’ learning, focusing on knowledge-based outcomes (KBO), skill-based outcomes (SBO), and affective outcomes (AO). Drawing upon Human Capital Theory (HCT) and the second level of the Kirkpatrick's training evaluation model, a pre-experimental research design and quantitative method were employed. Stratified random sampling technique is used to select a sample of 381 employees from eight Private Commercial Banks (PCBs) of four bank generations in Bangladesh who participated in pre-test and post-test assessments. Statistical techniques including comparison chart, normality tests (Shapiro-Wilk W Test and Skewness and Kurtosis Test), and paired t-test were utilized to analyze the data. Findings indicate significant improvements in KBO, SBO, and AO following training interventions, supported by statistical test conducted using MS Excel v.13. Later, findings from executing STATA v.12 also underscore the effect of training programs on upgrading employees’ learning in the PCBs. Limitation includes the absence of a control group that hinders the influence of potential external factors on learning outcomes, which is a new avenue for future research.

Enhancing Digital Pedagogy in the Millennial Classroom: Evaluating the Effectiveness of Minecraft Education through the Kirkpatrick Model

This study explores the effectiveness of Minecraft Education as a tool for enhancing digital pedagogy among teachers. Utilizing the Kirkpatrick model of training evaluation, the research focuses on the first two levels: reaction and learning. The study involved 10 primary teachers from Malaysia, engaging in a two-week Minecraft Education course designed to develop digital pedagogy skills. This study used a mixed method design that combined performance rubrics and questionnaires to assess participants' reactions, skills, attitudes, and knowledge. Findings indicate a positive reaction from participants, with high levels of engagement and perceived educational value. Performance analysis revealed diverse levels of task performance, with some participants excelling while others faced challenges, suggesting the need for differentiated instructional strategies. Quantitative data showed moderate levels of skills, knowledge, and attitude development among participants, highlighting Minecraft Education's effectiveness in fostering practical abilities and engagement, yet also pointing to areas needing improvement in content delivery. The study concludes that while Minecraft Education is a promising educational tool, its effectiveness varies across different learning dimensions. Future research is recommended to compare Minecraft Education with traditional teaching methods and explore specific pedagogical strategies within the Minecraft environment. This will deepen understanding of its optimal use in educational settings.

BeHERE’s effective virtual training to build capacity to support people who use drugs in non-substance use disorder settings

BackgroundHuman service settings not specifically focused on supporting people who use drugs (PWUD), especially those with a substance use disorder (SUD), such as probation and parole services, homeless shelters, and work re-entry and job training programs, offer a unique opportunity to assist this population and prevent overdose deaths. During the COVID-19 pandemic (pandemic), building capacity in such settings for overdose prevention, harm reduction, and to address barriers to treatment, recovery, and support services required that training vendors use a virtual format. Post-pandemic, virtual training remains a cost-effective and convenient alternative to in-person training. The Behavioral Health and Racial Equity (BeHERE) Training Initiative of Health Resources in Action, which offers eight training modules on prevention, recovery, and harm reduction, delivered 224 online trainings between April 2020 and June 2022.MethodsA mixed methods evaluation based upon the Kirkpatrick Training Evaluation Model was employed, which utilized post-training (n = 1272) and follow-up surveys (n = 62), and key informant interviews (n = 35).ResultsThe findings showed BeHERE’s trainings were relevant, engaging, and satisfying to trainees; increased their knowledge, skills, and confidence; and influenced workplace performance. Some participants also indicated that the training influenced the effectiveness of their work with clients and other staff.ConclusionsThe evaluation identified aspects of training that make a virtual format effective at improving the capacity of non-SUD settings to address substance use and support PWUD. Findings offer insights for those interested in delivery of virtual training, as well as training to influence the practice of human service providers across different settings to support PWUD.

Communication skills-based training about medication switch encounters: pharmacy staff and patients' experiences.

Non-medical medication switches can lead to difficult conversations. To support pharmacy staff, a communication training has been developed based on two strategies: 'positive message framing' to emphasize positive elements of the message and 'breaking bad news model' to break the news immediately and address emotions. To assess how patients and trained pharmacy staff experience the application of communication strategies for non-medical medication switch conversations and which are barriers and facilitators for the application. The Kirkpatrick training evaluation model, level 3 'behavior', including barriers and facilitators and 4 'results' was used. Trained pharmacy staff registered switch conversation characteristics and asked patients to complete a questionnaire. Semi-structured interviews with trained pharmacy staff members were conducted. Quantitative data were analyzed descriptively and interview data were analyzed thematically. Of the 39 trained pharmacy staff members, 21 registered characteristics of 71 conversations and 13 were interviewed; 31 patients completed questionnaires. Level 3: trained pharmacy staff self-reported they applied aspects of the strategies, though indicated this was not yet a standard process. Interviewees indicated signs of increased patient contact and job satisfaction. Time, face-to-face conversations and colleague support were facilitators. Level 4: pharmacy staff members were satisfied with most switch conversations (89%), particularly with addressing emotions (74%). Patients were (very) positive (77%) about the communication, particularly about clear explanations about the switch. Pharmacy staff's learned behavior includes being able to apply aspects of the strategies. The training results show first signs of better patient-pharmacy staff relationships and increased job satisfaction.

The impact of an ethics curriculum in a longitudinal integrated clerkship: a qualitative study of graduates

Background: Little is known about the effectiveness of ethics education in developing knowledge and skills in medical schools that persist into residency and beyond. While previous research has demonstrated that graduates of Longitudinal Integrated Clerkships (LICs) continue practices of patient-centeredness and empathy, to our knowledge, no studies have looked at effect of a medical ethics curriculum integrated into LICs. Methods: In 2011, the University of North Carolina School of Medicine-Asheville initiated an Ethics and Humanism course alongside the LIC. The goals of the program were for students to be able to identify ethical issues as they arose and exercise moral imagination in resolving these issues. Later, practices of mindfulness and writing were added. We conducted a qualitative case study using The Kirkpatrick Model of training evaluation to assess graduates of this course in their capacity to apply those skills in ongoing clinical practice. Semi-structured interviews were conducted, transcribed and independently coded. The Kirkpatrick Model informed the organization of codes into themes. Results: The qualitative analysis reached thematic saturation with review of 22 graduates’ interview transcripts. The findings of this study suggest that the Ethics and Humanism curriculum successfully shaped how respondents approached ethical tensions and the healthcare system more broadly. Furthermore, the interviews revealed that respondents used moral imagination in their approaches to conflict resolution and advocacy projects. Discussion: We found graduates of this ethics curriculum continued to identify ethical dilemmas in their daily clinical practice and using moral imagination to address and resolve conflicts. Our findings suggest that linking ethics education to the clinical curriculum may develop persistent skills in navigating ethical dilemmas and increase graduates’ capacity to be resilient, holistic physicians. The potential benefits of this program in preventing burnout are explored. We believe this demonstrates the value of a dedicated ethics curriculum integrated into an LIC.

Alcohol Prevention in Urgent and Emergency Care (APUEC): Development and Evaluation of Workforce Digital Training on Screening, Brief Intervention, and Referral for Treatment.

Excessive alcohol consumption carries a significant health, social and economic burden. Screening, brief intervention and referral to treatment (SBIRT) is one approach to identifying patients with excessive alcohol consumption and providing interventions to help them reduce their drinking. However, healthcare workers in urgent and emergency care settings do not routinely integrate SBIRT into clinical practice and raise a lack of training as a barrier to SBIRT delivery. Therefore, "Alcohol Prevention in Urgent and Emergency Care" (APUEC) training was developed, delivered, and evaluated. APUEC is a brief, stand-alone, multimedia, interactive digital training package for healthcare workers. The aim of APUEC is to increase positive attitudes, knowledge, confidence and skills related to SBIRT through the provision of (a) education on the impact of alcohol and the role of urgent and emergency care in alcohol prevention, and (b) practical guidance on patient assessment, delivery of brief advice and making referral decisions. Development involved collaborative-participatory design approaches and a rigorous six-step ASPIRE methodology (involving n = 28 contributors). APUEC was delivered to healthcare workers who completed an online survey (n = 18) and then participated in individual qualitative interviews (n = 15). Analysis of data was aligned with Levels 1-3 of the Kirkpatrick Model of Training Evaluation. Survey data showed that all participants (100%) found the training useful and would recommend it to others. Insights from the qualitative data showed that APUEC digital training increases healthcare workers' perceived knowledge, confidence and skills related to alcohol prevention in urgent and emergency care settings. Participants viewed APUEC to be engaging and relevant to urgent and emergency care workers. This digital training was perceived to be useful for workforce skills development and supporting the implementation of SBIRT in clinical practice. While the impact of APUEC on clinician behaviour and patient outcomes is yet to be tested, APUEC digital training could easily be embedded within education and continuing professional development programmes for healthcare workers and healthcare trainees of any discipline. Ultimately, this may facilitate the integration of SBIRT into routine care and contribute to population health improvement.

Simulation-Based Training in Emergency Obstetric Care in Sub-Saharan and Central Africa: A Scoping Review.

Every day approximately 810 women die from complications related to pregnancy and childbirth worldwide. Around two thirds of these deaths happen in sub-Saharan Africa. One of the strategies to decrease these numbers is improving the quality of care by emergency obstetric simulation-based training. The effectiveness of such training programs depends on the program's instructional design. This review gives an overview of studies about emergency obstetric simulation-based training and examines the applied instructional design of the training programs in sub-Saharan and Central Africa. We searched Medline, Embase and Cochrane Library from inception to May 2021. Peer-reviewed articles on emergency obstetric, postgraduate, simulation-based training in sub-Saharan and Central Africa were included. Outcome measures were categorized based on Kirkpatrick's levels of training evaluation. The instructional design was evaluated by using the ID-SIM questionnaire. In total, 47 studies met the inclusion criteria. Evaluation on Kirkpatrick level 1 showed positive reactions in 18 studies. Challenges and recommendations were considered. Results on knowledge, skills, and predictors for these results (Kirkpatrick level 2) were described in 29 studies. Retention as well as decay of knowledge and skills over time were presented. Results at Kirkpatrick level 3 were measured in 12 studies of which seven studies demonstrated improvements of skills on-the-job. Improvements of maternal and neonatal outcomes were described in fifteen studies and three studies reported on cost-estimations for training rollout (Kirkpatrick level 4). Instructional design items were heterogeneously applied and described. Results of 47 studies indicate evidence that simulation-based training in sub-Saharan and Central Africa can have a positive impact across all four levels of Kirkpatrick's training evaluation model. However, results were not consistent across all studies and the effects vary over time. A detailed description of instructional design features in future publications on simulation-based training will contribute to a deeper understanding of the underlying mechanisms that determine why certain training programs are more effective in improving maternal and neonatal healthcare outcomes than other.

EFFECTIVENESS OF BASIC SAFETY TRAINING AMONG RATINGS

Human error plays a serious role in causing accidents at sea. The human factor should be emphasized more than automation and technology with quality and effective training. In order to maintain the safety culture on board, it is necessary to standardize the effectiveness of the trainings. The aim of this study is to determine the effectiveness of the training of seafarers and prospective seafarers who take part in basic safety training. The participants in the study are 275 seafarers or prospective seafarers. It was used to evaluate the effectiveness of basic safety training in relation to the response of the Kirkpatrick Training Evaluation Model. The data obtained /rom the scale results were analyzed using non-parametric tests (Mann Whitney U test and Kruskal Wallis H test) on SPSS package program. As a result of the study, it was concluded that basic safety training at a high level was effective. The perceptions of effectiveness of seafarers who received training far the first time were higher than those of seafarers who received training to renew their certificates.

Facilitating pharmacy staff's conversations about non-medical medication switches: Development and testing of a communication training.

Non-medical medication switches, a change to another medicine or medication label not motivated by medical reasons, occur frequently. Switches often lead to negative patient emotions, such as confusion and anger. Pharmacy staff's communication, i.e. delivering the message and addressing patients' emotions is crucial, but experienced as difficult. To develop and test a communication training for the pharmacy team to facilitate medication switch conversations. A communication training was developed based on the 'breaking bad news model' and 'positive message framing' strategies, and incorporating needs and preferences from practice. The training consisted of an e-learning with theory and reflective exercises, a half-day live training session, and an online reflection session. The Kirkpatrick training evaluation model (levels one 'reaction' and two 'learning') was used to evaluate the training. Quantitative data were analyzed using descriptive statistics and interview data was transcribed verbatim and analyzed thematically. Twelve pharmacists and 27 pharmacy technicians from 15 Dutch pharmacies participated in the training. According to Kirkpatrick's model level one, the major learning outcome was to give space to patients to express their emotions and/or concerns (e.g. more silences in the conversations). For level two, most participants valued practicing the conversations, role-playing, and receiving feedback. The majority of the participants indicated that they had sufficient tools and practice during the live training to apply the strategies in daily practice. A few participants still needed time and practice, or missed examples to apply the strategies. The communication training based on the two strategies was well-received and participants felt well-equipped post-training. The take-away for participants was to give space to patients to express their emotions. Using these strategies and skills, pharmacy teams can tailor their medication counseling to patients' emotions and concerns during non-medical medication switches to better support patients in proper medication use.

Feasibility and acceptability of an online intervention to enhance hopefulness among oncology professionals.

Patients prefer medical communication including both hopefulness and realism, though health-care professional (HCPs) struggle to balance these. Providers could thus benefit from a detailed personal understanding of hope, allowing them to model and convey it to patients. Additionally, given that hope is associated with lower levels of burnout, HCPs may benefit from tools designed to enhance their own personal hopefulness. Several investigators have proposed offering HCPs interventions to augment hope. We developed an online workshop for this purpose. Feasibility and acceptability of the workshop were assessed in members of the SWOG Cancer Research Network. Three measures were used: the Was-It-Worth-It scale, a survey based on the Kirkpatrick Training Evaluation Model, and a single item prompting participants to rate the degree to which they believe concepts from the workshop should be integrated into SWOG studies. Twenty-nine individuals signed up for the intervention, which consisted of a single 2-hour session, and 23 completed measures. Results from Was-It-Worth-It items indicate that nearly all participants found the intervention relevant, engaging, and helpful. Mean ratings for Kirkpatrick Training Evaluation Model items were high, ranging from 6.91 to 7.70 on 8-point scales. Finally, participants provided a mean rating of 4.44 on a 5-point scale to the item "To what degree do you believe it may be useful to integrate concepts from this workshop into SWOG trials/studies?" An online workshop to enhance hopefulness is feasible and acceptable to oncology HCPs. The tool will be integrated into SWOG studies evaluating provider and patient well-being.

Developing a capacity building training model for public health managers of low and middle income countries.

The challenges faced by the low and middle-income countries (LMIC) in the field of public health management calls for the capacity building of qualified and trained public health managers in order to improve the effectiveness and efficiency of the health care delivery system. Most of the existing training programs for public health management are based in the settings of developed countries, which hinders their application in LMIC countries. The objective of this paper is to document the process of development and evaluation of a capacity building program for public health managers of various LMICs. A training program was developed using Kern's six-step framework with several innovative learning and assessment methodologies and evaluation using Kirkpatrick training evaluation model. Delphi technique was used for program development. This five to ten-day partly/fully funded six International Public Health Management Development Programs (IPHMDP) programs was conceptualized which enrolled 178 participants from 42 countries between years 2016 and 2019. Based upon the elaborative discussion in four rounds of Delphi technique, the problem and challenges faced by public health mangers and eight key competencies (viz. Leadership and governance, Project/ program planning, financial management, supply chain management, quality management, Human Resource management, monitoring and evaluation, and communication.) were identified. The group consensually agree upon a blended teaching methodology comprising of chalk and talk approach, inquiry based learning, participatory student based learning, small group instructions, gamification, project-based learning and field-based learning. There was a significant increase in participants' knowledge score (P<0.0001) after all programs especially in the competencies of monitoring and evaluation, followed by project/ program planning, supply chain management and quality management. The majority (90%) submitted their action plan one week following the program, out of which 64% implemented their action plans within six months. A majority (54.7%) of participants were able to implement their learning once they went back by conducting similar training/ workshop/webinars in their settings. The comprehensive public health management program in LMIC settings strengthens the competencies of public health managers which can be replicated in similar settings across LMIC to mitigate diverse challenges in public health management.

Extensiveness of the Entrepreneurship Education Skills Project on Enhancing Youth Employment among Secondary School Students in Kilimanjaro Region, Tanzania

The evaluation of the implementation of entrepreneurship incubator skills project on enhancing youth employment among 8 secondary school in Kilimanjaro region, Tanzania. The study was guided by the Kirkpatrick Training Evaluation Model by Donald Kirkpatrick (1959). This study adopted an ex post facto design with quantitative approach. The targeted population involved 480 students, 40 alumnae, six coordinating teachers, six heads of schools, one regional coordinator and one country director of the project. The sample of this study included 96 students, 8 alumnae, 6 coordinating teachers, 6 heads of schools, 1 regional coordinator, and 1 country director of the project. Simple random sampling, stratified random sampling, and census sampling techniques were used to sample the respondents. Research instruments included questionnaires, observation schedules, and document analysis schedules. Ethical issues relating to this study were considered accordingly. Validity for quantitative instruments was ensured by involving three (3) research experts from MWECAU. Reliability for quantitative instruments ensured by using the Cronbach’s Alpha technique. The researcher used descriptive statistics for data analysis in frequencies, percentages, and mean scores. The findings revealed that knowledge of entrepreneurship skills among youth is crucial and is highly needed by the extreme majority of the students since additional knowledge enables youth to avoid mismatches between theory and practice. The study concluded that Asante Africa Foundation’s entrepreneurship skills training has a positive impact on students’ critical thinking and self-confidence. The project was effective in providing students with cognitive skills, including critical thinking, problem-solving, and creativity, which are essential for success in the modern workplace. Governments and educational institutions should incorporate entrepreneurship education into their curricula to equip students with employability skills and improve job readiness among youth. Entrepreneurship education programs should involve real-life situations and be designed to enhance transferable skills that can be applied in various employment types.

Training Public Health Professionals on Adaptive Challenges-An Innovative Approach Using Remote Learning Modalities.

Remote learning opportunities increase access to public health training. Innovative approaches are needed to promote active engagement and learning when tackling more complex, adaptive challenges that are nuanced, lack quick and easy solutions, and require a transformation of values and beliefs. The Rocky Mountain Public Health Training Center's (RM-PHTC's) training framework considers the complexity of the challenge (technical to adaptive) when selecting the learning modality. Remote learning opportunities on adaptive challenges are addressed using virtual workshops, online courses, ECHO (Extension for Community Healthcare Outcomes) series, and communities of practice that include a combination of multiple sessions, cohort-based learning, peer learning, practice-based application, and live, interactive sessions. The specific modality chosen depends on the audience and amount of content delivery. Virtual workshops are single events that encourage peer learning using a blend of instruction and practice-based work. Online courses deliver content to learning cohorts through an online platform combined with multiple, facilitated "live learning sessions." ECHO series include a short delivery of content, combined with multiple, facilitated, face-to-face, case-based learning sessions with a cohort of peers. Communities of practice encourage a bidirectional peer exchange of resources, approaches, and challenges through online sharing platforms and virtual meetings. Using the Kirkpatrick training evaluation model, learners of public health reported that their understanding of the subject matter improved and that they had identified actions that they would apply to their work. In qualitative feedback, learners shared feedback on elements that facilitated learning on adaptive challenges, including interactive breakout groups with peers and practice-based applications. Remote learning opportunities increase access for public health professionals in rural areas and encourage knowledge exchange region-wide. The RM-PHTC's training framework matches learning opportunities that address adaptive challenges with modalities that include elements that encourage the sharing of ideas and cocreation of solutions.

Evaluation of Three Blended Learning Courses to Strengthen Health Professionals' Capacity in Primary Health Care, Management of Sexual and Reproductive Health Services and Research Methods in Guinea.

BackgroundThree blended courses on Primary Health Care (eSSP), Management of Sexual and Reproductive Health Services (eSSR), and Research Methods (eMR) were developed and implemented between 2017 and 2021 by the Maferinyah National Training and Research Center in Rural Health, a training and research institution of the Ministry of Health in Guinea. The study objectives were to evaluate the reasons for dropout and abstention, the learners' work behavior following the training, and the impact of the behavior change on the achievements of learners' organizations or services.MethodsWe evaluated the three implemented courses in 2021, focusing on levels 3 and 4 of the Kirkpatrick training evaluation model. Quantitative and qualitative data were collected through an open learning platform (Moodle), via an electronic questionnaire, during the face-to-face component of the courses (workshops), and at learners' workplaces. Descriptive statistics and thematic analysis were performed for quantitative and qualitative data, respectively.ResultsOut of 1,016 applicants, 543 including 137 (25%) women were enrolled in the three courses. Over the three courses, the completion rates were similar (67–69%) along with 20–29% dropout rates. Successful completion rates were 72% for eSSP, 83% for eMR and 85% for eSSR. Overall success rate (among all enrollees) ranged from 50% (eSSP) to 58% (eSSR). The majority (87%) of the learners reported applying the knowledge and skills they acquired during the courses through activities such as supervision (22%), service delivery (20%), and training workshops (14%). A positive impact of the training on utilization/coverage of services and increased revenues for their health facilities were also reported by some trainees.ConclusionThese findings showed fair success rates and a positive impact of the training on learners' work behavior and the achievements of their organizations.

Case-based, asynchronous sleep education outcomes among primary care nurse practitioner students.

Primary care nurse practitioners (NPs) receive little sleep education in graduate programs but are often first-line providers for patients presenting with sleep-related symptoms. A pre-/postevaluation study was conducted using asynchronous, case-based sleep education modules in a cohort of primary care NP students enrolled in a single academic institution's nursing master's degree program. Six virtual, case-based modules addressed adult sleep health and disorders, prioritized based on prevalence and primary care presentation. Kirkpatrick Training Evaluation Model guided outcome selection. Descriptive and paired comparative analyses were conducted. Participants were first-year NP students (n = 149; 88% female; 82% ≤ 35 years of age) in an adult primary care program that included psychiatric/mental health track. Participants reacted positively to course delivery methods and content. Insomnia was endorsed by 87% as most relevant to practice with healthy sleep (88%) and obstructive sleep apnea (50%) also frequently endorsed as practice relevant. Posttest knowledge scores significantly improved for all modules (P < .001). Self-rated confidence for future practice application was high. This novel asynchronous, virtual curriculum met Kirkpatrick levels 1 and 2 (positive reaction and knowledge transfer) in NP students who predicted an impact on their practice (Kirkpatrick level 3). Studies are needed to assess the benefits of increasing primary care NP knowledge in sleep medicine on quality of care and access to care (Kirkpatrick level 4). Future use of this novel sleep curriculum in other professional curricula, new-to-sleep clinical researchers, and practicing primary care providers may further potentiate care quality and sleep care access. Sawyer AM, Saconi B, Lyons MM, etal. Case-based, asynchronous sleep education outcomes among primary care nurse practitioner students. J Clin Sleep Med. 2022;18(10):2367-2376.

0369 Sleep Education for the Nurse Practitioner: Nurse Practitioner Student Focus Group Findings

Abstract Introduction Primary care nurse practitioners (NPs) receive little to no sleep education in graduate programs despite being first-line providers for patients presenting with sleep-related symptoms. Sleep curriculum has been consistently identified as a gap in nursing education and confirmed in recent survey studies of nurses and NPs. Methods Qualitative descriptive study to explore NP students’ reactions to an asynchronous, case-based sleep e-learning program. Data were collected as part of a larger pre-/post-study assessing the program. Six asynchronous online modules were offered to a cohort of primary care NP students in a single academic institution’s master’s degree in primary care nursing program. At the end of the course, students were invited to participate in one-hour, online, focus group sessions. Directed content analysis, guided by the Kirkpatrick training evaluation model, was used to analyze the qualitative data to understand NP students’ experience with the program and elicit their perspectives about sleep education. Results Participants in the course (N=67) were predominantly female (88%) and ≤35 years old (81%). Twenty-four students participated in the focus group sessions. Two overarching themes emerged, including positive reactions to (1) course design and (2) course content. Students reported the case-based scenarios and quizzes enhanced their learning and kept them engaged, noted the user-friendly format, and appreciated that the course was asynchronous. After completing the modules, students recognized they had a previous knowledge gap related to sleep and perceived the information they received to be relevant to their practice/patient population and to their own personal health/wellbeing. Students also discussed their intentions to incorporate sleep assessments into practice. Conclusion Given the increasing sleep health needs of the population and the growing shortages of sleep providers, there is a critical need to ensure NP’s have the proper education to recognize and identify implications of poor and disordered sleep in their patients. In this study, NP students enthusiastically embraced sleep education, identified knowledge gain, and had intentions to apply their learned skills in practice highlighting the feasibility of increasing curricular exposure to sleep medicine. Support (If Any) The work reported herein was supported by National Institutes of Health (R25HL120874 Rosen, PI).

Applying the New World Kirkpatrick Model to evaluate an online training course introducing a multi-sensory device

Online education offers care providers flexibility and convenience. Applying the New World Kirkpatrick Model of training evaluation, this descriptive study evaluated the design and content of a 30-40 minute online training course that introduces direct-care workers to a multi-sensory device to help manage dementia symptoms in older adults. Following course completion, an online survey obtained ratings of engagement (i.e., aesthetics, ease of use, novelty, and involvement), relevance, and knowledge/skills gained from the training using a 5-point Likert scale. A convenience sample of 72 undergraduate students enrolled in health science and human service programs at a Midwestern university participated. The majority were white (83.3%), non-Hispanic (81.9%) females (88.9%). Most participants agreed or strongly agreed (median=4) with positive statements related to engagement with the course. Statements about relevance to their intended career were rated even higher (median=5). Wilcoxon signed-rank tests for matched pairs revealed statistically significant improvements on self-reported pre-post knowledge/skills scores (p<0.005). The results indicated that participants found the training aesthetically pleasing, easy to use, novel, and that it encouraged user involvement. Participants thought the topics covered were relevant to the professional career they are pursing, and they learned new knowledge/skills. Responses to open-ended questions suggested improvements to the design (e.g., color choice) and content (e.g., expanded topics and resources). Future research will evaluate a revised course with direct-care workers who will use the multi-sensory device in long term care facilities. Subsequently, an intervention study will determine the effectiveness of the device in increasing the well-being of people with dementia.