Kirkpatrick's Training Evaluation Research Articles

Evaluation of an Online Course in 5 Languages for Inpatient Cardiac Care Providers on Promoting Cardiac Rehabilitation: REACH, EFFECTS, AND SATISFACTION.

Evidence proves that health care providers should promote cardiac rehabilitation (CR) to patients face-to-face to increase CR enrollment. An online course was designed to promote this at the bedside; it is evaluated herein in terms of reach, effect on knowledge, attitudes, discussion self-efficacy and practices, and satisfaction. Design was observational, one-group pre- and post-test. Some demographics were requested from learners taking all language versions of the 20-min course: English, Portuguese, French, Spanish, and simplified Chinese, available at: https://globalcardiacrehab.com/CR-Utilization. Investigator-generated items in the pre- and post-test and evaluation survey administered using Google Forms were based on Kirkpatrick's training evaluation model. The course was initiated by 522 learners from 33 of 203 (16%) countries; most commonly female (n = 341, 65%) nurses (n = 180, 34%) from high-income countries (n = 259, 57%) completing the English (n = 296, 57%) and Chinese (n = 108, 21%) versions. A total of 414 (79%) learners completed the post-test and 302 (58%) completed the evaluation. Median CR attitudes were 5 of 5 on the Likert scale at pre-test, suggesting some selection bias. Mean CR knowledge ([7.22 ± 2.14]/10), discussion self-efficacy ([3.86 ± 0.85]/5), and practice ([4.13 ± 1.11]/5) significantly improved after completion of the course (all P < .001). Satisfaction was high regardless of language version ([4.44 ± 0.64]/5; P = .593). This free, open-access course is effective in increasing CR knowledge, self-efficacy, and encouragement practices among participating inpatient cardiac providers, with high satisfaction. While testing impact on actual CR use is needed, it should be more broadly disseminated to increase reach, in an effort to increase patient enrollment in CR, to reduce morbidity and mortality.

Post-graduate Medical Training in Intellectual and Developmental Disabilities: a Systematic Review.

Despite the increasing number of people with autism-spectrum disorder (ASD), intellectual disabilities (ID), and developmental disabilities (DDs), individuals with these conditions continue to have high levels of unmet physical and mental health needs. Robust training of health professionals can help bridge this gap. A systematic review was conducted to describe the features and educational outcomes of existing postgraduate medical education curricula to inform the development of future training to address the growing unmet care needs of people with intellectual and developmental disabilities (IDD) such as ASD and ID. Four major databases were searched for peer-reviewed, English-language research focusing on post-graduate training in IDD education. Educational curricula and outcomes were summarized including Best Evidence in Medical Education (BEME) Quality of Evidence and Kirkpatrick training evaluation model. Sixteen studies were identified with a majority published after 2000 (69%). Pediatric departments were involved in 69%, Psychiatry 19%, Medicine-Pediatrics 19%, and Family Medicine 6.3%. Analysis of Kirkpatrick outcomes showed 31% were level 1 (satisfaction or comfort); 38% level 2 (change in objective knowledge or skills); 13% level 3 (change in behavior); and none at level 4. BEME analysis showed 19% of studies were grade 1 (no clear conclusions), 31% grade 2 (ambiguous results), and half (50%) grade 3 (conclusions can probably be based on findings), with none scoring four or higher. There is a paucity of objectively evaluated research in the area. Studies reviewed show clear promise for specialized, interdisciplinary, competency-based education which may be foundational for future curriculum development.

Wellness Interventions in Emergency Medicine Residency Programs: Review of the Literature Since 2017.

IntroductionRecent research demonstrates burnout prevalence rates as high as 76% in emergency medicine (EM) residents. In 2017 the Accreditation Council for Graduate Medical Education (ACGME) required that all training programs provide dedicated wellness education for their trainees as a requirement for accreditation. We aimed to conduct a systematic review of published wellness interventions conducted in EM residency programs following the implementation of the 2017 ACGME Common Program Requirements change in order to characterized published intervention and evaluate their effectiveness.MethodsWe applied a published approach to conducting systematic reviews of the medical education literature. We performed a search of the literature from January 1, 2017–February 1, 2020. Studies were included for final review if they described a specific intervention and reported outcomes with the primary goal of improving EM resident wellness. Outcomes were characterized using the Kirkpatrick training evaluation model.ResultsEight of 35 identified studies met inclusion criteria. Most described small convenience samples of EM residents from single training programs and used the satisfaction rates of participants as primary outcome data. Only quantitative assessment methods were used. Studies addressed only a limited number of factors affecting resident wellness. The majority of interventions focused on personal factors, although a few also included sociocultural factors and the learning and practice environment.ConclusionThere is a relative dearth of literature in the area of research focused on interventions designed to improve EM resident wellness. Furthermore, the studies we identified are narrow in scope, involve relatively few participants, and describe programmatic changes of limited variety. Future directions include an increase and emphasis on multi-institutional studies, randomized controlled trials, qualitative methodology, and opportunities for funded research.

Qualitative evaluation of a knowledge transfer training programme in maternal and child health in Burkina Faso, West Africa.

The study evaluated a first session of the knowledge transfer training programme organised for Ministry of Health workers in the area of maternal and child health in Burkina Faso. The objective was to obtain the participant's perceptive for improving the training programme. A qualitative study was conducted during and after the first training session, using the four levels of Kirkpatrick training evaluation framework. A group discussion was organised with 17 participants during the training and a few weeks after the first training session, 11 of them were interviewed again. A thematic analysis of the transcribed interviews was done in line with the objectives of the evaluation. We noted that the participants had a good impression of the training, in relation to their expectations, the adequacy and quality of the teaching. The 11 post-training interviewees stated that they had acquired knowledge and skills relevant to their work. Using the learner's behavioural change model, three groups emerged as a result of the training. The first group expressed intent to use the knowledge gained, the second group set conditions for the effective application of the knowledge acquired, including the enabling professional environment. While the third group felt that despite the knowledge and skills they had acquired, they would need assistance with the applying the skills at their jobs. In conclusion, the training improved the capacity and skills of health workers, but an enabling professional environment and support will facilitate the application of knowledge.

Evaluation of teaching practicum for pre-service English language teachers: A scale development study

The aim of this study is to develop a measurement tool that evaluates the teaching practicum to shed light on the expectations and attitudes of English teacher candidates in relation to teaching practicum. In this context, two measurement tools; expectation scale and reaction scale were prepared based on the amalgamation of Kirkpatrick Training Evaluation Theory (2000) and EPOSTL(2007). This study was conducted to have an overall critical analysis of the current practicum program applied in Turkey to determine the pre-service teachers’ views about the existing model and what could be done to make up for the deficits in the current implementation. Within the scope of the study, validity and reliability studies were conducted. For research validity; a pilot study was conducted at a state university with senior level ELT students at 2018-2019 spring academic year. As a result of the pilot study item writing and item pooling stages were completed. Expert opinion was received from three academicians from the field in order not to assess the face validity, content validity and construct validity. Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were conducted with 294 senior level ELT students who were taking the School Experience course at 2019-2020 fall term around various universities in Turkey. Working groups were determined based on maximum diversity sampling method. The expectation scale had 85 items at the beginning. According to the results of the structure validity of the research; the expectation scale had a) university supervisors, b) mentor teachers, c) school administrators, d) peer pre-service teachers, e) students, f) practicum school, g) teaching practicum and h) other stakeholders. Parts. The eight-factor structure scale consisted of 73 items. The reaction scale had 44 items at the beginning. As a result of the analysis, the reaction scale consisted of three-factor structure consisting of 39 items. These factors were determined as a) attitudes at the end of the practicum training as a teacher candidate, b) attitudes in relation to the teaching practicum and c) pre-service teachers reactions about teaching practicum. As a result of the CFA application, it has been observed that the scale shows good fit values in general. In line with the applications and analyzes, it can be said that the scales are valid, reliable and they have sufficient values in terms of psychometry.

Teaching Public and Population Health in Medical Education: An Evaluation Framework.

Curriculum models and training activities in medical education have been markedly enhanced to prepare physicians to address the health needs of diverse populations and to advance health equity. While different teaching and experiential learning activities in the public health and population health sciences have been implemented, there is no existing framework to measure the effectiveness of public and population health (PPH) education in medical education programs. In 2015, the Association of American Medical Colleges established the Expert Panel on Public and Population Health in Medical Education, which convened 20 U.S. medical faculty members whose goal was to develop an evaluation framework adapted from the New World Kirkpatrick Model. Institutional leaders can use this framework to assess the effectiveness of PPH curricula for learners, faculty, and community partners. It may also assist institutions with identifying opportunities to improve the integration of PPH content into medical education programs. In this article, the authors present outcomes metrics and practical curricular or institutional illustrations at each Kirkpatrick training evaluation level to assist institutions with the measurement of (1) reaction to the PPH education content, (2) learning accomplished, (3) application of knowledge and skills to practice, and (4) outcomes achieved as a result of PPH education and practice. A fifth level was added to measure the benefit of PPH curricula on the health system and population health. The framework may assist with developing a locally relevant evaluation to further integrate and support PPH education at U.S. medical schools and teaching hospitals.

THE ANALYSIS OF THE EFFECTIVENESS OF THE INDONESIAN NATIONAL POLICE’S INTELLIGENCE SPECIALIZED DEVELOPMENTAL EDUCATION USING KIRKPATRICK TRAINING EVALUATION MODEL

This study examines the effectiveness of the implementation of the Security Intelligence Specialized Developmental Education (Dikbangspes Intelkam) in the Indonesian National Police (INP)’s Security Intelligence Education Center in 2018 using Kirkpatrick training evaluation model. Conceptually, training center is important to improve the ability of human resources (HR) in the organization and in actual fact the INP’s security intelligence human resources constitute the critical and strategic assets as well as leading component in supporting the police organization especially in presenting information related to the investigation, security and promotion in order to support the main tasks and functions of the Police (Tupoksi) in the field of security and public order (Kamtibmas). This research was conducted with a mixed method, beginning with the collection and analysis of quantitative data then proceeded with the collection and analysis of qualitative data. The analysis used Kirkpatrick's training evaluation model, namely reaction, learning, behavior, and result. The quantitative analysis results show that the effectiveness of the INP’s Security Intelligence Specialized Developmental Education (Dikbangspes) is high; however, based on the qualitative analysis, the implementation of the Security Intelligence Specialized Developmental Education (Dikbangspes) in 2018 faces 3 (three) problems, namely: the attainment of educational goals has not yet optimal, the training management system having been developed in the yearly period has not yet had a strong reference, and the evaluation of educational outcomes has had no significant follow-up.

Qualification programmes for immigrant health professionals: A systematic review.

Immigrant health professionals are a particularly vulnerable group in a host country's labour market, as they face several barriers when re-entering their occupations. International studies indicate that early interventions can increase the employability of immigrants. Qualification programmes are one of these early interventions that can support the re-integration of these health professionals into the labour market. The purpose of this review is to identify international qualification programmes for immigrant health professionals, analyse their content and evaluate their effectiveness. Six international databases (PubMed, Web of Science, CINAHL, PsychInfo, EBSCO and ProQuest Social Sciences) were systematically searched. The search terms were identified using the PICOS-framework. The review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Articles were screened independently by two authors and discussed. Studies included in the final synthesis were assessed with the Mixed Methods Appraisal Tool (MMAT) and Kirkpatrick's Training Evaluation Model. Out of 10,371 findings, 31 articles were included in the final synthesis. The majority of them were addressed to international health care professionals and doctors. Two of them were addressed to refugee doctors. Three types of programme elements were identified: teaching, clinical practise and social support. The programmes' length ranged from 2 days to up to 2.5 years. Despite recommendations in its favour, pre- and post-programme support was scarce. Results also indicate a lack of transparency and quality in terms of evaluation. Effectiveness was mostly observed in the area of language improvement and an increase in self-confidence. This review points out the lack of systematically evaluated qualification programmes for immigrant health professionals. Programme providers should focus on implementing programmes for all health professionals as well as for underrepresented groups, such as refugees. In order to generate best practises it is necessary to evaluate these programmes. This requires the development of appropriate instruments when working with immigrant population in the context of educational programmes.

Impact of Using a 3D Visual Metaphor Serious Game to Teach History-Taking Content to Medical Students: Longitudinal Mixed Methods Pilot Study.

BackgroundHistory taking is a key component of clinical practice; however, this skill is often poorly performed by students and doctors.ObjectiveThe study aimed to determine whether Metaphoria, a 3D serious game (SG), is superior to another electronic medium (PDF text file) in learning the history-taking content of a single organ system (cardiac).MethodsIn 2015, a longitudinal mixed methods (quantitative and qualitative) pilot study was conducted over multiple sampling time points (10 weeks) on a group of undergraduate medical students at The University of Auckland Medical School, New Zealand. Assessors involved in the study were blinded to group allocation. From an initial sample of 83, a total of 46 medical students were recruited. Participants were assigned to either a PDF group (n=19) or a game group (n=27). In total, 1 participant left the PDF group after allocation was revealed and was excluded. A total of 24 students in the game group and 14 students in the PDF group completed follow-up 7 weeks later. Using an iterative design process for over a year, with input from a variety of clinical disciplines, a cardiac history-taking game and PDF file were designed and informed by Cognitive Load Theory. Each group completed its intervention in 40 min. A total of 3 levels of Kirkpatrick training evaluation model were examined using validated questionnaires: affective (perception and satisfaction), cognitive (knowledge gains and cognitive load), and behavioral attitudes (Objective Structured Clinical Exam) as well as qualitative assessment. A priori hypotheses were formulated before data collection.ResultsCompared with baseline, both groups showed significant improvement in knowledge and self-efficacy longitudinally (P<.001). Apart from the game group having a statistically significant difference in terms of satisfaction (P<.001), there were no significant differences between groups in knowledge gain, self-efficacy, cognitive load, ease of use, acceptability, or objective structured clinical examination scores. However, qualitative findings indicated that the game was more engaging and enjoyable, and it served as a visual aid compared with the PDF file.ConclusionsStudents favored learning through utilization of an SG with regard to cardiac history taking. This may be relevant to other areas of medicine, and this highlights the importance of innovative methods of teaching the next generation of medical students.

Evaluation of a Workshop on Social Determinants of Health based on Kirkpatrrick Model

Introduction: Evaluation provides truly useful information about the effectiveness of the educational programs. Kirkpatrick model provides a four-level training evaluation framework (reaction, learning, behavior, and results). The present research aimed to investigate the effectiveness of a workshop on “social determinants of health” held for the health staff of a medical center in Bandar Abbas Province, in the South of Iran. Methods: In the present research, a training workshop was held on social determinants of health for 40 staff of a medical center. Later, this workshop was evaluated based on the Kirkpatrick training evaluation model and a researcher-constructed questionnaire. Descriptive and inferential statistics were used to analyze the data in SPSS-19. Results: The participants' mean scores of satisfaction (reaction), learning, and behavior were 41.47±4.34, 14.9±1.93, and 10.72±3.8, respectively. The overall effectiveness (reaction, learning and behavior) was evaluated as optimal. The most effective items in reaction, learning, and behavior levels were awareness-raising, better performance of tasks at work, and use of the acquired content in monitoring work, respectively. Conclusion: Staff training to provide them with job-related practical knowledge not only satisfies them, but also improves their professional learning and performance. Kirkpatrick training evaluation model needs to be applied to other staff training programs at four levels of reaction, learning, behavior, and results. Those who develop and implement training courses need to be acquainted with these constructs.

Foresight Training: Moving from Design to Evaluation

The futures field has witnessed a rise in foresight training for government and business leaders. How should participants judge the quality of foresight training they receive? This article reviews the history of foresight training and explains why it is incomplete without evaluation. To conclude, practical steps are offered that foresight firms can take to upgrade their training impact through continuous improvement, based on the Kirkpatrick Training Evaluation Model.

Evaluation and Comparison of Short Form Methadone Maintenance Treatment Training Courses Among Iranian General Physician and Psychologist Participants

The evaluation of the effectiveness of education plays an important role in the improvement of training processes. The aim of the present study was to evaluate and compare the short form training courses for methadone maintenance treatment (MMT) in Iranian general physicians (GPs) and psychologists. This was a descriptive cross-sectional study with 203 participants (91 GPs and 112 psychologists) in three 10-day short form MMT training courses at the School of Behavioral Sciences and Mental Health - Tehran Institute of Psychiatry. After the courses, participants were asked to complete the Kirkpatrick Training Evaluation Questionnaire. Overall, the GPs evaluated the short form MMT training course significantly more positively than did the psychologists. Evaluation scores for Levels 1, 2, and 4 were good, average, and average, respectively. The Level 2 (learning) scores of both GPs and psychologists increased after the short form MMT training course compared with before the short form MMT training course. Similarities in the views of the GPs and the psychologists were more than differences between them. Research findings raised concerns about the implementation, management, monitoring, new guidelines, support, and training for this training course while providing valuable information in helping to facilitate more involvement of GPs and psychologists in the short form MMT training course.

Evaluating a mobile application for improving clinical laboratory test ordering and diagnosis

Mobile applications for improving diagnostic decision making often lack clinical evaluation. We evaluated if a mobile application improves generalist physicians' appropriate laboratory test ordering and diagnosis decisions and assessed if physicians perceive it as useful for learning. In an experimental, vignette study, physicians diagnosed 8 patient vignettes with normal prothrombin times (PT) and abnormal partial thromboplastin times (PTT). Physicians made test ordering and diagnosis decisions for 4 vignettes using each resource: a mobile app, PTT Advisor, developed by the Centers for Disease Control and Prevention (CDC)'s Clinical Laboratory Integration into Healthcare Collaborative (CLIHC); and usual clinical decision support. Then, physicians answered questions regarding their perceptions of the app's usefulness for diagnostic decision making and learning using a modified Kirkpatrick Training Evaluation Framework. Data from 368 vignettes solved by 46 physicians at 7 US health care institutions show advantages for using PTT Advisor over usual clinical decision support on test ordering and diagnostic decision accuracy (82.6 vs 70.2% correct; P < .001), confidence in decisions (7.5 vs 6.3 out of 10; P < .001), and vignette completion time (3:02 vs 3:53 min.; P = .06). Physicians reported positive perceptions of the app's potential for improved clinical decision making, and recommended it be used to address broader diagnostic challenges. A mobile app, PTT Advisor, may contribute to better test ordering and diagnosis, serve as a learning tool for diagnostic evaluation of certain clinical disorders, and improve patient outcomes. Similar methods could be useful for evaluating apps aimed at improving testing and diagnosis for other conditions.

Building Professionalism Through Management Training: New England Public Health Training Center's Low-Cost, High-Impact Model.

Evolving practices, accreditation, and priorities established in Public Health 3.0 are adding to the long-identified need for management training among public health practitioners. The New England Public Health Training Center is addressing this need with a flexible, open-source, 16-topic training program. The program is designed to build competencies for current and future managers, preparing them for their day-to-day tasks and for the kinds of adaptation suggested by Public Health 3.0 advocates. The training program uses live expert instructors for 10 webinars and 2 in-person trainings. Experts have also created the content for multiple self-paced E-Learnings that trainees undertake in addition to the instructor-led sessions. A webinar platform with breakout rooms and an advanced learning management system allows for online discussion and mentor interaction. The course has now been offered, evaluated, and modified 3 times, and the materials are available for noncommercial use by the public health community. Using the Kirkpatrick training evaluation model, the recent cohort was satisfied (87.5%) with the training, reported identifying actions to apply information learned to their work (85.8%), and experienced statistically significant knowledge gains. Earlier trainees reported work-related behavior change. Management training offers the hope of increasing professionalism; creating better, more effective workplaces and programs; and preparing practitioners for an evolving public health landscape. Early results indicate that NEPHTC's program, Managing Effectively in Today's Public Health Environment, is a useful tool in realizing that hope.

Is "modular" the way to go for small group learning in community medicine in undergraduate clinical postings?

Context:There is a need to shift from the didactic lecture-based instruction to more student-centered active learning methods for undergraduate teaching in community medicine.Aims:To compare didactic and modular method of learning on Level 1 and 2 on Kirkpatrick’s training evaluation model.Settings and Design:This was a two-arm educational intervention study for a small group of the 2nd year MBBS students in their 4th semester during clinical posting in the subject of community medicine.Subjects and Methods:The topic chosen was “rabies prophylaxis” in the 2nd clinical posting during 4th semester. With permission from Institutional Ethics Committee, first batch of 17 students was taught this topic by didactic method. Next batch of 22 students was taught by the modular method. A self-reading module was prepared for this study and validated by three teachers. What was different in modular teaching was a circular sitting arrangement, module reading by students, video presentation, and exercise using case vignettes.Statistical Analysis Used:Student’s t-test was used for pre- and post-test score comparison and Mann–Whitney U-test for students' responses on Likert scale.Results:The mean gain in obtained marks after modular learning (7.9/15) was significantly higher as compared to gain after didactic teaching (5.9/15) (P = 0.0038); more students asserted to be confident to manage a case in modular group compared to the didactic group (P < 0.05) indicating a higher level of learning through modular teaching.Conclusions:Modular teaching fares better than didactic method and hence should be used more frequently in community medicine clinical posting.

A Systematic Approach to Evaluating Public Health Training

Public health practitioners require new knowledge and skills to address the multilevel factors contributing to obesity. This article presents the systematic approach the Center of Excellence for Training and Research Translation (Center TRT) used both to assess practitioners' competencies to lead public health obesity prevention initiatives and to evaluate its annual, competency-based obesity prevention course. In 2006, Center TRT identified priority public health competencies for obesity prevention and then planned 7 annual courses to address the priority competencies progressively over time. Each year, a longitudinal evaluation based on Kirkpatrick's training evaluation framework was administered to course participants (n = 243) to assess perceptions of the course (daily), changes in self-reported competency (immediately pre- and postcourse), and course impact on practice over time (at 6 months). Participants rated the course highly for quality and relevance. Although many participants reported low levels of confidence prior to the course, following the course, at least 70% reported feeling confident to perform almost all competencies. At 6-month follow-up, the majority of participants reported completing at least 1 activity identified during course action planning. We identified practitioners' high-priority competency needs and then designed 7 annual courses to progressively address those needs and new needs as they arose. This approach resulted in trainings valued by practitioners and effective in increasing their sense of competence to lead public health obesity prevention initiatives. The course's continuing impact was evidenced by participants' high level of completion of their action plans at 6-month follow-up. Competency-based training is important to develop a skilled public health workforce.

The Preparedness and Emergency Response Learning Centers: advancing standardized evaluation of public health preparedness and response trainings.

The Centers for Disease Control and Prevention funded Preparedness and Emergency Response Learning Centers (PERLCs) across the United States. The PERLCs provide training to state, local, and tribal public health organizations to meet workforce development needs in the areas of public health preparedness and response, specialized training, education, and consultation. Using Donald Kirkpatrick's training evaluation model, the PERLC network established 4 evaluation working groups that developed evaluation criteria to address each level of the model. The purpose of the working groups was to inform and promote center-level and program-level evaluation across the PERLC network; identify common training evaluation methods and measures; and share materials, resources, and lessons learned with state, local, and tribal public health organizations for potential replication. The evaluation of education and training, irrespective of its modality (eg, in-person, online, webinars, seminars, symposia) can be accomplished using Kirkpatrick's 4-level taxonomy. The 4 levels aim to measure the following aspects of training programs: (1) trainees' reaction; (2) knowledge acquired, skills improved, or attitudes changed; (3) behavior changed; and (4) results or impact. To successfully evaluate emergency preparedness training, drills and exercises, it is necessary to understand the fundamental tenets of each level and how to apply each to measure training outcomes. The PERLC evaluators have adopted the basic schema of Kirkpatrick's 4-level model and applied its structure to a wide variety of preparedness and emergency response training and related activities. The PERLC evaluation working groups successfully developed and tested survey methods and instruments for each of the 4 levels of Kirkpatrick's training evaluation model. Each can be used for replication by state, local, and tribal public health professionals.

Going beyond Kirkpatrick's Training Evaluation Model: The role of workplace factors in distance learning transfer

This article emanates from a longitudinal study of the impact of a distance education programme for teacher training on graduates' job performance, in which the authors built on the findings of a previous pilot study. After using Kirkpatrick's Training Evaluation Model in a previous study, one of the authors found there to be a strong relationship between graduates' completion of the programme and their performance at school. However, the model does not probe factors that impede on transfer of learning. Quite a number of the graduate participants indicated that they were faced with this problem. In order to further probe this phenomenon, the authors fused Baldwin's Transfer of Training Model with the second level of Kirkpatrick's model by using a mixed-methods enquiry. It became clear that the organizational climate of schools has a strong influence on the transfer of learning in the workplace. Suggestions are presented on how educators and school managers can work together effectively.

The relationship between reaction and learning in leadership training

This study examined the relationship between ‘reaction’ and ‘learning’ based on the Kirkpatrick's training evaluation model, with an emphasis on the moderating effect of trainee's difference variable(goal-orientation, openness to experience, extraversion). The survey data from a job training class of 212 employees were analyzed. The results of the study are summarized as follows; First, the three dimensions of reaction(enjoyment, usefulness, difficulty) predicted subjective learning significantly. In other words, when trainees felt enjoyment or usefulness about their training program, effect of learning increased. However, learning was decreased as they felt difficulties in training program. Second, the result showed that among trainee's individual difference variables, mastery goal-orientation, performance goal-orientation, and extraversion had positive effect on learning while openness to experience had not any effect. Third, this result proved to have a moderating effect of some trainee's individual difference variables in the relationship between reaction and learning. Specifically, extraversion moderated the relationship between enjoyment and learning positively. Goal-orientation did not moderat the relationship between difficulty and learning. Extraversion had neigher main effect nor moderating effect on learning. Finally, limitation of the study and the direction for future research were discussed.

The Relationship Between Reaction and Learning in Leadership Training: Focus on Moderating Effect of Trainee's Difference Variable

This study examined the relationship between ‘reaction’ and ‘learning’ based on the Kirkpatrick's training evaluation model, with an emphasis on the moderating effect of trainee's difference variable(goal-orientation, openness to experience, extraversion). The survey data from a job training class of 212 employees were analyzed. The results of the study are summarized as follows; First, the three dimensions of reaction(enjoyment, usefulness, difficulty) predicted subjective learning significantly. In other words, when trainees felt enjoyment or usefulness about their training program, effect of learning increased. However, learning was decreased as they felt difficulties in training program. Second, the result showed that among trainee's individual difference variables, mastery goal-orientation, performance goal-orientation, and extraversion had positive effect on learning while openness to experience had not any effect. Third, this result proved to have a moderating effect of some trainee's individual difference variables in the relationship between reaction and learning. Specifically, extraversion moderated the relationship between enjoyment and learning positively. Goal-orientation did not moderat the relationship between difficulty and learning. Extraversion had neigher main effect nor moderating effect on learning. Finally, limitation of the study and the direction for future research were discussed.