Mastery Standard Research Articles

Assessing the Impact of Minimum Learning Mastery Standard on the Effectiveness of Follow-Up Programs in Madrasa

This study aims to analyse the assessment and follow-up processes of Arab learning at Madrasah Aliyah using qualitative methods with a phenomenological approach. It involved 24 Arabic language teachers from Madrasah Aliyah in South Kalimantan. The findings indicated that Minimum Learning Mastery Standards (MLMS) were not adjusted to accommodate student characteristics, subject competencies, and educational unit conditions. Consequently, learning assessments became less credible, often serving merely to fulfil MLMS requirements. Both enrichment and remedial follow-up were perceived as formalities, lacking a basis in the analysis of student assessments, which undermined students' motivation to learn. Teachers encountered ethical dilemmas in assessment and decision-making processes, exacerbated by external pressures for students to meet MLMS benchmarks. The study emphasizes the importance of reducing external pressures in determining MLMS, enhancing teacher competence in adopting new assessment methods, and improving evaluation systems. These improvements are crucial for developing evidence-based structured enrichment and remedial programs

Strategi Guru Fikih dalam Menggunakan Media Pembelajaran Berbasis Information and Communication Technology (ICT) di MTsS Hubbul Wathan Modal Bangsa Sei Bingai

The teacher uses teaching strategies to ensure that learning goes well, efficiently, and optimally. After evaluating, the teacher follows up to achieve the minimum mastery standards for students. This research aims to explain how teachers use teaching strategies in the classroom, focusing on class IX at MTsS Hubbul Wathan Modal Bangsa. The research obtained data through interviews and observations in class IX with several PAI subject teachers. The data validation method used is the triangulation technique. The data analysis technique consists of data reduction (data selection), data presentation (data display), and then drawing conclusions/verification. The results of the study show that the teacher's teaching has been proven to be efficient, effective, and optimal, with a well-organized learning process, good student understanding, and the use of IT-based media that helps students understand the material.

The PATH to PPE Mastery - Programme for Assessment and Training in HCID (High Consequence Infectious Diseases) PPE (Personal Protective Equipment), Mastery

High Consequence Infectious Diseases (HCIDs), have the potential to cause pandemics and require particular focus for preparedness due to their high mortality rates. The application of Personal Protective Equipment (PPE) for HCIDs is complex and carries significant risk of Health Care Worker (HCW) contamination if done incorrectly. Previous reviews have reported a lack of information on the nature of training provided and the ideal timing of repeat training to best retain skills. Simulation Based Mastery Learning (SBML) is a methodology for skill acquisition which encompasses deliberate practice and repeated assessment until the learner achieves a pre-set Mastery standard. SBML has been demonstrated to improve competence, skill retention and patient outcomes in other clinical procedures. SBML has not been previously studied or utilised in HCID PPE training. We aimed to increase the likelihood of safe clinical practice by evidencing that Lothian modified SBML for PPE effectively prepares our priority learners. A quasi-experimental within group post-test design was used. Learners undertook a modified SBML programme which included two-hour asynchronous and two-hour synchronous components. 11 learners (10 infectious diseases registrars and 1 infectious diseases consultant) were enrolled in the programme with 8 completing all stages, all of whom achieved the Mastery passing standard. The resources were highly rated by learners with the exemplar videos of skills highlighted as particularly useful. Self-assessed preparedness for each skill increased following pre-learning and synchronous sessions. Modified SBML can be used as an effective methodology for the training and assessment of HCWs in the donning and doffing of HCID PPE.

THE BLENDED LEARNING METHOD IN IMPROVING SPEAKING SKILLS OF MIDDLE SCHOOL STUDENTS

Implementing online learning causes a lack of enthusiasm and low motivation to learn English speaking skills. This study aims to analyze English speaking skills by implementing Hybrid learning-based in junior high school. This research used classroom action research (CAR) that determines the set of two cycles according to the characteristics of the material. Research in each Cycle was carried out through four stages. Application of the Learning Model Hybrid learning applies two activities which include bold and limited face-to-face learning. The number of students in the class was divided into odd and even absences. Data collection methods used observation, evaluation, and student responses. Methods of data analysis employed descriptive analysis. Preliminary learning outcomes before implementing hybrid learning mastery were only 54,84%. Cycle I showed an increase in learning completeness, reaching 84.2%. Cycle II also showed an increase in learning completeness with 94.7%. Data from Cycle I and Cycle II results reveal that implementing Hybrid learning can improve learning outcomes and the proportion of minimum mastery standards that is very significant. The implications of this research are expected to improve English speaking skills in class X students.

Simulation-based Mastery Learning Improves Critical Care Skills of Advanced Practice Providers.

Advanced practice providers (APPs) are essential members of intensive care unit (ICU) interprofessional teams and are expected to be competent in performing procedures. There are no published criteria for establishing when APPs can independently perform procedures. Simulation-based mastery learning (SBML) is an effective strategy for improving critical care skills but has not been applied to practicing ICU APPs. The purpose of this study was to evaluate if an SBML curriculum could improve the critical care skills and procedural self-confidence of ICU APPs. We performed a pretest-posttest study of central venous catheter (CVC) insertion, thoracentesis, and mechanical ventilation (MV) management skills among ICU APPs who participated in an SBML course at an academic hospital. For each skill, APPs underwent baseline skills assessments (pretests) on a simulator using previously published checklists, followed by didactic sessions and deliberate practice with individualized feedback. Within 2 weeks, participants were required to meet or exceed previously established minimum passing standards (MPS) on simulated skills assessments (posttests) using the same checklists. Further deliberate practice was provided for those unable to meet the MPS until they retested and met this standard. We compared pretest to posttest skills checklist scores and procedural confidence. All 12 eligible ICU APPs participated in internal jugular CVC, subclavian CVC, and MV training. Five APPs participated in thoracentesis training. At baseline, no APPs met the MPS on all skills. At training completion, all APPs achieved the mastery standard. Internal jugular CVC pretest performance improved from a mean of 67.2% (standard deviation [SD], 28.8%) items correct to 97.1% (SD, 3.8%) at posttest (P = 0.005). Subclavian CVC pretest performance improved from 29.2% (SD, 32.7%) items correct to 93.1% (SD 3.9%) at posttest (P < 0.001). Thoracentesis pretest skill improved from 63.9% (SD, 30.6%) items correct to 99.2% (SD, 1.7%) at posttest (P = 0.054). Pretest MV skills improved from 54.8% (SD, 19.7%) items correct to 92.3% (SD, 5.0%) at posttest (P < 0.001). APP procedural confidence improved for each skill from pre to posttest. SBML is effective for training APPs to perform ICU skills. Relying on traditional educational methods does not reliably ensure that APPs are adequately prepared to perform skills such as CVC insertion, thoracentesis, and MV management.

Pengaruh Pembelajaran IPS Materi Sejarah Terhadap Sikap Nasionalisme Berbasis Kearifan Lokal Pada Peserta Didik Di MTs Al Mansur Biangloe Kabupaten Bantaeng

The purpose of this study was to find out 1) Social studies learning history material at MTs Al-Mansur Biangloe Regency. Bantaeng 2) The attitude of nationalism based on local wisdom in students at MTs Al-Mansur Biangloe, Bantaeng Regency. 3) The effect of learning social studies history material on the attitude of nationalism based on local wisdom in students at MTs Al Mansur Biangloe, Bantaeng Regency. This research is a quantitative research with the type of correlational research. The population in this study were 60 students of class IX and a sample of 52 using random sampling technique. Data analysis used simple regression analysis and t-test. The results showed that 1) Social studies learning with historical material was carried out well where the learning outcomes of students had met the learning mastery standards. The learning process carried out by social studies teachers has met existing procedures such as lesson planning, implementation, and evaluation of learning 2) The attitude of nationalism based on local wisdom at MTs Al-Mansur Biangloe, Bantaeng Regency is in the high category. This shows that the attitude of nationalism is shown by an attitude of self-sacrifice, love for the homeland, pride as an Indonesian nation and unity and integrity. 3) There is a positive and significant influence between social studies learning and historical material on the attitude of nationalism based on local wisdom. That is, social studies learning with historical material can increase the attitude of nationalism based on local wisdom of students at MTs Al-Mansur Biangloe, Bantaeng Regency.

Developing a simulation-based training curriculum in transesophageal ultrasound with the use of the endobronchial ultrasound-endoscope

There is an increasing need to focus on how best to train respiratory physicians to perform EUS with bronchoscope-guided fine-needle aspiration biopsy (EUS-B-FNA). At current, training is mostly performed in the clinical environment under expert supervision; however, the advent of simulation-based education now provides a low-risk setting for novice trainees to learn and practice basic endosonography skills from identifying and understanding normal anatomy as well as pathology, maneuvering of endoscope, interpretation of images, and mastering of sampling techniques. In this descriptive educational paper, we used a six-step approach as a framework to describe the development of a structured training program combining EUS-B-FNA with the already well-established certification training program in endobronchial ultrasound transbronchial needle aspiration. This comprehensive training curriculum includes a theoretical course to achieve foundational knowledge, followed by simulation-based training until mastery standards are met, and supervised clinical apprenticeship. All steps should end with an objective assessment to achieve certification. This systematic development will hopefully encourage endosonography leaders and educators to collaborate and implement an evidence-based comprehensive endosonography curriculum that aims to provide the trainee with the essential EUS-B competencies to ensure that lung cancer patients are diagnosed and staged correctly.

Developing a simulation-based training curriculum in transesophageal ultrasound with the use of the endobronchial ultrasound-endoscope.

ABSTRACTThere is an increasing need to focus on how best to train respiratory physicians to perform EUS with bronchoscope-guided fine-needle aspiration biopsy (EUS-B-FNA). At current, training is mostly performed in the clinical environment under expert supervision; however, the advent of simulation-based education now provides a low-risk setting for novice trainees to learn and practice basic endosonography skills from identifying and understanding normal anatomy as well as pathology, maneuvering of endoscope, interpretation of images, and mastering of sampling techniques. In this descriptive educational paper, we used a six-step approach as a framework to describe the development of a structured training program combining EUS-B-FNA with the already well-established certification training program in endobronchial ultrasound transbronchial needle aspiration. This comprehensive training curriculum includes a theoretical course to achieve foundational knowledge, followed by simulation-based training until mastery standards are met, and supervised clinical apprenticeship. All steps should end with an objective assessment to achieve certification. This systematic development will hopefully encourage endosonography leaders and educators to collaborate and implement an evidence-based comprehensive endosonography curriculum that aims to provide the trainee with the essential EUS-B competencies to ensure that lung cancer patients are diagnosed and staged correctly.

Lesson Plan Needs for Arabic Essay Writing (Kitabah Hurrah) Using Common European Framework of Reference for Language

The lesson plan was a stage of instruction that regulates learning activities in the classroom that can have a positive impact on students' abilities in certain subjects. The purpose of this study was to identify the need for lesson plans in essay writing courses through the Common European Framework of Reference for Languages. The study used a qualitative approach through the descriptive method. Participants were students in the 3rd semester of 2020/2021 and lecturer. It was conducted in Arabic Language Education at the State University of Jakarta. Data is taken from the document, interview, and questionnaire. The data analysis process used data reduction, data display, and taking a conclusion. The results show that the lesson plans that have been made still have weaknesses and do not meet the Arabic language mastery standards from the Common European Framework of Reference for Languages ​​(CEFR) standards. However, the results of the study have implications for the level of understanding of the lecturer related to the concept of a lesson plan that must be designed to suit the needs of students, the current situation, and also students' understanding of each mastery of the material being studied.

THE USE OF GOOLE FORM AS A LEARNING EVALUATION TOOL IN ENGLISH LESSONS DURING PANDEMIC COVID-19

The Covid-19 pandemic has an impact on all aspects of life, including the provision of educational services. For the sake of the continuity of the education process and participating in breaking the chain of the spread of the covid-19 virus, learning and implementation of SMPN 168 Jakarta has been adjusted to the social distancing policy. Teaching activities are carried out at each student's home and through online media (online). This study aims to describe the implementation of online English learning activities, describe the implementation of online English assessment activities, and determine the effectiveness of the implementation of online English learning activities carried out in class VIII SMP 168 Jakarta during the Covid-19 pandemic. Use of the Google Form application to evaluate learning outcomes of English subjects. Through online activities, the average score of students' English is 77.31, and the proportion of students who exceed the KKM is 75.15%. SMP 168 Jakarta sets a minimum mastery standard of English at 75, and the proportion of students who pass the KKM exceeds 73%, so it can be concluded that online English learning at SMP 168 Jakarta is carried out effectively.

Mastery skill assessment in hepato-pancreato-biliary surgical ultrasound: It's a Matter of Entrustment

BackgroundA formative hepato-pancreato-biliary (HPB) ultrasound (US) skills practicum is offered annually to graduating HPB fellows, using entrustment assessments for open (IOUS) and laparoscopic (LAPUS) US. It is hypothesized that validity evidence will support the use of these assessments to determine if graduating fellows are well prepared to perform HPB US independently. MethodsExpert faculty were surveyed to set Mastery Entrustment standards for fellow performance. Standards were applied to fellow performances during two annual US skills practicums. Results11 faculty questionnaires were included. Mean Entrustment cut scores across all items were 4.9/5.0 and 4.8/5.0 and Global Entrustment cut scores were 5.0/5.0 and 4.8/5.0 for IOUS and LAPUS, respectively. 78.5% (29/37) fellows agreed to have their de-identified data evaluated. Mean fellow Entrustments (across all skills) were 4.1 (SD 0.6; 2.6–4.9) and 3.9 (SD 0.7; 2.7–5), while the Global Entrustments were 3.6 (SD 0.8; 2–5) and 3.5 (SD 1.0; 2–5) for IOUS and LAPUS, respectively. ConclusionsTwo cohorts of graduating HPB fellows are not meeting Mastery Standards for HPB US performance determined by a panel of expert faculty.

Multiple facets of learning a skill - Amalgamation of learning theories in cadaveric surgical skill lab.

Multiple facets of learning a skill - Amalgamation of learning theories in cadaveric surgical skill lab.

Psychometric Validation of Central Venous Catheter Insertion Mastery Learning Checklist Data and Decisions.

Resident physicians are expected to acquire competence at central venous catheter (CVC) insertion to a mastery standard. Valid competence decisions about resident physicians' CVC performance rely on reliable data and rigorous achievement standards. This study used data from 3 CVC simulation-based mastery learning studies involving internal medicine (IM) and emergency medicine (EM) residents to address 2 questions: What is the effectiveness of a CVC mastery learning education intervention? Are minimum passing standards (MPSs) set by faculty supported by item response theory (IRT) analyses? Pretraining and posttraining skills checklist data were drawn from 3 simulation-based mastery learning research reports about CVC internal jugular (IJ) and subclavian (SC) insertion skill acquisition. Residents were required to meet or exceed a posttest skills MPS. Generalized linear mixed effect models compared checklist performance from pre to postintervention. Minimum passing standards were determined by Angoff and Hofstee standard setting methods. Item response theory models were used for cut-score evaluation. Internal medicine and EM residents improved significantly on every IJ and SC checklist item after mastery learning. Item response theory analyses support the IJ and SC MPSs. Mastery learning is an effective education intervention to achieve clinical skill acquisition among IM and EM residents. Item response theory analyses reveal desirable measurement properties for the MPSs previously set by expert faculty panels. Item response theory analysis is useful for evaluating standards for mastery learning interventions. The CVC mastery learning curriculum, reliable outcome data, and high achievement standards together contribute to reaching valid decisions about the competence of resident physicians to perform the clinical procedure.

Using a Simulated Model and Mastery Learning Approach to Teach the Ultrasound-guided Serratus Anterior Plane Block to Emergency Medicine Residents: A Pilot Study.

The serratus anterior plane block (SAPB) is a safe, single-injection alternative for pain control in patients with rib fractures. This pilot study aims to teach the ultrasound-guided SAPB to emergency medicine (EM) residents using a mastery learning approach. A 19-item checklist was created and mastery was determined to be 17 of 19 items correct. This pass score was established using a Mastery Angoff standard-setting exercise with a group of EM experts. Learners participated in baseline testing on a simulated model and performance was assessed by two raters. Learners then watched an instructional video and participated in an individualized teaching session. Learners underwent deliberate practice followed by posttesting until mastery was achieved. Score differences in baseline testing and posttesting were analyzed using a paired t-test. Pre- and posttesting surveys were also completed by participants. Twenty-eight PGY-1 to -4 residents volunteered to participate in the study. The range of reported SAPBs seen previously was 0 to 5. The mean (±SD) number of items correct on the checklist for initial testing was 8.5 of 19 (±2.7), while the mean (±SD) final score was 18 of 19 (±0.6; p<0.001). All participants met mastery standards after the curriculum intervention. Median self-reported procedural confidence was 2 out of 5 on a 5-point Likert scale before the session and 5 out of 5 after the session (Z=-4.681, p<0.001). Using a mastery learning approach and simulated model, we were able to successfully train EM residents to perform the SAPB at a level of mastery and increase their overall confidence in executing this procedure.

Piloting the FIRE: A Novel Error Management Training Simulation Curriculum for Fasciotomy Instruction

Multiple studies have demonstrated poor performance of lower extremity fasciotomy (LEF), highlighted by missed and/or inadequately released compartments. Incorporating error management training (EMT) into surgical simulation has been promoted as a way to gain deeper understanding of procedural errors and overall performance. The purpose of this study was to evaluate LEF performance using a Fasciotomy Improvement through Recognition of Errors (FIRE) simulation training curriculum to train novice surgical trainees. A mastery learning-based EMT curriculum was developed, and surgical residents were enrolled and pretested with a multiple-choice question (MCQ) written test, and a simulated fasciotomy using a lower leg model. Each trainee then watched a 15-minute narrated presentation followed by 2 rounds of fasciotomy error recognition and management training exercises to a mastery standard. During each round, trainees performed hands-on assessment of unique premade fasciotomy leg models containing a variable number of procedural errors. They were required to identify and propose corrective action for all errors. Serial rounds of remediation were implemented until the mastery standard was attained on both error identification rounds. All trainees were post-tested with the same MCQ and another simulated fasciotomy. All 14 residents had minimal experience with only 0.3 ± 0.6 fasciotomies performed prior to instruction. There were 3 ± 1.6 missed or inadequately released compartments on the pretest. Residents examined 14 ± 2.5 legs, including 2 ± 2.5 legs during remediation to attain mastery. All residents demonstrated significant improvement following the FIRE of Error curriculum for the MCQ (57% ± 16% vs 78% ± 13%; p = 0.01; Cohen's d = 1.4), fasciotomy score (10 ± 7.1 vs 28 ± 1.9; p < 0.001; Cohen's d = 3.6), and achieving a complete fasciotomy (14% ± 36% vs 93% ± 27%; p < 0.001; Cohen's d = 2.5). Only a single cumulative compartment was missed on post-testing. Implementation of a mastery learning-based EMT curriculum for fasciotomy simulation training results in significant improvement in fasciotomy technique without reliance on repeated procedure performance nor clinical fasciotomy exposure. This curriculum is a highly effective option for surgical trainees lacking fasciotomy training during residency.

Notes From the Field: Automatic Item Generation, Standard Setting, and Learner Performance in Mastery Multiple-Choice Tests.

Mastery learning assessments have been described in simulation-based educational interventions; however, studies applying mastery learning to multiple-choice tests (MCTs) are lacking. This study investigates an approach to item generation and standard setting for mastery learning MCTs and evaluates the consistency of learner performance across sequential tests. Item models, variables for question stems, and mastery standards were established using a consensus process. Two test forms were created using item models. Tests were administered at two training programs. The primary outcome, the test-retest consistency of pass-fail decisions across versions of the test, was 94% (κ = .54). Decision-consistency classification was .85. Item-level consistency was 90% (κ = .77, SE = .03). These findings support the use of automatic item generation to create mastery MCTs which produce consistent pass-fail decisions. This technique broadens the range of assessment methods available to educators that require serial MCT testing, including mastery learning curricula.

The students’ mastery of fraction and its relation to the students’ abilities on its prerequisites

Fraction is one of the important topics in learning mathematics. However, it is still challenging for students. Lack of students’ mastery of the prerequisite topics might be one of the potential causes. For this reason, this study aims to: 1) Investigate whether students’ mastery of fractions and prerequisite topics in the 7th grade of a junior high school in Banda Aceh, Indonesia has reached the minimum learning mastery standard and 2) Investigate the relationship between students’ mastery of fractions and their abilities to understand the prerequisite topics in the 7th grade of a junior high school in Banda Aceh, Indonesia. This study used a quantitative approach. Data were collected from 50 students selected randomly from all of the 7th grade students in the junior high school. The results showed that the students’ mastery of fractions and their abilities to understand the prerequisites has achieved the minimum learning mastery standard. Also, there is a significant relationship between students’ fractions mastery and their abilities in the prerequisites. Finally, the lack of students’ mastery of the prerequisites resulted in students’ beliefs that the concept of fractions is difficult.

Application of "mastery learning" in medical education

Mastery learning is a competency-based mixed education method, which divides complex medical education contents into several teaching units according to the degree of difficulty and sets minimum passing standard for each teaching unit. Formative assessment was used to evaluate whether the learners had reached the mastery standard and finally mastered the knowledge through pass-continue or fail-repeat approach. The learners who had reached the mastery standard were usually able to use their knowledge more freely in clinical practice, and were able to accomplish more complex tasks through their own judgment with less supervision. Key words: Mastery learning; Competency; Deliberate practice; Feedback; Reflective discussion

Setting Mastery Standards In Hepato-Pancreato-Biliary Surgical Ultrasound

Presenter: Ellen Hagopian MD | Hackensack Meridian School of Medicine at Seton Hall University Background: A standardized curriculum in Hepato-Pancreato-Biliary (HPB) ultrasound (US), grounded in a Mastery skills framework, was developed in 2014 for HPB fellows in Americas Hepato-Pancreato-Biliary Association-Fellowship Council accredited programs. We previously developed two assessment tools for laparoscopic (LAPUS) and open US (IOUS) in HPB Surgery, each of which contains 14 items (skills) and a global performance item, using the O-Score entrustment scale, in which 1=“I had to do it”, 2= “I had to talk them through”, 3= “I had to prompt them from time to time”; 4=”I needed to be in the room just in case” and 5=“I did not need to be there.” A formative skills practicum utilizing these tools is offered annually to HPB fellows approaching graduation to provide individual performance feedback. The aim of this work was to establish Mastery standards for our two assessments, and to apply those standards to a sample of graduating fellows, providing consequential validity evidence within Messick’s framework. Methods: Standard setting: Faculty with expertise in HPB Surgery and HPB US from the membership of Americas Hepato-Pancreato-Biliary Association participated in standard setting exercises. Faculty were asked to rate each item and global US performance for both IOUS and LAPUS: What is the probability that a graduating HPB fellow, who is well-prepared for independent practice, is able to perform this skill? Please rate as a percentage. Item-level Mastery Angoff cut scores were determined as the mean rating across all judges. The overall Mastery Angoff cut score was determined as the mean cut score across all items. Mastery Angoff cut scores were converted to the 5-point entrustment scale (i.e., cut score*5). ellow performance: At the 2019 annual US skills practicum, fellow performances were evaluated using the assessment tools and the Mastery Angoff standards. This study met criteria for exemption by the University of Illinois-Chicago Institutional Review Board (2019-0265). Results: A total of 15 faculty were invited to participate, 12 of whom returned the questionnaire. To account for outliers, the highest and lowest faculty ratings were removed, thus standards were based on 10 faculty experts. For IOUS, the overall mean cut score across all items was 91.48% (SD 0.04) and for LAPUS, the overall mean cut score across all items was 89.96% (SD 0.04). The mean global IOUS performance cut score was 96.00% (SD 0.07), whereas the mean global LAPUS performance cut score was 87.80% (SD 0.14). (See Table) A total of 13 of 18 fellows who participated in the 2019 US skills practicum agreed to have their de-identified data evaluated. Overall, fellows did not meet standards in 11/14 IOUS items and in 13/14 LAPUS items. The mean global performance entrustment score was 3.62 (SD 0.65, range 2-4) in IOUS and 3.62 (SD 0.77, range 2-5) in LAPUS, which did not meet the entrustment standards of 4.80 and 4.39, respectively. Conclusion: AHPBA-FC HPB fellows prior to graduation are not meeting Mastery Standards for HPB US performance determined by a panel of expert faculty. A change in educational policy or curriculum may be needed to improve fellow performance.

Pretest Scores Uniquely Predict 1-Year-Delayed Performance in a Simulation-Based Mastery Course for Central Line Insertion.

Within simulation-based mastery learning (SBML) courses, there is inconsistent inclusion of learner pretesting, which requires considerable resources and is contrary to popular instructional frameworks. However, it may have several benefits, including its direct benefit as a form of deliberate practice and its facilitation of more learner-specific subsequent deliberate practice. We consider an unexplored potential benefit of pretesting: its ability to predict variable long-term learner performance. Twenty-seven residents completed an SBML course in central line insertion. Residents were tested on simulated central line insertion precourse, immediately postcourse, and after between 64 and 82 weeks. We analyzed pretest scores' prediction of delayed test scores, above and beyond prediction by program year, line insertion experiences in the interim, and immediate posttest scores. Pretest scores related strongly to delayed test scores (r = 0.59, P = 0.01; disattenuated ρ = 0.75). The number of independent central lines inserted also related to year-delayed test scores (r = 0.44, P = 0.02); other predictors did not discernibly relate. In a regression model jointly predicting delayed test scores, pretest was a significant predictor (β = 0.487, P = 0.011); number of independent insertions was not (β = 0.234, P = 0.198). This study suggests that pretests can play a major role in predicting learner variance in learning gains from SBML courses, thus facilitating more targeted refresher training. It also exposes a risk in SBML courses that learners who meet immediate mastery standards may be incorrectly assumed to have equal long-term learning gains.