Proficiency-based Progression Training Research Articles

International consensus panel for transurethral resection of bladder tumours metrics: assessment of face and content validity.

To develop performance metrics that objectively define a reference approach to a transurethral resection of bladder tumours (TURBT) procedure, seek consensus on the performance metrics from a group of international experts. The characterisation of a reference approach to a TURBT procedure was performed by identifying phases and explicitly defined procedure events (i.e., steps, errors, and critical errors). An international panel of experienced urologists (i.e., Delphi panel) was then assembled to scrutinise the metrics using a modified Delphi process. Based on the panel's feedback, the proposed metrics could be edited, supplemented, or deleted. A voting process was conducted to establish the consensus level on the metrics. Consensus was defined as the panel majority (i.e., >80%) agreeing that the metric definitions were accurate and acceptable. The number of metric units before and after the Delphi meeting were presented. A core metrics group (i.e., characterisation group) deconstructed the TURBT procedure. The reference case was identified as an elective TURBT on a male patient, diagnosed after full diagnostic evaluation with three or fewer bladder tumours of ≤3 cm. The characterisation group identified six procedure phases, 60 procedure steps, 43 errors, and 40 critical errors. The metrics were presented to the Delphi panel which included 15 experts from six countries. After the Delphi, six procedure phases, 63 procedure steps, 47 errors, and 41 critical errors were identified. The Delphi panel achieved a 100% consensus. Performance metrics to characterise a reference approach to TURBT were developed and an international panel of experts reached 100% consensus on them. This consensus supports their face and content validity. The metrics can now be used for a proficiency-based progression training curriculum for TURBT.

Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial.

To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity. GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations. In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves. The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments. Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.

PE047 - Single trainee experience of the ERUS proficiency-based progression training curriculum in an accredited ERUS Host centre

PE047 - Single trainee experience of the ERUS proficiency-based progression training curriculum in an accredited ERUS Host centre

Proficiency-based progression intern training to reduce critical blood sampling errors including ‘wrong blood in tube’

Background: Blood sampling errors including ‘wrong blood in tube’ (WBIT) may have adverse effects on clinical outcomes. WBIT errors occur when the blood sample in the tube is not that of the patient identified on the label. This study aims to determine the effect of proficiency-based progression (PBP) training in phlebotomy on the rate of blood sampling errors (including WBIT). Methods: A non-randomised controlled trial compared the blood sampling error rate of 43 historical controls who had not undergone PBP training in 2016 to 44 PBP trained interventional groups in 2017. In 2018, the PBP training programme was implemented and the blood sampling error rate of 46 interns was compared to the 43 historical controls in 2016. Data analysis was performed using logistic regression analysis adjusting for sample timing. Results: In 2016, 43 interns had a total blood sample error rate of 2.4%, compared to 44 interns in 2017, who had error rate of 1.2% (adjusted OR=0.50, 95% CI 0.36-0.70; &lt;0.01). In 2018, 46 interns had an error rate of 1.9% (adjusted OR=0.89, 95% CI 0.65-1.21; p=0.46) when compared to the 2016 historical controls. There were three WBITs in 2016, three WBITs in 2017 and five WBITs in 2018. Conclusions: The study demonstrates that PBP training in phlebotomy has the potential to reduce blood sampling errors. Trial registration number: NCT03577561

Proficiency-based progression training: implementing a novel approach to training for epidural analgesia in labour

BackgroundEpidural insertion is a challenging anaesthetic procedural skill to learn and may require up to 75 attempts to achieve competency. Proficiency-based progression (PbP) training based on unambiguously defined metrics was associated with a 53% reduction in epidural failure rate. The aim of this observational study was to examine the feasibility of implementation of innovative PbP training for labour epidural insertion performed by novices in a busy tertiary hospital. MethodsAll trainees who were scheduled to commence their obstetric anaesthesia training were invited to participate. Novices undertook intensive PbP training with one-to-one supervision by an anaesthetist trained in PbP. Trainees proceeded to the clinical phase only after attaining the pre-defined proficiency benchmark. All subsequent attempts at labour epidural catheter placement were evaluated. ResultsAll 12 novice trainees who were scheduled for their initial exposure to obstetric anaesthesia completed PbP training in epidural catheter insertion successfully. The average duration of the training courses was 70 (SD 11) min. Trainee characteristics were broadly similar. They performed a total of 180 labour epidural catheter placements with an overall epidural failure rate of 12.2% (22/180). The proportion of supervisor takeover was 6% (11/179). The incidence of complications was 4% (8/180) and difficulty in epidural catheter insertion due to patient factors was 16% (29/180). Patient satisfaction rates were 80% (satisfied or very satisfied), with 20% unsatisfied with their experience of epidural insertion. ConclusionIn our experience, PbP training in epidural placement is feasible within existing departmental resources in a busy tertiary teaching hospital setting.

Proficiency-based progression intern training to reduce critical blood sampling errors including ‘wrong blood in tube’

Background: Blood sampling errors including ‘wrong blood in tube’ (WBIT) may have adverse effects on clinical outcomes. WBIT errors occur when the blood sample in the tube is not that of the patient identified on the label. This study aims to determine the effect of proficiency-based progression (PBP) training in phlebotomy on the rate of blood sampling errors (including WBIT). Methods: A non-randomised controlled trial compared the blood sampling error rate of 43 historical controls who had not undergone PBP training in 2016 to 44 PBP trained interventional groups in 2017. In 2018, the PBP training programme was implemented and the blood sampling error rate of 46 interns was compared to the 43 historical controls in 2016. Data analysis was performed using logistic regression analysis adjusting for sample timing. Results: In 2016, 43 interns had a total blood sample error rate of 2.4%, compared to 44 interns in 2017, who had error rate of 1.2% (adjusted OR=0.50, 95% CI 0.36-0.70; &lt;0.01). In 2018, 46 interns had an error rate of 1.9% (adjusted OR=0.89, 95% CI 0.65-1.21; p=0.46) when compared to the 2016 historical controls. There were three WBITs in 2016, three WBITs in 2017 and five WBITs in 2018. Conclusions: The study demonstrates that PBP training in phlebotomy has the potential to reduce blood sampling errors. Trial registration number: NCT03577561

A Proficiency-Based Progression Simulation Training Curriculum to Acquire the Skills Needed in Performing Arthroscopic Bankart and Rotator Cuff Repairs—Implementation and Impact

To investigate the impact of a proficiency-based progression (PBP) curriculum employed to teach trainees in the skills needed to demonstrate proficiency for an arthroscopic Bankart repair (ABR) and an arthroscopic rotator cuff repair (ARCR) by objectively comparing pre- and immediate postcourse performances. In a prospective study, 16 arthroscopy/sports medicine fellows and 2 senior residents (complete group: N= 18) were randomly assigned to perform a precourse cadaveric ABR (Bankart subgroup: N= 6), ARCR (cuff subgroup: N= 6), or basic skills on a shoulder simulator (N= 6). After completing a PBP training curriculum, all 18 registrants performed both an ABR and ARCR scored in real time by trained raters using previously validated metrics. The Bankart subgroup made 58% fewer objectively assessed errors at the completion of the course than at baseline (P= .004, confidence interval -1.449 to-0.281), and performance variability was substantially reduced (standard deviation= 5.89 vs 2.81). The cuff subgroup also made 58% fewer errors (P= .001, confidence interval -1.376 to 0.382) and showed a similar reduction in performance variability (standard deviation= 5.42 vs 2.1). Only one subject's precourse baseline performance met the proficiency benchmark compared with 89% and 83% of the all registrants on the final ABR and ARCR cadaveric assessments, respectively. The results of this study reject the null hypothesis. They demonstrate that the implementation of a PBP simulation curriculum to train the skills necessary to perform arthroscopic Bankart and rotator cuff repairs results in a large and statistically significant improvement in the trainee's ability to meet the 2 related performance benchmarks. Proficiency was demonstrated by 89% and 83% of the trainees for an ABR and an ARCR, respectively, in a two- and one-half day course. Surgical training employing a PBP curriculum is efficient, effective, and has the potential to improve patient safety.

A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes.

To analyze all published prospective, randomized, and blinded clinical studies on the proficiency-based progression (PBP) training using objective performance metrics. The benefit of PBP methodology to learning clinical skills in comparison to conventional training is not settled. Search of PubMed, Cochrane library's Central, EMBASE, MEDLINE, and Scopus databases, from inception to 1st March 2020. Two independent reviewers extracted the data. The Medical Education Research Study Quality Instrument (MERSQI) was used to assess the methodological quality of included studies. Results were pooled using biased corrected standardized mean difference and ratio-of-means. Summary effects were evaluated using a series of fixed and random effects models. The primary outcome was the number of procedural errors performed comparing PBP and non-PBP-based training pathways. Secondary outcomes were the number of procedural steps completed and the time to complete the task/procedure. From the initial pool of 468 studies, 12 randomized clinical studies with a total of 239 participants were included in the analysis. In comparison to the non-PBP training, ratio-of-means results showed that PBP training reduced the number of performance errors by 60% (P < 0.001) and procedural time by 15% (P = 0.003) and increased the number of steps performed by 47% (P < 0.001). Our systematic review and meta-analysis confirms that PBP training in comparison to conventional or quality assured training improved trainees' performances, by decreasing procedural errors and procedural time, while increasing the number of correct steps taken when compared to standard simulation-based training.

Surgical training for anatomical endoscopic enucleation of the prostate.

Anatomical endoscopic enucleation of the prostate has been proposed as a potentially superior benign prostatic hyperplasia surgery than conventional transurethral resection of prostate. However, the learning curve of the procedure is steep, hence limiting its generalisability worldwide. In order to overcome the learning curve, a proper surgical training is extremely important. This review article discussed about various aspects of surgical training in anatomical endoscopic enucleation of the prostate. In summary, no matter what surgical technique or energy modality you use, the principle of anatomical enucleation should be followed. When one starts to perform prostate enucleation, a 50 to 80g prostate appears to be the 'best case' to begin with. Mentorship is extremely important to shorten the learning curve and to prevent drastic complications from the procedure. A proficiency-based progression training programme with the use of simulation and training models should be the best way to teach and learn about prostate enucleation. Enucleation ratio efficacy is the preferred measure for assessing skill level and learning curve of prostate enucleation. Morcellation efficiency is commonly used to assess morcellation performance, but the importance of safety rather than efficiency must be emphasised.

Proficiency-based progression training: an ‘end to end’ model for decreasing error applied to achievement of effective epidural analgesia during labour: a randomised control study

BackgroundTraining procedural skills using proficiency-based progression (PBP) methodology has consistently resulted in error reduction. We hypothesised that implementation of metric-based PBP training and a valid assessment tool would decrease the...

AO international consensus panel for metrics on a closed reduction and fixation of a 31A2 pertrochanteric fracture

BackgroundThe foundations of an effective and evidence-based training program are the metrics, which characterize optimal performance. PurposesTo develop, operationally define, and seek consensus from procedure experts on the metrics that best characterize a reference approach to the performance of a closed reduction and internal fixation of a 31A2 unstable pertrochanteric fracture with a cephalomedullary nail with distal locking through the proximal guide. MethodsA Metrics Group consisting of 3 senior orthopaedic surgeons, a surgeon/medical scientist, an education expert and a behavioural scientist deconstructed the performance of the selected fixation procedure and defined performance metrics. At a modified Delphi meeting, 32 senior orthopaedic and trauma surgeons from 18 countries critiqued these metrics and their operational definitions before reaching consensus. ResultsInitially performance metrics consisting of 14 Phases with 62 Steps, 84 errors and 20 Sentinel errors were identified that characterize the safe and effective performance of the procedure. During the Delphi panel meeting these were modified and consensus was reached on 15 Phases (1 added, p = 0.967)) with 75 Steps (14 added and 1 deleted; p = 0.028), 88 errors (10 added and 6 deleted; p = 0.47), and 28 Sentinel errors (8 added; p = 0.107). Pre and Post Delphi characterizations were highly correlated (r = 0.81–0.94). ConclusionsSurgical procedures can be broken down into constituent, essential, and elemental tasks necessary for the safe and effective completion of a reference approach to a specified procedure. Procedure experts from 18 countries reached consensus on performance metrics for the fixation procedure. This metric-based characterization should form the basis of more quantitative validation studies to guide the construction of a proficiency-based progression training curriculum.

A Proficiency-Based Progression Training Curriculum Coupled With a Model Simulator Results in the Acquisition of a Superior Arthroscopic Bankart Skill Set

To determine the effectiveness of proficiency-based progression (PBP) training using simulation both compared with the same training without proficiency requirements and compared with a traditional resident course for learning to perform an arthroscopic Bankart repair (ABR). In a prospective, randomized, blinded study, 44 postgraduate year 4 or 5 orthopaedic residents from 21 Accreditation Council for Graduate Medical Education-approved US orthopaedic residency programs were randomly assigned to 1 of 3 skills training protocols for learning to perform an ABR: group A, traditional (routine Arthroscopy Association of North America Resident Course) (control, n= 14); group B, simulator (modified curriculum adding a shoulder model simulator) (n= 14); or group C, PBP (PBP plus the simulator) (n= 16). At the completion of training, all subjects performed a 3 suture anchor ABR on a cadaveric shoulder, which was videotaped and scored in blinded fashion with the use of previously validated metrics. The PBP-trained group (group C) made 56% fewer objectively assessed errors than the traditionally trained group (group A) (P= .011) and 41% fewer than group B (P= .049) (both comparisons were statistically significant). The proficiency benchmark was achieved on the final repair by 68.7% of participants in group C compared with 36.7% in group B and 28.6% in group A. When compared with group A, group B participants were 1.4 times, group C participants were 5.5 times, and group C(PBP) participants (who metall intermediate proficiency benchmarks) were 7.5 times as likely to achieve the final proficiency benchmark. A PBP training curriculum and protocol coupled with the use of a shoulder model simulator and previously validated metrics produces a superior arthroscopic Bankart skill set when compared with traditional and simulator-enhanced training methods. Surgical training combining PBP and a simulator is efficient and effective. Patient safety could be improved if surgical trainees participated in PBP training using a simulator before treating surgical patients.