Certification Examination Research Articles

Revisiting the relationship between examination pass rate and eligibility pathway among candidates for pharmacist board certification

BackgroundPharmacists with specialized knowledge and skills are able to seek board certification from the Board of Pharmacy Specialties (BPS). In 2018, BPS conducted research to evaluate the relationship between eligibility pathways (i.e., completion of a postgraduate year 2 [PGY-2] specialty residency, completion of a postgraduate year 1 [PGY-1] residency plus 2 years of practice experience, or 4 years of practice experience) and certification examination pass rate. The study found statistically significant differences in pass rate based on eligibility criteria across all 6 specialties studied. There was a trend for higher pass rates in cohorts of examinees eligible for board certification based on completion of postgraduate residency training. ObjectivesThis study aimed to compare examination pass rates among different eligibility cohorts for board certification in recognized pharmacy practice specialties and compare contemporary findings with previously published findings. DesignThis cross-sectional study was conducted on data retrieved from BPS certification applications and examination administration records. Setting and participantsExaminees in the United States and Canada for 9 BPS certification programs. Outcome measuresA chi-square analysis was used to identify whether there were differences in pass rate among eligibility pathway cohorts. ResultsA total of 14,894 examinees met inclusion criteria; 6312 (42.4%) of the 14,894 examinees analyzed were eligible via practice experience, 5768 (38.7%) were eligible via PGY-1 completion, and 2814 (18.9%) were eligible via PGY-2 completion. A statistically significant difference was found for the relationship between pass rate and eligibility pathway for 8 of 9 BPS certification examinations analyzed (significant: ambulatory care pharmacy, critical care pharmacy, cardiology pharmacy, infectious diseases pharmacy, oncology pharmacy, psychiatric pharmacy, pediatric pharmacy, pharmacotherapy; not significant: solid organ transplantation pharmacy). Post hoc analyses showed that, in most cases, the PGY-2 eligibility cohort outperformed the PGY-1 eligibility cohort, which in turn outperformed the practice experience pathway cohort. ConclusionAnalysis of contemporary BPS certification examination administration data yields statistically significant differences among the pass rates by eligibility pathway, replicating previous findings and expanding the scope of the analysis.

Comparison of performance data between emergency medicine 1-3 and 1-4 program formats.

This study compares performance data from physicians completing 3-year versus 4-year emergency medicine residency training programs. Currently, there are 2 training formats and little is known about objective performance differences. This was a retrospective cross-sectional analysis of emergency residents and physicians. Multiple analyses were conducted comparing physicians' performances, including Accreditation Council of Graduate Medical Education Milestones and American Board of Emergency Medicine In-training Examination (ITE), Qualifying Examination (QE), Oral Certification Examination (OCE), and program extensions from 3-year and 4-year residency programs. Some confounding variables were not or could not be considered, such as rationale for medical students to choose one format over another, as well as application and final match rates. Milestone scores are higher for emergency medicine 3 residents in 1-3 programs (3.51) versus emergency medicine 3 residents in 1-4 programs (3.07; P<0.001, d=1.47) and highest for emergency medicine 4 residents (3.67). There was no significant difference in program extension rates (emergency medicine 1-3, 8.1%; emergency medicine 1-4, 9.6%; P=0.05, ω=0.02). ITE scores were higher for emergency medicine 1, 2, and 3 residents from 1-3 programs and emergency medicine 4 residents from 1-4 programs scored highest. Mean QE score was slightly higher for emergency 1-3 physicians (83.55 vs 83.00; P<0.01, d=0.10). QE pass rate was higher for emergency 1-3 physicians (93.1% vs 90.8%; P<0.001, ω=0.08). Mean OCE score was slightly higher for emergency 1-4 physicians (5.67 vs 5.65; P=0.03, d=-0.07) but did not reach a priori statistical significance (α<0.01). OCE pass rate was also slightly higher for emergency 1-4 physicians (96.9% vs 95.5%; P=0.06, ω=-0.07) but also non-significant. These results suggest that although performance measures demonstrate small differences between physicians from emergency medicine 1-3 and 1-4 programs, these differences are limited in their ability to make causal claims about performance on the basis of program format alone.

Identifying and Addressing Item Bias in the General Pediatrics Certifying Examination.

This article describes a 2-phase process implemented by the American Board of Pediatrics in 2021 to investigate and remove potential bias on its General Pediatrics Certifying Examination at the item (question) level based on gender or race and ethnicity. Phase 1 used a statistical technique known as differential item functioning (DIF) analysis to identify items in which 1 subgroup of the population outperformed another subgroup after controlling for overall knowledge level. Phase 2 involved a review of items flagged for statistical DIF by the American Board of Pediatrics' Bias and Sensitivity Review (BSR) panel, a diverse group of 12 voluntary subject matter experts tasked with identifying language or other characteristics of those items that may have contributed to the observed performance differences. Results indicated that no items on the 2021 examination were flagged for DIF by gender and 2.8% of the items were flagged for DIF by race and ethnicity. Of those items flagged for race and ethnicity, 14.3% (0.4% of total items administered) were judged by the BSR panel to contain biased language that may have undermined what the item was intending to measure and were therefore recommended to be removed from operational scoring. In addition to removing potentially biased items from the current pool of items, we hope that repeating the DIF/BSR process after each examination cycle will increase our understanding of how language nuances and other characteristics impact item performance so that we can improve our guidelines for developing future items.

Scaled Performance on the Vascular Qualifying Examination Does Not Correlate With Vascular Certifying Examination First Attempt Pass in a National Cohort of Residents and Fellows

Specialty board certification is an important indicator of surgeon competence and necessary for residency and fellowship programs to maintain accreditation with the Accreditation Council for Graduate Medical Education. Thus, identification of trainees at risk of board certification failure is important for both vascular surgery graduates and their parent program. Passing the written Vascular Qualifying Examination (VQE) is a prerequisite for eligibility to take the Vascular Certifying Examination (VCE). However, the association between performance on the VQE and ability to pass the VCE has not been examined. We sought to examine the correlation of VQE performance on first-time pass achievement of VCE. We utilized American Board of Surgery data to examine predictive associations of a national longitudinal cohort of all vascular surgery integrated residents (VIR) and fellows (VSF) who completed the VQE and VCE from 2016 through 2021. Descriptive statistics were used to examine trends. Chi-squared tests were used to examine proportional group differences. Predictive associations between VQE scaled score performance and first-time pass achievement on subsequent VCE attempts were examined using logistic regression. VQE scaled score performance and VCE pass rates were obtained for all VIR and VSF (n = 579; 176 residents and 403 fellows) from 151 programs (47 residency and 104 fellowship) during the study period. First-attempt pass rate for the VQE was 94% in the study cohort, while first-attempt pass rate for the VCE was 93%. There were no significant differences in pass-fail rates between VIR and VSF candidates taking the VQE (P = .133) or the VCE (P = .501), respectively. Trainee scaled score performance on VQE was not predictive of subsequent first-attempt pass achievement on VCE (odds ratio, 1.00; 95% confidence interval, 1.00-1.01; P = .077) (Figure). Stratified subgroup analyses by VIR and VSF cohorts also showed no significant predictive first-attempt VCE pass achievement (P = .125 and P = .171, respectively). In this national cohort of vascular trainees taking the VQE and VCE, VQE performance was not predictive of subsequent VCE first-time pass achievement. Although this finding highlights the necessity to identify other measures of trainee competence in order to indicate preparedness of trainees for board certification examinations, it is also reflective of the ability to assess distinctive competencies between the two examinations, with VQE focused on medical knowledge, and VCE on clinical judgement and interpersonal communication skills. This ensures certification of well-rounded vascular surgeons and maintenance of high-quality standards within our specialty.

Update on Educational Resources and Evaluation Tools for Orthopaedic Surgery Residents.

Innovations in orthopaedic resident educational resources and evaluation tools are essential to ensuring appropriate training and ultimately the graduation of competent orthopaedic surgeons. In recent years, there have been several advancements in comprehensive educational platforms within orthopaedic surgery. Orthobullets PASS, Journal of Bone and Joint Surgery Clinical Classroom, and American Academy of Orthopaedic Surgery Resident Orthopaedic Core Knowledge each have their own unique advantages in preparation for the Orthopaedic In-Training Examination and American Board of Orthopaedic Surgery board certification examinations. In addition, the Accreditation Council for Graduate Medical Education Milestones 2.0 and the American Board of Orthopaedic Surgery Knowledge Skills Behavior program each provide objective assessment of resident core competencies. Understanding and using these new platforms will help orthopaedic residents, faculty, residency programs, and program leadership to best train and evaluate their residents.

ASSESSMENT OF BRAIN BIOELECTRICAL ACTIVITY IN CLINICALLY HEALTHY FLIGHT PERSONNEL AND FLIGHT TRAINING CANDIDATES FOLLOWING COVID-19 INFECTION

Introduction: Electroneurophysiological tests, together with tests to evaluate structures of the central nervous system (CNS), are in neurological diagnostics among the basic additional specialized examinations for diagnostic, therapeutic and certification purposes. The electroencephalographic (EEG) test is one of the methods of elelectroneurophysiological evaluation of the CNS. EEG is a non-invasive objective test of the bioelectrical activity of the brain. Objective: To investigate whether the course of COVID-19 with clinical mild/moderate respiratory tract involvement, in non-hospitalized patients, may affect brain bioelectrical activity. Methods: Eighty-five subjects were selected for evaluation of EEG recordings from clinically initially healthy flight personnel and flight training candidates. Of all the male and female subjects, between 04 January 2021 to 31 January 2022, as part of periodic and qualifying certification examinations at the Military Institute of Aviation Medicine, for the purposes of the Regional Military Aviation and Medical Commission. The selected group consisted of individuals aged 14-60 (average age: 27 years and 6 months), which included 14 women (average age: 24 years and 2 months) and 71 men (average age 29 years and 3.5 months). The subjects were ill with COVID-19 (non-hospitalized) for approximately 3-4 weeks, during the period; June 2020 – December 2021. Routinely made recordings of bioelectrical brain activity with the Grass Technologies EEG TWin 4.5.3 system were visually evaluated. EEG examinations were performed in the supine position, in the before noon hours, in a darkened and acoustically isolated laboratory. Before the study, each person declared min. 6-8 hours of sleep and eating a meal (breakfast). Except for the flight training candidates and one person from flight personnel, all other subjects declared having received one and 29 subjects two SARS-CoV-2 vaccinations, prior to contracting COVID-19. Some subjects had one further revaccination after recovery from the disease. EEG examinations were performed between one and fourteen months after recovery from COVID-19. The mean time between recovery and EEG examination was 6.5 (6.47) months. Results: EEG recordings within normal limits (normal) were found in 79 subjects, while six subjects (7.05%) had abnormal brain bioelectrical activity recordings. Discussion and Conclusion: Infection with SARS-CoV-2 with clinical mild/moderate respiratory tract involvement by initially healthy individuals may adversely affect spontaneous brain bioelectrical activity in some individuals.

Position Paper: SGIM Sex- and Gender-Based Women's Health Core Competencies.

Women's health care has evolved significantly since it was first acknowledged as an integral part of internal medicine training more than two decades ago. To update and clarify core competencies in sex- and gender-based women's health for general internists, the Society of General Internal Medicine (SGIM) Women and Medicine Commission prepared the following Position Paper, approved by the SGIM council in 2023. Competencies were developed using several sources, including the 2021 Accreditation Council for Graduate Medical Education Program Requirements for Internal Medicine and the 2023 American Board of Internal Medicine Certification Examination Blueprint. These competencies are relevant to the care of patients who identify as women, as well as gender-diverse individuals to whom these principles apply. They align with pivotal advances in women's health and acknowledge the changing context of patients' lives, reaffirming the role of general internal medicine physicians in providing comprehensive care to women.

Shared Experiences

Shared Experiences

A comparative analysis of graduate preparedness for a career in General Internal Medicine before and after national subspecialty recognition to inform curricular changes: have we met the mark?

A survey of General Internal Medicine (GIM) graduates published in 2006 revealed large training gaps that informed the development of the first national GIM objectives of training in 2010. The first recognized GIM certification examination was written by candidates in 2014. The landscape is again changing with the introduction in 2019 of competency-by-design (CBD) to GIM training. This study aims to examine pre-existing and emerging training gaps with standardization of GIM curricula and identify new training needs to inform CBD curricula. GIM graduates from all 16 Canadian programs from 2014 -2019 were emailed a survey modeled after the original study published in 2006. Graduates were asked about their preparedness and importance ratings for various elements of practice. Many of the previously identified gaps (difference between importance and preparedness ratings) have been resolved in specific clinical areas (obstetrical and perioperative medicine) and skills (exercise stress testing) although some still require ongoing work in areas such as substance use disorders. Importantly, gaps still exist in preparedness for some intrinsic roles (e.g. managerial skills). The development of a national GIM curriculum has helped close some educational gaps but some still exist. Our study provides data needed to meet the evolving needs of our graduates.

Early validity and reliability evidence for the American Board of Emergency Medicine Virtual Oral Examination.

The American Board of Emergency Medicine (ABEM) in-person Oral Certification Examination (OCE) was halted abruptly in 2020 due to the COVID-19 pandemic. The OCE was reconfigured to be administered in a virtual environment starting in December 2020. The purpose of this investigation was to determine whether there was sufficient validity and reliability evidence to support the continued use of the ABEM virtual Oral Examination (VOE) for certification decisions. This retrospective, descriptive study used multiple data sources to provide validity evidence and reliability data. Validity evidence focused on test content, response processes, internal structure (e.g., internal consistency and item response theory), and the consequences of testing. A multifaceted Rasch reliability coefficient was used to measure reliability. Study data were from two 2019 in-person OCEs and the first four VOE administrations. There were 2279 physicians who took the 2019 in-person OCE examination and 2153 physicians who took the VOE during the study period. Among the OCE group, 92.0% agreed or strongly agreed that the cases on the examination were cases that an emergency physician should be expected to see; 91.1% of the VOE group agreed or strongly agreed. A similar pattern of responses given to a question about whether the cases on the examination were cases that they had seen. Additional evidence of validity was obtained by the use of the EM Model, the process for case development, the use of think-aloud protocols, and similar test performance patterns (e.g., pass rates). For reliability, the Rasch reliability coefficients for the OCE and the VOE during the study period were all >0.90. There was substantial validity evidence and reliability to support ongoing use of the ABEM VOE to make confident and defensible certification decisions.

Fear, health impacts, and life delays: residents' certification exam year experience.

Residency training is associated with risks of burnout and impaired well-being. This may be due to multiple factors, including navigating various transitions. Chief among these is the transition to independent practice which, in Canada, involves a certification exam administered by the Royal College of Physicians and Surgeons of Canada or the College of Family Physicians of Canada. This qualitative study explored the experience of residents in their examination year, including residents impacted by pandemic-related examination postponment, to understand how these experiences may impact residents' well-being. Qualitative description methodology was used for this study. Participants were residents and physicians in independent practice from McMaster University and the University of Toronto. In depth, semi-structured, one-on-one interviews were conducted by one of the investigators. Each was transcribed, reviewed, and coded by two members of the investigating team. Five themes were identified. Examinations were perceived to be a significant stressor, and the extent of preparation involved was viewed as a threat to one's physical and mental well-being. Participants identified a culture of fear surrounding the exam, as well as a perception that exam preparation requires significant sacrifice which can exacerbate the impacts of the exam year. Personal and professional supports were identified as important protective factors. This study has identified unique challenges in the examination year, and its impact on the well-being of residents immediately before they enter independent practice. Residents also experienced significant learning and a sense of accomplishment through their preparation for the examination. The COVID-19 pandemic had a unique impact on one cohort of residents. This should prompt medical education institutions to examine the support provided to residents, the culture surrounding certification examinations, and mitigation strategies for future examination disruptions.

Comprehension of fundamental knowledge about pediatric endoscopic surgery: a cross-sectional study in Japan.

Pediatric endoscopic surgery has become an alternative to conventional techniques with the development of medical equipment. However, there is no formal, standardized curriculum for pediatric endoscopic surgery, and its requirement remains elusive. The purpose of this study is to determine the baseline knowledge of pediatric surgeons that is considered essential for pediatric laparoscopic and thoracoscopic surgery. Sixteen web-based multiple-choice questions based on the fundamental cognitive knowledge of pediatric endoscopic surgery were administered. The questions were created based on the fields covered by the Fundamentals of Laparoscopic Surgery ™ (FLS) certification examination blueprints and eight specific diseases of pediatric surgery. Pediatric surgeons and pediatric surgical trainees participated in this study voluntarily. A total of 122 surgeons participated through the Japanese Society of Pediatric Surgeons. The response rate was 95% (122/128). The total mean examination score of all participants was 79.4% (77.3-81.4%). There were no significant differences in total scores between the board-certified pediatric surgeons without an endoscopic surgical skill qualification and the non-board-certified pediatric surgeons (80.4% vs. 77.1%, p = 0.12). The endoscopic surgical skill-qualified surgeons had significantly higher percentages of correct responses in specific subjects than board-certified pediatric surgeons and surgeons without pediatric board certification (94.3% vs. 82.9%, p = 0.02; 94.3% vs. 77.5%, p = 0.0002). The FLS original subjects' scores were not significantly different among them. The mean score of surgeons who had experienced more than 200 cases of endoscopic surgery, including adult cases, was 83.2% (80.4-85.9%). A knowledge gap exists between surgeons, board-certified pediatric surgeons, and endoscopic surgical skill-qualified surgeons in Japan. In the field of pediatric surgery, an effective formal curriculum, such as FLS, is required to help address this vast knowledge gap for the safe conduct of endoscopic surgeries.

The Barriers in Online Assessment for Professional Certification: A Case Study In Indonesia

Certification exams conducted online must be of the same quality as those conducted on-site. To achieve this, it is necessary to conduct a study in which assessors need to identify obstacles in implementing online certification examinations which have been implemented during the Covid-19 pandemic. The study aims to analyze the perspective of the assessors to identify the barrier factors, understand the interrelation among them, and propose improvements to the online application process. The method applied was the MICMAC (Matrix of Cross-impact Multiplications Applied to Classification), a structural model, to analyze the variables based on their influence and dependence driving force. A case study was conducted on the Indonesia Hydraulic Engineer Association. The data was collected from a focus-group discussion with the assessors. The result revealed that the barrier factors consist of technical aspects, such as internet connection delay, hackers and viruses, and unverified e-document; and social aspects, such as questioning skills limitation, non-verbal communication skills limitation, and environment distraction. The study found that internet connection delay and questioning skill limitations are the barriers with the highest influence beyond other barriers in the online assessment process. Also, the questioning skills limitation factor has the highest dependency on other barriers. Hence, it can be proposed that the curricula of the assessors’ training and/or certification program, should be incorporated with the questioning skill in an online assessment environment.

Academic Achievement and Competency in Rural and Urban Family Medicine Residents.

The quality of training in rural family medicine (FM) residencies has been questioned. Our objective was to assess differences in academic performance between rural and urban FM residencies. We used American Board of Family Medicine (ABFM) data from 2016-2018 residency graduates. Medical knowledge was measured by the ABFM in-training examination (ITE) and Family Medicine Certification Examination (FMCE). The milestones included 22 items across six core competencies. We measured whether residents met expectations on each milestone at each assessment. Multilevel regression models determined associations between resident and residency characteristics milestones met at graduation, FMCE score, and failure. Our final sample was 11,790 graduates. First-year ITE scores were similar between rural and urban residents. Rural residents passed their initial FMCE at a lower rate than urban residents (96.2% vs 98.9%) with the gap closing upon later attempts (98.8% vs 99.8%). Being in a rural program was not associated with a difference in FMCE score but was associated with higher odds of failure. Interactions between program type and year were not significant, indicating equal growth in knowledge. The proportions of rural vs urban residents who met all milestones and each of six core competencies were similar early in residency but diverged over time with fewer rural residents meeting all expectations. We found small, but persistent differences in measures of academic performance between rural- and urban-trained FM residents. The implications of these findings in judging the quality of rural programs are much less clear and warrant further study, including their impact on rural patient outcomes and community health.

학문적 이론의 실용성에 대한 학습자들의 인식 조사

This study explores Korean college students’ perceptions of the practicality of the Chomsky’s theory, focusing on generative grammar. Over 60 years the acknowledgement of the theory has been so firm in Korea that it has been the test range of the syntax section of the English Teacher Certification Examinations. This study investigates the students’ perceptions of the practical use of phrase structure(PS) rules and tree diagrams. There were 30 participants of the survey on their perceptions of tree diagram practice and its practicality. They also took two TOEIC L/C tests, one before tree diagram practice and the other after the practice. The results show that there is little difference between the two means of the TOEIC tests. In the survey results, the participants gave a positive reaction to the usefulness of tree diagram practice in the degrees of understanding the English structure and English sentences, and the utilization of chunking practice. These results indicate that the strong points of a theory strongly affect the participants’ perceptions of the practicality of the theory.

Expansion of MyDispense: A Descriptive Report of Simulation Activities and Assessment in a Certified Pharmacy Technician Training Program.

Yale New Haven Health (YNHH) implemented a pharmacy technician training program in 2016. The curriculum includes 14 weeks of combined didactic and simulation hours (280 h in total), followed by 360 h of experiential learning. MyDispense, an online pharmacy simulation, allows students to develop and practice their dispensing skills in a safe environment with minimal consequences for mistakes. We describe a novel innovation, expanding the functionality of MyDispense to the training of pharmacy technicians. Technician training coordinator, supervisor, faculty members with experience in MyDispense, and experiential pharmacy students created cases within the MyDispense software that were targeted towards pharmacy technician activities. Activities were aligned with current American Society of Health-System Pharmacists (ASHP)-Accreditation Council for Pharmacy Education (ACPE) Accreditation Standards for pharmacy technician education and training programs. A total of 14 cases were developed to be utilized in student technician training, and account for approximately 14 h of simulation. MyDispense is an innovative software that could allow students to access and complete exercises, and to continue developing dispensing skills in a safe, remote environment. We identified similarities between activities performed by student pharmacists and student pharmacy technicians, expanding MyDispense to a new learner group to practice, develop and be assessed on dispensing skills within their scope, as part of a formal technician training program and in preparation for the Pharmacy Technician Certification Examination (PTCE).

A half‐century check‐up for child neurology training

A half‐century check‐up for child neurology training

Successful Provision of a New York State Pharmacy Technician Certification Examination Review Program.

Successful Provision of a New York State Pharmacy Technician Certification Examination Review Program.

Preparing Preservice Teachers for Inclusive Education Through Standards-based Curriculum

This paper explores factors that impact inclusive teacher preparation programs in the United States, such as federal legislation, inclusive teacher shortage, state policies, and the standards developed by the accreditation organizations and specialty professional associations. It also discusses the process of developing and implementing a curriculum based on multiple sets of standards from different professional organizations to prepare preservice teachers for inclusive education in dual-certification teacher education programs in a mid-sized university located in the Northeastern US. Crosswalk examples between the Council for Accreditation of Educator Preparation (CAEP) standards, Council for Exceptional Children (CEC) standards, the New York State Teacher Certification Examination (NYSTCE) – Students with Disabilities Content Specialty Test (SWD CST) Framework, and an excerpt of a standards-based assessment rubric are shared to help provide insights into the preservice inclusive teacher preparation in New York State, the United States.

To Learn, Try Teaching

To Learn, Try Teaching