Oral Certification Research Articles

Comparison of performance data between emergency medicine 1-3 and 1-4 program formats.

This study compares performance data from physicians completing 3-year versus 4-year emergency medicine residency training programs. Currently, there are 2 training formats and little is known about objective performance differences. This was a retrospective cross-sectional analysis of emergency residents and physicians. Multiple analyses were conducted comparing physicians' performances, including Accreditation Council of Graduate Medical Education Milestones and American Board of Emergency Medicine In-training Examination (ITE), Qualifying Examination (QE), Oral Certification Examination (OCE), and program extensions from 3-year and 4-year residency programs. Some confounding variables were not or could not be considered, such as rationale for medical students to choose one format over another, as well as application and final match rates. Milestone scores are higher for emergency medicine 3 residents in 1-3 programs (3.51) versus emergency medicine 3 residents in 1-4 programs (3.07; P<0.001, d=1.47) and highest for emergency medicine 4 residents (3.67). There was no significant difference in program extension rates (emergency medicine 1-3, 8.1%; emergency medicine 1-4, 9.6%; P=0.05, ω=0.02). ITE scores were higher for emergency medicine 1, 2, and 3 residents from 1-3 programs and emergency medicine 4 residents from 1-4 programs scored highest. Mean QE score was slightly higher for emergency 1-3 physicians (83.55 vs 83.00; P<0.01, d=0.10). QE pass rate was higher for emergency 1-3 physicians (93.1% vs 90.8%; P<0.001, ω=0.08). Mean OCE score was slightly higher for emergency 1-4 physicians (5.67 vs 5.65; P=0.03, d=-0.07) but did not reach a priori statistical significance (α<0.01). OCE pass rate was also slightly higher for emergency 1-4 physicians (96.9% vs 95.5%; P=0.06, ω=-0.07) but also non-significant. These results suggest that although performance measures demonstrate small differences between physicians from emergency medicine 1-3 and 1-4 programs, these differences are limited in their ability to make causal claims about performance on the basis of program format alone.

Early validity and reliability evidence for the American Board of Emergency Medicine Virtual Oral Examination.

The American Board of Emergency Medicine (ABEM) in-person Oral Certification Examination (OCE) was halted abruptly in 2020 due to the COVID-19 pandemic. The OCE was reconfigured to be administered in a virtual environment starting in December 2020. The purpose of this investigation was to determine whether there was sufficient validity and reliability evidence to support the continued use of the ABEM virtual Oral Examination (VOE) for certification decisions. This retrospective, descriptive study used multiple data sources to provide validity evidence and reliability data. Validity evidence focused on test content, response processes, internal structure (e.g., internal consistency and item response theory), and the consequences of testing. A multifaceted Rasch reliability coefficient was used to measure reliability. Study data were from two 2019 in-person OCEs and the first four VOE administrations. There were 2279 physicians who took the 2019 in-person OCE examination and 2153 physicians who took the VOE during the study period. Among the OCE group, 92.0% agreed or strongly agreed that the cases on the examination were cases that an emergency physician should be expected to see; 91.1% of the VOE group agreed or strongly agreed. A similar pattern of responses given to a question about whether the cases on the examination were cases that they had seen. Additional evidence of validity was obtained by the use of the EM Model, the process for case development, the use of think-aloud protocols, and similar test performance patterns (e.g., pass rates). For reliability, the Rasch reliability coefficients for the OCE and the VOE during the study period were all >0.90. There was substantial validity evidence and reliability to support ongoing use of the ABEM VOE to make confident and defensible certification decisions.

Acute Chest Syndrome.

Acute Chest Syndrome.

Interrater Reliability in the American Board of Physical Medicine and Rehabilitation Part II Certification Examination: Impact of a New Assessment Design.

The design of medical board certification examinations continues to evolve with advances in testing innovations and psychometric analysis. The potential for subjectivity is inherent in the design of oral board examinations, making improvements in reliability and validity especially important. The purpose of this quality improvement study was to analyze the impact of using two examiners on the overall reliability of the oral certification examination in physical medicine and rehabilitation. This was a retrospective quality improvement study of 422 candidates for the American Board of Physical Medicine and Rehabilitation Part II Examination in 2020. Candidates were examined by examiner pairs, each of whom submitted independent scores. Training for all 116 examiners included examination case review, scoring guidelines, and bias mitigation. Examiner performance was analyzed for both internal consistency (intrarater reliability) and agreement with their paired examiner (interrater reliability). The reliability of the Part II Examination was high, ranging from 0.93 to 0.94 over three administrations. The analysis also demonstrated high interrater agreement and examiner internal consistency. A high degree of interrater agreement was found using a new, two-examiner format. Comprehensive examiner training is likely the most significant factor for this finding. The two-examiner format improved the overall reliability and validity of the Part II Examination.

American Board of Oral and Maxillofacial Pathology certification examination dates and deadlines

American Board of Oral and Maxillofacial Pathology certification examination dates and deadlines

Costs and Benefits of Initial Certification for Emergency Medicine Residency Graduates.

Graduates of emergency medicine residency programs can seek certification from the American Board of Emergency Medicine (ABEM), yet the costs and perceived value by residents is not clear. This report sought to better define the value of board certification by asking physicians taking the ABEM Oral Certification Examination (OCE) to describe its costs (eg, time, money) and perceived benefits. A descriptive, cross-sectional, voluntary, anonymous survey was administered to physicians taking the 2018 spring and fall ABEM OCEs. Response frequencies were used to report response rates. There were 2016 physicians who participated in the 2018 OCEs, of whom 1565 (78%) completed a survey. With respect to preparation, 38% (599 of 1565 responses) spent more than 30 hours preparing for the examination. Regarding the expense of preparing for the examination, 21% (328) spent nothing, 50% (776) spent less than $1,000, and 2% (38) spent more than $3,000. Most physicians (80%, 1254) reported a learning benefit to preparing for and taking the OCE. There were 49% (765) of respondents who reported that preparing for the examination reinforced their knowledge of emergency medicine; 20% (311) reported no learning benefit. Most physicians (92%, 1442) reported that ABEM certification provided a career benefit, the most common of which was more career opportunities (69%, 1076). Initial certification requires a considerable investment of time and money. Physicians seeking initial ABEM certification found both learning and professional benefits, with the most frequently reported being reinforcement of medical knowledge and more career opportunities.

USMLE Scores Predict Success in ABEM Initial Certification: A Multicenter Study.

IntroductionThere are no existing data on whether performance on the United States Medical Licensing Examination (USMLE) predicts success in American Board of Emergency Medicine (ABEM) certification. The aim of this study was to determine the presence of any association between USMLE scores and first-time success on the ABEM qualifying and oral certification examinations.MethodsWe retrospectively collected USMLE Step 1, Step 2 Clinical Knowledge (CK) scores and pass/fail results from the first attempt at ABEM qualifying and oral examinations from residents graduating between 2009 and 2011 from nine EM programs. A composite score was defined as the sum of USMLE Step 1 and Step 2 CK scores.ResultsSample was composed of 197 residents. Median Step 1, Step 2 CK and composite scores were 218 ([IQR] 207–232), 228 (IQR 217–239) and 444 (IQR 427–468). First-time pass rates were 95% for the qualifying examination and 93% for both parts of the examination. Step 2 CK and composite scores were better predictors of achieving ABEM initial certification compared to Step 1 score (area under the curve 0.800, 0.759 and 0.656). Step 1 score of 227, Step 2 CK score of 225 and composite score of 444 predicted a 95% chance of passing both boards.ConclusionHigher USMLE Step 1, Step 2 CK and composite scores are associated with better performance on ABEM examinations, with Step 2 CK being the strongest predictor. Cutoff scores for USMLE Step 1, Step 2 CK and composite score were established to predict first-time success on ABEM initial certification.

Initial Validity Analysis of the American Board of Emergency Medicine Enhanced Oral Examination.

The American Board of Emergency Medicine (ABEM) has introduced a new testing format for the oral certification examination (OCE): the enhanced oral or "eOral" format. The purpose of this study was to perform initial validity analyses of the eOral format. The two hypotheses were: 1) the case content in the eOral format was sufficiently similar to clinical practice and 2) the eOral case materials were sufficiently similar to clinical practice. The eOral and traditional formats were compared for these characteristics. This was a prospective survey study. The survey was administered as a voluntary postexamination activity at the end of the 2015 spring (April 25-27) and fall (October 10-13) ABEM OCEs. The survey is a routine part of the ABEM oral examination experience. For 2015, two additional questions were added to gauge the similarity of the eOral format to clinical practice. Validity was defined by content and substantive elements within Messick's model of construct validity as well as portions of Kane's validity model. Of the 1,746 physicians who took the oral examination, 1,380 physicians (79.0%) completed all or part of the study survey questions. The majority of respondents agreed the patient presentations in the cases were similar (strongly agreed or agreed) to cases seen in clinical practice, in both the traditional cases (95.1%) and the eOral cases (90.1%). Likewise, the majority of respondents answered that the case materials (e.g., laboratory, radiographs) were similar (strongly agreed or agreed) to what they encounter in clinical practice, both in the traditional format (85.8%) and in the eOral cases (93.7%). Most emergency physicians reported that the types of cases tested in the traditional and eOral formats were similar to cases encountered in clinical practice. In addition, most physicians found the case materials to be similar to what is seen in clinical practice. This study provides early validity evidence for the eOral format.

Current dilemmas in court interpreting: improving quality and access through smarter testing and administration protocols

Court interpreting certifying bodies face a plethora of challenges in their quest to identify competent judicial interpreters so that speakers of all languages might be assured of due process under the law and equal access to justice. For the entities which develop and administer the oral certification exams which act as gateways to the profession of court interpreting, two such dilemmas are of particular interest: the first is high rates of exam failure, with a frustrating number of candidates not meeting minimum levels of qualification to practice in court. The second is an increasing need for qualified interpreters of languages of lesser diffusion. In the face of ubiquitous budget constraints, this article explores an abbreviated testing model as a mitigator of extreme exam failure at the same time as it reveals the results of a recent pilot project which focused on centralizing interpreting services protocols while prioritizing interpreter quality.

Situación actual y retrospectiva de los quesos genuinos de Chiautla de Tapia, México.

En el municipio de Chiautla de Tapia, estado de Puebla, México, se elaboran dos tipos de queso, uno fresco y otro añejo, de manera artesanal, desde hace aproximadamente 100 años, y hoy forman parte de la historia e identidad de la población. En los últimos años se ha evidenciado un proceso gradual de extinción de este derivado lácteo, aspecto que se verifica en su menor presencia en los mercados locales. Los factores que contribuyen a la desaparición del queso son complejos y no han sido suficientemente estudiados; su comprensión resulta de gran importancia para diseñar e implementar estrategias para el rescate y la valorización de este patrimonio alimentario. Para identificar estos factores desde una visión multidisciplinaria se empleó la metodología propuesta por Grass, Cervantes y Palacios (2012) para el enfoque SIAL, que utiliza la Historia oral, el Método genealógico, la Trayectoria tecnológica, el Análisis de cadena, la Calificación y certificación del producto, y el Análisis de redes para acopiar información que se analiza posteriormente de manera integral. Se concluye que estos quesos son inocuos para el consumidor aunque se elaboran con leche cruda y que los factores que inciden en su desaparición tienen relación con aspectos geográficos, técnicos, económicos y sociales.

Performance of Candidates Selecting the Subspecialty of Spine Surgery for the Part II American Board of Orthopaedic Surgery Oral Certification Examination

The American Board of Orthopaedic Surgery (ABOS) was founded in 1934 to establish standards for the certification of orthopaedic surgeons and educational requirements for postgraduate training in orthopaedic surgery. Educational standards have been well established for residency training, with specific requirements spelled out to successfully complete the five-year accredited orthopaedic residency program. These requirements are clearly noted on the ABOS web site1. Over the past fifteen years, there has been a rapid increase in the number of postgraduate fellowship programs encompassing the multiple subspecialties of orthopaedic surgery. At present, >90% of graduates take a postgraduate fellowship upon completion of their residency training according to ABOS internal documentation obtained from application material. Fellowships in the subspecialties of orthopaedic surgery accredited by the Accreditation Council for Graduate Medical Education (ACGME) are available in the Graduate Medical Education Directory published by the American Medical Association2. There is a wide disparity in the number of accredited or non-accredited fellowships depending on the subspecialty selected. For instance, surgery of the hand and sports medicine have almost 100% of their fellowships accredited by the ACGME with stringent educational, training, and faculty requirements overseen by the Residency Review Committee for Orthopaedic Surgery. Other subspecialties have far fewer accredited programs. These include spine surgery, which has eighteen accredited fellowships of approximately seventy-five available programs. Presently, the ABOS has not mandated educational and training requirements for fellowships as it has previously for resident training. The ABOS recognizes that postgraduate fellowship training has become an accepted path following completion of residency. The board is also aware that non-accredited fellowships are not subject to the same educational and training requirement reviews that accredited fellowships undergo on a regular basis. From 2006 to 2010, there were 359 candidates who sat for the ABOS Part II oral examination as …

American Board of Physical Medicine and Rehabilitation 2012 Part II (Oral) Certification Examination

American Board of Physical Medicine and Rehabilitation 2012 Part II (Oral) Certification Examination

Judgesʼ Perception of Candidatesʼ Organization and Communication, in Relation to Oral Certification Examination Ratings

To determine (1) whether judges differed in the levels of severity they exercised when rating candidates' performance in an oral certification exam, (2) to what extent candidates' clinical competence ratings were related to their organization/communication ratings, and (3) to what extent clinical competence ratings could predict organization/communication ratings. Six hundred eighty-four physicians participated in a medical specialty board's 2002 oral examination. Ninety-nine senior members of the medical specialty served as judges, rating candidates' performances. Candidates' clinical competence ratings were analyzed using multifaceted Rasch measurement to investigate judge severity. A Pearson correlation was calculated to examine the relationship between ratings of clinical competence and organization/communication. Logistic regression was used to determine to what extent clinical competence ratings predicted organization/communication ratings. There were about three statistically distinct strata of judge severity; judges were not interchangeable. There was a moderately strong relationship between the two sets of candidate ratings. Higher clinical competence ratings were associated with an organization/communication rating of acceptable, whereas lower clinical competence ratings were associated with an organization/communication rating of unacceptable. The judges' clinical competence ratings correctly predicted 61.9% of the acceptable and 88.3% of the unacceptable organization/communication ratings. Overall, the clinical competence ratings correctly predicted 80% of the organization/communication ratings. The close association between the two sets of ratings was possibly due to a "halo" effect. Several explanations for this relationship were explored, and the authors considered the implications for their understanding of how judges carry out this complex rating task.

American Board of Physical Medicine and Rehabilitation 2009 Part II (Oral) Certification Examination

American Board of Physical Medicine and Rehabilitation 2009 Part II (Oral) Certification Examination

American Board of Physical Medicine and Rehabilitation 2009 Part II (Oral) Certification Examination

American Board of Physical Medicine and Rehabilitation 2009 Part II (Oral) Certification Examination

S222: Evaluation and Rejuvenation of the Upper Facial Third

Cosmetic surgery is now a fundamental part of an oral and maxillofacial surgeon's training. In fact, proficiency in cosmetic surgery follows the OMS's experience with trauma management and orthognathic surgery, and is included in the same section of the American Board of Oral and Maxillofacial Surgery's oral certification examination.

Relationship between candidate communication ability and oral certification examination scores

Structured case-based oral examinations are widely used in medical certifying examinations in the USA. These orals assess the candidate's decision-making skills using real or realistic patient cases. Frequently mentioned but not empirically evaluated is the potential bias introduced by the candidate's communication ability. This study aimed to assess the relationship between candidate communication ability and medical certification oral examination scores. Non-doctor communication observers rated a random sample of 90 candidates on communication ability during a medical oral certification examination. The multi-facet Rasch model was used to analyse the communication survey and the oral examination data. The multi-facet model accounts for observer and examiner severity bias. anova was used to measure differences in communication ability between passing and failing candidates and candidates grouped by level of communication ability. Pearson's correlations were used to compare candidate communication ability and oral certification examination performance. Candidate separation reliability values for the communication survey and the oral examination were 0.85 and 0.97, respectively, suggesting accurate candidate measurement. The correlation between communication scores and oral examination scores was 0.10. No significant difference was found between passing and failing candidates for measured communication ability. When candidates were grouped by high, moderate and low communication ability, there was no significant difference in their oral certification examination performance. Candidates' communication ability has little relationship to candidate performance on high-stakes, case-based oral examinations. Examiners for this certifying examination focused on assessing candidate decision-making ability and were not influenced by candidate communication ability.

Relationship of Candidate Communication and Organization Skills to Oral Certification Examination Scores

This study investigated the relationship between candidate's proficiency in communication/organization and ability measures derived from judges' oral examination ratings on a medical specialty certification examination. Judges who rated candidates on the oral examination also provided a separate rating for candidates' communication/organization skills. ANOVA was used to examine differences among levels of communication/organizational skills with respect to candidates' ability measures on the oral examination. There was a statistically significant increase in oral examination measures for candidates with higher levels of communication/organization skills. This supports the hypothesis that candidates who were more proficient in organizing and presenting their responses were more likely to have better performance on the oral examination. Given communication and oral examination ratings were provided by the same judges, future research should investigate whether these preliminary findings generalize to situations in which communication ratings are obtained from independent ratings during and outside the oral examination.

American Board of Physical Medicine and Rehabilitation 2008 Part II (Oral) Certification Examination

American Board of Physical Medicine and Rehabilitation 2008 Part II (Oral) Certification Examination

S212: Evaluation and Rejuvenation of the Upper Facial Third

Cosmetic surgery is now a fundamental part of an Oral and Maxillofacial Surgeon's training. In fact, proficiency in cosmetic surgery follows the OMS's experience with trauma management and orthognathic surgery, and is included in the same section of the American Board of Oral and Maxillofacial Surgery's oral certification examination.