Ophthalmic Knowledge Assessment Program Research Articles

ChatGPT and Beyond: An overview of the growing field of large language models and their use in ophthalmology.

ChatGPT, an artificial intelligence (AI) chatbot built on large language models (LLMs), has rapidly gained popularity. The benefits and limitations of this transformative technology have been discussed across various fields, including medicine. The widespread availability of ChatGPT has enabled clinicians to study how these tools could be used for a variety of tasks such as generating differential diagnosis lists, organizing patient notes, and synthesizing literature for scientific research. LLMs have shown promising capabilities in ophthalmology by performing well on the Ophthalmic Knowledge Assessment Program, providing fairly accurate responses to questions about retinal diseases, and in generating differential diagnoses list. There are current limitations to this technology, including the propensity of LLMs to "hallucinate", or confidently generate false information; their potential role in perpetuating biases in medicine; and the challenges in incorporating LLMs into research without allowing "AI-plagiarism" or publication of false information. In this paper, we provide a balanced overview of what LLMs are and introduce some of the LLMs that have been generated in the past few years. We discuss recent literature evaluating the role of these language models in medicine with a focus on ChatGPT. The field of AI is fast-paced, and new applications based on LLMs are being generated rapidly; therefore, it is important for ophthalmologists to be aware of how this technology works and how it may impact patient care. Here, we discuss the benefits, limitations, and future advancements of LLMs in patient care and research.

Evaluating the Performance of ChatGPT in Ophthalmology: An Analysis of Its Successes and Shortcomings

Foundation models are a novel type of artificial intelligence algorithms, in which models are pretrained at scale on unannotated data and fine-tuned for a myriad of downstream tasks, such as generating text. This study assessed the accuracy of ChatGPT, a large language model (LLM), in the ophthalmology question-answering space. Evaluation of diagnostic test or technology. ChatGPT is a publicly available LLM. We tested 2 versions of ChatGPT (January 9 "legacy" and ChatGPT Plus) on 2 popular multiple choice question banks commonly used to prepare for the high-stakes Ophthalmic Knowledge Assessment Program (OKAP) examination. We generated two 260-question simulated exams from the Basic and Clinical Science Course (BCSC) Self-Assessment Program and the OphthoQuestions online question bank. We carried out logistic regression to determine the effect of the examination section, cognitive level, and difficulty index on answer accuracy. We also performed a post hoc analysis using Tukey's test to decide if there were meaningful differences between the tested subspecialties. We reported the accuracy of ChatGPT for each examination section in percentage correct by comparing ChatGPT's outputs with the answer key provided by the question banks. We presented logistic regression results with a likelihood ratio (LR) chi-square. We considered differences between examination sections statistically significant at a P value of < 0.05. The legacy model achieved 55.8% accuracy on the BCSC set and 42.7% on the OphthoQuestions set. With ChatGPT Plus, accuracy increased to 59.4% ± 0.6% and 49.2% ± 1.0%, respectively. Accuracy improved with easier questions when controlling for the examination section and cognitive level. Logistic regression analysis of the legacy model showed that the examination section (LR, 27.57; P= 0.006) followed by question difficulty (LR, 24.05; P < 0.001) were most predictive of ChatGPT's answer accuracy. Although the legacy model performed best in general medicine and worst in neuro-ophthalmology (P < 0.001) and ocular pathology (P= 0.029), similar post hoc findings were not seen with ChatGPT Plus, suggesting more consistent results across examination sections. ChatGPT has encouraging performance on a simulated OKAP examination. Specializing LLMs through domain-specific pretraining may be necessary to improve their performance in ophthalmic subspecialties. Proprietary or commercial disclosure may be found after the references.

Flipped classroom approach to global outreach: cross-cultural teaching of horizontal strabismus to Chinese ophthalmology residents.

To evaluate the flipped classroom model for teaching horizontal strabismus didactics in an ophthalmology residency program in China as part of a visiting professorship from the United States. Residents from an ophthalmology residency program in China were invited to participate in flipped classroom sessions taught by an experienced American ophthalmology faculty in 2018. Residents were instructed to watch a pre-class video lecture prior to the in-class-case-based activity. Content tests (5 Ophthalmic Knowledge Assessment Program style questions) and surveys were administered before and after the classroom sessions (100% response rate). These results were compared to that of an American cohort who were taught the same content. The Chinese cohort of 12 residents preferred the flipped classroom to the traditional classroom at higher rates than the American cohort of 40 residents (92% vs 55%, P=0.04) and felt that all ophthalmology topics would be appropriate for the flipped classroom teaching style (P-values between 0.008 and <0.001). In both Chinese and American cohorts, we found that the exotropia curriculum saw a small but significant improvement in performance following the flipped classroom session (P=0.025 for Chinese residents; P=0.001 for US residents), whereas scores in both groups for the esotropia course did not significantly improve. This is the first study to evaluate the flipped classroom model implemented by a visiting ophthalmology professor in a global outreach setting. The flipped classroom sessions are viewed favorably by the Chinese residents relative to the US cohort with a modest impact on knowledge. Decreased in-person interpreter requirement and increased student engagement make this model valuable in cross-cultural visiting professorship settings. Finally, the flipped classroom may lend itself well to a virtual format to prevent the transmission of COVID-19, although such a format requires further study.

Association Between Parental Leave and Ophthalmology Resident Physician Performance

Although parental leave is essential in enhancing resident wellness and fostering inclusive workplace environments, residents may often feel discouraged from using parental leave owing to perceived stigma and concerns about possible negative effects on their training. To examine parental leave usage across multiple institutions and compare residency performance metrics between residents who took parental leave vs their peers who did not take leave. This was a retrospective cross-sectional analysis conducted from April 1, 2020, to July 28, 2022, of educational records. Multicenter data were obtained from 10 Accreditation Council for Graduate Medical Education (ACGME)-accredited ophthalmology programs across the US. Included ophthalmology residents graduated between 2015 and 2019. Data were analyzed from August 15, 2021, to July 25, 2022. Performance metrics of residents who used parental leave during residency were compared with those of residents who did not take parental leave. Measures of performance included the Ophthalmic Knowledge Assessment Program (OKAP) scores, ACGME milestones scores, board examination pass rates, research activity, and surgical volumes. Of the 283 ophthalmology residents (149 male [52.7%]) included in the study, 44 (15.5%) took a median (IQR) parental leave of 4.5 (2-6) weeks. There were no differences in average OKAP percentiles, research activity, average ACGME milestones scores, or surgical volume between residents who took parental leave and those who did not. Residents who pursued fellowship were less likely to have taken parental leave (odds ratio [OR], 0.43; 95% CI, 0.27-0.68; P < .001), and residents who practiced in private settings after residency were more likely to have taken parental leave (OR, 3.56; 95% CI, 1.79-7.08; P < .001). When stratified by sex, no differences were identified in performance between female residents who took parental leave compared with residents who did not take leave, except a mild surgical number difference in 1 subspecialty category of keratorefractive procedures (difference in median values, -2; 95% CI, -3.7 to -0.3; P = .03). In this multicenter cross-sectional study, no differences in performance metrics were identified between residents taking parental leave compared with their peers. These findings may provide reassurance to trainees and program directors regarding the unlikelihood, on average, that taking adequate parental leave will affect performance metrics adversely.

Perceived Barriers to Increasing Diversity within Oculofacial Plastic Surgery.

Purpose Physician diversity is limited in ophthalmology and oculofacial plastic surgery. Determination of barriers within the application process for oculofacial plastic surgery may help target efforts to improve the recruitment of underrepresented groups. This study aimed to illuminate perceived barriers to increasing diversity in oculofacial plastic surgery trainees, according to the American Society of Ophthalmic Plastic and Reconstructive Surgery (ASOPRS) fellows and fellowship program directors (FPDs). Methods During the month of February 2021, we sent surveys out to 54 current oculofacial plastic surgery fellows and 56 FPDs at 56 oculofacial plastic surgery programs recognized by the ASOPRS nationwide using a 15-question Qualtrics survey. Results Sixty-three individuals (57%) responded to the survey: 34 fellows (63%) and 29 FPDs (52%). Eighty-eight percent of fellows and 68% of FPDs identified as non-underrepresented in medicine (UiM). Forty-four percent of fellows and 25% of FPDs identified as men. FPDs most commonly noted, "Not enough minorities applying to our program" and "The objective data (Ophthalmic Knowledge Assessment Program score, United States Medical Licensing Examination Step scores, clinical honors, Alpha Omega Alpha status, letter of recommendation) for minority applicants often do not meet the threshold required to offer an interview or to be ranked to match" as barriers. Among fellows, the lowest-rated considerations when applying to oculofacial plastic surgery were "Racially/ethnically diverse faculty" and "Perceptions of minority candidates by fellowship programs," whereas "Likelihood of matching in program of choice" was ranked highest in considerations. Fellows identifying as men indicated greater concern for "Financial factors related to fellowship (e.g., loans, salary, cost of living, or cost of interviewing)" compared to fellows identifying as women who noted greater concern for "Program or preceptor acceptance of starting or having a family during fellowship." Conclusion Responses from FPDs suggest that efforts focused on recruiting and supporting diverse students to medicine and ophthalmology, mentoring applicants interested in oculofacial plastic surgery, and restructuring the application process to decrease bias, may improve diversity within the subspecialty. The lack of UiM representation in this study, 6% fellows and 7.4% FPDs identified as UiM, shows both the stark underrepresentation and the need for further research into this topic.

Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency.

Purpose Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to "Pass/Fail" removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS). Design Retrospective analysis of application characteristics versus residency performance was conducted. Participants Residents who graduated from the Dean McGee Eye Institute/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study. Methods Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS. Main Outcome Measures Characteristics from residency applications that correlate with resident performance were the primary outcome of this study. Results Fifty-five residents and 21 faculty members were included. A grade of "A" or "Honors" in the obstetrics/gynecology (OB/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance. Conclusions Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking.

Objective Resident Characteristics Associated with Performance on the Ophthalmic Knowledge Assessment Program Examination.

Background To determine objective resident characteristics that correlate with Ophthalmic Knowledge Assessment Program (OKAP) performance, as well as to correlate OKAP performance with Accreditation Council for Graduate Medical Education (ACGME) milestone assessments, written qualifying examination (WQE) scores, and oral board pass rates. Methods Review of administrative records at an ACGME-accredited ophthalmology residency training program at an urban, tertiary academic medical center. Results The study included data from a total of 50 resident physicians who completed training from 2012 to 2018. Mean (standard deviation) OKAP percentile performance was 60.90 (27.51), 60.46 (28.12), and 60.55 (27.43) for Years 1, 2, and 3 examinations, respectively. There were no statistically significant differences based on sex, marital status, having children, MD/PhD degree, other additional degree, number of publications, number of first author publications, or grades on medical school medicine and surgery rotations. OKAP percentile scores were significantly associated with United States Medical Licensing Examination (USMLE) Step 1 scores (linear regression coefficient 0.88 [0.54-1.18], p = 0.008). Finally, continuous OKAP scores were significantly correlated with WQE ( r s = 0.292, p = 0.049) and oral board ( r s = 0.49, p = 0.001) scores. Conclusion Higher OKAP performance is correlated with passage of both WQE and oral board examinations during the first attempt. USMLE Step 1 score is the preresidency academic factor with the strongest association with success on the OKAP examination. Programs can utilize this information to identify those who may benefit from additional OKAP, WQE, and oral board preparation assistance.

Relationship between Didactic Attendance and Average Ophthalmic Knowledge Assessment Program Performance

Abstract Background The Accreditation Council for Graduate Medical Education's ophthalmology-specific program requirements indicate that a resident must participate in 360 hours of educational activities over the course of residency. The requirements allow a variety of options by which this educational process may occur. Furthermore, the current generation of learners has used a large array of learning tools in their careers: digital media, online resources, question banks, flipped classrooms, problem-based learning, and asynchronous lectures in addition to the traditional face-to-face didactic lectures. Objectives This article determines if the traditional face-to-face didactic lectures play a significant role in the education of ophthalmology residents, attempts to quantitate this role, and estimates the relative importance of this method. Methods Lecture hours beginning immediately after an Ophthalmic Knowledge Assessment Program (OKAP) examination and up to the subsequent exam was tabulated for 40 residents over 8 years. From 2012 through 2019 each of the residents took the OKAP exam one to three times. The average overall OKAP score for each resident was then regressed against his or her logged hours of didactics that year. This rate of average OKAP score to lecture hours was then evaluated for significant differences on the basis of postgraduate year (PGY). “STATA 13.1 IC,” was used for all statistical analysis. Conclusion Traditional synchronous didactic lectures play a statistically significant role in the education of ophthalmology residents. For each 3.5 hours of lectures per year there is a 1% increase in the average OKAP score, p = 0.0064. However, the amount of variability explained by this single factor is low, R 2 = 0.067, and other factors are presumed to play a large part in learning. PGY subgroup analysis showed significant difference in the OKAP performance to lecture hour relationship, the number of lecture hours attended, and the OKAP performance.

Guiding Residency Program Educational Goals Using Institutional Keyword Reports from the Ophthalmic Knowledge Assessment Program Examination

Abstract Introduction Residency programs receive an institutional keyword report following the annual Ophthalmic Knowledge Assessment Program (OKAP) examination containing the raw number of incorrectly answered questions. Programs would benefit from a method to compare relative performance between subspecialty sections. We propose a technique of normalizing the keyword report to determine relative subspecialty strengths and weaknesses in trainee performance. Methods We retrospectively reviewed our institutional keyword reports from 2017 to 2019. We normalized the percentage of correctly answered questions for each postgraduate year (PGY) level by dividing the percent of correctly answered questions for each subspecialty by the percent correct across all subsections for that PGY level. We repeated this calculation for each PGY level in each subsection for each calendar year of analysis. Results There was a statistically significant difference in mean performance between the subspecialty sections (p = 0.038). We found above average performance in the Uveitis and Ocular Inflammation section (95% confidence interval [CI]: 1.02–1.18) and high variability of performance in the Clinical Optics section (95% CI: 0.76–1.34). Discussion The OKAP institutional keyword reports are extremely valuable for residency program self-evaluation. Performance normalized for PGY level and test year can reveal insightful trends into the relative strengths and weaknesses of trainee knowledge and guide data-driven curriculum improvement.

Completing the Basic and Clinical Science Course as a First-Year Ophthalmology Resident

Abstract Background The Basic and Clinical Science Course (BCSC) is the primary educational curriculum for ophthalmology resident physicians in the United States. The Ophthalmic Knowledge Assessment Program (OKAP) examination is an annual evaluation completed by residents that is based primarily on the BCSC curriculum. First-year ophthalmology residents are encouraged to complete the 13 volume BCSC series in preparation for the OKAP examination while balancing a steep clinical learning curve and substantial call schedule. By calculating the daily time commitment necessary to read each volume in the series, we hope to help residents create a realistic study plan to compete the entire BSCS series before the OKAP examination. Methods We determine the word counts of each volume using an electronic copy of the 2018–2019 BCSC series. We include all text sections and legends, and we exclude all figures and tables. We calculate the time per day of dedicated reading required to complete a goal number of BCSC books between the start of ophthalmology residency (postgraduate year 2 [PGY2]) and the OKAP examination by developing a formula that is a function of self-assessed reading speed. Results A first-year ophthalmology (PGY2) resident with an average reading speed of 250 words per minute must read for 25.0 minutes per day to complete the entire BCSC series before the OKAP examination. If studying is initiated at the beginning of intern (PGY1) year, the resident must read for 10.2 minutes per day. We introduce a formula and provide a table to guide residents on the amount of time needed to dedicate to reading the BCSC each day as a function of self-assessed reading speed. Discussion Completion of all volumes of the BCSC requires a daily commitment with little room for missed sessions. The commitment is substantially more realistic if initiated during the PGY1 year; thus, residency programs should encourage an early start to OKAP preparation. We hope with a better understanding of the daily time commitment involved in completing the BCSC series, ophthalmology residents will be able to develop more successful study plans.

Resident and program characteristics that impact performance on the Ophthalmic Knowledge Assessment Program (OKAP)

BackgroundTo determine which resident and program characteristics correlate with ophthalmic knowledge, as assessed by resident Ophthalmic Knowledge Assessment Program (OKAP) performance.MethodsAn online survey was sent in June 2017 to all US ophthalmology residents who took the OKAP in April 2017.ResultsThe survey response rate was 13.8% (192/1387 residents). The mean respondent age was 30.4 years, and 57.3% were male. The mean [SD] self-reported 2017 OKAP percentile was 61.9 [26.7]. OKAP performance was found to have a significant positive correlation with greater number of hours spent/week studying for the OKAPs (p = 0.007), with use of online question banks (p < 0.001), with review sessions and/or lectures arranged by residency programs (p < 0.001), and with OKAP-specific didactics (p = 0.002). On multivariable analysis, factors most predictive of residents scoring ≥75th percentile were, higher step 1 scores (OR = 2.48, [95% CI: 1.68–3.64, p < 0.001]), presence of incentives (OR = 2.75, [95% CI: 1.16–6.56, p = 0.022]), greater number of hours/week spent studying (OR = 1.09, [95% CI:1.01–1.17, p = 0.026]) and fewer hours spent in research 3 months prior to examination (OR = 1.08, [95% CI: 1.01–1.15, p = 0.020]. Lastly, residents less likely to depend on group study sessions as a learning method tended to score higher (OR = 3.40, [95% CI: 1.16–9.94, p = 0.026]).ConclusionsPrograms wishing to improve resident OKAP scores might consider offering incentives, providing effective access to learning content e.g. online question banks, and adjusting the curriculum to highlight OKAP material. Step 1 scores may help educators identify residents who might be at risk of not performing as well on the OKAP.

Teaching Ophthalmology Residents Clinical Optics Via a Flipped Classroom Curriculum

Background Clinical optics is an essential part of ophthalmology resident education that can be challenging for both learners and teachers when taught using a lecture format. The effectiveness of a flipped classroom approach in this context has not been formally evaluated. Objective The main purpose of this article is to compare the effectiveness of flipped classroom versus lecture-based clinical optics curricula in a graduate medical education setting. Design Retrospective, nonrandomized, pre- and post-interventional study from 2009 to 2016. Setting, Participants Ophthalmology residency program at the Casey Eye Institute, Oregon Health &amp; Science University, an academic medical center in the United States. Participants included all ophthalmology residents able to take at least one Ophthalmic Knowledge Assessment Program (OKAP) examination during the years 2009 to 2016. Methods The clinical optics curriculum was changed from a lecture-based series to a flipped classroom curriculum and moved from the fall to winter during the 2012 to 2013 academic year. No major changes were made to the curriculum in other subject areas during the study period. Resident performance on the OKAP annual national in-service examination for the 4 years before and after the optics curriculum change was compared. Specifically, the scaled subtest scores from the Optics, Refraction, and Contact Lens subsection were examined, while scores from the 10 nonoptics subsections served as controls. Results Scores from 57 resident test administrations before the optics curriculum change and 59 after the optic curriculum change were available for comparison. The Optics, Refraction, and Contact Lens subsection mean scores were 50.37 ± 2.31 and 57.27 ± 2.47 before and after the optics curriculum change, respectively (mean ± 95% confidence interval). This was the only subsection score to show a statistically significant difference after the optics curriculum change (p = 0.00008). Conclusions and Relevance In comparison to a lecture-based curriculum, a flipped classroom approach to clinical optics education was found to be associated with higher ophthalmology resident performance on the optics subsection of the OKAP examination. Our study suggests that a flipped classroom format may be more effective than traditional lectures for teaching clinical optics in a graduate medical education setting.

Perspective of ophthalmology residents in the United States about residency programs and competency in relation to the International Council of Ophthalmology guidelines

PurposeTo evaluate the perspective of ophthalmology residents in the US about their residency programs and compare the competency of residency programs to international competency levels set by the International Council of Ophthalmology (ICO). MethodsA cross-sectional web-based survey extracted from the ICO published competency standards was sent to program directors of ophthalmology residency programs in the US to forward it to current PGY-3, 4 residents, and residency graduates from 2011 to 2014. ResultsEighty-seven responses were received, comprising 61 residents and 26 graduates. Most respondents were highly satisfied with their programs (93.6%). Clinic-based training was rated satisfactorily. Insufficient exposure to low-vision rehabilitation (38.5%), refraction and contact lenses prescription (38.5%), and vitreo-retinal surgeries (38.5%) was reported. Respondents were satisfied with their overall surgical experiences, with the vast majority (>83%) rating case volume, complexity, and variety as satisfactory or better. A significant group stated they had insufficient exposure to extra-capsular cataract extraction (26.3%), refractive surgery (19.7%), and orbital surgery (64.5%). All graduates surveyed passed their Ophthalmic Knowledge Assessment Program (OKAP) examinations, and 72% felt their residency programs adequately prepared them for the examinations. All respondents reported insufficient training in certain nonclinical areas, such as practice management, staffing, and administration skills. ConclusionsOphthalmology residents in the US express high levels of satisfaction with their residency training programs. While most programs adequately address most ICO core objectives, certain curriculum modifications should be considered.

A Multicenter Analysis of the Ophthalmic Knowledge Assessment Program and American Board of Ophthalmology Written Qualifying Examination Performance

To compare the performance on the American Board of Ophthalmology Written Qualifying Examination (WQE) with the performance on step 1 of the United States Medical Licensing Examination (USMLE) and the Ophthalmic Knowledge Assessment Program (OKAP) examination for residents in multiple residency programs. Comparative case series. Fifteen residency programs with 339 total residents participated in this study. The data were extracted from the 5-year American Board of Ophthalmology report to each participating program in 2009 and included residency graduating classes from 2003 through 2007. Residents were included if data were available for the USMLE, OKAP examination in ophthalmology years 1 through 3, and the WQE score. Residents were excluded if one or more of the test scores were not available. Two-sample t tests, logistic regression analysis, and receiver operating characteristic (ROC) curves were used to examine the association of the various tests (USMLE, OKAP examination year 1, OKAP examination year 2, OKAP examination year 3, and maximum OKAP examination score) as a predictor for a passing or failing grade on the WQE. The primary outcome measure of this study was first time pass rate for the WQE. Using ROC analysis, the OKAP examination taken at the third year of ophthalmology residency best predicted performance on the WQE. For the OKAP examination taken during the third year of residency, the probability of passing the WQE was at least 80% for a score of 35 or higher and at least 95% for a score of 72 or higher. The OKAP examination, especially in the third year of residency, can be useful to residents to predict the likelihood of success on the high-stakes WQE examination.

A Comparative Study of Resident Performance on Standardized Training Examinations and the American Board of Ophthalmology Written Examination

To investigate the relationships between ophthalmology resident performance on the United States Medical Licensing Examination (USMLE), the Ophthalmic Knowledge Assessment Program (OKAP) exam and the American Board of Ophthalmology written qualifying examination (ABO-WQE). Cohort study. We included 76 residents from 15 consecutive training classes (1991-2006) at 1 ophthalmologic residency training program. Numeric scores on the USMLE Step 1 and OKAP examinations during the 3 years of residency, and first attempt pass rate on the ABO-WQE were recorded for 76 residents. Age and gender were also noted. Spearman's rank correlations and univariate and multivariate logistic analyses were performed to determine relevant associations. First-time attempt pass rate on the ABO-WQE and/or successful completion of the ABO-WQE within 3 years of graduation from the residency program. The ABO-WQE first-attempt pass rate was 72.6%, consistent with the national average. Resident USMLE scores were not significantly associated with ABO-WQE performance. The ABO-WQE pass rate was significantly associated with OKAP examination scores during the 3 residency years (year 1: odds ratio [OR], 8.85 and 95% confidence interval [CI] 1.82-42.79; year 2: OR, 5.28 and 95% CI, 1.15-25.27; year 3: OR, 11.08 and 95% CI, 1.86-68.96). Passing the OKAP examinations in all 3 years during residency training was associated with 5.43-fold increased odds of passing the ABO-WQE and failing all 3 OKAP examinations was associated with >9-fold lower odds of passing the ABO-WQE on the first attempt. Our results suggest that OKAP examination performance is a predictor of a resident's success in passing the ABO-WQE on the first attempt, as well as within 3 years of graduation from an ophthalmologic training program. Awareness of this association may permit identification of residents at risk for failing the ABO-WQE and encourage educational remediation to prevent this failure.

Clinical Cases in Ophthalmology

This concise and easy-to-read book will be useful to all ophthalmology residents. It succinctly covers both rare and common diseases of the eye. Each case begins with an illustrative color clinical photograph, followed by a discussion of the pertinent findings, a differential diagnosis, diagnostic workup, management options, and the prognosis. The case usually concludes with a few tips and 1 or 2 references. This is an excellent quick review for the Ophthalmic Knowledge Assessment Program examination. However, the title is somewhat misleading. The reader is not provided with a clinical history associated with the photograph. Fortunately, the photographs are classic enough that the history of the case can usually be inferred. Future editions might be improved by presenting the photographs as unknowns and not revealing the disease process until the applicable findings have been described and discussed.

New Directions for the Ophthalmic Knowledge Assessment Program (OKAP) Examination

The Ophthalmic Knowledge Assessment Program (OKAP) Task Force of the American Academy of Ophthalmology has reviewed the present OKAP examination and addressed the goals of this resident-in-training examination. Simultaneously, the American Board of Ophthalmology has been assessing its Written Qualifying Examination (WQE) as a certifying tool. The history of the OKAP examination is chronicled along with a discussion of the goals of this examination as a stimulus for resident education. The new OKAP examination is structured specifically for residents-in-training and is now administered separately from the WQE. The OKAP examination is moving in the direction of a criterion referenced examination based on a defined content outline. Twelve subtest areas are tested. Future goals are to improve subtest reliability and increase feedback to residents and program directors.

The Selection of Residents

To Editor. —Dr Volle, 1 in his editorial in Jan 8,1988, issue, correctly states that ophthalmology programs use scores from Part I of National Board of Medical Examiners tests as part of resident selection process. However, he is incorrect as to motive. It is not because National Board scores are assumed (falsely) to correlate with the quality of later professional life. Perhaps he would not have erred if Dr McCollister, 2 in his article in same issue, would have used a questionnaire asking programs why as well as if they used National Board scores. The reason I, and others in my department, use Part I National Board scores in residency selection process is that I believe they correlate with scores on Ophthalmic Knowledge Assessment Program (OKAP) examinations given yearly to residents by American Academy of Ophthalmology. Before we began using

The Ophthalmic Knowledge Assessment Program (OKAP): A personal view

This essay relates the historic process that led to the development of the Ophthalmic Knowledge Assessment Program (OKAP) as an established component of ophthalmic training in the U.S.A. Begun in the early 1960s as an evaluative examination for residents at the University of Florida, the first national OKAP examination took place in May 1968, with over 85% of all eye training programs participating. It was later incorporated into the Continuing Education program of the American Academy of Ophthalmology and Otolaryngology, where it prospered. Beginning January 1980 the OKAP was combined with American Board of Ophthalmology exam, thus establishing a single, unified national exam.