Performance of generative large language models in answering questions from the Brazilian Retina and Vitreous Society certification exam.
Natural language models and chatbots, particularly OpenAI's Generative Pre-Trained Transformer architecture, have transformed human interaction with digital interfaces. The latest versions, including ChatGPT-4o, offer enhanced functionalities compared to their predecessors. This study evaluates the accuracy of ChatGPT-4, ChatGPT-4o, and Claude 3.5 Sonnet in answering questions from the Brazilian Retina and Vitreous Society certification exam. We compiled 200 multiple-choice questions from the Brazilian Retina and Vitreous Society 2018 and 2019 exams. Questions were categorized into three domains: Anatomy and Physiology of the Retina, Retinal Pathology, and Diagnosis and Treatment. Using a standardized prompt developed according to prompt design guidelines, we tested ChatGPT-4, ChatGPT-4o, and Claude 3.5 Sonnet, recording their first responses as final. Three retina specialists performed a qualitative analysis of the answers. Accuracy was determined by comparing responses to the official correct answers. Statistical analysis was conducted using chi-square tests and Cohen's Kappa. Claude 3.5 Sonnet achieved the highest overall accuracy (72.5%), followed by ChatGPT-4o (66.0%) and ChatGPT-4 (55.5%). Claude 3.5 Sonnet and ChatGPT-4o significantly outperformed ChatGPT-4 (p<0.01 and p=0.03, respectively), while no significant difference was observed between Claude 3.5 Sonnet and ChatGPT-4o (p=0.16). Model responses agreed 74.5% of the time, with a Cohen's κ of 0.47. Retinal Pathology was the best-performing domain for all models, whereas Anatomy and Physiology of the Retina and Diagnosis and Treatment were the weakest domains for Claude 3.5 Sonnet and ChatGPT-4, respectively. This study is the first to assess Claude 3.5 Sonnet, ChatGPT-4, and ChatGPT-4o in retina specialist certification exams. Claude 3.5 Sonnet and ChatGPT-4o significantly outperformed ChatGPT-4, highlighting their potential as effective tools for studying retina specialist board exams. These findings suggest that the enhanced functionalities of Claude 3.5 Sonnet and ChatGPT-4o offer substantial improvements in medical education contexts.
- Research Article
42
- 10.1016/j.oret.2020.12.022
- Jan 1, 2021
- Ophthalmology Retina
Standardization of OCT Angiography Nomenclature in Retinal Vascular Diseases: First Survey Results
- Research Article
- 10.1097/icu.0000000000000792
- Sep 1, 2021
- Current Opinion in Ophthalmology
Editorial introductions
- Research Article
- 10.1097/icu.0000000000000697
- Sep 1, 2020
- Current Opinion in Ophthalmology
Current Opinion in Ophthalmology was launched in 1990. It is one of a successful series of review journals whose unique format is designed to provide a systematic and critical assessment of the literature as presented in the many primary journals. The field of Ophthalmology is divided into nine sections that are reviewed once a year. Each section is assigned a Section Editor, a leading authority in the area, who identifies the most important topics at that time. Here we are pleased to introduce the Journal's Section Editor for this issue. SECTION EDITOR Judy E. KimJudy E. KimDr Judy E. Kim, MD is an internationally well-known vitreoretinal specialist. She is a graduate of the University of Chicago, Johns Hopkins University School of Medicine, and Howard Hughes Medical Institute-National Institutes of Health Research Scholars Program. She completed her ophthalmology residency at the Bascom Palmer Eye Institute of the University of Miami and vitreoretinal fellowship at the Medical College of Wisconsin (MCW). Currently she is a Professor of Ophthalmology and Visual Sciences with tenure and a Professor of the Graduate School of Biomedical Sciences at the MCW. As a Korean-American who immigrated from South Korea, she has achieved many firsts. She is the first Korean-American to become a member of the American Ophthalmological Society which is the oldest medical society in the country, first to become a member of all three major US retina societies, the first to become a Board of Trustee member for American Academy of Ophthalmology (AAO), and the second to become a Professor of Ophthalmology in the United States and the first to do so in retina. She is only the second woman to serve on the executive committee of American Society of Retina Specialists (ASRS), the largest retina organization in the world with members from 60 countries and will be only the second woman to serve as the President of ASRS. She is currently the chair of Women in Retina. She has held leadership positions in multiple committees of AAO, Association for Research and Vision in Ophthalmology, Macula Society, ASRS, Retina Society, and Milwaukee Ophthalmological Society. She serves on the editorial board of JAMA Ophthalmology, OSLI and Ocular Surgery News and is a member of the Board of NAEVR/AEVR and the National Eye Health Education Group of National Institutes of Health. Dr Kim has received numerous awards and honors, especially for her clinical excellence, leadership, and service to organizations. She has received Honor Award and Senior Honor Award from ASRS, Achievement Award and Senior Achievement Award from AAO, Heed Foundation Fellowship, and Women Pioneers Research Award. She is the recipient of the prestigious Suzanne Veronneau-Troutman Award from Women in Ophthalmology. She has been named in the “Best Doctor” annually since 2003, has been named in the “Retina Hall of Fame”. She has published over 200 papers and given over 400 presentations, including 150 invited national and international presentations. She has mentored numerous students, residents, fellows, and international retina specialists. She has been actively involved with over 60 multicenter clinical trials and has served as a vice-chair of Diabetic Retinopathy Clinical Research Retina network, in which she is currently a national study chair for Protocol AE. She leads the TeleEye Health Collaborative in Wisconsin. She initiated a program called the “Eyes on the Future” to mentor under-represented minority middle school students. In addition to diabetic retinopathy and age-related macular degeneration, her research interests include surgical retina, telemedicine, ocular imaging, and community engaged research. She is married to Dr. John K. Hur and has a daughter who is an OB/GYN resident and a son who is a first lieutenant in Marine Corps, studying to become a pilot. In her spare time, she enjoys singing, gardening, playing the piano, photography, traveling, and culinary adventures. Ehsan RahimyEhsan RahimyDr Ehsan Rahimy is a surgical and medical retina specialist practicing in the San Francisco Bay Area. He completed his ophthalmology residency at the Jules Stein Eye Institute at UCLA, and his surgical retina fellowship at the Wills Eye Hospital. Dr Rahimy is passionate about the interplay between technology and medicine, and how ongoing advancements will transform healthcare delivery in the near future. Dr Rahimy is frequently consulted for collaborative research endeavors and advises on numerous early stage companies involved in ophthalmology, telemedicine, A.I., and other medtech innovation. Allen ChiangAllen ChiangDr Allen Chiang graduated Magna Cum Laude from Brown University. He received his Doctor of Medicine degree from the New York University School of Medicine, where he was elected to the Alpha Omega Alpha Medical Honor Society. He then completed a transitional internship at Scripps Mercy Hospital in San Diego, the oldest hospital and one of the best in that region. Thereafter, he completed a residency in ophthalmology at the prestigious Stein Eye Institute at the University of California at Los Angeles (UCLA), perennially the #1 eye hospital in the western US. At UCLA, his performance was dually recognized with the Stein Resident Awards for Surgical Excellence and Clinical Research. He then completed a two-year medical and surgical retina fellowship at Wills Eye Hospital in Philadelphia and was the recipient of the William Tasman, MD Outstanding Fellow Award. Prior to joining Mid Atlantic Retina, Dr Chiang was in private practice in the San Francisco Bay Area as a retina surgeon and garnered recognition as a Top Doctor by Oakland Magazine. In 2013, he returned to Philadelphia to join Mid Atlantic Retina and serve as an active member of the Retina Service of Wills Eye Hospital. He is an Assistant Professor of Ophthalmology at Sidney Kimmel Medical College of Thomas Jefferson University and is actively involved in the education and training of the next generation of ophthalmologists and retina specialists. Board-certified by the American Board of Ophthalmology since 2010, Dr Chiang is an active member of the American Academy of Ophthalmology (AAO), American Society of Retina Specialists (ASRS), Association for Research in Vision and Ophthalmology, Pennsylvania Academy of Ophthalmology, Montgomery County Medical Society, and the Vit-Buckle Society. In addition, he is an elected member of the esteemed Retina Society. He is a member of the editorial boards of Retina Today and the AAO's Ophthalmic News and Education (ONE) Network, which is the Academy's global platform for continuing medical education for over 30,000 ophthalmologists. Dr Chiang has contributed over 50 original papers, editorials, and textbook chapters on vitreoretinal diseases to help advance the field. He serves as an ad hoc reviewer for multiple journals including RETINA, American Journal of Ophthalmology, and British Journal of Ophthalmology. With a clinical interest in developing new treatments for retinal diseases, he is a Principal Investigator or co-investigator at Wills Eye Hospital for many collaborative national and international clinical trials for conditions such as age-related macular degeneration, diabetic retinopathy, and surgical retinal diseases. He has been invited to speak and present scientific papers at national ophthalmic meetings and earned an Honor Award from the ASRS for this work.
- Research Article
80
- 10.1155/2017/8234186
- Jan 1, 2017
- Journal of Ophthalmology
A group of members of the Spanish Retina and Vitreous Society (SERV) and of the Working Group of Ocular Health of the Spanish Society of Diabetes (SED) updated knowledge regarding the diagnosis and treatment of diabetic retinopathy (DR) based on recent evidence reported in the literature. A synthesis of this consensus forms the basis of the present review, which is intended to inform clinicians on current advances in the field of DR and their clinical applicability to patients with this disease. Aspects presented in this article include screening procedures of DR, new technologies in the early diagnosis of DR, control of risk factors in the different stages of the disease, indications of panretinal laser photocoagulation, efficacy of intravitreal antiangiogenic agents and steroids, and surgical options for treating DR-related complications. Practical information regarding periodicity of screening procedures in patients with type 1 and type 2 diabetes, ophthalmological controls according to the stage of retinopathy and complications, and criteria and degree of urgency for referral of a DR patient to the ophthalmologist are also presented.
- Research Article
19
- 10.5935/0004-2749.20150009
- Jan 1, 2015
- Arquivos Brasileiros de Oftalmologia
To evaluate and describe the precautions involved in the technique of intravitreal injection of antiangiogenic drugs adopted by the ophthalmologists who are members of the Brazilian Society of Retina and Vitreous (SBRV). A questionnaire containing 22 questions related to precautions taken before, during, and after intravitreal injection was sent electronically to 920 members of SBRV between November 15, 2013 and April 31, 2014. 352 responses (38%) were obtained. There was a predominance of men (76%) from the southwest region of Brazil (51%). The professional experience varied between 6 and 15 years after medical specialization (50%). Most professionals (76%) performed an average of 1 to 10 intravitreal injections a week, and 88% of the procedures were performed in the operating room using povidone iodine (99%), sterile gloves, and blepharostat (94%). For inducing topical anesthesia, usage of anesthetic eye drops was the most used technique (65%). Ranibizumab (Lucentis®) was the most common drug (55%), and age-related macular degeneration (AMD) was the most treated disease (57%). Regarding the complications treated, 6% of the ophthalmologists had treated at least one case of retinal detachment, 20% had treated cases of endophthalmitis, 9% had treated cases of vitreous hemorrhage, and 12% had encountered cases of crystalline lens touch. Intravitreal injection is a procedure routinely performed by retina specialists and has a low incidence of complications. Performing the procedure in the operating room using an aseptic technique was preferred by most of the respondents. Ranibizumab was the most used drug, and AMD was the most treated disease.
- Research Article
- 10.1007/s10384-026-01348-x
- May 6, 2026
- Japanese journal of ophthalmology
To investigate differences in work styles and career satisfaction between women and men physicians in the retina subspecialty in Japan using a questionnaire survey. Cross-sectional study. A web-based questionnaire was sent as an e-mail link to all 3,027 current members of the Japanese Retina and Vitreous Society in August 2023 with responses collected anonymously in September 2023. The questionnaire consisted of 4 main categories: baseline characteristics, job satisfaction, life events and career advancement, and suitability as a retina specialist and workplace gender equality. Valid responses were received from 614 members, representing a response rate of 20.3%. One-fifth of men (19.0%) were "very satisfied" with their job, much higher than the same for women (7.5%) (P < 0.001). Furthermore, there was a significant difference between men and women who "strongly agreed" or "agreed" that life events were an impediment to one's career (44.3% vs 74.9%) (P < 0.001). There was also a significant difference between men and women who answered "strongly agree" when asked whether they felt they were suited to work in the retina subspecialty (31.0% vs 13.4%) (P < 0.001). Our study identified and quantified marked differences in work style and career satisfaction between women and men. Such differences highlight gender-based challenges in the retina subspecialty that could be improved in order to enhance career satisfaction and career longevity of women retina specialists in Japan.
- Conference Article
6
- 10.1109/robot.2010.5509532
- May 1, 2010
This paper describes an novel approach towards linguistic processing for robots through integration of a motion language model and a natural language model. The motion language model works for association of words from motion symbols. The natural language model is one used for a morphological analysis, which has been developed in natural language community. The natural language model is optimized using a enormous amount of words. So this model is scalable architecture. The motion language model and the natural language model can be integrated since both models are represented graphically. The integration of the motion language model and the natural language model allows robots not only to interpret motion patterns as sentences but also to generate motions from sentences. This paper demonstrates the validity of our proposed framework even in the case that large-scale word corpus is needed processing through experiments of interpreting motion patterns as sentences and generating motion patterns from sentences.
- Research Article
54
- 10.1016/j.jcjo.2014.03.009
- May 24, 2014
- Canadian Journal of Ophthalmology
Survey of intravitreal injection techniques and treatment protocols among retina specialists in Canada
- Conference Article
23
- 10.1109/icra.2012.6225331
- May 1, 2012
The language is a symbolic system unique to human being. The acquisition of language, which has its meanings in the real world, is important for robots to understand the environment and communicate with us in our daily life. This paper proposes a novel approach to establish a fundamental framework for the robots which can understand language through their whole body motions. The proposed framework is composed of three modules: “motion symbol”, “motion language model”, and “natural language model”. In the motion symbol module, motion data are symbolized by Hidden Markov Models (HMMs). Each HMM represents abstract motion patterns. Then the HMMs are defined as motion symbols. The motion language model is stochastically designed for links between motion symbols and words. This model consists of three layers of motion symbols, latent states and words. The connections between the motion symbol and the latent state, and between the latent state and the words are denoted by two kinds of probabilities respectively. One connection is represented by the probability that the motion symbol generates the latent state, and the other connection is represented by the probability that the latent state generates the word. Therefore, the motion language model can connect the motion symbols to the words through the latent state. The natural language model stochastically represents sequences of words. In this paper, a bigram, which is a special case of N-gram model, is adopted as the natural language model. This model has the words as nodes and transitions between two words as edges. Therefore sentence structure is expressed as transitions among words. The integration of the motion language model and natural language model can be implemented by the search computation for sentences corresponding to motions and for motions corresponding to sentences. Especially, the usage of the bigram as the natural language model provides a simple search computation so that appropriate and fast bidirectional computation between the motions and language can be achieved. Our approach makes it possible for humanoid robots not only to interpret motions as sentences but also to generate motions from sentences. The tests by using various motions and words validate our framework for the language acquisition of humanoid robots.
- Research Article
51
- 10.1177/0278364915587923
- Jul 24, 2015
- The International Journal of Robotics Research
This paper describes a novel approach to linguistic mutual inference, which enables robots not only to linguistically interpret the motion patterns in the form of sentences but also to generate the motions from the sentences. The inference can be established based on two modules, the motion language model and the natural language model. The motion language model stochastically represents an association structure between symbols of motion patterns and the words in sentences assigned to the motion. This is a statistical model with a three layered structure of motion symbols, latent states and words. The natural language model statistically represents a structure of sentences based on word bigrams. The motion language model and the natural language model correspond to semantics and syntax respectively. An approach to the integration of motion language model with the natural language model allows the linguistic mutual inference for the robots. The two kinds of inference can be made by solving search problems, search for a sequence of words corresponding to a motion and search for a symbol of motion pattern corresponding to a sentence. The proposed approach to interpretation of motion patterns as sentences and generation of motion patterns from the sentences through the integration of motion language model with the natural language model is validated by an experiment on the human behavioral data.
- Conference Article
30
- 10.1109/robot.2009.5152574
- May 1, 2009
This paper describes the linguistic model based on symbolization of motion patterns for humanoid robots. The model consists of two kinds of stochastic models : the motion language model and the natural language model. The motion language model stochastically connects the symbols of motion patterns to the morpheme words through the latent states which represent the underlying linguistic structure such as semantic contents. The natural language model represents the dynamics of the word classes. The motion language model and the natural language model correspond to semantics and syntax respectively. The integration of the motion language model and the natural language model allows robots not only to linguistically interpret the motion patterns as sentences but also to generate the motions from the sentences. The two kinds of linguistic processes of the interpretation and the generation can be obtained by solving search problems: search for a sequence of morpheme words and a symbol of motion pattern. The proposed approach to interpretation of motion patterns as sentences and generation of motion patterns from the sentences through integration of the motion language model and the natural language model is validated by the experiment on the human behavioral data.
- Research Article
1
- 10.7210/jrsj.30.674
- Jan 1, 2012
- Journal of the Robotics Society of Japan
This paper describes a novel approach to linguistic mutual inference, which enables robots not only to linguistically interpret the motion patterns as sentences but also to recall the motions from the sentences. The inference can be established based on the motion language model and the natural language model. The motion language model stochastically connects the symbols of motion patterns to the words through the latent states which represent the underlying linguistic structure. The natural language model represents sequences of words. The motion language model and the natural language model correspond to semantics and syntax respectively. The integration of the motion language model and the natural language model allows the linguistic mutual inference for the robots . The two kinds of inference can be made by solving search problems: search for a sequence of words and a symbol of motion pattern. The proposed approach to interpretation of motion patterns as sentences and recall of motion patterns from the sentences through integration of the motion language model and the natural language model is validated by the experiment on the human behavioral data.
- Conference Article
24
- 10.1109/ichr.2008.4755976
- Dec 1, 2008
This paper describes the novel approach to integration of natural language processing and symbolization of motion patterns in order to allow for humanoid robotpsilas acquisition of language. This framework consists of two models : motion language model and natural language model. In the motion language model, morpheme words are stochastically associated with symbolized motion patterns via latent variables. The association is defined by probability that the motion pattern generates the latent variable and probability that the latent variable generates the morpheme word. The natural language model represents order relation among the morpheme words via word classes by using hidden Markov model. The motion language model and the natural language model are equivalent to semantics and syntax respectively. Integration of the motion language model and the natural language model achieves linguistic interpretation of motion patterns by composing semantically and syntactically appropriate sentence. The efficient algorithm for the composition is proposed. The validity of the motion language model, the natural language model and the integration is demonstrated by testing the implemented algorithm on human motion data.
- Research Article
27
- 10.1167/tvst.9.4.13
- Mar 16, 2020
- Translational Vision Science & Technology
PurposeSubretinal fibrosis (SRFib) is an important cause of permanent loss-of-vision diseases with submacular neovascularization, but a reliable diagnostic method is currently missing. This study uses polarization-sensitive optical coherence tomography (PS-OCT) to detect SRFib within retinal lesions by measurement of its birefringent collagen fibers.MethodsTwenty-five patients were enrolled with retinal pathology in one or both eyes containing (1) suspected SRFib, (2) lesions suspected not to be fibrotic, or (3) lesions with doubtful presence of SRFib. All eyes were evaluated for SRFIb using conventional diagnostics by three retinal specialists. PS-OCT images were visually evaluated for SRFib based on cumulative phase retardation, local birefringence, and optic axis uniformity.ResultsTwenty-nine eyes from 22 patients were scanned successfully. In 13 eyes, SRFib was diagnosed by all retinal specialists; of these, 12 were confirmed by PS-OCT and one was inconclusive. In nine eyes, the retinal specialists expected no SRFib, which was confirmed by PS-OCT in all cases. In seven eyes, the retinal specialists’ evaluations were inconsistent with regard to the presence of SRFib. PS-OCT confirmed the presence of SRFib in four of these eyes and the absence of SRFib in two eyes and was inconclusive in one eye.ConclusionsIn 21 out of 22 eyes, PS-OCT confirmed the evaluation of retinal specialists regarding the presence of SRFib. PS-OCT provided additional information to distinguish SRFib from other tissues within subretinal neovascular lesions in 6 out of 7 eyes.Translational RelevancePS-OCT can identify and quantify SRFib in doubtful cases for which a reliable diagnosis is currently lacking.
- Conference Article
- 10.18653/v1/2024.acl-long.542
- Jan 1, 2024
Recent studies have shown that integrating constructional information can improve the performance of pre-trained language models (PLMs) in natural language understanding.However, exploration into leveraging constructional information to enhance generative language models for natural language generation has been limited.Additionally, probing studies indicate that PLMs primarily grasp the syntactic structure of constructions but struggle to capture their semantics.In this work, we encode constructions as inductive biases to explicitly embed constructional semantics and guide the generation process.We begin by presenting a construction grammar induction framework designed to automatically identify constructions from corpora.Subsequently, we propose the Construction-Enhanced Language Model (CoELM).It introduces a construction-guided language modeling approach that employs a dynamic sequence reassembly strategy during pre-training.Extensive experiments have demonstrated the superiority of CoELM across various benchmarks.