Radiology Residents Research Articles

Efficacy of Fine-Tuned Large Language Model in CT Protocol Assignment as Clinical Decision-Supporting System

Abstract Accurate CT protocol assignment is crucial for optimizing medical imaging procedures. The integration of large language models (LLMs) may be helpful, but its efficacy as a clinical decision support system for protocoling tasks remains unknown. This study aimed to develop and evaluate fine-tuned LLM specifically designed for CT protocoling, as well as assess its performance, both standalone and in concurrent use, in terms of effectiveness and efficiency within radiological workflows. This retrospective study included radiology tests for contrast-enhanced chest and abdominal CT examinations (2829/498/941 for training/validation/testing). Inputs involve the clinical indication section, age, and anatomic coverage. The LLM was fine-tuned for 15 epochs, selecting the best model by macro sensitivity in validation. Performance was then evaluated on 800 randomly selected cases from the test dataset. Two radiology residents and two radiologists assigned CT protocols with and without referencing the output of LLM to evaluate its efficacy as a clinical decision support system. The LLM exhibited high accuracy metrics, with top-1 and top-2 accuracies of 0.923 and 0.963, respectively, and a macro sensitivity of 0.907. It processed each case in an average of 0.39 s. The LLM, as a clinical decision support tool, improved accuracy both for residents (0.913 vs. 0.936) and radiologists (0.920 vs. 0.926 without and with LLM, respectively), with the improvement for residents being statistically significant (p = 0.02). Additionally, it reduced reading times by 14% for residents and 12% for radiologists. These results indicate the potential of LLMs to improve CT protocoling efficiency and diagnostic accuracy in radiological practice.

Impact of software-assisted structured reporting on radiology residents approaching prostate MRI.

To evaluate the potential advantages of software-assisted structured reporting for radiology residents approaching multiparametric prostate MRI (mpMRI). MpMRI scans from 100 patients, performed for prostate cancer (PCa) detection or staging, were anonymized, and reviewed by six second-year radiology residents without previous experience in prostate MRI, following 6h of intensive training. The dataset was split into two subsets of 50 cases each. All residents were asked to report scans from the first subset using a basic text processor (narrative reports -NR-). For the second subset, one group used a dedicated software to produce structured reports (SR) while the other continued with NR. Report completeness was assessed using a PI-RADS-based checklist, and statistical analyses, including Wilcoxon rank sum and Pearson's Chi-squared tests, were performed to compare word count, reporting time, and concordance with an expert radiologist's findings. All readers adopting SR in the second batch demonstrated a significant increase in word count and a decrease in reporting time compared to the first batch. Image quality and final impressions were missing from all NR, while gland size, lesion description, and PI-RADS score were consistently included in nearly all reports (96-100%). One of the three residents using SR showed a statistically significant improvement in concordance with the expert radiologist on index lesion location and clinically significant PCa presence (p=0.001), while the other two exhibited positive trends (p=0.061-0.078). The adoption of SR allowed radiology residents to decrease their reporting time and improve the comprehensiveness of their reports, while increasing concordance with an expert radiologist.

Abstract TMP59: Artificial Intelligence Improves Detection Sensitivity for Challenging Acute Ischemic Stroke Lesions on Diffusion-weighted Imaging

Introduction: Diffusion-weighted imaging (DWI) is key for detecting acute ischemic brain lesions but struggles with hyperacute or small lesions that mimic artifacts. This randomized crossover trial assessed whether an artificial intelligence (AI) solution enhances diagnostic accuracy for these challenging lesions compared to conventional interpretation. Methods: From February 2017 to November 2021, 4,071 suspected acute ischemic stroke patients underwent initial and follow-up DWI. A neurologist assessed ischemic stroke based on medical records, and a neuroradiologist established the ground truth using the neurologist's evaluation, MRI reports, and AI-marked DWI images. The accuracy of AI and MRI reports was then evaluated against this ground truth. For a reader performance study, 874 challenging cases were selected: (1) infarct volume < 0.5 mL in the posterior circulation on follow-up DWI or (2) initial DWI within 3 hours of onset with infarct volume < 1.0 mL in the anterior circulation. Additionally, 80 negative and 40 positive control cases were included. Five readers (a neuroradiologist, two radiology residents, and two neurology residents) interpreted the DWIs, half with AI assistance and half without. After a 4-week washout, cases were re-evaluated with the groups reversed. We compared the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and Dice similarity coefficient (DSC) between readings with and without AI. Results: Of 3,981 patients, 3,465 had acute ischemic strokes, and 516 had MRI-negative transient ischemic attacks. The AI alone showed a positive predictive value (PPV) of 93.1% and a negative predictive value (NPV) of 81.3%. The PPV of AI varied significantly with predicted infarction volume: 81.7% for ≤ 0.5 mL vs. 99.5% for > 0.5 mL. With AI, the sensitivity of MRI reports could potentially increase from 98.5% to 99.7%, ensuring identification of all false negatives. In the challenging case reader study, AI significantly increased AUC (0.848 vs. 0.927; p < 0.001) and sensitivity (74.59% to 90.59%; p < 0.001), with minimal impact on specificity (88.75% vs. 84.00%; p = 0.0496). AI-assisted segmentation also showed higher DSC compared to non-AI segmentation (0.742 vs. 0.523; p < 0.001). Conclusions: AI significantly improved the diagnostic performance for challenging acute ischemic lesions on DWI, demonstrating the potential to enhance stroke care.

Upskilling or deskilling? Measurable role of an AI-supported training for radiology residents: a lesson from the pandemic

ObjectivesThis article aims to evaluate the use and effects of an artificial intelligence system supporting a critical diagnostic task during radiology resident training, addressing a research gap in this field.Materials and methodsWe involved eight residents evaluating 150 CXRs in three scenarios: no AI, on-demand AI, and integrated-AI. The considered task was the assessment of a multi-regional severity score of lung compromise in patients affected by COVID-19. The chosen artificial intelligence tool, fully integrated in the RIS/PACS, demonstrated superior performance in scoring compared to the average radiologist. Using quantitative metrics and questionnaires, we measured the ‘upskilling’ effects of using AI support and residents’ resilience to ‘deskilling,’ i.e., their ability to overcome AI errors.ResultsResidents required AI in 70% of cases when left free to choose. AI support significantly reduced severity score errors and increased inter-rater agreement by 22%. Residents were resilient to AI errors above an acceptability threshold. Questionnaires indicated high tool usefulness, reliability, and explainability, with a preference for collaborative AI scenarios.ConclusionWith this work, we gathered quantitative and qualitative evidence of the beneficial use of a high-performance AI tool that is well integrated into the diagnostic workflow as a training aid for radiology residents.Critical relevance statementBalancing educational benefits and deskilling risks is essential to exploit AI systems as effective learning tools in radiology residency programs. Our work highlights metrics for evaluating these aspects.Key PointsInsights into AI tools’ effects in radiology resident training are lacking.Metrics were defined to observe residents using an AI tool in different settings.This approach is advisable for evaluating AI tools in radiology training.Graphical

CompositIA: an open-source automated quantification tool for body composition scores from thoraco-abdominal CT scans

BackgroundBody composition scores allow for quantifying the volume and physical properties of specific tissues. However, their manual calculation is time-consuming and prone to human error. This study aims to develop and validate CompositIA, an automated, open-source pipeline for quantifying body composition scores from thoraco-abdominal computed tomography (CT) scans.MethodsA retrospective dataset of 205 contrast-enhanced thoraco-abdominal CT examinations was used for training, while 54 scans from a publicly available dataset were used for independent testing. Two radiology residents performed manual segmentation, identifying the centers of the L1 and L3 vertebrae and segmenting the corresponding axial slices. MultiResUNet was used to identify CT slices intersecting the L1 and L3 vertebrae, and its performance was evaluated using the mean absolute error (MAE). Two U-nets were used to segment the axial slices, with performance evaluated through the volumetric Dice similarity coefficient (vDSC). CompositIA’s performance in quantifying body composition indices was assessed using mean percentage relative error (PRE), regression, and Bland–Altman analyses.ResultsOn the independent dataset, CompositIA achieved a MAE of about 5 mm in detecting slices intersecting the L1 and L3 vertebrae, with a MAE < 10 mm in at least 85% of cases and a vDSC greater than 0.85 in segmenting axial slices. Regression and Bland–Altman analyses demonstrated a strong linear relationship and good agreement between automated and manual scores (p values < 0.001 for all indices), with mean PREs ranging from 5.13% to 15.18%.ConclusionCompositIA facilitated the automated quantification of body composition scores, achieving high precision in independent testing.Relevance statementCompositIA is an automated, open-source pipeline for quantifying body composition indices from CT scans, simplifying clinical assessments, and expanding their applicability.Key PointsManual body composition assessment from CTs is time-consuming and prone to errors.CompositIA was trained on 205 CT scans and tested on 54 scans.CompositIA demonstrated mean percentage relative errors under 15% compared to manual indices.CompositIA simplifies body composition assessment through an artificial intelligence-driven and open-source pipeline.Graphical

The use of large language models in detecting Chinese ultrasound report errors

This retrospective study evaluated the efficacy of large language models (LLMs) in improving the accuracy of Chinese ultrasound reports. Data from three hospitals (January-April 2024) including 400 reports with 243 errors across six categories were analyzed. Three GPT versions and Claude 3.5 Sonnet were tested in zero-shot settings, with the top two models further assessed in few-shot scenarios. Six radiologists of varying experience levels performed error detection on a randomly selected test set. In zero-shot setting, Claude 3.5 Sonnet and GPT-4o achieved the highest error detection rates (52.3% and 41.2%, respectively). In few-shot, Claude 3.5 Sonnet outperformed senior and resident radiologists, while GPT-4o excelled in spelling error detection. LLMs processed reports faster than the quickest radiologist (Claude 3.5 Sonnet: 13.2 s, GPT-4o: 15.0 s, radiologist: 42.0 s per report). This study demonstrates the potential of LLMs to enhance ultrasound report accuracy, outperforming human experts in certain aspects.

Take your PIC (Program Improvement Committee): The benefits of a resident-run initiative to improve graduate medical education.

The evolving landscape of graduate medical education (GME) necessitates innovative approaches to residency program evaluation and improvement. At Detroit Medical Center/Wayne State University, a novel resident-led Program Improvement Committee (PIC) was established in June 2022 within the Diagnostic Radiology Residency Program. The PIC serves as a flexible, resident-driven structure designed to enhance engagement, provide continuous feedback, and implement actionable solutions in collaboration with residency leadership. Unlike traditional Program Evaluation Committees (PECs) and Clinical Competency Committees (CCCs), the PIC promotes a more democratized approach, empowering residents to take ownership of their training. By convening monthly and fostering fluid subcommittees, the PIC has successfully driven initiatives including increased research engagement, mentorship programs, social media enhancement, and system-level improvements such as the establishment of a dedicated WiFi network for hospital employees. The PIC represents a paradigm shift in residency enhancement, offering a responsive, adaptable mechanism for fostering innovation, inclusivity, and collaboration. Further research is needed to evaluate its broader impact, but the PIC has the potential to revolutionize residency training and improve medical education.

P0460 Intestinal ultrasound in inflammatory bowel disease: Do we really need experts to monitor our patients?

Abstract Background Intestinal ultrasound (IUS) has emerged as an important tool in monitoring inflammatory bowel disease (IBD) due to its accessibility, cost-effectiveness, and non-invasive nature, making it an attractive option for gastroenterologists. Our study aimed to assess the agreement between young sonographers and experts in the interpretation of IUS examinations in patients with IBD Methods We video-recorded the IUS of a patient with ulcerative colitis in severe acute relapse. A cine-loop of the sigmoid colon was made from the video including a longitudinal and transverse B-mode and CDS-mode. The cine-loop and an electronic form, were sent to 30 radiology residents (Ten in their 2nd year, ten 3rd year and ten 4th year, chosen randomly), along with an IUS expert with 10 years’ experience. Participants were asked to complete the electronic form including key parameters in IUS, such as bowel wall thickness [BWT], color Doppler signal [CDS] (using the modified Limberg score), inflammatory fat [i-fat], bowel wall stratification loss [BWS], haustration loss and presence of lymph nodes. Two scores covering these parameters were also used: Milan ultrasound criteria (MUC) and the International Bowel Ultrasound Segmental Activity Score [IBUS-SAS]. Inter-observer reliability was analyzed using Cohen’s Kappa coefficient (Jamovi version 2.3) Results The expert’s findings compared to the average values reported by all residents were as follows: BWT (6.04mm vs 6.3mm ± 3.63), MUC (10.46 vs 10.7 ± 4.81), IBUS (87.17 vs 65.6 ± 21.7). A strong positive association was found between the 3rd and 4th years for BWT (r= 0.562), although not statistically significant (p= 0.091). Fair agreement was found for BWS [κ = 0.329, p= 0.003]. No significant differences in BWT, MUC, IBUS, and modified Limberg score values were found between the three groups of residents (p= 0.762, p= 0.759, p= 0.922, and p= 0.781 respectively). When compared with the expert’s values, there were no significant differences between the BWT and MUC values (2nd year p= 0.625, 3rd year p = 0.492, 4th year p= 0.375). In contrast, the residents’ IBUS and modified Limberg score values were significantly different from the expert’s values (IBUS: 2nd year p= 0.002, 3rd year p= 0.05, 4th year p= 0.01, Limberg: 2nd year p= 0.01, 3rd year: p= 0.017, 4th year p= 0.025) Conclusion This study has shown us that our young sonographers are providing encouraging results in the monitoring of IBD. However, the presence of an expert in intestinal ultrasound remains essential in the evaluation of the various ultrasound parameters. Current recommendations in gastroenterology place IUS for IBD in the spotlight, and it is essential that our collaborators can easily help us in the monitoring of these patients

The Art Museum as a Training Ground for Residents in Nuclear Medicine and Radiology

ABSTRACT For the past seven years, we – a museum educator and an academic radiologist – have co-taught Seeing in Art and Medical Imaging, a first-of-its-kind medical humanities curriculum for residents in nuclear medicine and radiology based at the Harvard Art Museums. The program aims to foster residents’ close looking and clinical skills, self-reflection, and humanistic qualities. Here we describe some of the ways we engage trainee imaging physicians with works of art to help them prepare for a fulfilling, engaged, and sustainable career. We also offer some initial takeaways from the program’s anecdotal data. Finally, we discuss the Seeing in Art and Medicine exhibition, on view at the Harvard Art Museums in Fall 2023, which gave the public a window into this work.

Machine Learning-Aided Diagnosis Enhances Human Detection of Perilunate Dislocations.

Perilunate/lunate injuries are frequently misdiagnosed. We hypothesize that utilization of a machine learning algorithm can improve human detection of perilunate/lunate dislocations. Participants from emergency medicine, hand surgery, and radiology were asked to evaluate 30 lateral wrist radiographs for the presence of a perilunate/lunate dislocation with and without the use of a machine learning algorithm, which was used to label the lunate. Human performance with and without the machine learning tool was evaluated using sensitivity, specificity, accuracy, and F1 score. A total of 137 participants were recruited, with 55 respondents from emergency medicine, 33 from radiology, and 49 from hand surgery. Thirty-nine participants were attending physicians or fellows, and 98 were residents. Use of the machine learning tool improved specificity from 88% to 94%, accuracy from 89% to 93%, and F1 score from 0.89 to 0.92. When stratified by training level, attending physicians and fellows had an improvement in specificity from 93% to 97%. For residents, use of the machine learning tool resulted in improved accuracy from 86% to 91% and specificity from 86% to 93%. The performance of surgery and radiology residents improved when assisted by the tool to achieve similar accuracy to attendings, and their assisted diagnostic performance reaches levels similar to that of the fully automated artificial intelligence tool. Use of a machine learning tool improves resident accuracy for radiographic detection of perilunate dislocations, and improves specificity for all training levels. This may help to decrease misdiagnosis of perilunate dislocations, particularly when subspecialist evaluation is delayed.

Impact of trauma center status on radiology resident performance in detecting non-accidental trauma on the WIDI SIM exam.

To evaluate whether adult and pediatric trauma center status, as well as the presence of dedicated child protection teams, influences radiology resident performance in detecting non-accidental trauma on the Emergent/Critical Care Imaging Simulation (WIDI SIM) exam. We retrospectively analyzed 639 WIDI SIM exam scores for four pediatric non-accidental trauma cases completed by radiology residents across 33 programs. Residents were stratified by level (R1-R4) and institutional factors, including adult trauma center status, pediatric trauma center status, and child protection team presence. Additionally, 2022 pediatric physical abuse counts per state were obtained from the National Child Abuse and Neglect Data System and normalized between 0 and 1. We employed a negative binomial generalized linear model (GLM). The outcome was regressed on resident level, trauma center status variables, child protection team presence, and normalized abuse counts. Higher resident training level was significantly associated with better performance (R1=3.6, R2=5, R3=6.1, R4=7.9; P<0.001). Adult trauma center accreditation was also associated with higher scores (4.6 vs. 3.3; P<0.001). In contrast, pediatric trauma center status, child protection team presence, and normalized state-level abuse counts were not significantly associated with resident performance (all P>0.05). Resident experience and adult trauma center accreditation were significantly associated with better radiology resident performance in detecting non-accidental trauma in a simulated call environment. Pediatric trauma center status, child protection team presence, and statewide abuse prevalence were not associated with detection performance. These findings suggest the need for targeted educational interventions at non-adult trauma centers and emphasize the importance of experience in developing diagnostic proficiency for non-accidental trauma.

Evaluation of a content-based image retrieval system for radiologists in high-resolution CT of interstitial lung diseases

BackgroundThis retrospective study aims to evaluate the impact of a content-based image retrieval (CBIR) application on diagnostic accuracy and confidence in interstitial lung disease (ILD) assessment using high-resolution computed tomography CT (HRCT).MethodsTwenty-eight patients with verified pattern-based ILD diagnoses were split into two equal datasets (1 and 2). The images were assessed by two radiology residents (3rd and 5th year) and one expert radiologist in four sessions. Dataset 1 was used for sessions A and C, assessing diagnostic accuracy and confidence with mandatory and without CBIR software. Dataset 2 was used for sessions B and D with optional CBIR use, assessing time spending and frequency of CBIR usage. Accuracy was assessed on the CT pattern level, comparing readers’ diagnoses with reference diagnoses and CBIR results with region-of-interest (ROI) patterns.ResultsDiagnostic accuracy and confidence of readers showed an increasing trend with CBIR use compared to no CBIR use (53.6% versus 35.7% and 50.0% versus 32.2%, respectively). Time for reading significantly decreased in both datasets (A versus C: 104 s versus 54 s, p < 0.001; B versus D: 88.5 s versus 70 s, p = 0.009), whereas time for research increased with CBIR software use (A versus C: 31 s versus 81 s, p = 0.040). CBIR results showed a high pattern-based accuracy of overall 73.4%. Comparison between readers indicates a slightly higher accuracy of CBIR results when more than one ROI was used as input (77.7% versus 70.1%).ConclusionCBIR software improves in-training radiologist diagnostic accuracy and confidence while reducing interpretation time in ILD assessment.Relevance statementContent-based image retrieval software improves the assessment of interstitial lung diseases (ILD) in high-resolution CT, especially for radiology residents, by increasing diagnostic accuracy and confidence while reducing interpretation time. This can provide educational benefits and more time-efficient management of complex cases.Key PointsA content-based image retrieval (CBIR) software improves diagnostic accuracy and confidence for in-training radiologists for interstitial lung disease (ILD) assessment on computed tomography (CT).A CBIR application provides condensed information about similar HRCT cases reducing time for ILD assessment.CBIR algorithms benefit from the input of multiple regions of interest per ILD case.Graphical

Radiology resident competency in orthopedic trauma detection in simulated on-call scenarios.

To evaluate radiology residents' ability to accurately identify three specific types of orthopedic trauma using radiographic imaging within a simulated on-call environment. We utilized the Wisdom in Diagnostic Imaging Emergent/Critical Care Radiology Simulation (WIDI SIM) to assess residents' preparedness for independent radiology call. The simulation included 65 cases, with three focusing on orthopedic trauma: sacral ala, femoral neck, and pediatric tibial/Toddler's fractures. Faculty graded residents' responses using a standardized 10-point rubric and categorized errors as observational (failing to identify key findings) or interpretive (incorrect conclusions despite correct identification of findings). 321 residents evaluated sacral ala fracture radiographs and received an average score of 1.29/10, with 8.71 points lost to observational errors. Only 6% produced effective reports (scores ≥ 7), while 80% made critical errors (scores < 2). For femoral neck fracture CT images (n = 316 residents), the average score was 2.48/10, with 6.71 points lost to observational errors. 25% produced effective reports, and 66% made critical errors. Pediatric tibial/Toddler's fracture radiographs (n = 197 residents) yielded an average score of 2.94/10, with 6.60 points lost to observational errors. 29% generated effective reports, while 71% made critical errors. Radiology residents demonstrated significant difficulty in identifying these orthopedic trauma cases, with errors primarily attributed to observational deficiencies. These findings suggest a need for targeted educational interventions in radiology residency programs to improve the identification of these fractures.

Effectiveness of Social Media in Promoting Diversity, Equity, and Inclusion in Radiology Residency Programs.

Effectiveness of Social Media in Promoting Diversity, Equity, and Inclusion in Radiology Residency Programs.

Injury Spectrum and Radiology Department Response Assessment Following the Beirut Blast.

The catastrophic Beirut blast on August 4, 2020 at 6:07 pm resulted in extensive damage. Our study aims to categorize the injuries of patients who were transferred to the radiology department in the first 12 hours following the blast and to evaluate the disaster preparedness of the radiology department at Hôtel-Dieu de France Hospital in order to implement corrective action process. A total of 97 patients underwent imaging examinations, comprising 77 CT scans and 20 radiographs, which were retrospectively reviewed by 4 senior radiology residents. Patient injuries were classified according to blast injury categories. A full assessment of our disaster plan was done including staff shortage, examination time, patients triage and diagnostic performance of radiologists. Tertiary injuries were the most prevalent (47.6%). Maxillofacial fractures and intracranial hemorrhage were the most common (54.3%). The radiology department encountered numerous challenges, including communication difficulties, staffing shortages and infrastructure damage. Nevertheless, the disaster management plan enabled the department to effectively navigate these obstacles despite many flaws and many lessons were learned. This study underscores the importance of an efficient response of the radiology team during mass casualty incidents and can inform future disaster preparedness efforts in health care settings.

Evaluation of an image-rich quiz-based iOS app as a study resource for the ABR Core exam.

The American Board of Radiology Core exam requires that trainees demonstrate knowledge of critical concepts across 12 domains spanning a range of imaging modalities and anatomic regions. Mobile apps have become popular components of medical and radiology education since the inception of smartphones. Numerous medical educational apps are accessible via smartphone devices and tablets, regardless of operating system, for medical training and learning purposes. For over two decades, CTisus has served as an informational and educational radiology website containing image-rich materials and resources dedicated to the use of body CT. We conducted a study to evaluate the perceived utility of the CTisus iQuiz app as a study resource for the American Board of Radiology Core exam. The overall rating of the app was above average with 50 % of respondents characterizing the app as "Good" and 29 % evaluating the app as "Excellent." Further, 85 % of survey respondents found the app easy to understand and use, with related pearls deemed helpful by 75 % of participants, the video discussions found to be clear and beneficial by 79 %. Mobile apps are a valuable tool for the current generation of medical trainees, with quizzes shown to be an effective method to evaluate and enhance knowledge. The CTisus iQuiz app may benefit radiology residents studying for the ABR Core exam by providing access to image-rich, multiple choice-based self-assessments with in-depth explanations and feedback in an accessible interface, allowing for asynchronous learning and repeated practice.

The Role of Artificial Intelligence in Radiology Residency Training: A National Survey Study

Objective: Artificial Intelligence (AI) offers opportunities for radiologists to enhance workflow efficiency, perform faster and repeatable segmentation, and detect lesions more easily. The aim of this study is to investigate the current knowledge and general attitudes of radiology resident physicians towards AI. Additionally, it seeks to assess the current state of AI/ML/DL education in radiology residency, the awareness and use of available educational resources. Methods: A cross-sectional study was conducted using an online survey from October 2023 to February 2024. The survey included demographic data, AI knowledge, attitudes towards AI, and the role of AI in medical education. Survey questions were developed based on literature and reviewed by experts in medical education and radiology. Results: The study included 155 participants (38.7% female) with an average age of 28.81±4.77 years. About 80.6% were aware of AI terms, with a mean knowledge score of 3.02±1.39 on a 7-point Likert scale. Most participants (90.3%) had no programming knowledge. Only 22.6% used AI tools occasionally. The majority (73.4%) believed AI would change radiology's future, though only 10.3% felt radiologists' jobs were at risk. Regarding AI education, 84.5% reported no formal training, and awareness of online resources was low. Conclusion: The study found that while awareness of AI among radiology residents is high, their knowledge and practical use of AI tools are limited. AI education is largely absent from residency programs, and awareness of online educational resources is low. These findings highlight the need for integrating AI training into radiology education and increasing awareness of available resources.

Classification of Interventional Radiology Reports into Technique Categories with a Fine-Tuned Large Language Model.

The aim of this study is to develop a fine-tuned large language model that classifies interventional radiology reports into technique categories and to compare its performance with readers. This retrospective study included 3198 patients (1758 males and 1440 females; age, 62.8 ± 16.8years) who underwent interventional radiology from January 2018 to July 2024. Training, validation, and test datasets involved 2292, 250, and 656 patients, respectively. Input data involved texts in clinical indication, imaging diagnosis, and image-finding sections of interventional radiology reports. Manually classified technique categories (15 categories in total) were utilized as reference data. Fine-tuning of the Bidirectional Encoder Representations model was performed using training and validation datasets. This process was repeated 15 times due to the randomness of the learning process. The best-performed model, which showed the highest accuracy among 15 trials, was selected to further evaluate its performance in the independent test dataset. The report classification involved one radiologist (reader 1) and two radiology residents (readers 2 and 3). The accuracy and macrosensitivity (average of each category's sensitivity) of the best-performed model in the validation dataset were 0.996 and 0.994, respectively. For the test dataset, the accuracy/macrosensitivity were 0.988/0.980, 0.986/0.977, 0.989/0.979, and 0.988/0.980 in the best model, reader 1, reader 2, and reader 3, respectively. The model required 0.178s required for classification per patient, which was 17.5-19.9 times faster than readers. In conclusion, fine-tuned large language model classified interventional radiology reports into technique categories with high accuracy similar to readers within a remarkably shorter time.

RADHawk-an AI-based knowledge recommender to support precision education, improve reporting productivity, and reduce cognitive load.

Using artificial intelligence (AI) to augment knowledge is key to establishing precision education in modern radiology training. Our department has developed a novel AI-derived knowledge recommender, the first reported precision education program in radiology, RADHawk (RH), that augments the training of radiology residents and fellows by pushing personalized and relevant educational content in real-time and in context with the case being interpreted. To assess the impact on trainees of an AI-based knowledge recommender compared to traditional knowledge sourcing for radiology reporting through reporting time, quality, cognitive load, and learning experiences. A mixed methods prospective study allocated trainees to intervention and control groups, working with and without access to RH, respectively. Validated questionnaires and observed and graded simulated picture archiving and communication system (PACS)-based reporting at the start and end of a month's rotation assessed technology acceptance, case report quality, case report time and sourcing time, cognitive load, and attitudes toward modified learning strategies. Non-parametric regression analyses and Mann-Whitney tests were used to compare outcomes between groups, with significance set at P<0.05. The intervention group (n=28) demonstrated astatistically significant reduction in the case report time by -162s per case (95%CI -275.76s to -52.40s) (P-value = 0.002) and an increase of 14% (95%CI 8.1-19.8%) (P-value <0.001) in accuracy scores compared to the control group (n=29) at the end of the rotation. The intervention group also showed lower levels of mental demand (P=0.030) and experienced less effort (P=0.030) and frustration (P=0.030) while reporting. Additionally, >78% of the intervention group gave positive ratings on RH's effectiveness, increase in productivity, job usefulness, and ease of use. Eighty-nine percent of participants in the intervention group requested access to RH for their next rotation. This study demonstrates that RH, as the first reported AI-derived knowledge recommender for radiology education, significantly reduces reporting time and improves reporting accuracy while reducing overall workload and mental demand for radiology trainees. The high acceptance among trainees suggests its potential for supporting self-directed learning. Further testing of a larger external cohort will support more widespread implementation of RH for precision education.

Radiology resident proficiency in identifying misplaced lines, tubes, and devices: a simulation-based study using WIDI SIM.

Accurate placement of medical devices is crucial in critical care to prevent severe complications. This study aims to evaluate radiology residents' proficiency in identifying four specific critical misplacements of medical devices using the Wisdom in Diagnostic Imaging Simulation (WIDI SIM). A retrospective analysis was conducted on 1,102 responses from radiology residents who participated in the WIDI SIM between 2010 and 2022. The majority were first- and second-year residents from multiple institutions. The simulation presented four specific cases featuring misplacements of an endotracheal tube in the esophagus, an intrauterine device embedded in the myometrium, a peripherally inserted central catheter in the right internal jugular vein, and an umbilical venous catheter in the splenic vein. Residents provided free-text interpretations scored on a 0-10 scale by subspecialty radiologists. Errors were categorized as observational (failure to identify misplacement) or interpretive (misinterpretation of identified misplacement). Statistical analyses were performed using Kruskal-Wallis and Dunn's multiple comparisons tests. Across all cases, residents' average scores did not meet the acceptable standard of 7 points. Observational errors were predominant, indicating a failure to recognize these specific device misplacements. Effective report rates were low: 58% for the endotracheal tube case, 35% for the intrauterine device, 19% for the peripherally inserted central catheter, and 25% for the umbilical venous catheter. Significant performance improvements were observed between first- and second-year residents in three of the four cases (p-values ranging from < 0.0001 to 0.0238), but overall proficiency remained suboptimal even among senior residents. This study reveals gaps in radiology residents' ability to identify these specific misplaced lines, tubes, and devices accurately. The consistent pattern of underperformance, primarily due to observational errors, suggests a need for targeted educational interventions to improve resident proficiency in this aspect of emergency radiology.