Radiology Residents Research Articles

Artificial intelligence-based graded training of pulmonary nodules for junior radiology residents and medical imaging students

BackgroundTo evaluate the efficiency of artificial intelligence (AI)-assisted diagnosis system in the pulmonary nodule detection and diagnosis training of junior radiology residents and medical imaging students.MethodsThe participants were divided into three groups. Medical imaging students of Grade 2020 in the Jinzhou Medical University were randomly divided into Groups 1 and 2; Group 3 comprised junior radiology residents. Group 1 used the traditional case-based teaching mode; Groups 2 and 3 used the ‘AI intelligent assisted diagnosis system’ teaching mode. All participants performed localisation, grading and qualitative diagnosed of 1,057 lung nodules in 420 cases for seven rounds of testing after training. The sensitivity and number of false positive nodules in different densities (solid, pure ground glass, mixed ground glass and calcification), sizes (less than 5 mm, 5–10 mm and over 10 mm) and positions (subpleural, peripheral and central) of the pulmonary nodules in the three groups were detected. The pathological results and diagnostic opinions of radiologists formed the criteria. The detection rate, diagnostic compliance rate, false positive number/case, and kappa scores of the three groups were compared.ResultsThere was no statistical difference in baseline test scores between Groups 1 and 2, and there were statistical differences with Group 3 (P = 0.036 and 0.011). The detection rate of solid, pure ground glass and calcified nodules; small-, medium-, and large-diameter nodules; and peripheral nodules were significantly different among the three groups (P<0.05). After seven rounds of training, the diagnostic compliance rate increased in all three groups, with the largest increase in Group 2. The average kappa score increased from 0.508 to 0.704. The average kappa score for Rounds 1–4 and 5–7 were 0.595 and 0.714, respectively. The average kappa scores of Groups 1,2 and 3 increased from 0.478 to 0.658, 0.417 to 0.757, and 0.638 to 0.791, respectively.ConclusionThe AI assisted diagnosis system is a valuable tool for training junior radiology residents and medical imaging students to perform pulmonary nodules detection and diagnosis.

Subspecialized medical team mode facilitates radiology resident training

AbstractPurposeOur institution developed the subspecialized medical team (SMT) mode in radiology reporting workflows in July 2022 to strengthen professionalism in radiology residency training. This study aimed to investigate whether adopting the SMT mode facilitated radiology resident training.MethodThis observational study was conducted in October 2022. An online questionnaire was distributed among first‐to third‐year radiology residents to determine their levels of perceived stress, anxiety, competency in image interpretation, and learning effectiveness in using the SMT mode and other approaches. Responses were on 4 or 5 point Likert scales, and multiple‐choice responses were used for semi‐open questions. The turnaround time for radiology reports was measured for both residents and senior radiologists. Statistical significance (p &lt; 0.05) was determined using chi‐square or Fisher's exact tests, one‐way analysis of variance, pairwise t‐tests, and the Kruskal‒Wallis H test.ResultsIn total, 71 residents completed the questionnaire survey. The turnaround times for radiology reports were collected for these residents and 23 senior radiologists. Residents' stress and anxiety scores, perceived levels of competency in image interpretation, and time spent writing reports were significantly increased in the SMT mode. In contrast, the time senior radiologists spent reviewing a single report was decreased for the majority of report types after 1 year of implementing the SMT mode.ConclusionThese findings suggest adopting the SMT mode could facilitate the training of radiology residents. However, residents' mental health and well‐being should also be considered.

Improving the diagnostic performance of inexperienced readers for thyroid nodules through digital self-learning and artificial intelligence assistance.

Data-driven digital learning could improve the diagnostic performance of novice students for thyroid nodules. To evaluate the efficacy of digital self-learning and artificial intelligence-based computer-assisted diagnosis (AI-CAD) for inexperienced readers to diagnose thyroid nodules. Between February and August 2023, a total of 26 readers (less than 1 year of experience in thyroid US from various departments) from 6 hospitals participated in this study. Readers completed an online learning session comprising 3,000 thyroid nodules annotated as benign or malignant independently. They were asked to assess a test set consisting of 120 thyroid nodules with known surgical pathology before and after a learning session. Then, they referred to AI-CAD and made their final decisions on the thyroid nodules. Diagnostic performances before and after self-training and with AI-CAD assistance were evaluated and compared between radiology residents and readers from different specialties. AUC (area under the receiver operating characteristic curve) improved after the self-learning session, and it improved further after radiologists referred to AI-CAD (0.679 vs 0.713 vs 0.758, p<0.05). Although the 18 radiology residents showed improved AUC (0.7 to 0.743, p=0.016) and accuracy (69.9% to 74.2%, p=0.013) after self-learning, the readers from other departments did not. With AI-CAD assistance, sensitivity (radiology 70.3% to 74.9%, others 67.9% to 82.3%, all p<0.05) and accuracy (radiology 74.2% to 77.1%, others 64.4% to 72.8%, all p <0.05) improved in all readers. While AI-CAD assistance helps improve the diagnostic performance of all inexperienced readers for thyroid nodules, self-learning was only effective for radiology residents with more background knowledge of ultrasonography. Online self-learning, along with AI-CAD assistance, can effectively enhance the diagnostic performance of radiology residents in thyroid cancer.

Flipping the script: A multiyear assessment and comparison of flipped classroom versus traditional didactic teaching at two academic institutions

Rationale and objectivesHistorically radiology resident education has taken the form of workstation and didactic teaching. Due to increasing clinical demand and administrative burden for academic radiologists, the need for more efficient and effective teaching has increased. Flipped classroom teaching, where trainees independently learn material prior to interactive teaching sessions with faculty, is a possible alternative. While the use of flipped teaching in radiology has been studied in the medical student setting, its use in the radiology residency setting has been less published. Materials and methodsAt two academic institutions (University of Washington and Northwestern), exam scores from five PGY-2 Core rotations were collected. Flipped teaching was used for one rotation at the University of Washington (FR). The influence of teaching method, rotation, and institution on exam score was examined. Resident surveys were also collected to understand perceptions of flipped classroom teaching. ResultsAt the University of Washington, the mean exam score for the flipped rotation was significantly higher than the majority of other rotations utilizing traditional teaching (p<0.05). Between the University of Washington and Northwestern, there was no difference in exam scores when comparing comparable rotations. Among residents at the University of Washington, the flipped teaching rotation was perceived as more educationally valuable than traditional teaching rotations. ConclusionFlipped classroom teaching is at least as effective as the traditional teaching model and associated with better performance on standardized exams at one institution. Among residents, flipped learning is also associated with higher perceived educational value.

Artificial Intelligence Application in Skull Bone Fracture with Segmentation Approach.

This study aims to evaluate an AI model designed to automatically classify skull fractures and visualize segmentation on emergent CT scans. The model's goal is to boost diagnostic accuracy, alleviate radiologists' workload, and hasten diagnosis, thereby enhancing patient outcomes. Unique to this research, both pediatric and post-operative patients were not excluded, and diagnostic durations were analyzed. Our testing dataset for theobserver studies involved 671 patients, with a mean age of 58.88 years and fairly balanced gender representation. Model 1 of our AI algorithm, trained with 1499 fracture-positive cases, showed a sensitivity of 0.94 and specificity of 0.87, with a DICE score of 0.65. Implementing post-processing rules (specifically Rule B) improved the model's performance, resulting in a sensitivity of 0.94, specificity of 0.99, and a DICE score of 0.63. AI-assisted diagnosis resulted in significantly enhanced performance for all participants, with sensitivity almost doubling for junior radiology residents and other specialists. Additionally, diagnostic durations were significantly reduced (p < 0.01) with AI assistance across all participant categories. Our skull fracture detection model, employing a segmentation approach, demonstrated high performance, enhancing diagnostic accuracy and efficiency for radiologists and clinical physicians. This underlines the potential of AI integration in medical imaging analysis to improve patient care.

Comparison of Artificial Intelligence-based learning with the traditional method in the diagnosis of COVID-19 chest radiographs among postgraduate radiology residents

Comparison of Artificial Intelligence-based learning with the traditional method in the diagnosis of COVID-19 chest radiographs among postgraduate radiology residents

Unsolicited scam invitations from predatory publications and fraudulent conferences: Radiology-in-training experience

PurposeRadiology faculty across various specialties have been reported to receive an average of 20.7 invitations to submit manuscripts to bogus journals and 4.1 invitations to speak at unsuitable events over a two-week span. Radiology trainees also receive a fair number of unsolicited invitations from unknown senders to submit manuscripts and speak at meetings. Trainees can be more vulnerable to predatory invitations due to potential naivety. We aimed to determine the prevalence of these spam invitations received by radiology trainees. Material and methodsThe designed survey for evaluating the experience of radiology trainees regarding phishing scams of predatory publications and conferences was sent to radiology residency and neuroradiology fellowship program leadership to redistribute amongst their trainees, and was advertised on social media platforms. The survey was first sent out on September 28, 2023, and was closed two weeks later October 12, 2023. Spearman’s correlation, univariable and multivariable linear regression analyses were performed. ResultsOur study included 151 respondents who completed the survey. Of the survey respondents, 53 % reported receiving unsolicited emails from predatory publications (mean = 6.76 ± 7.29), and 32 % reported receiving emails from fraudulent conferences (mean = 5.61 ± 5.77). Significant positive correlation was observed between number of unsolicited email invitations with number of PubMed indexed publications, number as corresponding author, number in open access journals and number of abstract presentations. ConclusionsTrainees in radiology receive many unsolicited invitations to publish papers as well as to present at meetings that are not accredited. This could lead to wasted time and financial resources for unsuspecting trainees.

AI-based automated evaluation of image quality and protocol tailoring in patients undergoing MRI for suspected prostate cancer

PurposeTo develop and validate an artificial intelligence (AI) application in a clinical setting to decide whether dynamic contrast-enhanced (DCE) sequences are necessary in multiparametric prostate MRI. MethodsThis study was approved by the institutional review board and requirement for study-specific informed consent was waived. A mobile app was developed to integrate AI-based image quality analysis into clinical workflow. An expert radiologist provided reference decisions. Diagnostic performance parameters (sensitivity and specificity) were calculated and inter-reader agreement was evaluated. ResultsFully automated evaluation was possible in 87% of cases, with the application reaching a sensitivity of 80% and a specificity of 100% in selecting patients for multiparametric MRI. In 2% of patients, the application falsely decided on omitting DCE. With a technician reaching a sensitivity of 29% and specificity of 98%, and resident radiologists reaching sensitivity of 29% and specificity of 93%, the use of the application allowed a significant increase in sensitivity. ConclusionThe presented AI application accurately decides on a patient-specific MRI protocol based on image quality analysis, potentially allowing omission of DCE in the diagnostic workup of patients with suspected prostate cancer. This could streamline workflow and optimize time utilization of healthcare professionals.

No spectators in a pandemic: A comparison of radiologists, radiology residents, and CT technicians in interpreting chest CT scans for COVID-19

ObjectiveTo compare the diagnostic accuracy of radiology trainees, CT technicians, and radiologists in interpreting chest CT scans for the detection of COVID-19. MethodsRetrospective data were analyzed from 269 reading assessments of chest CT scans for COVID-19. These were conducted by 160 radiologists, 41 radiology residents, and 68 CT technicians from diverse countries and performed online using DetectedX software between March 30, 2020, and January 1, 2022. The readers examined and assessed realistic, de-identified, digital chest CT images—15 positive and 15 negative for COVID-19. Information about the readers' clinical roles (radiologist, radiology resident, or CT technician) was collected alongside measures of their performance in distinguishing between positive and negative cases (sensitivity, specificity, and ROC AUC). The data were evaluated with Kruskal–Wallis tests to identify any differences in performance between the roles. Post hoc comparisons using Bonferroni-corrected Dunn's tests were then used to identify any significant pairwise differences. ResultsStatistically significant differences were observed in specificity (p < 0.001) and ROC AUC (p < 0.001) when comparing the three groups. In the pairwise comparisons, differences in specificity were only found between CT technicians and radiologists (p = 0.001), while ROC AUC was significantly different between CT technicians and radiologists (p = 0.001) and between CT technicians and radiology trainees (p = 0.016). No statistically significant differences were observed in sensitivity (p = 0.309) between the groups. ConclusionsThe study identifies both diagnostic potential and areas for improvement for radiology residents and CT technicians. Notably, the performance of residents mirrored that of radiologists, underscoring their crucial role in early case management. However, while CT technicians demonstrated comparable sensitivity, their specificity and ROC AUC were lower, highlighting an opportunity for targeted training to enhance their ability to differentiate COVID-19 from other pathologies.

“Building bridges”—communication education for residents in radiology: a scoping review

BackgroundGood communication is an important professional attribute for radiologists. However, explorations of communication education and their outcomes in radiology residents are sparse. This scoping review aims to evaluate the existing literature on communication education for radiology residents, identify gaps in current practices, and suggest directions for future studies.MethodsA scoping review following the six-step approach of Arksey and O’Malley was undertaken. We searched through PubMed, Embase, ERIC, and Web of Science databases, focusing on communication education in radiology residents.ResultsSixteen of the 3096 identified articles were included in the analysis. Most studies (13/16) originated from the United States. The studies varied in study design, including quantitative, qualitative and mixed-methods approaches. The sample sizes of most studies were small to moderate, with more than half of the studies had fewer than 30 participants. The identified studies predominantly focused on communication with patients and healthcare professionals. The need for communication education, the efficacy of specific communication education programs, and the capability of some assessment tools for evaluating residents’ communication skills were investigated.ConclusionsThis scoping review reveals the gap between the need for communication education and the lack of comprehensive education programs in radiology residents globally. Future studies should develop tailored interventions and use reliable assessment tools, engaging more participants with extended follow-up periods, and expand the scope of communication training to include all relevant stakeholders.

The Digital Disruption: YouTube’s Role in Information on Radiology Residency

The Digital Disruption: YouTube’s Role in Information on Radiology Residency

Multi-sequence MRI-based radiomics: An objective method to diagnose early-stage osteonecrosis of the femoral head

ObjectivesThis study investigated the use of radiomics for diagnosing early-stage osteonecrosis of the femoral head (ONFH) by extracting features from multiple MRI sequences and constructing predictive models. Materials and methodsWe conducted a retrospective review, collected MR images of early-stage ONFH (102 from institution A and 20 from institution B) and healthy femoral heads (102 from institution A and 20 from institution B) from two institutions. We extracted radiomics features, handled batch effects using Combat, and normalized features using z-score. We employed the Least absolute shrinkage and selection operator (LASSO) algorithm, along with Max-Relevance and Min-Redundancy (mRMR), to select optimal features for constructing radiomics models based on single, double, and multi-sequence MRI data. We evaluated performance using receiver operating characteristic (ROC) and precision-recall (PR) curves, and compared area under curve of ROC (AUC-ROC) values with the DeLong test. Additionally, we studied the diagnostic performance of the multi-sequence radiomics model and radiologists, compared the diagnostic outcomes of the model and radiologists using the Fisher exact test. ResultsWe studied 122 early-stage ONFH and 122 normal femoral heads. The multi-sequence model exhibited the best diagnostic performance among all models (AUC-ROC, PR-AUC for training set: 0.96, 0.961; validation set: 0.96, 0.97; test set: 0.94, 0.94), and it outperformed three resident radiologists on the external testing group with an accuracy of 87.5 %, sensitivity of 85.00 %, and specificity of 90.00 % (p < 0.01), highlighting the robustness of our findings. ConclusionsOur study underscored the novelty of the multi-sequence radiomics model in diagnosing early-stage ONFH. By leveraging features extracted from multiple imaging sequences, this approach demonstrated high efficacy, indicating its potential to advance early diagnosis for ONFH. These findings provided important guidance for enhancing early diagnosis of ONFH through radiomics methods, offering new avenues and possibilities for clinical practice and patient care.

Finding Waldo: Sonographic systematic approach to localize the appendix in children.

To describe a systematic scanning approach using anatomical landmarks followed by an assessment of radiology trainees' ability to identify the normal appendix in healthy children. Uncontrolled pre and post study assessing radiology residents' sonographic skills in detecting the normal appendix in healthy children. Initial questionnaire for the trainees' demographics, perceptions and experiences in detecting the appendix with ultrasound in children followed by a precourse test on healthy volunteers. Hands-on training was conducted by describing a systematic sonographic approach to identify the appendix using anatomical landmarks, and then a postcourse test was carried out. The primary outcome was unprompted ability to identify the appendix. Subjective self-scoring of confidence was also recorded. A three-hour hands-on workshop was conducted. Sixteen radiology trainees participated and were randomly distributed to four stations, each with different ultrasound machines and healthy volunteers. Fifteen had a precourse assessment, and 12 completed the postcourse assessment. Before the course, 3/15 (20%) identified the appendix, while 10/12 (83%) identified the appendix afterward. After the course, participants perceived finding the appendix easier than before. There was no statistically significant difference in the participants' perceived confidence in detecting the appendix. With the described scanning technique, most of the participants were able to identify the normal appendix after receiving short hands-on training. This highlights the importance of targeted training of radiology trainees and nonradiologists.

Comparing the Diagnostic Performance of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and Radiologists in Challenging Neuroradiology Cases.

To compare the diagnostic performance among Generative Pre-trained Transformer (GPT)-4-based ChatGPT, GPT‑4 with vision (GPT-4V) based ChatGPT, and radiologists in challenging neuroradiology cases. We collected 32consecutive "Freiburg Neuropathology Case Conference" cases from the journal Clinical Neuroradiology between March 2016 and December 2023. We input the medical history and imaging findings into GPT-4-based ChatGPT and the medical history and images into GPT-4V-based ChatGPT, then both generated adiagnosis for each case. Six radiologists (three radiology residents and three board-certified radiologists) independently reviewed all cases and provided diagnoses. ChatGPT and radiologists' diagnostic accuracy rates were evaluated based on the published ground truth. Chi-square tests were performed to compare the diagnostic accuracy of GPT-4-based ChatGPT, GPT-4V-based ChatGPT, and radiologists. GPT‑4 and GPT-4V-based ChatGPTs achieved accuracy rates of 22% (7/32) and 16% (5/32), respectively. Radiologists achieved the following accuracy rates: three radiology residents 28% (9/32), 31% (10/32), and 28% (9/32); and three board-certified radiologists 38% (12/32), 47% (15/32), and 44% (14/32). GPT-4-based ChatGPT's diagnostic accuracy was lower than each radiologist, although not significantly (all p > 0.07). GPT-4V-based ChatGPT's diagnostic accuracy was also lower than each radiologist and significantly lower than two board-certified radiologists (p = 0.02 and 0.03) (not significant for radiology residents and one board-certified radiologist [all p > 0.09]). While GPT-4-based ChatGPT demonstrated relatively higher diagnostic performance than GPT-4V-based ChatGPT, the diagnostic performance of GPT‑4 and GPT-4V-based ChatGPTs did not reach the performance level of either radiology residents or board-certified radiologists in challenging neuroradiology cases.

Investigating Canadian Radiology Residents' Personal Financial Literacy: A Nation-Wide Assessment.

Purpose: Canadian resident physicians carry large debt to finance their education, which impacts their wellness and their future decision making. The objective of this observational study is to assess the financial literacy of Canadian radiology residents through testing their financial knowledge and examining their current financial status. Methods: A survey was designed to assess the financial literacy and current financial status of radiology residents, which was distributed to Canadian radiology residents via Google Forms. Descriptive analyses on preliminary data and the association between level of training and financial quiz scores were obtained. Results: 104 valid responses from 16 universities were received. The majority (53%) of residents indicated that their debt was greater than $150 000. Residents on average scored 71% on the financial quiz and the scores were not associated with training level (P = .71). The majority (89%) of residents indicated a strong interest in a formal financial literacy curriculum, with 80% preferring a physician-led curriculum. Conclusion: Overall, residents face a high debt burden. Current resident physicians value a formal financial literacy curriculum as a part of their residency program despite existing financial knowledge. Most importantly, residents feel that a curriculum created with involvement of other physicians would be optimal.

Going Global: Scaling the Artificial Intelligence Literacy Course to an International Audience

Introduction: Applications of artificial intelligence (AI) in radiology continue to increase every year, however most radiology residencies lack a dedicated AI education curriculum. Fundamental AI education resources are even more sparse for trainees in low- to middle-income countries and under-resourced healthcare systems. The AI Literacy Course assesses the effectiveness and scalability of a free, remote AI education curriculum to increase understanding of fundamental AI terms, methods, and applications in radiology among radiology trainees in the United States and internationally. Method: A week-long AI in radiology literacy course for radiology trainees was held October 3-7, 2022. Ten 30-minute lectures utilizing a remote learning format covered basic AI terms and methods, clinical applications of AI in radiology by three different subspecialties, and special topics lectures. A proctored, hands-on clinical AI session allowed participants to directly use an FDA-cleared, AI-assisted viewer and reporting system for advanced cancer. Pre- and post-course electronic surveys were distributed to assess participants&amp;rsquo; knowledge of AI terminology and applications, as well as their interest in AI education. Results: A total of 25 residency programs throughout the US participated in the course with participants attending from 10 countries. An average of 150 participants viewed the course per day. Nearly all participants reported insufficient exposure to AI in their radiology training (95.8%). Participant knowledge of fundamental AI terms and methods increased after completion of the course, with an average pre-course evaluation of 8.3/15 and a post-course evaluation of 10.0/15 (p=0.01). Conclusion: The scalability of the AI Literacy Course demonstrates a viable model to bring accessible fundamental AI education to radiology trainees in the United States and internationally.

Improved differentiation of cavernous malformation and acute intraparenchymal hemorrhage on CT using an AI algorithm

This study aimed to evaluate the utility of an artificial intelligence (AI) algorithm in differentiating between cerebral cavernous malformation (CCM) and acute intraparenchymal hemorrhage (AIH) on brain computed tomography (CT). A retrospective, multireader, randomized study was conducted to validate the performance of an AI algorithm in differentiating AIH from CCM on brain CT. CT images of CM and AIH (< 3 cm) were identified from the database. Six blinded reviewers, including two neuroradiologists, two radiology residents, and two emergency department physicians, evaluated CT images from 288 patients (CCM, n = 173; AIH, n = 115) with and without AI assistance, comparing diagnostic performance. Brain CT interpretation with AI assistance resulted in significantly higher diagnostic accuracy than without (86.92% vs. 79.86%, p < 0.001). Radiology residents and emergency department physicians showed significantly improved accuracy of CT interpretation with AI assistance than without (84.21% vs. 75.35%, 80.73% vs. 72.57%; respectively, p < 0.05). Neuroradiologists showed a trend of higher accuracy with AI assistance in the interpretation but lacked statistical significance (95.83% vs. 91.67%, p = 0.56). The use of an AI algorithm can enhance the differentiation of AIH from CCM in brain CT interpretation, particularly for nonexperts in neuroradiology.

Evaluation of radiology resident well-being at a single institution with a dedicated wellness curriculum

IntroductionIncreasing rates of physician burnout over recent years have resulted in the need for formal tools and programming dedicated to physician well-being. The Accreditation Council for Graduate Medical Education (ACGME) has taken measures to prioritize trainee well-being by revising its common program requirements. Widespread departmental initiatives have been developed in line with these changes. At the authors' institution, a committee was developed to design and implement a holistic wellness curriculum for radiology trainees. ObjectiveThe objective of this study was to assess overall well-being in a cohort of radiology residents at a training program with a dedicated wellness curriculum. MethodsA wellness curriculum for radiology residents was developed and implemented. Over a 3-year period, data was collected using the Maslach Burnout Inventory (MBI), Brief Resilience Scale (BRS). ResultsRates amongst respondents were low, compared to average, for emotional exhaustion (below 17, average 25.2), depersonalization (6, average 10), and of personal accomplishment were moderate to high (37.5, average 32.9). PGY-4 level residents had the highest rates of burnout (p = 0.042 for depersonalization, p = 0.006 for emotional exhaustion). PGY-4 residents also had the lowest rates of resilience based on the BRS, and PGY-3 the highest (p = 0.037). There were no statistically significant differences between gender or differing relationship status for MBI or BRS. The most commonly cited barriers to wellness practices included fatigue, competing responsibilities, and not knowing where to start. ConclusionRadiology residents at a single institution with a dedicated wellness curriculum demonstrated overall lower rates of burnout compared to their peers.

Deep learning-based platform performs high detection sensitivity of intracranial aneurysms in 3D brain TOF-MRA: An external clinical validation study

PurposeTo evaluate the diagnostic efficacy of a developed artificial intelligence (AI) platform incorporating deep learning algorithms for the automated detection of intracranial aneurysms in time-of-flight (TOF) magnetic resonance angiography (MRA). MethodThis retrospective study encompassed 3D TOF MRA images acquired between January 2023 and June 2023, aiming to validate the presence of intracranial aneurysms via our developed AI platform. The manual segmentation results by experienced neuroradiologists served as the “gold standard”. Following annotation of MRA images by neuroradiologists using InferScholar software, the AI platform conducted automatic segmentation of intracranial aneurysms. Various metrics including accuracy (ACC), balanced ACC, area under the curve (AUC), sensitivity (SE), specificity (SP), F1 score, Brier Score, and Net Benefit were utilized to evaluate the generalization of AI platform. Comparison of intracranial aneurysm identification performance was conducted between the AI platform and six radiologists with experience ranging from 3 to 12 years in interpreting MR images. Additionally, a comparative analysis was carried out between radiologists’ detection performance based on independent visual diagnosis and AI-assisted diagnosis. Subgroup analyses were also performed based on the size and location of the aneurysms to explore factors impacting aneurysm detectability. Results510 patients were enrolled including 215 patients (42.16 %) with intracranial aneurysms and 295 patients (57.84 %) without aneurysms. Compared with six radiologists, the AI platform showed competitive discrimination power (AUC, 0.96), acceptable calibration (Brier Score loss, 0.08), and clinical utility (Net Benefit, 86.96 %). The AI platform demonstrated superior performance in detecting aneurysms with an overall SE, SP, ACC, balanced ACC, and F1 score of 91.63 %, 92.20 %, 91.96 %, 91.92 %, and 90.57 % respectively, outperforming the detectability of the two resident radiologists. For subgroup analysis based on aneurysm size and location, we observed that the SE of the AI platform for identifying tiny (diameter＜3mm), small (3 mm ≤ diameter＜5mm), medium (5 mm ≤ diameter＜7mm) and large aneurysms (diameter ≥ 7 mm) was 87.80 %, 93.14 %, 95.45 %, and 100 %, respectively. Furthermore, the SE for detecting aneurysms in the anterior circulation was higher than that in the posterior circulation. Utilizing the AI assistance, six radiologists (i.e., two residents, two attendings and two professors) achieved statistically significant improvements in mean SE (residents: 71.40 % vs. 88.37 %; attendings: 82.79 % vs. 93.26 %; professors: 90.07 % vs. 97.44 %; P < 0.05) and ACC (residents: 85.29 % vs. 94.12 %; attendings: 91.76 % vs. 97.06 %; professors: 95.29 % vs. 98.82 %; P < 0.05) while no statistically significant change was observed in SP. Overall, radiologists’ mean SE increased by 11.40 %, mean SP increased by 1.86 %, and mean ACC increased by 5.88 %, mean balanced ACC promoted by 6.63 %, mean F1 score grew by 7.89 %, and Net Benefit rose by 12.52 %, with a concurrent decrease in mean Brier score declined by 0.06. ConclusionsThe deep learning algorithms implemented in the AI platform effectively detected intracranial aneurysms on TOF-MRA and notably enhanced the diagnostic capabilities of radiologists. This indicates that the AI-based auxiliary diagnosis model can provide dependable and precise prediction to improve the diagnostic capacity of radiologists.

Automated Detection of Steno-Occlusive Lesion on Time-of-Flight MR Angiography: An Observer Performance Study.

Intracranial steno-occlusive lesions are responsible for acute ischemic stroke. However, the clinical benefits of artificial intelligence (AI)-based methods for detecting pathologic lesions in intracranial arteries have not been evaluated. We aimed to validate the clinical utility of an AI model for detecting steno-occlusive lesions in the intracranial arteries. Overall, 138 TOF-MRA images were collected from 2 institutions, which served as internal (n = 62) and external (n = 76) test sets, respectively. Each study was reviewed by 5 radiologists (2 neuroradiologists and 3 radiology residents) to compare the usage and nonusage of our proposed AI model for TOF-MRA interpretation. They identified the steno-occlusive lesions and recorded their reading time. Observer performance was assessed by using the area under the jackknife free-response receiver operating characteristic curve (AUFROC) and reading time for comparison. The average AUFROC for the 5 radiologists demonstrated an improvement from 0.70 without AI to 0.76 with AI (P = .027). Notably, this improvement was most pronounced among the 3 radiology residents, whose performance metrics increased from 0.68 to 0.76 (P = .002). Despite an increased reading time by using AI, there was no significant change among the readings by radiology residents. Moreover, the use of AI resulted in improved interobserver agreement among the reviewers (the intraclass correlation coefficient increased from 0.734 to 0.752). Our proposed AI model offers a supportive tool for radiologists, potentially enhancing the accuracy of detecting intracranial steno-occlusion lesions on TOF-MRA. Less experienced readers may benefit the most from this model.