Human Readers Research Articles

STEED: A data mining tool for automated extraction of experimental parameters and risk of bias items from in vivo publications.

Systematic reviews, i.e., research summaries that address focused questions in a structured and reproducible manner, are a cornerstone of evidence-based medicine and research. However, certain steps in systematic reviews, such as data extraction, are labour-intensive, which hampers their feasibility, especially with the rapidly expanding body of biomedical literature. To bridge this gap, we aimed to develop a data mining tool in the R programming environment to automate data extraction from neuroscience in vivo publications. The function was trained on a literature corpus (n = 45 publications) of animal motor neuron disease studies and tested in two validation corpora (motor neuron diseases, n = 31 publications; multiple sclerosis, n = 244 publications). Our data mining tool, STEED (STructured Extraction of Experimental Data), successfully extracted key experimental parameters such as animal models and species, as well as risk of bias items like randomization or blinding, from in vivo studies. Sensitivity and specificity were over 85% and 80%, respectively, for most items in both validation corpora. Accuracy and F1-score were above 90% and 0.9 for most items in the validation corpora, respectively. Time savings were above 99%. Our text mining tool, STEED, can extract key experimental parameters and risk of bias items from the neuroscience in vivo literature. This enables the tool's deployment for probing a field in a research improvement context or replacing one human reader during data extraction, resulting in substantial time savings and contributing towards the automation of systematic reviews.

Utilizing a domain-specific large language model for LI-RADS v2018 categorization of free-text MRI reports: a feasibility study

ObjectiveTo develop a domain-specific large language model (LLM) for LI-RADS v2018 categorization of hepatic observations based on free-text descriptions extracted from MRI reports.Material and methodsThis retrospective study included 291 small liver observations, divided into training (n = 141), validation (n = 30), and test (n = 120) datasets. Of these, 120 were fictitious, and 171 were extracted from 175 MRI reports from a single institution. The algorithm’s performance was compared to two independent radiologists and one hepatologist in a human replacement scenario, and considering two combined strategies (double reading with arbitration and triage). Agreement on LI-RADS category and dichotomic malignancy (LR-4, LR-5, and LR-M) were estimated using linear-weighted κ statistics and Cohen’s κ, respectively. Sensitivity and specificity for LR-5 were calculated. The consensus agreement of three other radiologists served as the ground truth.ResultsThe model showed moderate agreement against the ground truth for both LI-RADS categorization (κ = 0.54 [95% CI: 0.42–0.65]) and the dichotomized approach (κ = 0.58 [95% CI: 0.42–0.73]). Sensitivity and specificity for LR-5 were 0.76 (95% CI: 0.69–0.86) and 0.96 (95% CI: 0.91–1.00), respectively. When the chatbot was used as a triage tool, performance improved for LI-RADS categorization (κ = 0.86/0.87 for the two independent radiologists and κ = 0.76 for the hepatologist), dichotomized malignancy (κ = 0.94/0.91 and κ = 0.87) and LR-5 identification (1.00/0.98 and 0.85 sensitivity, 0.96/0.92 and 0.92 specificity), with no statistical significance compared to the human readers’ individual performance. Through this strategy, the workload decreased by 45%.ConclusionLI-RADS v2018 categorization from unlabelled MRI reports is feasible using our LLM, and it enhances the efficiency of data curation.Critical relevance statementOur proof-of-concept study provides novel insights into the potential applications of LLMs, offering a real-world example of how these tools could be integrated into a local workflow to optimize data curation for research purposes.Key PointsAutomatic LI-RADS categorization from free-text reports would be beneficial to workflow and data mining.LiverAI, a GPT-4-based model, supported various strategies improving data curation efficiency by up to 60%.LLMs can integrate into workflows, significantly reducing radiologists’ workload.Graphical

Abstract 4119869: Automatic assignment of CAD-RADS categories in coronary CTA reports using large language model

Background: The effectiveness of different large language models (LLM) in extracting important information from CT angiography (CTA) radiology reports is unknown. Purpose: The Coronary Artery Disease-Reporting and Data System (CAD-RADS) is a standardized reporting system for CTA that categorizes the severity of coronary artery disease to aid in clinical decision-making. To evaluate the agreement between human readers and LLMs in assigning CAD-RADS categories based on CTA reports. Materials and Methods: This retrospective study analyzed reports from patients who underwent coronary CTA for screening or diagnosis at Johns Hopkins Hospital. Reports with CAD-RADS 0–5 findings were collected from Jan. 2020 to Oct. 2023. Initially, 6,212 CTA reports were retrieved. After excluding non-organized reports and retaining only those with CAD-RADS scores, 590 reports remained. Board-certified radiologists and three large language models (GPT-3.5, GPT-4, and Llama3) assigned CAD-RADS categories based on the original radiologists' findings. Two radiologists reviewed the CAD scores, with only one discrepancy in the 590 reports, indicating high inter-agreement. Agreement between human readers and LLMs for CAD-RADS categories was assessed using the Gwet agreement coefficient (AC1 value). Frequencies of changes in CAD-RADS category assignments that could potentially affect clinical management (CAD-RADS 0-2 vs. 3-5) were calculated and compared. Typically, CAD-RADS 0-2 indicates mild disease leading to conservative management, while CAD-RADS 3-5 denotes further testing or disease requiring aggressive intervention to prevent adverse cardiovascular events. Results: Among 590 reports, agreement between original and reviewing radiologists was almost perfect (AC1 = 0.985). Good agreement was also found between original radiologists and Llama3, GPT-3.5, and GPT-4 (AC1 = 0.861, 0.907, 0.941, respectively). Differences in CAD-RADS category upgrades/downgrades potentially affecting clinical management were observed: 0 of 590 (0%) for human readers, 56 of 590 (9.4%) for Llama3, 33 of 590 (5.5%) for GPT-3.5, and 23 of 590 (3.9%) for GPT-4 upgrading CAD-RADs; 8 of 590 (1.4%) for human readers, 19 of 590 (3.2%) for Llama3, 22 of 590 (3.7%) for GPT-3.5, and 12 of 590 (2.0%) for GPT-4 downgrading RAD-CADs. Conclusion: LLMs showed good agreement with human readers in assigning CAD-RADS categories on CTA reports. However, they were more likely to upgrade CAD-RADS category when compared to radiologists.

Abstract 4144553: Fully Automated Machine Learning Based Echocardiographic Assessment of Global Longitudinal Strain in Breast Cancer Patients Receiving Cardiotoxic Chemotherapy

Background: Global longitudinal strain (GLS) is considered the most sensitive echocardiographic marker for detecting cardiotoxicity and subclinical changes in cardiac function in patients receiving cardiotoxic therapies. Despite literature that has shown less inter-observer variability for GLS compared to left ventricular ejection fraction, there remains significant variability in GLS measurement across observers and vendors. US2.AI is a novel, machine learning-based artificial intelligence (AI) algorithm for automated echocardiographic interpretation. We aim to demonstrate the feasibility of AI-based GLS assessment and hypothesize that AI-based GLS measurement is more reproducible compared to human-based measurement Methods: A total of 27 transthoracic echocardiograms were selected at random from a population of female cardio-oncology breast cancer patients and were run through the US2.AI algorithm twice to calculate GLS. The same echocardiograms were then analyzed by two board-certified cardiologists in a blinded fashion to calculate GLS using QLAB software (Philips). The two methods of GLS measurement (AI versus humans) were compared using linear regression, as assessed by the Pearson correlation coefficient. Reproducibility for each method was assessed using the intra-class correlation coefficient (ICC). Results: A total of 27 studies were analyzed with a mean age of 55.6 ± 11.3 and all patients were female. The feasibility of US2.AI for the assessment of GLS was demonstrated with a strong correlation between AI-based and human reads (r = 0.80, p < 0.00001). The reproducibility of AI-based GLS assessment was superior to that of human readers as assessed by an ICC of 1.0 versus 0.52, respectively. Conclusion: AI-based assessment of GLS was shown to be both feasible and more reproducible compared to expert human readers in a population of breast cancer patients receiving cardiotoxic chemotherapy. Given the important clinical implications of GLS in cardio-oncology patients, more widespread implementation of AI-based strain assessment has the potential to significantly improve the reproducibility of GLS measurement in this unique patient population.

NIMG-63. LEVERAGING LLMS FOR ACCURATE DIFFERENTIATION OF RADIATION NECROSIS AND TUMOR PROGRESSION IN BRAIN MRI REPORTS: A STUDY ON AUTOMATED SCORING AND CLINICAL IMPLICATIONS

Abstract BACKGROUND Accurate differentiation between radiation necrosis and tumor progression or recurrence in patients treated with stereotactic radiosurgery for brain metastases is critical for guiding clinical management. This study leverages the advanced natural language processing capabilities of Meta Llama3, an artificial intelligence (AI) large language model (LLM), combined with prompt engineering, to rapidly categorize brain magnetic resonance imaging (MRI) radiology reports. Our objective was to develop an automated scoring system to classify concern for radiation necrosis, tumor progression, equivocal findings, or stable exams. METHODS Using a comprehensive dataset of reports annotated by expert radiologists, we ran inference on a 70-billion parameter Llama3 model (temperature 0.2, top_p 0.9) with specific prompts designed to capture the nuanced language and diagnostic criteria related to radiation necrosis or tumor progression. RESULTS The first pass was performed on a training dataset of 107 reports and did not predefine the clinical conditions. This demonstrated 43.4% accuracy in scoring when compared to a human user’s classification of each report. Agreement between the human reader and Llama3 was assessed using the Gwet agreement coefficient, AC1=0.411 (99%CI 0.396-0.426). Multiple iterations of targeted prompt engineering were then employed to narrow the definition of radiation necrosis and tumor progression, with specific examples and nuanced language used to achieve a higher degree of accuracy at 72.0%, AC1=0.719 (99%CI 0.717-0.722). DISCUSSION/FUTURE DIRECTIONS This surpasses recent demonstration of lower human-LLM agreement in radiographic score assignment, with further room for calibration on a dataset of several thousand reports. Next, we will correlate automated interpretations with actual clinical management decisions for radiation necrosis (e.g., initiation of steroids, bevacizumab, Laser Interstitial Thermal Therapy, and/or repeat imaging). This automated scoring system holds significant potential for LLMs in clinical applications. Future work will focus on integrating this model into clinical workflows and expanding its capabilities to include longitudinal monitoring of patient outcomes.

A critical comparative study of the performance of three AI-assisted programs for bone age determination.

To date, AI-supported programs for bone age (BA) determination for medical use in Europe have almost only been validated separately, according to Greulich and Pyle (G&P). Therefore, the current study aimed to compare the performance of three programs, namely BoneXpert, PANDA, and BoneView, on a single Central European population. For this retrospective study, hand radiographs of 306 children aged 1-18 years, stratified by gender and age, were included. A subgroup consisting of the age group accounting for 90% of examinations in clinical practice was formed. The G&P BA was estimated by three human experts-as ground truth-and three AI-supported programs. The mean absolute deviation, the root mean squared error (RMSE), and dropouts by the AI were calculated. The correlation between all programs and the ground truth was prominent (R2 ≥ 0.98). In the total group, BoneXpert had a lower RMSE than BoneView and PANDA (0.62 vs. 0.65 and 0.75 years) with a dropout rate of 2.3%, 20.3% and 0%, respectively. In the subgroup, there was less difference in RMSE (0.66 vs. 0.68 and 0.65 years, max. 4% dropouts). The standard deviation between the AI readers was lower than that between the human readers (0.54 vs. 0.62 years, p < 0.01). All three AI programs predict BA after G&P in the main age range with similar high reliability. Differences arise at the boundaries of childhood. Question There is a lack of comparative, independent validation for artificial intelligence-based bone age estimation in children. Findings Three commercially available programs estimate bone age after Greulich and Pyle with similarly high reliability in a central European cohort. Clinical relevance The comparative study will help the reader choose a software for bone age estimation approved for the European market depending on the targeted age group and economic considerations.

Deep Learning Algorithms for Breast Cancer Detection in a UK Screening Cohort: As Stand-alone Readers and Combined with Human Readers.

Background Deep learning (DL) algorithms have shown promising results in mammographic screening either compared to a single reader or, when deployed in conjunction with a human reader, compared with double reading. Purpose To externally validate the performance of three DL algorithms as mammographic screen readers in an independent UK data set. Materials and Methods Three commercial DL algorithms (DL-1, DL-2, and DL-3) were retrospectively investigated from January 2022 to June 2022 using consecutive full-field digital mammograms collected at two UK sites during 1 year (2017). Normal cases with 3-year follow-up and histopathologically proven cancer cases detected either at screening (that round or next) or within the 3-year interval were included. A preset specificity threshold equivalent to a single reader was applied. Performance was evaluated for stand-alone DL reading compared with single human reading, and for DL reading combined with human reading compared with double reading, using sensitivity and specificity as the primary metrics. P < .025 was considered to indicate statistical significance for noninferiority testing. Results A total of 26 722 cases (median patient age, 59.0 years [IQR, 54.0-63.0 years]) with mammograms acquired using machines from two vendors were included. Cases included 332 screen-detected, 174 interval, and 254 next-round cancers. Two of three stand-alone DL algorithms achieved noninferior sensitivity (DL-1: 64.8%, P < .001; DL-2: 56.7%, P = .03; DL-3: 58.9%, P < .001) compared with the single first reader (62.8%), and specificity was noninferior for DL-1 (92.8%; P < .001) and DL-2 (96.8%; P < .001) and superior for DL-3 (97.9%; P < .001) compared with the single first reader (96.5%). Combining the DL algorithms with human readers achieved noninferior sensitivity (67.0%, 65.6%, and 65.4% for DL-1, DL-2, and DL-3, respectively; P < .001 for all) compared with double reading (67.4%), and superior specificity (97.4%, 97.6%, and 97.6%; P < .001 for all) compared with double reading (97.1%). Conclusion Use of stand-alone DL algorithms in combination with a human reader could maintain screening accuracy while reducing workload. Published under a CC BY 4.0 license. Supplemental material is available for this article.

Diagnostic performance of an artificial intelligence model for the detection of pneumothorax at chest X-ray

PurposePneumothorax (PTX) is a common clinical urgency, its diagnosis is usually performed on chest radiography (CXR), and it presents a setting where artificial intelligence (AI) methods could find terrain in aiding radiologists in facing increasing workloads. Hence, the purpose of our study was to test an AI system for the detection of PTX on CXR examinations, to review its diagnostic performance in such setting alongside that of reading radiologists. MethodWe retrospectively ran the AI system on CXR examinations of patients who were imaged for the suspicion of PTX, and who also underwent computed tomography (CT) within the same day, the latter being used as reference standard. The performance of the proposed AI system was compared to that of reading radiologists, obtained from CXR reports. ResultsOverall, the AI system achieved an accuracy of 74 % (95%CI 68–79 %), with a sensitivity of 66 % (95%CI 59–73 %) and a specificity of 93 % (95%CI 85–97 %). Human readers displayed a comparable accuracy (77 %, 95%CI 71–82 %, p = 0.355), with higher sensitivity (73 %, 95%CI 66–79 %, p = 0.040), albeit lower specificity (85 %, 95%CI 75–91 %, p = 0.034). The performance of AI was influenced by patient positioning at CXR (p = 0.040). ConclusionsThe proposed tool could represent an aid to radiologists in detecting PTX, improving specificity. Further improvement with training on more challenging cases may pave the way for its use as a screening or standalone tool.

Toward Foundation Models in Radiology? Quantitative Assessment of GPT-4V's Multimodal and Multianatomic Region Capabilities.

Background Large language models have already demonstrated potential in medical text processing. GPT-4V, a large vision-language model from OpenAI, has shown potential for medical imaging, yet a quantitative analysis is lacking. Purpose To quantitatively assess the performance of GPT-4V in interpreting radiologic images using unseen data. Materials and Methods This retrospective study included single representative abnormal and healthy control images from neuroradiology, cardiothoracic radiology, and musculoskeletal radiology (CT, MRI, radiography) to generate reports using GPT-4V via the application programming interface from February to March 2024. The factual correctness of free-text reports and the performance in detecting abnormalities in binary classification tasks were assessed using accuracy, sensitivity, and specificity. The binary classification performance was compared with that of a first-year nonradiologist in training and four board-certified radiologists. Results A total of 515 images in 470 patients (median age, 61 years [IQR, 44-71 years]; 267 male) were included, of which 345 images were abnormal. GPT-4V correctly identified the imaging modality and anatomic region in 100% (515 of 515) and 99.2% (511 of 515) of images, respectively. Diagnostic accuracy in free-text reports was between 0% (0 of 33 images) for pneumothorax (CT and radiography) and 90% (45 of 50 images) for brain tumor (MRI). In binary classification tasks, GPT-4V showed sensitivities between 56% (14 of 25 images) for ischemic stroke and 100% (25 of 25 images) for brain hemorrhage and specificities between 8% (two of 25 images) for brain hemorrhage and 52% (13 of 25 images) for pneumothorax, compared with a pooled sensitivity of 97.2% (1103 of 1135 images) and pooled specificity of 97.2% (1084 of 1115 images) for the human readers across all tasks. The model exhibited a clear tendency to overdiagnose abnormalities, with 86.5% (147 of 170 images) and 67.7% (151 of 223 images) false-positive rates for the free-text and binary classification tasks, respectively. Conclusion GPT-4V, in its earliest version, recognized medical image content and reliably determined the modality and anatomic region from single images. However, GPT-4V failed to detect, classify, or rule out abnormalities in image interpretation. © RSNA, 2024 Supplemental material is available for this article.

"Artificial intelligence Reading Digital Mammogram: Enhancing Detection and Differentiation of Suspicious Microcalcifications".

To investigate the impact of artificial intelligence (AI) on enhancing the sensitivity of digital mammograms in the detection and specification of grouped microcalcifications. The study is a retrospective analysis of grouped microcalcifications for 447 patients. Grouped microcalcifications detected were correlated with AI, which was applied to the initial mammograms. AI provided a heat map, demarcation, and quantitative evaluation for abnormalities according to the degree of suspicion of malignancy. Histopathology was the standard for confirmation of malignancy. AI showed a high correlation percentage of 67.5% between the red color of the color hue bar and malignant microcalcifications (p value <0.001). The scoring of probable cancer was suggested (ie, more than 50% abnormality scoring) in 39.5% of true cancer lesions. The diagnostic performance of mammography for grouped microcalcifications revealed a sensitivity of 94.7% and a negative predictive value of 82.1%. False negatives were only 12 out of 228 that proved malignant calcifications. The agreement of cancer probability between standard mammograms and examinations read by AI presented a Kappa value of -0.094 and a p value of < 0.001. The used AI system enhanced the sensitivity of mammograms in detecting suspicious microcalcifications, yet an expert human reader is required for proper specification. Grouped calcifications could be early breast cancer on a mammogram. The morphology and distribution are correlated with the nature of breast diseases. AI is a potential decision support for the detection and classification of grouped microcalcifications and thus positively affects the control of breast cancer.

AI-based algorithm for assessing coronary artery calcium score on contrast-enhanced cardiac computed tomography scans

Abstract Introduction In cardiac computed tomography (CT) examinations, non-contrast scans are often performed in addition to contrast-enhanced scans to quantify the coronary artery calcification (CAC) score. These calcium score scans are associated with additional exposure to ionizing radiation. Purpose We sought to develop a fully automated artificial intelligence (AI)-based algorithm capable of determining CAC score based solely on contrast-enhanced CT scans, eliminating the need for additional non-contrast scans. Methods An automated CAC scoring algorithm on contrast-enhanced CT scans was developed and trained using a dataset of 297 cardiac CT studies. Coronary artery calcifications were manually segmented in contrast-enhanced scans using a threshold of 2 standard deviations above the mean attenuation value of the ascending aorta. On non-contrast scans, CAC was manually assessed using a standard threshold of ≥130 Hounsfield Units. A correction factor for calcium score results assessed on contrast-enhanced scans was determined using linear regression. The algorithm was tested on an independent set of 90 contrast-enhanced CT scans (four manufacturers, eight scanner models). We compared automated CAC scores to manual expert reader reference assessments on non-contrast CT scans. CAC scores were categorized into four risk categories following the Society of Cardiovascular Computed Tomography recommendations: 0, 1-100, 101-300, and &amp;gt;300 Agatston Units. Results In the assessment of 90 CT studies (mean age 61.0±11.4 years, 46.7% males), the AI-model detected CAC in 69 contrast-enhanced scans (76.7%), comparable to the human reader's detection rate of CAC in 71 non-contrast scans (78.9%) (p = 0.63). The CAC score was initially calculated using AI-based segmentation of coronary calcification on contrast-enhanced scans and then the result was multiplied by established linear correction factor of 1.97. There was an excellent correlation between AI-model and manual reference total CAC scores (Pearson’s r = 0.96, 95% CI 0.94–0.97, p &amp;lt; 0.001). The model correctly classified 77 patients (85.6%) into the same CAC risk category as the human reader (Figure 1). Among 19 patients (21.1%) with a CAC score of zero, only 1 patient (5.3%) was reclassified with a non-zero CAC score by the AI-model. Cohen’s kappa value for CAC score risk categorization was 0.80 (p &amp;lt; 0.001), indicating very good agreement (Figure 2). Bland–Altman analysis revealed minimal bias of -9.7 Agatston Unit with 95% limits of agreement ranging from -184.8 to 165.5 Agatston Unit. Conclusions CAC score can be accurately quantified on contrast-enhanced cardiac CT scans using an automated AI-based algorithm. This approach has the potential to eliminate the necessity for an additional non-contrast CT scan, thereby reducing the patient's exposure to ionizing radiation.

Validation of Inter-Reader Agreement/Consistency for Quantification of Ellipsoid Zone Integrity and Sub-RPE Compartmental Features Across Retinal Diseases.

An unmet need exists when clinically assessing retinal and layer-based features of retinal diseases. Therefore, quantification of retinal-layer-thicknesses/fluid volumes using deep-learning-augmented platforms to reproduce human-obtained clinical measurements is needed. In this analysis, 210 spectral-domain optical coherence tomography (SD-OCT) scans (30 without pathology, 60 dry age-related macular degeneration [AMD], 60 wet AMD, and 60 diabetic macular edema [total 23,625 B-scans]) were included. A fully automated segmentation platform segmented four retinal layers for compartmental assessment (internal limiting membrane, ellipsoid zone [EZ], retinal pigment epithelium [RPE], and Bruch's membrane). Two certified OCT readers independently completed manual segmentation and B-scan level validation of automated segmentation, with segmentation correction when needed (semi-automated). Certified reader metrics were compared to gold standard metrics using intraclass correlation coefficients (ICCs) to assess overall agreement. Across different diseases, several metrics generated from automated segmentations approached or matched human readers performance. Absolute ICCs for retinal mean thickness measurements showed excellent agreement (range 0.980-0.999) across four cohorts. EZ-RPE thickness values and sub-RPE compartment ICCs demonstrated excellent agreement (ranges of 0.953-0.987 and 0.944-0.997, respectively) for full dataset, dry-AMD, and wet-AMD cohorts. Analyses demonstrated high reliability and consistency of segmentation of outer retinal compartmental features using a completely human/manual approach or a semi-automated approach to segmentation. These results support the critical role that measuring features, such as photoreceptor preservation through EZ integrity, in future clinical trials may optimize clinical care.

Post-deployment performance of a deep learning algorithm for normal and abnormal chest X-ray classification: A study at visa screening centers in the United Arab Emirates

BackgroundChest radiographs (CXRs) are widely used to screen for infectious diseases like tuberculosis and COVID-19 among migrants. At such high-volume settings, manual CXR reporting is challenging and integrating artificial intelligence (AI) algorithms into the workflow help to rule out normal findings in minutes, allowing radiologists to focus on abnormal cases. MethodsIn this post-deployment study, all the CXRs acquired during the visa screening process across 33 centers in United Arab Emirates from January 2021 to June 2022 (18 months) were included. The qXR v2.1 chest X-ray interpretation software was used to classify the scans into normal and abnormal, and its agreement against radiologist was evaluated. Additionally, a digital survey was conducted among 20 healthcare professionals with prior AI experience to understand real-world implementation challenges and impact. ResultsThe analysis of 1309,443 CXRs from 1309,431 patients (median age: 35 years; IQR [29–42]; 1030,071 males [78.7 %]) in this study revealed a Negative Predictive Value (NPV) of 99.92 % (95 % CI: 99.92, 99.93), Positive Predictive Value (PPV) of 5.06 % (95 % CI: 4.99, 5.13) and overall percent agreement of the AI with radiologists of 72.90 % (95 % CI: 72.82, 72.98). In the survey, majority (88.2 %) of the radiologists agreed to turnaround time reduction after AI integration, while 82 % suggested that the AI improved their diagnostic accuracy. DiscussionIn contrast with the existing studies, this research uses a substantially large data. A high NPV and satisfactory agreement with human readers indicate that AI can reliably identify normal CXRs, making it suitable for routine applications.

“Let faith oust fact; let fancy oust memory”: Melville, Media and Narratives

Herman Melville’s birthday (On August 1, 1819) was a foreshadowing gesture of the present special issue, having come out of a long –somewhat protracted— collaboration, which coincides with the original publication of Moby-Dick: The Whale on October 18, 1851. The Journal of Narrative and Language Studies takes the opportunity to celebrate Melville’s resonating myth of the human being at the threshold of posthumanism, which explored the human psyche and mysterious potentials from dignity to evil, having been protracted for centuries but eventually led to oblivion. Since the time when he thoroughly, painstakingly, peculiarly created Moby-Dick” and crafted a world of consuming human desire that represents the universal human condition, readers, writers, and artists have been enormously inspired to envisage, fancy, create, design and produce verbal or nonverbal arts in the form of visual works or other means of imagination.

Automatic structuring of radiology reports with on-premise open-source large language models.

Structured reporting enhances comparability, readability, and content detail. Large language models (LLMs) could convert free text into structured data without disrupting radiologists' reporting workflow. This study evaluated an on-premise, privacy-preserving LLM for automatically structuring free-text radiology reports. We developed an approach to controlling the LLM output, ensuring the validity and completeness of structured reports produced by a locally hosted Llama-2-70B-chat model. A dataset with de-identified narrative chest radiograph (CXR) reports was compiled retrospectively. It included 202 English reports from a publicly available MIMIC-CXR dataset and 197 German reports from our university hospital. Senior radiologist prepared a detailed, fully structured reporting template with 48 question-answer pairs. All reports were independently structured by the LLM and two human readers. Bayesian inference (Markov chain Monte Carlo sampling) was used to estimate the distributions of Matthews correlation coefficient (MCC), with [-0.05, 0.05] as the region of practical equivalence (ROPE). The LLM generated valid structured reports in all cases, achieving an average MCC of 0.75 (94% HDI: 0.70-0.80) and F1 score of 0.70 (0.70-0.80) for English, and 0.66 (0.62-0.70) and 0.68 (0.64-0.72) for German reports, respectively. The MCC differences between LLM and humans were within ROPE for both languages: 0.01 (-0.05 to 0.07), 0.01 (-0.05 to 0.07) for English, and -0.01 (-0.07 to 0.05), 0.00 (-0.06 to 0.06) for German, indicating approximately comparable performance. Locally hosted, open-source LLMs can automatically structure free-text radiology reports with approximately human accuracy. However, the understanding of semantics varied across languages and imaging findings. Question Why has structured reporting not been widely adopted in radiology despite clear benefits and how can we improve this? Findings A locally hosted large language model successfully structured narrative reports, showing variation between languages and findings. Critical relevance Structured reporting provides many benefits, but its integration into the clinical routine is limited. Automating the extraction of structured information from radiology reports enables the capture of structured data while allowing the radiologist to maintain their reporting workflow.

Coronary artery disease detection using deep learning and ultrahigh-resolution photon-counting coronary CT angiography

PurposeThe purpose of this study was to evaluate the diagnostic performance of automated deep learning in the detection of coronary artery disease (CAD) on photon-counting coronary CT angiography (PC-CCTA). Materials and methodsConsecutive patients with suspected CAD who underwent PC-CCTA between January 2022 and December 2023 were included in this retrospective, single-center study. Non-ultra-high resolution (UHR) PC-CCTA images were analyzed by artificial intelligence using two deep learning models (CorEx, Spimed-AI), and compared to human expert reader assessment using UHR PC-CCTA images. Diagnostic performance for global CAD assessment (at least one significant stenosis ≥ 50 %) was estimated at patient and vessel levels. ResultsA total of 140 patients (96 men, 44 women) with a median age of 60 years (first quartile, 51; third quartile, 68) were evaluated. Significant CAD on UHR PC-CCTA was present in 36/140 patients (25.7 %). The sensitivity, specificity, accuracy, positive predictive value), and negative predictive value of deep learning-based CAD were 97.2 %, 81.7 %, 85.7 %, 64.8 %, and 98.9 %, respectively, at the patient level and 96.6 %, 86.7 %, 88.1 %, 53.8 %, and 99.4 %, respectively, at the vessel level. The area under the receiver operating characteristic curve was 0.90 (95 % CI: 0.83–0.94) at the patient level and 0.92 (95 % CI: 0.89–0.94) at the vessel level. ConclusionAutomated deep learning shows remarkable performance for the diagnosis of significant CAD on non-UHR PC-CCTA images. AI pre-reading may be of supportive value to the human reader in daily clinical practice to target and validate coronary artery stenosis using UHR PC-CCTA.

Assessment of the Breast Density Prevalence in Swiss Women with a Deep Convolutional Neural Network: A Cross-Sectional Study

Background/Objectives: High breast density is a risk factor for breast cancer and can reduce the sensitivity of mammography. Given the influence of breast density on patient risk stratification and screening accuracy, it is crucial to monitor the prevalence of extremely dense breasts within local populations. Moreover, there is a lack of comprehensive understanding regarding breast density prevalence in Switzerland. Therefore, this study aimed to determine the prevalence of breast density in a selected Swiss population. Methods: To overcome the potential variability in breast density classifications by human readers, this study utilized commercially available deep convolutional neural network breast classification software. A retrospective analysis of mammographic images of women aged 40 years and older was performed. Results: A total of 4698 mammograms from women (58 ± 11 years) were included in this study. The highest prevalence of breast density was in category C (heterogeneously dense), which was observed in 41.5% of the cases. This was followed by category B (scattered areas of fibroglandular tissue), which accounted for 22.5%. Conclusion: Notably, extremely dense breasts (category D) were significantly more common in younger women, with a prevalence of 34%. However, this rate dropped sharply to less than 10% in women over 55 years of age.

Galileo-an Artificial Intelligence tool for evaluating pre-implantation kidney biopsies.

Pre-transplant procurement biopsy interpretation is challenging, also because of the low number of renal pathology experts. Artificial intelligence (AI) can assist by aiding pathologists with kidney donor biopsyassessment. Herein we present the "Galileo" AI tool, designed specifically to assist the on-call pathologist with interpreting pre-implantation kidney biopsies. A multicenter cohort of whole slide images acquired from core-needle and wedge biopsies of the kidney was collected. A deep learning algorithm was trained to detect the main findings evaluated in the pre-implantation setting (normal glomeruli, globally sclerosed glomeruli, ischemic glomeruli, arterioles and arteries). The model obtained on the Aiforia Create platform was validated on an external dataset by three independent pathologists to evaluate the performance of the algorithm. Galileo demonstrated a precision, sensitivity, F1 score and total area error of 81.96%, 94.39%, 87.74%, 2.81% and 74.05%, 71.03%, 72.5%, 2% in the training and validation sets, respectively. Galileo was significantly faster than pathologists, requiring 2min overall in the validation phase (vs 25, 22 and 31min by 3 separate human readers, p < 0.001). Galileo-assisted detection of renal structures and quantitative information was directly integrated in the final report. The Galileo AI-assisted tool shows promise inspeeding up pre-implantation kidney biopsy interpretation, as well as inreducing inter-observer variability. This toolmay represent a starting point for further improvements based on hardendpoints such as graft survival.

Evaluation of an AI-Driven Deep Learning Model in Predicting Mammographic Breast Density

Abstract Background Increased mammographic breast density is an established risk factor for breast cancer. Given the high workload of mammography screening programs, Mounting attention is being directed at establishing and fine- tunning AI algorithms capable of improving the performance of human readers. Aim of the Work To emphasize the role of AI-driven deep learning algorithms in accurate prediction of breast density on digital mammograms. Patients and Methods The study was performed on 62 digital mammography films of women undergoing breast cancer screening. Mammograms were assessed by three human readers of different levels of experience in comparison to a commercially available AI algorithm (Lunit INSIGHT MMG, version 1.1.7.2; Lunit Inc). Results This clinical investigation demonstrated that AI softwares (namely LUNIT INSIGHT in our study) can predict mammographic breast density with about 67.7%, 69.4 % and 75.81 % accuracy for HR1(least experienced), HR2 and HR3 (most experienced) respectively, and reliability (Cronbach's Alpha) 0.887, 0.892 and 0.906 for HR1(least experienced), HR2 and HR3 (most experienced) respectively. This result offers a robust way to overcome the variability of human visual assessment and improve the overall performance of less experienced radiologists allowing them to reach expert like levels. Conclusion Application of AI softwares on screening mammograms can overcome the variability of human visual assessment and improve the overall performance of less experienced radiologists allowing them to reach expert like levels.

Deep learning-based detection and classification of intracranial tumors on magnetic resonance imaging

AbstractBackgroundWe evaluated the capability of an AI application to independently detect, segment and classify intracranial tumors in MRI.MethodsIn this retrospective single-centre study, 138 patients (65 female and 73 male) with a mean age of 35 ± 26y were included. 97 were diagnosed with an intracranial neoplasm, while 41 exhibited no intracranial pathology. Inclusion criteria were a 1.5 or 3.0T MRI dataset with the following sequences: T2 axial, T1 axial pre- and post-contrast with a slice thickness between 3 and 6 mm and no previous brain surgery. Image analysis was performed by two human readers (R1 = 5 years and R2 = 10 years of experience in brain MRI) and a deep learning (DL)-based AI model. Sensitivity, specificity and accuracy of the AI model and the human readers to detect and correctly classify brain tumors were measured. Histological results served as the gold standard.ResultsThe AI model reached a sensitivity of 93.81% [87.02–97.70] and a specificity of 63.41% [46.94–77.88], while human readers reached 100% [96.27–100.00] and 100% [91.40–100.00], respectively. Human readers provided a significantly higher accuracy rate with R1 0.93 (95% CI: 0.88–0.97) and R2 0.98 (95% CI: 0.94, 0.99) vs. 0.74 (95% CI: 0.66–0.81) for the AI model (P-value &lt;0.001).ConclusionThe underlying DL-based AI algorithm can independently identify and segment intracranial tumors while providing satisfactory results for establishing the correct diagnosis. Despite its current inferiority compared to experienced radiologists, it still experiences ongoing development and it is a step towards developing an artificial intelligence-augmented radiology workflow.