Artificial Intelligence Diagnostic System Research Articles

Performance of a Chest Radiograph AI Diagnostic Tool for COVID-19: A Prospective Observational Study.

PurposeTo conduct a prospective observational study across 12 U.S. hospitals to evaluate real-time performance of an interpretable artificial intelligence (AI) model to detect COVID-19 on chest radiographs.Materials and MethodsA total of 95 363 chest radiographs were included in model training, external validation, and real-time validation. The model was deployed as a clinical decision support system, and performance was prospectively evaluated. There were 5335 total real-time predictions and a COVID-19 prevalence of 4.8% (258 of 5335). Model performance was assessed with use of receiver operating characteristic analysis, precision-recall curves, and F1 score. Logistic regression was used to evaluate the association of race and sex with AI model diagnostic accuracy. To compare model accuracy with the performance of board-certified radiologists, a third dataset of 1638 images was read independently by two radiologists.ResultsParticipants positive for COVID-19 had higher COVID-19 diagnostic scores than participants negative for COVID-19 (median, 0.1 [IQR, 0.0–0.8] vs 0.0 [IQR, 0.0–0.1], respectively; P < .001). Real-time model performance was unchanged over 19 weeks of implementation (area under the receiver operating characteristic curve, 0.70; 95% CI: 0.66, 0.73). Model sensitivity was higher in men than women (P = .01), whereas model specificity was higher in women (P = .001). Sensitivity was higher for Asian (P = .002) and Black (P = .046) participants compared with White participants. The COVID-19 AI diagnostic system had worse accuracy (63.5% correct) compared with radiologist predictions (radiologist 1 = 67.8% correct, radiologist 2 = 68.6% correct; McNemar P < .001 for both).ConclusionAI-based tools have not yet reached full diagnostic potential for COVID-19 and underperform compared with radiologist prediction.Keywords: Diagnosis, Classification, Application Domain, Infection, Lung Supplemental material is available for this article.. © RSNA, 2022

Validation of artificial intelligence prediction models for skin cancer diagnosis using dermoscopy images: the 2019 International Skin Imaging Collaboration Grand Challenge

BackgroundPrevious studies of artificial intelligence (AI) applied to dermatology have shown AI to have higher diagnostic classification accuracy than expert dermatologists; however, these studies did not adequately assess clinically realistic scenarios, such as how AI systems behave when presented with images of disease categories that are not included in the training dataset or images drawn from statistical distributions with significant shifts from training distributions. We aimed to simulate these real-world scenarios and evaluate the effects of image source institution, diagnoses outside of the training set, and other image artifacts on classification accuracy, with the goal of informing clinicians and regulatory agencies about safety and real-world accuracy.MethodsWe designed a large dermoscopic image classification challenge to quantify the performance of machine learning algorithms for the task of skin cancer classification from dermoscopic images, and how this performance is affected by shifts in statistical distributions of data, disease categories not represented in training datasets, and imaging or lesion artifacts. Factors that might be beneficial to performance, such as clinical metadata and external training data collected by challenge participants, were also evaluated. 25 331 training images collected from two datasets (in Vienna [HAM10000] and Barcelona [BCN20000]) between Jan 1, 2000, and Dec 31, 2018, across eight skin diseases, were provided to challenge participants to design appropriate algorithms. The trained algorithms were then tested for balanced accuracy against the HAM10000 and BCN20000 test datasets and data from countries not included in the training dataset (Turkey, New Zealand, Sweden, and Argentina). Test datasets contained images of all diagnostic categories available in training plus other diagnoses not included in training data (not trained category). We compared the performance of the algorithms against that of 18 dermatologists in a simulated setting that reflected intended clinical use.Findings64 teams submitted 129 state-of-the-art algorithm predictions on a test set of 8238 images. The best performing algorithm achieved 58·8% balanced accuracy on the BCN20000 data, which was designed to better reflect realistic clinical scenarios, compared with 82·0% balanced accuracy on HAM10000, which was used in a previously published benchmark. Shifted statistical distributions and disease categories not included in training data contributed to decreases in accuracy. Image artifacts, including hair, pen markings, ulceration, and imaging source institution, decreased accuracy in a complex manner that varied based on the underlying diagnosis. When comparing algorithms to expert dermatologists (2460 ratings on 1269 images), algorithms performed better than experts in most categories, except for actinic keratoses (similar accuracy on average) and images from categories not included in training data (26% correct for experts vs 6% correct for algorithms, p<0·0001). For the top 25 submitted algorithms, 47·1% of the images from categories not included in training data were misclassified as malignant diagnoses, which would lead to a substantial number of unnecessary biopsies if current state-of-the-art AI technologies were clinically deployed.InterpretationWe have identified specific deficiencies and safety issues in AI diagnostic systems for skin cancer that should be addressed in future diagnostic evaluation protocols to improve safety and reliability in clinical practice.FundingMelanoma Research Alliance and La Marató de TV3.

Diagnostic value of circulating genetically abnormal cells to support computed tomography for benign and malignant pulmonary nodules

BackgroundThe accuracy of CT and tumour markers in screening lung cancer needs to be improved. Computer-aided diagnosis has been reported to effectively improve the diagnostic accuracy of imaging data, and recent studies have shown that circulating genetically abnormal cell (CAC) has the potential to become a novel marker of lung cancer. The purpose of this research is explore new ways of lung cancer screening.MethodsFrom May 2020 to April 2021, patients with pulmonary nodules who had received CAC examination within one week before surgery or biopsy at First Affiliated Hospital of Zhengzhou University were enrolled. CAC counts, CT scan images, serum tumour marker (CEA, CYFRA21–1, NSE) levels and demographic characteristics of the patients were collected for analysis. CT were uploaded to the Pulmonary Nodules Artificial Intelligence Diagnostic System (PNAIDS) to assess the malignancy probability of nodules. We compared diagnosis based on PNAIDS, CAC, Mayo Clinic Model, tumour markers alone and their combination. The combination models were built through logistic regression, and was compared through the area under (AUC) the ROC curve.ResultsA total of 93 of 111 patients were included. The AUC of PNAIDS was 0.696, which increased to 0.847 when combined with CAC. The sensitivity (SE), specificity (SP), and positive (PPV) and negative (NPV) predictive values of the combined model were 61.0%, 94.1%, 94.7% and 58.2%, respectively. In addition, we evaluated the diagnostic value of CAC, which showed an AUC of 0.779, an SE of 76.3%, an SP of 64.7%, a PPV of 78.9%, and an NPV of 61.1%, higher than those of any single serum tumour marker and Mayo Clinic Model. The combination of PNAIDS and CAC exhibited significantly higher AUC values than the PNAIDS (P = 0.009) or the CAC (P = 0.047) indicator alone. However, including additional tumour markers did not significantly alter the performance of CAC and PNAIDS.ConclusionsCAC had a higher diagnostic value than traditional tumour markers in early-stage lung cancer and a supportive value for PNAIDS in the diagnosis of cancer based on lung nodules. The results of this study offer a new mode of screening for early-stage lung cancer using lung nodules.

Implementation of artificial intelligence in the histological assessment of pulmonary subsolid nodules.

BackgroundClinical management of subsolid nodules (SSNs) is defined by the suspicion of tumor invasiveness. We sought to develop an artificial intelligent (AI) algorithm for invasiveness assessment of lung adenocarcinoma manifesting as radiological SSNs. We investigated the performance of this algorithm in classification of SSNs related to invasiveness.MethodsA retrospective chest computed tomography (CT) dataset of 1,589 SSNs was constructed to develop (85%) and internally test (15%) the proposed AI diagnostic tool, SSNet. Diagnostic performance was evaluated in the hold-out test set and was further tested in an external cohort of 102 SSNs. Three thoracic surgeons and three radiologists were required to evaluate the invasiveness of SSNs on both test datasets to investigate the clinical utility of the proposed SSNet.ResultsIn the differentiation of invasive adenocarcinoma (IA), SSNet achieved a similar area under the curve [AUC; 0.914, 95% confidence interval (CI): 0.813–0.987] with that of the 6 doctors (0.900, 95% CI: 0.867–0.922). When interpreting with the assistance of SSNet, the sensitivity of junior doctors, specificity of senior doctor, and their accuracy were significantly improved. In the external test, SSNet (AUC: 0.949, 95% CI: 0.884–1.000) achieved a better AUC than doctors (AUC: 0.883, 95% CI: 0.826–0.939) whose AUC increased (AUC: 0.908, 95% CI: 0.847–0.982) with SSNet assistance. In the histological subtype classifications, SSNet achieved better performance than practicing doctors. The AUCs of doctors were significantly improved with the assistance of SSNet in both 4-category and 3-category classifications to 0.836 (95% CI: 0.811–0.862) and 0.852 (95% CI: 0.825–0.882), respectively.ConclusionsThe AI diagnostic system achieved non-inferior performance to doctors, and will potentially improve diagnostic performance and efficiency in SSN evaluation.

An Artificial Intelligence System for the Detection of Bladder Cancer via Cystoscopy: A Multicenter Diagnostic Study.

Cystoscopy plays an important role in bladder cancer (BCa) diagnosis and treatment, but its sensitivity needs improvement. Artificial intelligence has shown promise in endoscopy, but few cystoscopic applications have been reported. We report a Cystoscopy Artificial Intelligence Diagnostic System (CAIDS) for BCa diagnosis. In total, 69 204 images from 10 729 consecutive patients from 6 hospitals were collected and divided into training, internal validation, and external validation sets. The CAIDS was built using a pyramid scene parsing network and transfer learning. A subset (n = 260) of the validation sets was used for a performance comparison between the CAIDS and urologists for complex lesion detection. The diagnostic accuracy, sensitivity, specificity, and positive and negative predictive values and 95% confidence intervals (CIs) were calculated using the Clopper-Pearson method. The diagnostic accuracies of the CAIDS were 0.977 (95% CI = 0.974 to 0.979) in the internal validation set and 0.990 (95% CI = 0.979 to 0.996), 0.982 (95% CI = 0.974 to 0.988), 0.978 (95% CI = 0.959 to 0.989), and 0.991 (95% CI = 0.987 to 0.994) in different external validation sets. In the CAIDS vs urologists' comparisons, the CAIDS showed high accuracy and sensitivity (accuracy = 0.939, 95% CI = 0.902 to 0.964; sensitivity = 0.954, 95% CI = 0.902 to 0.983) with a short latency of 12 seconds, much more accurate and quicker than the expert urologists. The CAIDS achieved accurate BCa detection with a short latency. The CAIDS may provide many clinical benefits, from increasing the diagnostic accuracy for BCa, even for commonly misdiagnosed cases such as flat cancerous tissue (carcinoma in situ), to reducing the operation time for cystoscopy.

Efficiency of a deep learning-based artificial intelligence diagnostic system in spontaneous intracerebral hemorrhage volume measurement

BackgroundAccurate measurement of hemorrhage volume is critical for both the prediction of prognosis and the selection of appropriate clinical treatment after spontaneous intracerebral hemorrhage (ICH). This study aimed to evaluate the performance and accuracy of a deep learning-based automated segmentation algorithm in segmenting spontaneous intracerebral hemorrhage (ICH) volume either with or without intraventricular hemorrhage (IVH) extension. We compared this automated pipeline with two manual segmentation techniques.MethodsWe retrospectively reviewed 105 patients with acute spontaneous ICH. Depending on the presence of IVH extension, patients were divided into two groups: ICH without (n = 56) and with IVH (n = 49). ICH volume of the two groups were segmented and measured using a deep learning-based artificial intelligence (AI) diagnostic system and computed tomography-based planimetry (CTP), and the ABC/2 score were used to measure hemorrhage volume in the ICH without IVH group. Correlations and agreement analyses were used to analyze the differences in volume and length of processing time among the three segmentation approaches.ResultsIn the ICH without IVH group, the ICH volumes measured using AI and the ABC/2 score were comparable to CTP segmentation. Strong correlations were observed among the three segmentation methods (r = 0.994, 0.976, 0.974; P < 0.001; concordance correlation coefficient [CCC] = 0.993, 0.968, 0.967). But the absolute error of the ICH volume measured by the ABC/2 score was greater than that of the algorithm (P < 0.05). In the ICH with IVH group, there is no significant differences were found between algorithm and CTP(P = 0.614). The correlation and agreement between CTP and AI were strong (r = 0.996, P < 0.001; CCC = 0.996). The AI segmentation took a significantly shorter amount of time than CTP (P < 0.001), but was slightly longer than ABC/2 score technique (P = 0.002).ConclusionsThe deep learning-based AI diagnostic system accurately quantified volumes of acute spontaneous ICH with high fidelity and greater efficiency compared to the CTP measurement and more accurately than the ABC/2 scores. We believe this is a promising tool to help physicians achieve precise ICH quantification in practice.

Development of an artificial intelligence diagnostic system for lower urinary tract dysfunction in men.

To establish an artificial intelligence diagnostic system for lower urinary tract function in men with lower urinary tract symptoms using only uroflowmetry data and to evaluate its usefulness. Uroflowmetry data of 256 treatment-naive men with detrusor underactivity, bladder outlet obstruction, or detrusor underactivity+bladder outlet obstruction were used for artificial intelligence learning and validation using neural networks. An optimal artificial intelligence diagnostic model was established using 10-fold stratified cross-validation and data augmentation. Correlations of bladder contractility index and bladder outlet obstruction index values for the artificial intelligence system and pressure flow study values were examined using Spearman's correlation coefficients. Additionally, diagnostic accuracy was compared between the established artificial intelligence system and trained urologists with uroflowmetry data of 25 additional patients by χ2 -tests. Detrusor underactivity was defined as bladder contractility index ≤100 and bladder outlet obstruction index ≤40, bladder outlet obstruction was defined as bladder contractility index >100 and bladder outlet obstruction index >40, and detrusor underactivity+bladder outlet obstruction was defined as bladder contractility index ≤100 and bladder outlet obstruction index >40. The artificial intelligence system's estimated bladder contractility index and bladder outlet obstruction index values showed significant positive correlations with pressure flow study values (bladder contractility index: r=0.60, P<0.001; bladder outlet obstruction index: r=0.46, P<0.001). The artificial intelligence system's detrusor underactivity diagnosis had a sensitivity and specificity of 79.7% and 88.7%, respectively, and those for bladder outlet obstruction diagnosis were 76.8% and 84.7%, respectively. The artificial intelligence system's average diagnostic accuracy was 84%, which was significantly higher than that of urologists (56%). Our artificial intelligence diagnostic system developed using the uroflowmetry waveform distinguished between detrusor underactivity and bladder outlet obstruction with high sensitivity and specificity in men with lower urinary tract symptoms.

Development of a deep learning-based image quality control system to detect and filter out ineligible slit-lamp images: A multicenter study

Background and objectivePrevious studies developed artificial intelligence (AI) diagnostic systems only using eligible slit-lamp images for detecting corneal diseases. However, images of ineligible quality (including poor-field, defocused, and poor-location images), which are inevitable in the real world, can cause diagnostic information loss and thus affect downstream AI-based image analysis. Manual evaluation for the eligibility of slit-lamp images often requires an ophthalmologist, and this procedure can be time-consuming and labor-intensive when applied on a large scale. Here, we aimed to develop a deep learning-based image quality control system (DLIQCS) to automatically detect and filter out ineligible slit-lamp images (poor-field, defocused, and poor-location images). MethodsWe developed and externally evaluated the DLIQCS based on 48,530 slit-lamp images (19,890 individuals) that were derived from 4 independent institutions using different types of digital slit lamp cameras. To find the best deep learning model for the DLIQCS, we used 3 algorithms (AlexNet, DenseNet121, and InceptionV3) to train models. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy were leveraged to assess the performance of each algorithm for the classification of poor-field, defocused, poor-location, and eligible images. ResultsIn an internal test dataset, the best algorithm DenseNet121 had AUCs of 0.999, 1.000, 1.000, and 1.000 in the detection of poor-field, defocused, poor-location, and eligible images, respectively. In external test datasets, the AUCs of the best algorithm DenseNet121 for identifying poor-field, defocused, poor-location, and eligible images were ranged from 0.997 to 0.997, 0.983 to 0.995, 0.995 to 0.998, and 0.999 to 0.999, respectively. ConclusionsOur DLIQCS can accurately detect poor-field, defocused, poor-location, and eligible slit-lamp images in an automated fashion. This system may serve as a prescreening tool to filter out ineligible images and enable that only eligible images would be transferred to the subsequent AI diagnostic systems.

Abstract S11-04: Using CXR-Net to detect COVID-19 and non-COVID-19 patients

Abstract A critical step in the fight against COVID-19 pandemic is the screening and rapid recognition of those affected by Severe Acute Respiratory Syndrome CoronaVirus 2 (SARS-CoV-2). The main screening method used for detecting COVID-19 infection is the reverse transcriptase-polymerase chain reaction (RT-PCR). Furthermore, the main screening methods for SARS-CoV-2 are chest X-ray (CXR) and computed tomography (CT). While CTs provide greater diagnostic accuracy, CXRs are more readily available and enable rapid triaging of patients in the most affected areas. Since CXR imaging is typically carried out as part of a standard procedure for patients with a respiratory complaint, it represents an ideal complement to RT-PCR testing. For this reason, much effort is ongoing into the development of Artificial Intelligence diagnostic systems based on Neural Networks (NN) that can aid radiologists in accurately interpreting CXRs in SARS-CoV-2 cases. Here we present CXR-Net, a two-module pipeline for SARS-CoV-2 detection. Module I is based on Res-CR-Net, a type of NN originally developed for the semantic segmentation of microscopy images (doi: 10.1088/2632-2153/aba8e8), which can process images of any size, retaining the original resolution of the input images in the feature maps of all layers and in the final output. Module I was trained on a dataset of 6395 Antero/Posterior CXRs with radiologist annotated lung contours to generate accurate masks of the lungs that overlap the heart and large vasa, and are minimally influenced by regions of consolidation or other texture alterations due to underlying pathologies. Module II is a hybrid convnet in which the first convolutional layer with learned coefficients is replaced by a layer with fixed coefficients provided by the Wavelet Scattering Transform (WST). A particular advantage of this hybrid net is the removal of deep network instabilities associated with adversarial images (noise or small deformations in the input images that are visually insignificant, but that the network does not reduce correctly, leading to incorrect classification). This net converges more rapidly than an end-to-end learned architecture, does not suffer from vanishing or exploding gradients, and prevents overfitting, leading to better generalization. Module II takes as inputs the patients’ CXRs and corresponding lung masks calculated by Module I, and produces as outputs a class assignment (Covid or non-Covid) and high resolution heat maps that identify the SARS associated lung regions. In preliminary work, CXR-Net was tested on a small dataset of CXRs from non-Covid and RT-PCR confirmed Covid patients acquired at HFHS Hospital in Detroit. Module II was trained against 405 CXRs, and validated against 84 CXRs (validation statistics: accuracy = 0.810; precision = 0.833; recall = 0.932; F1 score = 0.88, ROC_auc = 0.942). Citation Format: Benjamen Huber, Abdulah Haikal, Domenico Gatti, Hamid Soltanian-Zadeh. Using CXR-Net to detect COVID-19 and non-COVID-19 patients [abstract]. In: Proceedings of the AACR Virtual Meeting: COVID-19 and Cancer; 2021 Feb 3-5. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(6_Suppl):Abstract nr S11-04.

Artificial intelligence diagnostic system predicts multiple Lugol-voiding lesions in the esophagus and patients at high risk for esophageal squamous cell carcinoma.

It is known that an esophagus with multiple Lugol-voiding lesions (LVLs) after iodine staining is high risk for esophageal cancer; however, it is preferable to identify high-risk cases without staining because iodine causes discomfort and prolongs examination times. This study assessed the capability of an artificial intelligence (AI) system to predict multiple LVLs from images that had not been stained with iodine as well as patients at high risk for esophageal cancer. We constructed the AI system by preparing a training set of 6634 images from white-light and narrow-band imaging in 595 patients before they underwent endoscopic examination with iodine staining. Diagnostic performance was evaluated on an independent validation dataset (667 images from 72 patients) and compared with that of 10 experienced endoscopists. The sensitivity, specificity, and accuracy of the AI system to predict multiple LVLs were 84.4 %, 70.0 %, and 76.4 %, respectively, compared with 46.9 %, 77.5 %, and 63.9 %, respectively, for the endoscopists. The AI system had significantly higher sensitivity than 9/10 experienced endoscopists. We also identified six endoscopic findings that were significantly more frequent in patients with multiple LVLs; however, the AI system had greater sensitivity than these findings for the prediction of multiple LVLs. Moreover, patients with AI-predicted multiple LVLs had significantly more cancers in the esophagus and head and neck than patients without predicted multiple LVLs. The AI system could predict multiple LVLs with high sensitivity from images without iodine staining. The system could enable endoscopists to apply iodine staining more judiciously.

Development of a deep learning-based image eligibility verification system for detecting and filtering out ineligible fundus images: A multicentre study

BackgroundRecent advances in artificial intelligence (AI) have shown great promise in detecting some diseases based on medical images. Most studies developed AI diagnostic systems only using eligible images. However, in real-world settings, ineligible images (including poor-quality and poor-location images) that can compromise downstream analysis are inevitable, leading to uncertainty about the performance of these AI systems. This study aims to develop a deep learning-based image eligibility verification system (DLIEVS) for detecting and filtering out ineligible fundus images. MethodsA total of 18,031 fundus images (9,188 subjects) collected from 4 clinical centres were used to develop and evaluate the DLIEVS for detecting eligible, poor-location, and poor-quality fundus images. Four deep learning algorithms (AlexNet, DenseNet121, Inception V3, and ResNet50) were leveraged to train models to obtain the best model for the DLIEVS. The performance of the DLIEVS was evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, as compared with a reference standard determined by retina experts. ResultsIn the internal test dataset, the best algorithm (DenseNet121) achieved AUCs of 1.000, 0.999, and 1.000 for the classification of eligible, poor-location, and poor-quality images, respectively. In the external test datasets, the AUCs of the best algorithm (DenseNet121) for detecting eligible, poor-location, and poor-quality images were ranged from 0.999–1.000, 0.997–1.000, and 0.997–0.999, respectively. ConclusionsOur DLIEVS can accurately discriminate poor-quality and poor-location images from eligible images. This system has the potential to serve as a pre-screening technique to filter out ineligible images obtained from real-world settings, ensuring only eligible images will be applied in the subsequent image-based AI diagnostic analyses.

Deep learning from \u201cpassive feeding\u201d to \u201cselective eating\u201d of real-world data

Artificial intelligence (AI) based on deep learning has shown excellent diagnostic performance in detecting various diseases with good-quality clinical images. Recently, AI diagnostic systems developed from ultra-widefield fundus (UWF) images have become popular standard-of-care tools in screening for ocular fundus diseases. However, in real-world settings, these systems must base their diagnoses on images with uncontrolled quality (“passive feeding”), leading to uncertainty about their performance. Here, using 40,562 UWF images, we develop a deep learning–based image filtering system (DLIFS) for detecting and filtering out poor-quality images in an automated fashion such that only good-quality images are transferred to the subsequent AI diagnostic system (“selective eating”). In three independent datasets from different clinical institutions, the DLIFS performed well with sensitivities of 96.9%, 95.6% and 96.6%, and specificities of 96.6%, 97.9% and 98.8%, respectively. Furthermore, we show that the application of our DLIFS significantly improves the performance of established AI diagnostic systems in real-world settings. Our work demonstrates that “selective eating” of real-world data is necessary and needs to be considered in the development of image-based AI systems.

Efficacy of endoscopic ultrasound with artificial intelligence for the diagnosis of gastrointestinal stromal tumors.

Although endoscopic ultrasound (EUS) is reported to be suitable for determining the layer from which subepithelial lesions (SELs) originate, it is difficult to distinguish gastrointestinal stromal tumor (GIST) from non-GIST using only EUS images. If artificial intelligence (AI) can be used for the diagnosis of SELs, it should provide several benefits, including objectivity, simplicity, and quickness. In this pilot study, we propose an AI diagnostic system for SELs and evaluate its efficacy. Thirty sets each of EUS images with SELs ≥ 20mm or < 20mm were prepared for diagnosis by an EUS diagnostic system with AI (EUS-AI) and three EUS experts. The EUS-AI and EUS experts diagnosed the SELs using solely the EUS images. The concordance rates of the EUS-AI and EUS experts' diagnoses were compared with the pathological findings of the SELs. The accuracy, sensitivity, and specificity for SELs < 20mm were 86.3, 86.3, and 62.5%, respectively for the EUS-AI, and 73.3, 68.2, and 87.5%, respectively, for the EUS experts. In contrast, accuracy, sensitivity, and specificity for SELs ≥ 20mm were 90.0, 91.7, and 83.3%, respectively, for the EUS-AI, and 53.3, 50.0, and 83.3%, respectively, for the EUS experts. The area under the curve for the diagnostic yield of the EUS-AI for SELs ≥ 20mm (0.965) was significantly higher than that (0.684) of the EUS experts (P = 0.007). EUS-AI had a good diagnostic yield for SELs ≥ 20mm. EUS-AI has potential as a good option for the diagnosis of SELs.

Artificial intelligence based on deep learning for differential diagnosis between benign and malignant pulmonary nodules: A real-world, multicenter, diagnostic study.

9037 Background: Lung cancer is the most common cancer worldwide. Artificial intelligence (AI) platform using deep learning algorithms have made a remarkable progress in improving diagnostic accuracy of lung cancer. But AI diagnostic performance in identifying benign and malignant pulmonary nodules still needs improvement. We aimed to validate a Pulmonary Nodules Artificial Intelligence Diagnostic System (PNAIDS) by analyzing computed tomography (CT) imaging data. Methods: This real-world, multicentre, diagnostic study was done in five different tier hospitals in China. The CT images of patients, who were aged over 18 years and never had previous anti-cancer treatments, were retrieved from participating hospitals. 534 eligible patients with 5-30mm diameter pulmonary nodules identified by CT were planning to confirm with histopathological diagnosis. The performance of PNAIDS was also compared with respiratory specialists and radiologists with expert or competent degrees of expertise as well as Mayo Clinic’s model by area under the curve (AUC) and evaluated differences by calculating the 95% CIs using the Z-test method. 11 selected participants were tested circulating genetically abnormal cells (CACs) before surgery with doctors suggested. Results: 611 lung CT images from 534 individuals were used to test PNAIDS. The diagnostic accuracy, valued by AUC, in identifying benign and malignant pulmonary nodules was 0.765 (95%CI [0.729 - 0.798]). The diagnostic sensitivity of PNAIDS is 0.630(0.579 – 0.679), specificity is 0.753 (0.693 – 0.807). PNAIDS achieved diagnostic accuracy similar to that of the expert respiratory specialists (AUC difference: 0.0036 [-0.0426 - 0.0497]; p = 0.8801) and superior when compared with Mayo Clinic’s model (0.120 [0.0649 - 0.176], p &lt; 0·0001), expert radiologists (0.0620 [0.0124 - 0.112], p = 0.0142) and competent radiologists (0.0751 [0.0248 - 0.125], p = 0.0034). 11 selected participants were suggested negative in AI results but positive in respiratory specialists’ result. 8 of them were malignant in histopathological diagnosis with tested more than 3 CACs in their blood. Conclusions: PNAIDS achieved high diagnostic accuracy in differential diagnoses between benign and malignant pulmonary nodules, with diagnostic accuracy similar to that of expert respiratory specialists and was superior to that of Mayo Clinic’s model and radiologists. CACs may be able to assist CT-based AI in improving their effectiveness but it still need more data to be proved. Clinical trial information: ChiCTR1900026233.

Real-time artificial intelligence for detection of upper gastrointestinal cancer by endoscopy: a multicentre, case-control, diagnostic study.

Upper gastrointestinal cancers (including oesophageal cancer and gastric cancer) are the most common cancers worldwide. Artificial intelligence platforms using deep learning algorithms have made remarkable progress in medical imaging but their application in upper gastrointestinal cancers has been limited. We aimed to develop and validate the Gastrointestinal Artificial Intelligence Diagnostic System (GRAIDS) for the diagnosis of upper gastrointestinal cancers through analysis of imaging data from clinical endoscopies. This multicentre, case-control, diagnostic study was done in six hospitals of different tiers (ie, municipal, provincial, and national) in China. The images of consecutive participants, aged 18 years or older, who had not had a previous endoscopy were retrieved from all participating hospitals. All patients with upper gastrointestinal cancer lesions (including oesophageal cancer and gastric cancer) that were histologically proven malignancies were eligible for this study. Only images with standard white light were deemed eligible. The images from Sun Yat-sen University Cancer Center were randomly assigned (8:1:1) to the training and intrinsic verification datasets for developing GRAIDS, and the internal validation dataset for evaluating the performance of GRAIDS. Its diagnostic performance was evaluated using an internal and prospective validation set from Sun Yat-sen University Cancer Center (a national hospital) and additional external validation sets from five primary care hospitals. The performance of GRAIDS was also compared with endoscopists with three degrees of expertise: expert, competent, and trainee. The diagnostic accuracy, sensitivity, specificity, positive predictive value, and negative predictive value of GRAIDS and endoscopists for the identification of cancerous lesions were evaluated by calculating the 95% CIs using the Clopper-Pearson method. 1 036 496 endoscopy images from 84 424 individuals were used to develop and test GRAIDS. The diagnostic accuracy in identifying upper gastrointestinal cancers was 0·955 (95% CI 0·952-0·957) in the internal validation set, 0·927 (0·925-0·929) in the prospective set, and ranged from 0·915 (0·913-0·917) to 0·977 (0·977-0·978) in the five external validation sets. GRAIDS achieved diagnostic sensitivity similar to that of the expert endoscopist (0·942 [95% CI 0·924-0·957] vs 0·945 [0·927-0·959]; p=0·692) and superior sensitivity compared with competent (0·858 [0·832-0·880], p<0·0001) and trainee (0·722 [0·691-0·752], p<0·0001) endoscopists. The positive predictive value was 0·814 (95% CI 0·788-0·838) for GRAIDS, 0·932 (0·913-0·948) for the expert endoscopist, 0·974 (0·960-0·984) for the competent endoscopist, and 0·824 (0·795-0·850) for the trainee endoscopist. The negative predictive value was 0·978 (95% CI 0·971-0·984) for GRAIDS, 0·980 (0·974-0·985) for the expert endoscopist, 0·951 (0·942-0·959) for the competent endoscopist, and 0·904 (0·893-0·916) for the trainee endoscopist. GRAIDS achieved high diagnostic accuracy in detecting upper gastrointestinal cancers, with sensitivity similar to that of expert endoscopists and was superior to that of non-expert endoscopists. This system could assist community-based hospitals in improving their effectiveness in upper gastrointestinal cancer diagnoses. The National Key R&D Program of China, the Natural Science Foundation of Guangdong Province, the Science and Technology Program of Guangdong, the Science and Technology Program of Guangzhou, and the Fundamental Research Funds for the Central Universities.

Using Real-Time Artificial Intelligence to Detect Upper Gastrointestinal Cancer by Endoscopy

Background: Upper gastrointestinal (UGI) cancers (including esophageal cancer, gastric cancer, etc.) are the most common cancers worldwide. Detection at early stage is the key for increasing the overall survival. Artificial intelligence (AI) using deep learning algorithms have made remarkable progress in medical imaging. Here we constructed a Gastrointestinal Artificial Intelligence Diagnostic System (GRAIDS) to improve the diagnostic accuracy of upper gastrointestinal (UGI) cancer by analyzing imaging data from clinical endoscopies. Methods: A multi-institutional, diagnostic study from six hospitals of different tiers across China was initiated. We included a total of 1,036,496 endoscopy images from 84,424 individuals to develop and validate the GRAIDS. We evaluated the performance of the GRAIDS using an internal validation set and a prospective validation set from a national hospital and five external validation sets from primary care hospitals. We also compared the diagnostic performance of the GRAIDS with that of endoscopists with three degrees of expertise (expert, competent, and trainee). Findings: The diagnostic accuracy in identifying UGI cancer was 0.955 in the internal validation set and 0.927 in the prospective set, and ranged from 0.915 to 0.977 in the five external validation sets. Compared with the expert endoscopists, the GRAIDS achieved a comparable sensitivity to experts (0.942 versus 0.945; P=0.692) and a superior sensitivity to competent (0.858, P<0.001) and trainee endoscopists (0.722, P<0.001). Interpretation: The GRAIDS achieved high diagnostic accuracy in detecting UGI cancer, with a sensitivity comparable to experts and was superior to nonexpert endoscopists. This system could support nonexpert endoscopists from primary basic hospitals by improving their diagnostic accuracy to the level of the experts, thus providing an opportunity for community-based hospitals to improve their effectiveness in UGI cancer diagnosis. Funding Statement: This research was supported by the National Key R&D Program of China (2018YFC1313300), the Natural Science Foundation of Guangdong Province (2017A030313485, 2014A030312015), the Science and Technology Program of Guangdong (2015B020232008), the Science and Technology Program of Guangzhou (201508020250, 201604020003, 2019B020227002). Declaration of Interests: No competing interests are declared. Ethics Approval Statement: This study was approved by all the relevant independent institutional review boards (IRB) and was performed according to the Helsinki declaration.

Multi-modal Imaging and Artificial Intelligence Diagnostic System for Multi-level Clinical Application

Multi-modal Imaging and Artificial Intelligence Diagnostic System for Multi-level Clinical Application

Better diagnostic performance using computer-assisted diagnostic support systems in internal medicine

The recent application of artificial intelligence（AI）to clinical medicine has confirmed the usefulness of AI for diagnostic imaging, histopathological examinations, and dermatologic screening. Clinical decision support systems are another promising area to which AI could contribute toward better clinical decisions. We have developed computer-assisted diagnostic support systems to reduce human diagnostic errors such as delayed diagnoses, misdiagnoses, and overdiagnoses. Our three Diagnosis Reminder（DR）systems include two AI systems that use machine learning in their diagnosis algorithms. Here, we compared the diagnostic accuracy of a DR-supported group with that of an unassisted physicians group, using three difficult patient cases provided by experts in general medicine. Our analyses revealed that the three AI diagnostic systems could not provide accurate differential diagnoses up to top 10 in all three patient cases because of incomplete data inputs for machine learning. However, the first DR system, which was developed by an experienced diagnostician over the last 35 years, showed very useful performance in reducing human diagnostic errors when it was used by an expert physician. The use of AI diagnostic systems by knowledgeable physicians will lead to better diagnostic performance. We also discuss the current scenario, future challenges, and prospects for AI diagnostic systems herein.

A Novel Fault Diagnosis System on Polymer Insulation of Power Transformers Based on 3-stage GA⁻SA⁻SVM OFC Selection and ABC⁻SVM Classifier.

Dissolved gas analysis (DGA) has been widely used in various scenarios of power transformers’ online monitoring and diagnoses. However, the diagnostic accuracy of traditional DGA methods still leaves much room for improvement. In this context, numerous new DGA diagnostic models that combine artificial intelligence with traditional methods have emerged. In this paper, a new DGA artificial intelligent diagnostic system is proposed. There are two modules that make up the diagnosis system. The two modules are the optimal feature combination (OFC) selection module based on 3-stage GA–SA–SVM and the ABC–SVM fault diagnosis module. The diagnosis system has been completely realized and embodied in its outstanding performances in diagnostic accuracy, reliability, and efficiency. Comparing the result with other artificial intelligence diagnostic methods, the new diagnostic system proposed in this paper performed superiorly.

Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices

Artificial Intelligence (AI) has long promised to increase healthcare affordability, quality and accessibility but FDA, until recently, had never authorized an autonomous AI diagnostic system. This pivotal trial of an AI system to detect diabetic retinopathy (DR) in people with diabetes enrolled 900 subjects, with no history of DR at primary care clinics, by comparing to Wisconsin Fundus Photograph Reading Center (FPRC) widefield stereoscopic photography and macular Optical Coherence Tomography (OCT), by FPRC certified photographers, and FPRC grading of Early Treatment Diabetic Retinopathy Study Severity Scale (ETDRS) and Diabetic Macular Edema (DME). More than mild DR (mtmDR) was defined as ETDRS level 35 or higher, and/or DME, in at least one eye. AI system operators underwent a standardized training protocol before study start. Median age was 59 years (range, 22–84 years); among participants, 47.5% of participants were male; 16.1% were Hispanic, 83.3% not Hispanic; 28.6% African American and 63.4% were not; 198 (23.8%) had mtmDR. The AI system exceeded all pre-specified superiority endpoints at sensitivity of 87.2% (95% CI, 81.8–91.2%) (>85%), specificity of 90.7% (95% CI, 88.3–92.7%) (>82.5%), and imageability rate of 96.1% (95% CI, 94.6–97.3%), demonstrating AI’s ability to bring specialty-level diagnostics to primary care settings. Based on these results, FDA authorized the system for use by health care providers to detect more than mild DR and diabetic macular edema, making it, the first FDA authorized autonomous AI diagnostic system in any field of medicine, with the potential to help prevent vision loss in thousands of people with diabetes annually. ClinicalTrials.gov NCT02963441