Expert-level Performance Research Articles

A novel endoscopic artificial intelligence system to assist in the diagnosis of autoimmune gastritis: a multicenter study.

Autoimmune gastritis (AIG), distinct from Helicobacter pylori-associated atrophic gastritis (HpAG), is underdiagnosed due to limited awareness. This multicenter study aims to develop a novel endoscopic artificial intelligence (AI) system assisting in AIG diagnosis. Patients diagnosed with AIG, as well as HpAG and non-atrophic gastritis (NAG), were retrospectively enrolled from six centers. Endoscopic images with relevant demographic and medical data, were collected for the development of AI-assisted system, SEER-SCOPE AI, based on multi-site feature fusion model. The diagnostic performance of SEER-SCOPE AI was evaluated in the internal and external datasets. Endoscopists' performance with and without AI support was tested and compared using Mann-Whitney U test. Heatmap analysis was performed to interpret SEER-SCOPE AI. 1 070 patients (294 AIG, 386 HpAG, 390 NAG) with 18 828 endoscopy images were collected. SEER-SCOPE AI achieved strong performance for identifying AIG, with 96.9% sensitivity, 92.2% specificity and an AUROC of 0.990 internally, and 90.3% sensitivity, 93.1% specificity and an AUROC of 0.973 externally. The performance of SEER-SCOPE AI (sensitivity 91.3%) was comparable to experts (87.3%) and significantly outperformed non-experts (70.0%). With AI support, the overall performance of endoscopists was improved (sensitivity: 90.3% [95% CI 86.0%-93.2%] vs. 78.7% [95% CI 73.6%-83.2%], p=0.008). Heatmap analysis revealed consistent focus of SEER-SCOPE AI on regions corresponding to atrophic areas. SEER-SCOPE AI demonstrated expert-level performance in identifying AIG, and enhanced the diagnostic ability of endoscopists. Its application holds promise as a potent endoscopy-assisted tool for guiding biopsy sampling and early detection of AIG.

Ultrasomics differentiation of malignant and benign focal liver lesions based on contrast-enhanced ultrasound

ObjectivesTo establish a nomogram for differentiating malignant and benign focal liver lesions (FLLs) using ultrasomics features derived from contrast-enhanced ultrasound (CEUS).Methods527 patients were retrospectively enrolled. On the training cohort, ultrasomics features were extracted from CEUS and b-mode ultrasound (BUS). Automatic feature selection and model development were performed using the Ultrasomics-Platform software, outputting the corresponding ultrasomics scores. A nomogram based on the ultrasomics scores from artery phase (AP), portal venous phase (PVP) and delayed phase (DP) of CEUS, and clinical factors were established. On the validation cohort, the diagnostic performance of the nomogram was assessed and compared with seniorexpert and resident radiologists.ResultsIn the training cohort, the AP, PVP and DP scores exhibited better differential performance than BUS score, with area under the curve (AUC) of 84.1-85.1% compared with the BUS (74.6%, P < 0.05). In the validation cohort, the AUC of combined nomogram and expert was significantly higher than that of the resident (91.4% vs. 89.5% vs. 79.3%, P < 0.05). The combined nomogram had a comparable sensitivity with the expert and resident (95.2% vs. 98.4% vs. 97.6%), while the expert had a higher specificity than the nomogram and the resident (80.6% vs. 72.2% vs. 61.1%, P = 0.205).ConclusionsA CEUS ultrasomics based nomogram had an expert level performance in FLL characterization.

Joint path planning and power allocation of a cellular-connected UAV using apprenticeship learning via deep inverse reinforcement learning

This paper investigates an interference-aware joint path planning and power allocation mechanism for a cellular-connected unmanned aerial vehicle (UAV) in a sparse suburban environment. The UAV’s goal is to fly from an initial point and reach a destination point by moving along the cells to guarantee the required quality of service (QoS). In particular, the UAV aims to maximize its uplink throughput and minimize interference to the ground user equipment (UEs) connected to neighboring cellular base stations (BSs), considering both the shortest path and limitations on flight resources. Expert knowledge is used to experience the scenario and define the desired behavior for the sake of the agent (i.e., UAV) training. To solve the problem, an apprenticeship learning method is utilized via inverse reinforcement learning (IRL) based on both Q-learning and deep reinforcement learning (DRL). The performance of this method is compared to learning from a demonstration technique called behavioral cloning (BC) using a supervised learning approach. Simulation and numerical results show that the proposed approach can achieve expert-level performance. We also demonstrate that, unlike the BC technique, the performance of our proposed approach does not degrade in unseen situations.

Automated echocardiography view classification and quality assessment with recognition of unknown views.

Interpreting echocardiographic exams requires substantial manual interaction as videos lack scan-plane information and have inconsistent image quality, ranging from clinically relevant to unrecognizable. Thus, a manual prerequisite step for analysis is to select the appropriate views that showcase both the target anatomy and optimal image quality. To automate this selection process, we present a method for automatic classification of routine views, recognition of unknown views, and quality assessment of detected views. We train a neural network for view classification and employ the logit activations from the neural network for unknown view recognition. Subsequently, we train a linear regression algorithm that uses feature embeddings from the neural network to predict view quality scores. We evaluate the method on a clinical test set of 2466 echocardiography videos with expert-annotated view labels and a subset of 438 videos with expert-rated view quality scores. A second observer annotated a subset of 894 videos, including all quality-rated videos. The proposed method achieved an accuracy of for the joint objective of routine view classification and unknown view recognition, whereas a second observer reached an accuracy of 87.6%. For view quality assessment, the method achieved a Spearman's rank correlation coefficient of 0.71, whereas a second observer reached a correlation coefficient of 0.62. The proposed method approaches expert-level performance, enabling fully automatic selection of the most appropriate views for manual or automatic downstream analysis.

Leveraging Large Language Models for Precision Monitoring of Chemotherapy-Induced Toxicities: A Pilot Study with Expert Comparisons and Future Directions.

Large Language Models (LLMs), such as the GPT model family from OpenAI, have demonstrated transformative potential across various fields, especially in medicine. These models can understand and generate contextual text, adapting to new tasks without specific training. This versatility can revolutionize clinical practices by enhancing documentation, patient interaction, and decision-making processes. In oncology, LLMs offer the potential to significantly improve patient care through the continuous monitoring of chemotherapy-induced toxicities, which is a task that is often unmanageable for human resources alone. However, existing research has not sufficiently explored the accuracy of LLMs in identifying and assessing subjective toxicities based on patient descriptions. This study aims to fill this gap by evaluating the ability of LLMs to accurately classify these toxicities, facilitating personalized and continuous patient care. This comparative pilot study assessed the ability of an LLM to classify subjective toxicities from chemotherapy. Thirteen oncologists evaluated 30 fictitious cases created using expert knowledge and OpenAI's GPT-4. These evaluations, based on the CTCAE v.5 criteria, were compared to those of a contextualized LLM model. Metrics such as mode and mean of responses were used to gauge consensus. The accuracy of the LLM was analyzed in both general and specific toxicity categories, considering types of errors and false alarms. The study's results are intended to justify further research involving real patients. The study revealed significant variability in oncologists' evaluations due to the lack of interaction with fictitious patients. The LLM model achieved an accuracy of 85.7% in general categories and 64.6% in specific categories using mean evaluations with mild errors at 96.4% and severe errors at 3.6%. False alarms occurred in 3% of cases. When comparing the LLM's performance to that of expert oncologists, individual accuracy ranged from 66.7% to 89.2% for general categories and 57.0% to 76.0% for specific categories. The 95% confidence intervals for the median accuracy of oncologists were 81.9% to 86.9% for general categories and 67.6% to 75.6% for specific categories. These benchmarks highlight the LLM's potential to achieve expert-level performance in classifying chemotherapy-induced toxicities. The findings demonstrate that LLMs can classify subjective toxicities from chemotherapy with accuracy comparable to expert oncologists. The LLM achieved 85.7% accuracy in general categories and 64.6% in specific categories. While the model's general category performance falls within expert ranges, specific category accuracy requires improvement. The study's limitations include the use of fictitious cases, lack of patient interaction, and reliance on audio transcriptions. Nevertheless, LLMs show significant potential for enhancing patient monitoring and reducing oncologists' workload. Future research should focus on the specific training of LLMs for medical tasks, conducting studies with real patients, implementing interactive evaluations, expanding sample sizes, and ensuring robustness and generalization in diverse clinical settings. This study concludes that LLMs can classify subjective toxicities from chemotherapy with accuracy comparable to expert oncologists. The LLM's performance in general toxicity categories is within the expert range, but there is room for improvement in specific categories. LLMs have the potential to enhance patient monitoring, enable early interventions, and reduce severe complications, improving care quality and efficiency. Future research should involve specific training of LLMs, validation with real patients, and the incorporation of interactive capabilities for real-time patient interactions. Ethical considerations, including data accuracy, transparency, and privacy, are crucial for the safe integration of LLMs into clinical practice.

Identifying retinopathy in optical coherence tomography images with less labeled data via contrastive graph regularization.

Retinopathy detection using optical coherence tomography (OCT) images has greatly advanced with computer vision but traditionally requires extensive annotated data, which is time-consuming and expensive. To address this issue, we propose a novel contrastive graph regularization method for detecting retinopathies with less labeled OCT images. This method combines class prediction probabilities and embedded image representations for training, where the two representations interact and co-evolve within the same training framework. Specifically, we leverage memory smoothing constraints to improve pseudo-labels, which are aggregated by nearby samples in the embedding space, effectively reducing overfitting to incorrect pseudo-labels. Our method, using only 80 labeled OCT images, outperforms existing methods on two widely used OCT datasets, with classification accuracy exceeding 0.96 and an Area Under the Curve (AUC) value of 0.998. Additionally, compared to human experts, our method achieves expert-level performance with only 80 labeled images and surpasses most experts with just 160 labeled images.

Endoscopic Artificial Intelligence for Image Analysis in Gastrointestinal Neoplasms.

Artificial intelligence (AI) using deep learning systems has recently been utilized in various medical fields. In the field of gastroenterology, AI is primarily implemented in image recognition and utilized in the realm of gastrointestinal (GI) endoscopy. In GI endoscopy, computer-aided detection/diagnosis (CAD) systems assist endoscopists in GI neoplasm detection or differentiation of cancerous or noncancerous lesions. Several AI systems for colorectal polyps have already been applied in colonoscopy clinical practices. In esophagogastroduodenoscopy, a few CAD systems for upper GI neoplasms have been launched in Asian countries. The usefulness of these CAD systems in GI endoscopy has been gradually elucidated. In this review, we outline recent articles on several studies of endoscopic AI systems for GI neoplasms, focusing on esophageal squamous cell carcinoma (ESCC), esophageal adenocarcinoma (EAC), gastric cancer (GC), and colorectal polyps. In ESCC and EAC, computer-aided detection (CADe) systems were mainly developed, and a recent meta-analysis study showed sensitivities of 91.2% and 93.1% and specificities of 80% and 86.9%, respectively. In GC, a recent meta-analysis study on CADe systems demonstrated that their sensitivity and specificity were as high as 90%. A randomized controlled trial (RCT) also showed that the use of the CADe system reduced the miss rate. Regarding computer-aided diagnosis (CADx) systems for GC, although RCTs have not yet been conducted, most studies have demonstrated expert-level performance. In colorectal polyps, multiple RCTs have shown the usefulness of the CADe system for improving the polyp detection rate, and several CADx systems have been shown to have high accuracy in colorectal polyp differentiation. Most analyses of endoscopic AI systems suggested that their performance was better than that of nonexpert endoscopists and equivalent to that of expert endoscopists. Thus, endoscopic AI systems may be useful for reducing the risk of overlooking lesions and improving the diagnostic ability of endoscopists.

Deep learning-based fully automatic Risser stage assessment model using abdominal radiographs.

Artificial intelligence has been increasingly used in medical imaging and has demonstrated expert level performance in image classification tasks. To develop a fully automatic approach for determining the Risser stage using deep learning on abdominal radiographs. In this multicenter study, 1,681 supine abdominal radiographs (age range, 9-18years, 50% female) obtained between January 2019 and April 2022 were collected retrospectively from three medical institutions and graded manually using the United States Risser staging system. A total of 1,577 images from Hospitals 1 and 2 were used for development, and 104 images from Hospital 3 for external validation. From each radiograph, right and left iliac crest patch images were extracted using the pelvic bone segmentation model DeepLabv3 + with the EfficientNet-B0 encoder trained with 90 digitally reconstructed radiographs from pelvic computed tomography scans with a pelvic bone mask. Using these patch images, ConvNeXt-B was trained to grade according to the Risser classification. The model's performance was evaluated using accuracy, area under the receiver operating characteristic curve (AUROC), and mean absolute error. The fully automatic Risser stage assessment model showed an accuracy of 0.87 and 0.75, mean absolute error of 0.13 and 0.26, and AUROC of 0.99 and 0.95 on internal and external test sets, respectively. We developed a deep learning-based, fully automatic segmentation and classification model for Risser stage assessment using abdominal radiographs.

Classification of LAMOST spectra of B-type and hot subdwarf stars using kernel support vector machine

Machine learning has emerged as a leading field in artificial intelligence, demonstrating expert-level performance in various domains. Astronomy has benefited from machine learning techniques, particularly in classifying and identifying stars based on their features. This study focuses on the spectra-based classification of 11,408 B-type and 2422 hot subdwarf stars. The study employs baseline correction using Asymmetric Least Squares (ALS) to enhance classification accuracy. It applies the Pan-Core concept to identify 500 unique patterns or ranges for both types of stars. These patterns are the foundation for creating Support Vector Machine (SVM) models, including the linear (L-SVM), polynomial (P-SVM), and radial basis (R-SVM) kernels. Parameter tuning for the SVM models is achieved through cross-validation. Evaluation of the SVM models on test data reveals that the linear kernel SVM achieves the highest accuracy (87.0%), surpassing the polynomial kernel SVM (84.1%) and radial kernel SVM (80.1%). The average calibrated accuracy falls within the range of 90–95%. These results demonstrate the potential of using spectrum-based classification to aid astronomers in improving and expanding their understanding of stars, with a specific focus on the identification of hot subdwarf stars. This study presents a valuable investigation for astronomers, as it enables the classification of stars based on their spectra, leveraging machine learning techniques to enhance their knowledge and insights in astronomy.

Artificial Intelligence in Spinal Imaging and Patient Care: A Review of Recent Advances.

Artificial intelligence (AI) is transforming spinal imaging and patient care through automated analysis and enhanced decision-making. This review presents a clinical task-based evaluation, highlighting the specific impact of AI techniques on different aspects of spinal imaging and patient care. We first discuss how AI can potentially improve image quality through techniques like denoising or artifact reduction. We then explore how AI enables efficient quantification of anatomical measurements, spinal curvature parameters, vertebral segmentation, and disc grading. This facilitates objective, accurate interpretation and diagnosis. AI models now reliably detect key spinal pathologies, achieving expert-level performance in tasks like identifying fractures, stenosis, infections, and tumors. Beyond diagnosis, AI also assists surgical planning via synthetic computed tomography generation, augmented reality systems, and robotic guidance. Furthermore, AI image analysis combined with clinical data enables personalized predictions to guide treatment decisions, such as forecasting spine surgery outcomes. However, challenges still need to be addressed in implementing AI clinically, including model interpretability, generalizability, and data limitations. Multicenter collaboration using large, diverse datasets is critical to advance the field further. While adoption barriers persist, AI presents a transformative opportunity to revolutionize spinal imaging workflows, empowering clinicians to translate data into actionable insights for improved patient care.

Exploring the psychology of LLMs’ moral and legal reasoning

Large language models (LLMs) exhibit expert-level performance in tasks across a wide range of different domains. Ethical issues raised by LLMs and the need to align future versions makes it important to know how state of the art models reason about moral and legal issues. In this paper, we employ the methods of experimental psychology to probe into this question. We replicate eight studies from the experimental literature with instances of Google's Gemini Pro, Anthropic's Claude 2.1, OpenAI's GPT-4, and Meta's Llama 2 Chat 70b. We find that alignment with human responses shifts from one experiment to another, and that models differ amongst themselves as to their overall alignment, with GPT-4 taking a clear lead over all other models we tested. Nonetheless, even when LLM-generated responses are highly correlated to human responses, there are still systematic differences, with a tendency for models to exaggerate effects that are present among humans, in part by reducing variance. This recommends caution with regards to proposals of replacing human participants with current state-of-the-art LLMs in psychological research and highlights the need for further research about the distinctive aspects of machine psychology.

MFNet: Meta-learning based on frequency-space mix for MRI segmentation in nasopharyngeal carcinoma.

Deep learning techniques have been applied to medical image segmentation and demonstrated expert-level performance. Due to the poor generalization abilities of the models in the deployment in different centres, common solutions, such as transfer learning and domain adaptation techniques, have been proposed to mitigate this issue. However, these solutions necessitate retraining the models with target domain data and annotations, which limits their deployment in clinical settings in unseen domains. We evaluated the performance of domain generalization methods on the task of MRI segmentation of nasopharyngeal carcinoma (NPC) by collecting a new dataset of 321 patients with manually annotated MRIs from two hospitals. We transformed the modalities of MRI, including T1WI, T2WI and CE-T1WI, from the spatial domain to the frequency domain using Fourier transform. To address the bottleneck of domain generalization in MRI segmentation of NPC, we propose a meta-learning approach based on frequency domain feature mixing. We evaluated the performance of MFNet against existing techniques for generalizing NPC segmentation in terms of Dice and MIoU. Our method evidently outperforms the baseline in handling the generalization of NPC segmentation. The MF-Net clearly demonstrates its effectiveness for generalizing NPC MRI segmentation to unseen domains (Dice = 67.59%, MIoU = 75.74% T1W1). MFNet enhances the model's generalization capabilities by incorporating mixed-feature meta-learning. Our approach offers a novel perspective to tackle the domain generalization problem in the field of medical imaging by effectively exploiting the unique characteristics of medical images.

USN: A Robust Imitation Learning Method against Diverse Action Noise

Learning from imperfect demonstrations is a crucial challenge in imitation learning (IL). Unlike existing works that still rely on the enormous effort of expert demonstrators, we consider a more cost-effective option for obtaining a large number of demonstrations. That is, hire annotators to label actions for existing image records in realistic scenarios. However, action noise can occur when annotators are not domain experts or encounter confusing states. In this work, we introduce two particular forms of action noise, i.e., state-independent and state-dependent action noise. Previous IL methods fail to achieve expert-level performance when the demonstrations contain action noise, especially the state-dependent action noise. To mitigate the harmful effects of action noises, we propose a robust learning paradigm called USN (Uncertainty-aware Sample-selection with Negative learning). The model first estimates the predictive uncertainty for all demonstration data and then selects sampleswith high loss based on the uncertainty measures. Finally, it updates the model parameters with additional negative learning on the selected samples. Empirical results in Box2D tasks and Atari games show that USN consistently improves the final rewards of behavioral cloning, online imitation learning, and offline imitation learning methods under various action noises. The ratio of significant improvements is up to 94.44%. Moreover, our method scales to conditional imitation learning with real-world noisy commands in urban driving

RAPHIA: A deep learning pipeline for the registration of MRI and whole-mount histopathology images of the prostate

Image registration can map the ground truth extent of prostate cancer from histopathology images onto MRI, facilitating the development of machine learning methods for early prostate cancer detection. Here, we present RAdiology PatHology Image Alignment (RAPHIA), an end-to-end pipeline for efficient and accurate registration of MRI and histopathology images. RAPHIA automates several time-consuming manual steps in existing approaches including prostate segmentation, estimation of the rotation angle and horizontal flipping in histopathology images, and estimation of MRI-histopathology slice correspondences. By utilizing deep learning registration networks, RAPHIA substantially reduces computational time. Furthermore, RAPHIA obviates the need for a multimodal image similarity metric by transferring histopathology image representations to MRI image representations and vice versa. With the assistance of RAPHIA, novice users achieved expert-level performance, and their mean error in estimating histopathology rotation angle was reduced by 51% (12 degrees vs 8 degrees), their mean accuracy of estimating histopathology flipping was increased by 5% (95.3% vs 100%), and their mean error in estimating MRI-histopathology slice correspondences was reduced by 45% (1.12 slices vs 0.62 slices). When compared to a recent conventional registration approach and a deep learning registration approach, RAPHIA achieved better mapping of histopathology cancer labels, with an improved mean Dice coefficient of cancer regions outlined on MRI and the deformed histopathology (0.44 vs 0.48 vs 0.50), and a reduced mean per-case processing time (51 vs 11 vs 4.5 min). The improved performance by RAPHIA allows efficient processing of large datasets for the development of machine learning models for prostate cancer detection on MRI. Our code is publicly available at: https://github.com/pimed/RAPHIA.

Towards trustworthy seizure onset detection using workflow notes

A major barrier to deploying healthcare AI is trustworthiness. One form of trustworthiness is a model’s robustness across subgroups: while models may exhibit expert-level performance on aggregate metrics, they often rely on non-causal features, leading to errors in hidden subgroups. To take a step closer towards trustworthy seizure onset detection from EEG, we propose to leverage annotations that are produced by healthcare personnel in routine clinical workflows—which we refer to as workflow notes—that include multiple event descriptions beyond seizures. Using workflow notes, we first show that by scaling training data to 68,920 EEG hours, seizure onset detection performance significantly improves by 12.3 AUROC (Area Under the Receiver Operating Characteristic) points compared to relying on smaller training sets with gold-standard labels. Second, we reveal that our binary seizure onset detection model underperforms on clinically relevant subgroups (e.g., up to a margin of 6.5 AUROC points between pediatrics and adults), while having significantly higher FPRs (False Positive Rates) on EEG clips showing non-epileptiform abnormalities (+19 FPR points). To improve model robustness to hidden subgroups, we train a multilabel model that classifies 26 attributes other than seizures (e.g., spikes and movement artifacts) and significantly improve overall performance (+5.9 AUROC points) while greatly improving performance among subgroups (up to +8.3 AUROC points) and decreasing false positives on non-epileptiform abnormalities (by 8 FPR points). Finally, we find that our multilabel model improves clinical utility (false positives per 24 EEG hours) by a factor of 2×.

Development and validation of a deep-learning model for the detection of non-displaced femoral neck fractures with anteroposterior and lateral hip radiographs.

Hip fractures, including femoral neck fractures, are a significant cause of morbidity and mortality in the elderly population and are typically diagnosed using plain radiography. However, diagnosing non-displaced femoral neck fractures can be challenging due to their subtle appearance on hip radiographs. Previous deep-learning models have shown low accuracy in identifying these fractures on anteroposterior (AP) radiographs; however, no studies have used lateral radiographs. This study aimed to evaluate the potential of using deep-learning with both AP and lateral hip radiographs to automatically identify non-displaced femoral neck fractures. We conducted a retrospective analysis of patients with femoral neck fractures at The First Affiliated Hospital of Xiamen University. All the hip radiographs were reviewed, and cases of non-displaced femoral neck fractures were included in the study. Additionally, 439 participants with normal hip radiographs were also included in the study. A vision transformer (Vit) model was developed using 1,536 AP and lateral hip radiograph. The model's performance was compared to the performance of two groups of human observers: an expert group comprising orthopedic surgeons and radiologists, and a non-expert group, including emergency physicians and general practice doctors. We also carried out the external validation using two additional data sets to assess the generalizability of the model. The Vit model showed exceptional performance in detecting non-displaced femoral neck fractures on paired AP and lateral hip radiographs, achieving a binary accuracy of 95.8% [95% confidence interval (CI): 94.9%, 96.8%] and an area under the curve (AUC) of 0.988. Compared to the human observers, the model had a higher accuracy of 96.7% (95% CI: 93.9%, 99.5%) on the paired AP and lateral hip radiographs, while the accuracy of the expert group was 90.5% (95% CI: 85.7%, 95.2%). Further, the model maintained good performance during the external validation, with an AUC of 0.959 on the paired AP and lateral views. Our Vit model showed expert-level performance in identifying non-displaced femoral neck fractures on paired AP and lateral hip radiographs. This model has the potential to enhance diagnosis accuracy and improve patient outcomes by reducing the need for additional examinations and preoperative time.

Measuring the binary thickness of buccal bone of anterior maxilla in low-resolution cone-beam computed tomography via a bilinear convolutional neural network.

The thickness of the buccal bone of the anterior maxilla is an important aesthetic-determining factor for dental implant, which is divided into the thick (≥1 mm) and thin type (<1 mm). However, as a micro-scale structure that is evaluated through low-resolution cone-beam computed tomography (CBCT), its thickness measurement is error-prone under the circumstance of enormous patients and relatively inexperienced primary dentists. Further, the challenges of deep learning-based analysis of the binary thickness of buccal bone include the substantial real-world variance caused by pixel error, the extraction of fine-grained features, and burdensome annotations. This study built bilinear convolutional neural network (BCNN) with 2 convolutional neural network (CNN) backbones and a bilinear pooling module to predict the binary thickness of buccal bone (thick or thin) of the anterior maxilla in an end-to-end manner. The methods of 5-fold cross-validation and model ensemble were adopted at the training and testing stages. The visualization methods of Gradient Weighted Class Activation Mapping (Grad-CAM), Guided Grad-CAM, and layer-wise relevance propagation (LRP) were used for revealing the important features on which the model focused. The performance metrics and efficacy were compared between BCNN, dentists of different clinical experience (i.e., dental student, junior dentist, and senior dentist), and the fusion of BCNN and dentists to investigate the clinical feasibility of BCNN. Based on the dataset of 4,000 CBCT images from 1,000 patients (aged 36.15±13.09 years), the BCNN with visual geometry group (VGG)16 backbone achieved an accuracy of 0.870 [95% confidence interval (CI): 0.838-0.902] and an area under the receiver operating characteristic (ROC) curve (AUC) of 0.924 (95% CI: 0.896-0.948). Compared with the conventional CNNs, BCNN precisely located the buccal bone wall over irrelevant regions. The BCNN generally outperformed the expert-level dentists. The clinical diagnostic performance of the dentists was improved with the assistance of BCNN. The application of BCNN to the quantitative analysis of binary buccal bone thickness validated the model's excellent ability of subtle feature extraction and achieved expert-level performance. This work signals the potential of fine-grained image recognition networks to the precise quantitative analysis of micro-scale structures.

Development of LuxIA, a Cloud-Based AI Diabetic Retinopathy Screening Tool Using a Single Color Fundus Image.

Diabetic retinopathy (DR) is the leading cause of vision impairment in working-age adults. Automated screening can increase DR detection at early stages at relatively low costs. We developed and evaluated a cloud-based screening tool that uses artificial intelligence (AI), the LuxIA algorithm, to detect DR from a single fundus image. Color fundus images that were previously graded by expert readers were collected from the Canarian Health Service (Retisalud) and used to train LuxIA, a deep-learning-based algorithm for the detection of more than mild DR. The algorithm was deployed in the Discovery cloud platform to evaluate each test set. Sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve were computed using a bootstrapping method to evaluate the algorithm performance and compared through different publicly available datasets. A usability test was performed to assess the integration into a clinical tool. Three separate datasets, Messidor-2, APTOS, and a holdout set from Retisalud were evaluated. Mean sensitivity and specificity with 95% confidence intervals (CIs) reached for these three datasets were 0.901 (0.901-0.902) and 0.955 (0.955-0.956), 0.995 (0.995-0.995) and 0.821 (0.821-0.823), and 0.911 (0.907-0.912) and 0.880 (0.879-0.880), respectively. The usability test confirmed the successful integration of LuxIA into Discovery. Clinical data were used to train the deep-learning-based algorithm LuxIA to an expert-level performance. The whole process (image uploading and analysis) was integrated into the cloud-based platform Discovery, allowing more patients to have access to expert-level screening tools. Using the cloud-based LuxIA tool as part of a screening program may give diabetic patients greater access to specialist-level decisions, without the need for consultation.

EP02.46: An AI system (SonoLyst) achieves expert level performance when categorising images for adherence to ISUOG mid‐trimester screening guidelines

EP02.46: An AI system (SonoLyst) achieves expert level performance when categorising images for adherence to ISUOG mid‐trimester screening guidelines

Performance of ChatGPT on the Peruvian National Licensing Medical Examination: Cross-Sectional Study.

ChatGPT has shown impressive performance in national medical licensing examinations, such as the United States Medical Licensing Examination (USMLE), even passing it with expert-level performance. However, there is a lack of research on its performance in low-income countries' national licensing medical examinations. In Peru, where almost one out of three examinees fails the national licensing medical examination, ChatGPT has the potential to enhance medical education. We aimed to assess the accuracy of ChatGPT using GPT-3.5 and GPT-4 on the Peruvian National Licensing Medical Examination (Examen Nacional de Medicina [ENAM]). Additionally, we sought to identify factors associated with incorrect answers provided by ChatGPT. We used the ENAM 2022 data set, which consisted of 180 multiple-choice questions, to evaluate the performance of ChatGPT. Various prompts were used, and accuracy was evaluated. The performance of ChatGPT was compared to that of a sample of 1025 examinees. Factors such as question type, Peruvian-specific knowledge, discrimination, difficulty, quality of questions, and subject were analyzed to determine their influence on incorrect answers. Questions that received incorrect answers underwent a three-step process involving different prompts to explore the potential impact of adding roles and context on ChatGPT's accuracy. GPT-4 achieved an accuracy of 86% on the ENAM, followed by GPT-3.5 with 77%. The accuracy obtained by the 1025 examinees was 55%. There was a fair agreement (κ=0.38) between GPT-3.5 and GPT-4. Moderate-to-high-difficulty questions were associated with incorrect answers in the crude and adjusted model for GPT-3.5 (odds ratio [OR] 6.6, 95% CI 2.73-15.95) and GPT-4 (OR 33.23, 95% CI 4.3-257.12). After reinputting questions that received incorrect answers, GPT-3.5 went from 41 (100%) to 12 (29%) incorrect answers, and GPT-4 from 25 (100%) to 4 (16%). Our study found that ChatGPT (GPT-3.5 and GPT-4) can achieve expert-level performance on the ENAM, outperforming most of our examinees. We found fair agreement between both GPT-3.5 and GPT-4. Incorrect answers were associated with the difficulty of questions, which may resemble human performance. Furthermore, by reinputting questions that initially received incorrect answers with different prompts containing additional roles and context, ChatGPT achieved improved accuracy.