Internal Test Dataset Research Articles

Abstract 4135928: Externally Validated Deep Learning Model for Patent Ductus Arteriosus Detection by Echocardiography in Preterm Infants

Introduction: Patent ductus arteriosus (PDA) is a common form of congenital heart disease in premature infants and a source of significant morbidity. Echocardiography is the mainstay modality for diagnosis and monitoring but is resource-intensive and requires expertise for interpretation. Aims: To develop and externally validate a deep learning model to identify PDA in preterm infants by echocardiography. Methods: We trained, validated, and tested a convolutional neural network to detect PDA in raw echocardiogram clips from infants (gestational age <37 weeks) with and without PDAs and no significant additional congenital heart disease at the Medical University of South Carolina (MUSC). For training and validation, we employed a 12-fold cross-validation procedure. Experienced cardiologists defined the ground truth labels. We assessed classification performance on an internal model naïve dataset using the area under the receiver operator curve (AUROC), sensitivity, specificity, positive predictive value, and negative predictive value. For external multicenter testing, we used studies from Children’s Hospital of Orange County (CHOC) and Boston Children’s Hospital (BCH), both of which utilize a different ultrasound system vendor. Results: Training and validation were completed using 1,145 color/color-compare clips (661 clips with PDA and 484 no PDA) from 66 patients. The internal test dataset consisted of 142 clips from 16 patients. The external CHOC and BCH cohorts consisted of 146 clips from 50 patients and 294 clips from 68 patients, respectively. Model performance was similar for internal and external testing (Table 1). Conclusions: Our deep learning model detected the presence of PDA with acceptable performance. This performance generalized well across multiple institutions and ultrasound vendors. This work shows promise for the future development of an automated tool for PDA diagnosis enabling the democratization of pediatric cardiology resources.

Automatic AI tool for opportunistic screening of vertebral compression fractures on chest frontal radiographs: A multicenter study.

Vertebral compression fractures (VCFs) are the most common type of osteoporotic fractures, yet they are often clinically silent and undiagnosed. Chest frontal radiographs (CFRs) are frequently used in clinical practice and a portion of VCFs can be detected through this technology. This study aimed to develop an automatic artificial intelligence (AI) tool using deep learning (DL) model for the opportunistic screening of VCFs from CFRs. The datasets were collected from four medical centers, comprising 19,145 vertebrae (T6-T12) from 2735 patients. Patients from Center 1, 2 and 3 were divided into the training and internal testing datasets in an 8:2 ratio (n = 2361, with 16,527 vertebrae). Patients from Center 4 were used as the external test dataset (n = 374, with 2618 vertebrae). Model performance was assessed using sensitivity, specificity, accuracy and the area under the curve (AUC). A reader study with five clinicians of different experience levels was conducted with and without AI assistance. In the internal testing dataset, the model achieved a sensitivity of 83.0 % and an AUC of 0.930 at the fracture level. In the external testing dataset, the model demonstrated a sensitivity of 78.4 % and an AUC of 0.942 at the fracture level. The model's sensitivity outperformed that of five clinicians with different levels of experience. Notably, AI assistance significantly improved sensitivity at the patient level for both junior clinicians (from 56.1 % without AI to 81.6 % with AI) and senior clinicians (from 65.0 % to 85.6 %). In conclusion, the automatic AI tool significantly increases clinicians' sensitivity in diagnosing fractures on CFRs, showing great potential for the opportunistic screening of VCFs.

A New Intraocular Lens Power Formula Integrating an Artificial Intelligence-Powered Estimation for Effective Lens Position Based on Chinese Eyes.

To develop and evaluate a new intraocular lens (IOL) formula based on Chinese eyes. A training dataset of 709 eyes undergoing uneventful cataract surgery was used to train the algorithm for effective lens position estimation. The algorithm was then integrated with Gaussian optics to develop the new IOL formula (Jin-AI). From the same center, 177 eyes served as an internal test dataset. An independent dataset of 557 eyes served as an external test dataset. The standard deviation (SD) of prediction errors was compared among the Jin-AI formula, traditional third-generation formulas (SRK/T, Holladay 1, Hoffer Q), and newer generation formulas (Kane, Barrett Universal II [BUII], Hill-radial basis function [RBF] 3.0, and PEARL-DGS). In the internal test dataset, the Jin-AI formula showed the lowest SD (0.25 D), followed by the BUII (0.31 D), Kane (0.33 D), and PEARL-DGS (0.33 D) formulas. In the external test dataset, the Jin-AI, Kane, and PEARL-DGS formulas had the lowest SD (0.38 D), followed by the BUII (0.39 D), Hill-RBF 3.0 (0.39 D), SRK/T (0.45 D), Holladay 1 (0.48 D), and Hoffer Q (0.48 D) formulas. The SD of the Jin-AI formula was significantly lower than the third-generation formulas and comparable to the four newer generation formulas. Predictive accuracy of the Jin-AI formula was similar to the newer generation formulas across all axial length, keratometry, and anterior chamber depth ranges. The new formula has exhibited good performance in predicting postoperative refractions. Its overall predictive accuracy was better than the third-generation formulas and comparable to the newer generation ones. The Jin-AI formula could be a reliable alternative for IOL power calculation in Chinese.

Fully automated measurement of noise, signal-to-noise ratio, and contrast-to-noise ratio on chest CT images: feasibility and efficiency.

Rapid and accurate measurement of computed tomography (CT) image noise, signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) is a clinical challenge. To explore the feasibility of intelligent measurement of chest CT image noise, SNR, and CNR. A total of 300 chest CT scans were included in the study, which was divided into research dataset, internal test dataset, and external test dataset. Based on the research dataset, automatically segment and measure the average CT values and standard deviation (SD) of CT values for background air and lung field under different thresholds to obtain noise, SNR, and CNR results. Using the results of manual measurements as the reference standard, we determine the optimal threshold with the highest consistency. Using internal and external test datasets, validate the consistency of automated measurements of noise, SNR, and CNR at the optimal CT threshold with reference standards. With background air set at -900 HU and lung field at -800 HU as thresholds, the automated measurements of noise, SNR, and CNR demonstrate the highest consistency with the reference standards. At the optimal threshold, the noise, SNR, and CNR measured automatically on both the internal (intraclass correlation coefficient [ICC] = 0.85-0.96) and external (ICC = 0.75-0.85) test datasets exhibit high consistency with their respective reference standards. The method we explored can intelligently measure the noise, SNR, and CNR of chest CT images, exhibits high consistency with radiologists, and offers a novel tool for image quality evaluation and analysis.

Association between risk factors and bone mineral density and the development of a self-assessment tool for early osteoporosis screening in postmenopausal women with type 2 diabetes.

Both diabetes and osteoporosis have developed into major global public health problems due to the increasing aging population. It is crucial to screen populations at higher risk of developing osteoporosis for disease prevention and management in postmenopausal women with type 2 diabetes (T2D). This study aims to quantitatively investigate the association between risk factors and bone mineral density (BMD) and develop a self-assessment tool for early osteoporosis screening in postmenopausal women with T2D. We retrospectively enrolled 1,309 postmenopausal women with T2D. Linear regression methods were used to assess the association between risk factors and BMD. Additionally, a multivariate logistic regression analysis was performed to identify independent risk factors associated with osteoporosis. Utilizing the logistic regression machine learning algorithm, we developed an osteoporosis screening tool that categorizes the population into three risk regions based on age and body mass index (BMI), indicating low, moderate, and high prevalence of osteoporosis in the age-BMI plane. Older age and lower BMI were independently associated with decreased BMD. The BMD at the total hip, femur neck, and lumbar spine differed by 12.9, 10.9, and 15.5 mg/cm2 for each 1 unit increase in BMI, respectively. Both age and BMI were identified as independent predictors of osteoporosis. The osteoporosis screening tool was developed by using two straight lines with equations of BMI = 0.56 * age-4.12 and BMI = 0.56 * age-10.88; there were no significant differences in the prevalence of osteoporosis among the training, internal test, and external test datasets in the low-, moderate-, and high-risk regions. We have successfully developed and validated a self-assessment tool for early osteoporosis screening in postmenopausal women with T2D for the first time. BMI was identified as a significant modifiable risk factor. Our study may improve awareness of osteoporosis and is valuable for disease prevention and management for postmenopausal women with T2D.

Development and Validation of a Computed Tomography-Based Model for Noninvasive Prediction of the T Stage in Gastric Cancer: Multicenter Retrospective Study.

As part of the TNM (tumor-node-metastasis) staging system, T staging based on tumor depth is crucial for developing treatment plans. Previous studies have constructed a deep learning model based on computed tomographic (CT) radiomic signatures to predict the numberoflymphnodemetastases and survival in patients with resected gastric cancer (GC). However, few studies have reported the combination of deep learning and radiomics in predicting T staging in GC. This study aimed to develop a CT-based model for automatic prediction of the T stage of GC via radiomics and deep learning. A total of 771 GC patients from 3 centers were retrospectively enrolled and divided into training, validation, and testing cohorts. Patients with GC were classified into mild (stage T1 and T2), moderate (stage T3), and severe (stage T4) groups. Three predictive models based on the labeled CT images were constructed using the radiomics features (radiomics model), deep features (deep learning model), and a combination of both (hybrid model). The overall classification accuracy of the radiomics model was 64.3% in the internal testing data set. The deep learning model and hybrid model showed better performance than the radiomics model, with overall classification accuracies of 75.7% (P=.04) and 81.4% (P=.001), respectively. On the subtasks of binary classification of tumor severity, the areas under the curve of the radiomics, deep learning, and hybrid models were 0.875, 0.866, and 0.886 in the internal testing data set and 0.820, 0.818, and 0.972 in the external testing data set, respectively, for differentiating mild (stage T1~T2) from nonmild (stage T3~T4) patients, and were 0.815, 0.892, and 0.894 in the internal testing data set and 0.685, 0.808, and 0.897 in the external testing data set, respectively, for differentiating nonsevere (stage T1~T3) from severe (stage T4) patients. The hybrid model integrating radiomics features and deep features showed favorable performance in diagnosing the pathological stage of GC.

Development and validation of a deep learning model for detecting signs of tuberculosis on chest radiographs among US-bound immigrants and refugees.

Immigrants and refugees seeking admission to the United States must first undergo an overseas medical exam, overseen by the US Centers for Disease Control and Prevention (CDC), during which all persons ≥15 years old receive a chest x-ray to look for signs of tuberculosis. Although individual screening sites often implement quality control (QC) programs to ensure radiographs are interpreted correctly, the CDC does not currently have a method for conducting similar QC reviews at scale. We obtained digitized chest radiographs collected as part of the overseas immigration medical exam. Using radiographs from applicants 15 years old and older, we trained deep learning models to perform three tasks: identifying abnormal radiographs; identifying abnormal radiographs suggestive of tuberculosis; and identifying the specific findings (e.g., cavities or infiltrates) in abnormal radiographs. We then evaluated the models on both internal and external testing datasets, focusing on two classes of performance metrics: individual-level metrics, like sensitivity and specificity, and sample-level metrics, like accuracy in predicting the prevalence of abnormal radiographs. A total of 152,012 images (one image per applicant; mean applicant age 39 years) were used for model training. On our internal test dataset, our models performed well both in identifying abnormalities suggestive of TB (area under the curve [AUC] of 0.97; 95% confidence interval [CI]: 0.95, 0.98) and in estimating sample-level counts of the same (-2% absolute percentage error; 95% CIC: -8%, 6%). On the external test datasets, our models performed similarly well in identifying both generic abnormalities (AUCs ranging from 0.89 to 0.92) and those suggestive of TB (AUCs from 0.94 to 0.99). This performance was consistent across metrics, including those based on thresholded class predictions, like sensitivity, specificity, and F1 score. Strong performance relative to high-quality radiological reference standards across a variety of datasets suggests our models may make reliable tools for supporting chest radiography QC activities at CDC.

AI-based strategies in breast mass ≤ 2 cm classification with mammography and tomosynthesis

PurposeTo evaluate the diagnosis performance of digital mammography (DM) and digital breast tomosynthesis (DBT), DM combined DBT with AI-based strategies for breast mass ≤ 2 cm. Materials and methodsDM and DBT images in 483 patients including 512 breast masses were acquired from November 2018 to November 2019. Malignant and benign tumours were determined by biopsies using histological analysis and follow-up within 24 months. The radiomics and deep learning methods were employed to extract the breast mass features in images and finally for benign and malignant classification. The DM, DBT and DM combined DBT (DM + DBT) images were fed into radiomics and deep learning models to construct corresponding models, respectively. The area under the receiver operating characteristic curve (AUC) was employed to estimate model performance. An external dataset of 146 patients from March 2021 to December 2022 from another center was enrolled for external validation. ResultsIn the internal testing dataset, compared with the DM model and the DBT model, the DM + DBT models based on radiomics and deep learning both showed statistically significant higher AUCs [0.810 (RA-DM), 0.823 (RA-DBT) and 0.869 (RA-DM + DBT), P ≤ 0.001; 0.867 (DL-DM), 0.871 (DL-DBT) and 0.908 (DL-DM + DBT), P = 0.001]. The deep learning models present superior to the radiomics models in the experiments with only DM (0.867 vs 0.810, P = 0.001), only DBT (0.871 vs 0.823, P = 0.001) and DM + DBT (0.908 vs 0.869, P = 0.003). ConclusionsDBT has a clear additional value for diagnosing breast mass less than 2 cm compared with only DM. AI-based methods, especially deep learning, can help achieve excellent performance.

Data-driven risk stratification and precision management of pulmonary nodules detected on chest computed tomography.

The widespread implementation of low-dose computed tomography (LDCT) in lung cancer screening has led to the increasing detection of pulmonary nodules. However, precisely evaluating the malignancy risk of pulmonary nodules remains a formidable challenge. Here we propose a triage-driven Chinese Lung Nodules Reporting and Data System (C-Lung-RADS) utilizing a medical checkup cohort of 45,064 cases. The system was operated in a stepwise fashion, initially distinguishing low-, mid-, high- and extremely high-risk nodules based on their size and density. Subsequently, it progressively integrated imaging information, demographic characteristics and follow-up data to pinpoint suspicious malignant nodules and refine the risk scale. The multidimensional system achieved a state-of-the-art performance with an area under the curve (AUC) of 0.918 (95% confidence interval (CI) 0.918-0.919) on the internal testing dataset, outperforming the single-dimensional approach (AUC of 0.881, 95% CI 0.880-0.882). Moreover, C-Lung-RADS exhibited a superior sensitivity compared with Lung-RADS v2022 (87.1% versus 63.3%) in an independent cohort, which was screened using mobile computed tomography scanners to broaden screening accessibility in resource-constrained settings. With its foundation in precise risk stratification and tailored management, this system has minimized unnecessary invasive procedures for low-risk cases and recommended prompt intervention for extremely high-risk nodules to avert diagnostic delays. This approach has the potential to enhance the decision-making paradigm and facilitate a more efficient diagnosis of lung cancer during routine checkups as well as screening scenarios.

A deep learning model for differentiating paediatric intracranial germ cell tumour subtypes and predicting survival with MRI: a multicentre prospective study

BackgroundThe pretherapeutic differentiation of subtypes of primary intracranial germ cell tumours (iGCTs), including germinomas (GEs) and nongerminomatous germ cell tumours (NGGCTs), is essential for clinical practice because of distinct treatment strategies and prognostic profiles of these diseases. This study aimed to develop a deep learning model, iGNet, to assist in the differentiation and prognostication of iGCT subtypes by employing pretherapeutic MR T2-weighted imaging.MethodsThe iGNet model, which is based on the nnUNet architecture, was developed using a retrospective dataset of 280 pathologically confirmed iGCT patients. The training dataset included 83 GEs and 117 NGGCTs, while the retrospective internal test dataset included 31 GEs and 49 NGGCTs. The model’s diagnostic performance was then assessed with the area under the receiver operating characteristic curve (AUC) in a prospective internal dataset (n = 22) and two external datasets (n = 22 and 20). Next, we compared the diagnostic performance of six neuroradiologists with or without the assistance of iGNet. Finally, the predictive ability of the output of iGNet for progression-free and overall survival was assessed and compared to that of the pathological diagnosis.ResultsiGNet achieved high diagnostic performance, with AUCs between 0.869 and 0.950 across the four test datasets. With the assistance of iGNet, the six neuroradiologists’ diagnostic AUCs (averages of the four test datasets) increased by 9.22% to 17.90%. There was no significant difference between the output of iGNet and the results of pathological diagnosis in predicting progression-free and overall survival (P = .889).ConclusionsBy leveraging pretherapeutic MR imaging data, iGNet accurately differentiates iGCT subtypes, facilitating prognostic evaluation and increasing the potential for tailored treatment.Graphical

Prediction of H3K27M alteration in midline gliomas of brain using radiomics- A multi-institute study

Abstract Background Non-invasive prediction of H3K27M-altered Diffuse midline gliomas is important because of the involvement of deep locations and proximity to eloquent structures. We aim to predict H3K27M alteration in midline gliomas using radiomics features of T2W images. Methods Radiomics features extracted from 124 subjects (69 H3K27M-altered/55 H3K27M-wildtype). T2W-images were resampled to 1x1x1mm voxel-size, pre-processed, and normalized for artefact correction, intensity variations. Feature-set was normalized, subjected to reduction by variance thresholding, correlation coefficient thresholding, and sequential feature selector. Adaptive synthesis oversampling technique was used to over-sample the training data. Random forest classifier (RFC), Decision tree classifier (DTC) and K-nearest neighbors classifier (KNN) were trained over the training dataset and the performance was assessed over the internal test dataset and external test data set (52 subjects: 33 H3K27M-altered/19-H3K27M-wildtype). Results DTC achieved validation score of 77.33% (5-fold cross-validation) and accuracy of 80.64%, 75% on internal and external test dataset. RFC achieved validation score of 80.7% (5-fold cross-validation) and accuracy of 80.6%, 73% on internal and external test dataset. DTC achieved validation score of 78.67% (5-fold cross-validation) and accuracy of 80.64%, 61.53% on internal and external test dataset. Accuracy score of DTC, RFC, and KNN on internal test dataset was approximately 80% while on the external test dataset, DTC achieved 75% accuracy, RFC achieved 73% accuracy and KNN achieved 65.1% accuracy. Conclusion H3K27M alteration is a potential immunotherapeutic marker and is associated with poor prognosis and radiomics features extracted from conventional T2W-images can help in identifying H3K27M-altered cases non-invasively with high precision.

Artificial Intelligence-Based Surgery Support Model Using Intraoperative Radiographs for Assessing the Acetabular Component Angle

BackgroundThis study aimed to develop an artificial intelligence–based surgical support model for assessing the acetabular component angle using intraoperative radiographs during total hip arthroplasty and verify its accuracy. MethodsA total of 268 hips were analyzed. At first, 268 preoperative and intraoperative anteroposterior pelvic radiographs were amplified to 536. These radiographs were used to create a learning model to estimate the acetabular component angle from the radiographs intraoperatively. The ground truth was the anteversion and inclination angles obtained from the computed tomography–based navigation system intraoperatively. Bone landmarks on the preoperative and intraoperative radiographs were manually annotated. The distances and angles between each landmark were used as predictor variables. The estimation accuracy was assessed for internal and external test datasets. Mean absolute error (MAE) and R2 values were used as accuracy measures. ResultsThe MAE and R2 for the internal test set showed 2.19 and 0.850 for anteversion, and 1.18 and 0.805 for inclination, respectively. The MAE and R2 for the external test set showed 2.78 and 0.789 for anteversion, and 1.56 and 0.744 for inclination, respectively. ConclusionsWe developed an artificial intelligence–based surgical support model for accurately assessing the acetabular component angle using intraoperative radiographs. Excellent estimation accuracy was confirmed for the external test set. In the future, the model may help to reduce the risk of adverse postoperative events.

Renal tumor segmentation, visualization, and segmentation confidence using ensembles of neural networks in patients undergoing surgical resection.

To develop an automatic segmentation model for solid renal tumors on contrast-enhanced CTs and to visualize segmentation with associated confidence to promote clinical applicability. The training dataset included solid renal tumor patients from two tertiary centers undergoing surgical resection and receiving CT in the corticomedullary or nephrogenic contrast media (CM) phase. Manual tumor segmentation was performed on all axial CT slices serving as reference standard for automatic segmentations. Independent testing was performed on the publicly available KiTS 2019 dataset. Ensembles of neural networks (ENN, DeepLabV3) were used for automatic renal tumor segmentation, and their performance was quantified with DICE score. ENN average foreground entropy measured segmentation confidence (binary: successful segmentation with DICE score > 0.8 versus inadequate segmentation ≤ 0.8). N = 639/n = 210 patients were included in the training and independent test dataset. Datasets were comparable regarding age and sex (p > 0.05), while renal tumors in the training dataset were larger and more frequently benign (p < 0.01). In the internal test dataset, the ENN model yielded a median DICE score = 0.84 (IQR: 0.62-0.97, corticomedullary) and 0.86 (IQR: 0.77-0.96, nephrogenic CM phase), and the segmentation confidence an AUC = 0.89 (sensitivity = 0.86; specificity = 0.77). In the independent test dataset, the ENN model achieved a median DICE score = 0.84 (IQR: 0.71-0.97, corticomedullary CM phase); and segmentation confidence an accuracy = 0.84 (sensitivity = 0.86 and specificity = 0.81). ENN segmentations were visualized with color-coded voxelwise tumor probabilities and thresholds superimposed on clinical CT images. ENN-based renal tumor segmentation robustly performs in external test data and might aid in renal tumor classification and treatment planning. Ensembles of neural networks (ENN) models could automatically segment renal tumors on routine CTs, enabling and standardizing downstream image analyses and treatment planning. Providing confidence measures and segmentation overlays on images can lower the threshold for clinical ENN implementation. Ensembles of neural networks (ENN) segmentation is visualized by color-coded voxelwise tumor probabilities and thresholds. ENN provided a high segmentation accuracy in internal testing and in an independent external test dataset. ENN models provide measures of segmentation confidence which can robustly discriminate between successful and inadequate segmentations.

Predicting 1, 2 and 3 year emergent referable diabetic retinopathy and maculopathy using deep learning

BackgroundPredicting diabetic retinopathy (DR) progression could enable individualised screening with prompt referral for high-risk individuals for sight-saving treatment, whilst reducing screening burden for low-risk individuals. We developed and validated deep learning systems (DLS) that predict 1, 2 and 3 year emergent referable DR and maculopathy using risk factor characteristics (tabular DLS), colour fundal photographs (image DLS) or both (multimodal DLS).MethodsFrom 162,339 development-set eyes from south-east London (UK) diabetic eye screening programme (DESP), 110,837 had eligible longitudinal data, with the remaining 51,502 used for pretraining. Internal and external (Birmingham DESP, UK) test datasets included 27,996, and 6928 eyes respectively.ResultsInternal multimodal DLS emergent referable DR, maculopathy or either area-under-the receiver operating characteristic (AUROC) were 0.95 (95% CI: 0.92–0.98), 0.84 (0.82–0.86), 0.85 (0.83–0.87) for 1 year, 0.92 (0.87–0.96), 0.84 (0.82–0.87), 0.85 (0.82–0.87) for 2 years, and 0.85 (0.80–0.90), 0.79 (0.76–0.82), 0.79 (0.76–0.82) for 3 years. External multimodal DLS emergent referable DR, maculopathy or either AUROC were 0.93 (0.88–0.97), 0.85 (0.80–0.89), 0.85 (0.76–0.85) for 1 year, 0.93 (0.89–0.97), 0.79 (0.74–0.84), 0.80 (0.76–0.85) for 2 years, and 0.91 (0.84–0.98), 0.79 (0.74–0.83), 0.79 (0.74–0.84) for 3 years.ConclusionsMultimodal and image DLS performance is significantly better than tabular DLS at all intervals. DLS accurately predict 1, 2 and 3 year emergent referable DR and referable maculopathy using colour fundal photographs, with additional risk factor characteristics conferring improvements in prognostic performance. Proposed DLS are a step towards individualised risk-based screening, whereby AI-assistance allows high-risk individuals to be closely monitored while reducing screening burden for low-risk individuals.

Conditional generative adversarial network-assisted system for radiation-free evaluation of scoliosis using a single smartphone photograph: a model development and validation study

Adolescent idiopathic scoliosis (AIS) is the most common spinal disorder in children, characterized by insidious onset and rapid progression, which can lead to severe consequences if not detected in a timely manner. Currently, the diagnosis of AIS primarily relies on X-ray imaging. However, due to limitations in healthcare access and concerns over radiation exposure, this diagnostic method cannot be widely adopted. Therefore, we have developed and validated a screening system using deep learning technology, capable of generating virtual X-ray images (VXI) from two-dimensional Red Green Blue (2D-RGB) images captured by a smartphone or camera to assist spine surgeons in the rapid, accurate, and non-invasive assessment of AIS. We included 2397 patients with AIS and 48 potential patients with AIS who visited four medical institutions in mainland China from June 11th 2014 to November 28th 2023. Participants data included standing full-spine X-ray images captured by radiology technicians and 2D-RGB images taken by spine surgeons using a camera. We developed a deep learning model based on conditional generative adversarial networks (cGAN) called Swin-pix2pix to generate VXI on retrospective training (n=1842) and validation (n=100) dataset, then validated the performance of VXI in quantifying the curve type and severity of AIS on retrospective internal (n=100), external (n=135), and prospective test datasets (n=268). The prospective test dataset included 268 participants treated in Nanjing, China, from April 19th, 2023, to November 28th, 2023, comprising 220 patients with AIS and 48 potential patients with AIS. Their data underwent strict quality control to ensure optimal data quality and consistency. Our Swin-pix2pix model generated realistic VXI, with the mean absolute error (MAE) for predicting the main and secondary Cobb angles of AIS significantly lower than other baseline cGAN models, at 3.2° and 3.1° on prospective test dataset. The diagnostic accuracy for scoliosis severity grading exceeded that of two spine surgery experts, with accuracy of 0.93 (95% CI [0.91, 0.95]) in main curve and 0.89 (95% CI [0.87, 0.91]) in secondary curve. For main curve position and curve classification, the predictive accuracy of the Swin-pix2pix model also surpassed that of the baseline cGAN models, with accuracy of 0.93 (95% CI [0.90, 0.95]) for thoracic curve and 0.97 (95% CI [0.96, 0.98]), achieving satisfactory results on three external datasets as well. Our developed Swin-pix2pix model holds promise for using a single photo taken with a smartphone or camera to rapidly assess AIS curve type and severity without radiation, enabling large-scale screening. However, limited data quality and quantity, a homogeneous participant population, and rotational errors during imaging may affect the applicability and accuracy of the system, requiring further improvement in the future. National Key R&D Program of China, Natural Science Foundation of Jiangsu Province, China Postdoctoral Science Foundation, Nanjing Medical Science and Technology Development Foundation, Jiangsu Provincial Key Research and Development Program, and Jiangsu Provincial Medical Innovation Centre of Orthopedic Surgery.

Development of a multi-modal learning-based lymph node metastasis prediction model for lung cancer

PurposeThis study proposed a three-dimensional (3D) multi-modal learning-based model for the automated prediction and classification of lymph node metastasis in patients with non-small cell lung cancer (NSCLC) using computed tomography (CT) images and clinical information. MethodsWe utilized clinical information and CT image data from 4239 patients with NSCLC across multiple institutions. Four deep learning algorithm-based multi-modal models were constructed and evaluated for lymph node classification. To further enhance classification performance, a soft-voting ensemble technique was applied to integrate the outcomes of multiple multi-modal models. ResultsA comparison of the classification performance revealed that the multi-modal model, which integrated CT images and clinical information, outperformed the single-modal models. Among the four multi-modal models, the Xception model demonstrated the highest classification performance, with an area under the curve (AUC) of 0.756 for the internal test dataset and 0.736 for the external validation dataset. The ensemble model (SEResNet50_DenseNet121_Xception) exhibited even better performance, with an AUC of 0.762 for the internal test dataset and 0.751 for the external validation dataset, surpassing the multi-modal model's performance. ConclusionsIntegrating CT images and clinical information improved the performance of the lymph node metastasis prediction models in patients with NSCLC. The proposed 3D multi-modal lymph node prediction model can serve as an auxiliary tool for evaluating lymph node metastasis in patients with non-pretreated NSCLC, aiding in patient screening and treatment planning.

Diagnosing Solid Lesions in the Pancreas With Multimodal Artificial Intelligence

Diagnosing solid lesions in the pancreas via endoscopic ultrasonographic (EUS) images is challenging. Artificial intelligence (AI) has the potential to help with such diagnosis, but existing AI models focus solely on a single modality. To advance the clinical diagnosis of solid lesions in the pancreas through developing a multimodal AI model integrating both clinical information and EUS images. In this randomized crossover trial conducted from January 1 to June 30, 2023, from 4 centers across China, 12 endoscopists of varying levels of expertise were randomly assigned to diagnose solid lesions in the pancreas with or without AI assistance. Endoscopic ultrasonographic images and clinical information of 439 patients from 1 institution who had solid lesions in the pancreas between January 1, 2014, and December 31, 2022, were collected to train and validate the joint-AI model, while 189 patients from 3 external institutions were used to evaluate the robustness and generalizability of the model. Conventional or AI-assisted diagnosis of solid lesions in the pancreas. In the retrospective dataset, the performance of the joint-AI model was evaluated internally and externally. In the prospective dataset, diagnostic performance of the endoscopists with or without the AI assistance was compared. The retrospective dataset included 628 patients (400 men [63.7%]; mean [SD] age, 57.7 [27.4] years) who underwent EUS procedures. A total of 130 patients (81 men [62.3%]; mean [SD] age, 58.4 [11.7] years) were prospectively recruited for the crossover trial. The area under the curve of the joint-AI model ranged from 0.996 (95% CI, 0.993-0.998) in the internal test dataset to 0.955 (95% CI, 0.940-0.968), 0.924 (95% CI, 0.888-0.955), and 0.976 (95% CI, 0.942-0.995) in the 3 external test datasets, respectively. The diagnostic accuracy of novice endoscopists was significantly enhanced with AI assistance (0.69 [95% CI, 0.61-0.76] vs 0.90 [95% CI, 0.83-0.94]; P < .001), and the supplementary interpretability information alleviated the skepticism of the experienced endoscopists. In this randomized crossover trial of diagnosing solid lesions in the pancreas with or without AI assistance, the joint-AI model demonstrated positive human-AI interaction, which suggested its potential to facilitate a clinical diagnosis. Nevertheless, future randomized clinical trials are warranted. ClinicalTrials.gov Identifier: NCT05476978.

SLO-Net: Enhancing Multiple Sclerosis Diagnosis Beyond Optical Coherence Tomography Using Infrared Reflectance Scanning Laser Ophthalmoscopy Images.

Several machine learning studies have used optical coherence tomography (OCT) for multiple sclerosis (MS) classification with promising outcomes. Infrared reflectance scanning laser ophthalmoscopy (IR-SLO) captures high-resolution fundus images, commonly combined with OCT for fixed B-scan positions. However, no machine learning research has utilized IR-SLO images for automated MS diagnosis. This study utilized a dataset comprised of IR-SLO images and OCT data from Isfahan, Iran, encompassing 32 MS and 70 healthy individuals. A number of convolutional neural networks (CNNs)-namely, VGG-16, VGG-19, ResNet-50, ResNet-101, and a custom architecture-were trained with both IR-SLO images and OCT thickness maps as two separate input datasets. The highest performing models for each modality were then integrated to create a bimodal model that receives the combination of OCT thickness maps and IR-SLO images. Subject-wise data splitting was employed to prevent data leakage among training, validation, and testing sets. Overall, images of the 102 patients from the internal dataset were divided into test, validation, and training subsets. Subsequently, we employed a bootstrapping approach on the training data through iterative sampling with replacement. The performance of the proposed bimodal model was evaluated on the internal test dataset, demonstrating an accuracy of 92.40% ± 4.1% (95% confidence interval [CI], 83.61-98.08), sensitivity of 95.43% ± 5.75% (95% CI, 83.71-100.0), specificity of 92.82% ± 3.72% (95% CI, 81.15-96.77), area under the receiver operating characteristic (AUROC) curve of 96.99% ± 2.99% (95% CI, 86.11-99.78), and area under the precision-recall curve (AUPRC) of 97.27% ± 2.94% (95% CI, 86.83-99.83). Furthermore, to assess the model generalization ability, we examined its performance on an external test dataset following the same bootstrap methodology, achieving promising results, with accuracy of 85.43% ± 0.08% (95% CI, 71.43-100.0), sensitivity of 97.33% ± 0.06% (95% CI, 83.33-100.0), specificity of 84.6% ± 0.10% (95% CI, 71.43-100.0), AUROC curve of 99.67% ± 0.02% (95% CI, 95.63-100.0), and AUPRC of 99.65% ± 0.02% (95% CI, 94.90-100.0). Incorporating both modalities improves the performance of automated diagnosis of MS, showcasing the potential of utilizing IR-SLO as a complementary tool alongside OCT. Should the results of our proposed bimodal model be validated in future work with larger and more diverse datasets, diagnosis of MS based on both OCT and IR-SLO can be reliably integrated into routine clinical practice.

The use of artificial intelligence in musculoskeletal ultrasound: a systematic review of the literature

PurposeTo systematically review the use of artificial intelligence (AI) in musculoskeletal (MSK) ultrasound (US) with an emphasis on AI algorithm categories and validation strategies.Material and MethodsAn electronic literature search was conducted for articles published up to January 2024. Inclusion criteria were the use of AI in MSK US, involvement of humans, English language, and ethics committee approval.ResultsOut of 269 identified papers, 16 studies published between 2020 and 2023 were included. The research was aimed at predicting diagnosis and/or segmentation in a total of 11 (69%) out of 16 studies. A total of 11 (69%) studies used deep learning (DL)-based algorithms, three (19%) studies employed conventional machine learning (ML)-based algorithms, and two (12%) studies employed both conventional ML- and DL-based algorithms. Six (38%) studies used cross-validation techniques with K-fold cross-validation being the most frequently employed (n = 4, 25%). Clinical validation with separate internal test datasets was reported in nine (56%) papers. No external clinical validation was reported.ConclusionAI is a topic of increasing interest in MSK US research. In future studies, attention should be paid to the use of validation strategies, particularly regarding independent clinical validation performed on external datasets.

Semi-Self-Supervised Domain Adaptation: Developing Deep Learning Models with Limited Annotated Data for Wheat Head Segmentation

Precision agriculture involves the application of advanced technologies to improve agricultural productivity, efficiency, and profitability while minimizing waste and environmental impacts. Deep learning approaches enable automated decision-making for many visual tasks. However, in the agricultural domain, variability in growth stages and environmental conditions, such as weather and lighting, presents significant challenges to developing deep-learning-based techniques that generalize across different conditions. The resource-intensive nature of creating extensive annotated datasets that capture these variabilities further hinders the widespread adoption of these approaches. To tackle these issues, we introduce a semi-self-supervised domain adaptation technique based on deep convolutional neural networks with a probabilistic diffusion process, requiring minimal manual data annotation. Using only three manually annotated images and a selection of video clips from wheat fields, we generated a large-scale computationally annotated dataset of image–mask pairs and a large dataset of unannotated images extracted from video frames. We developed a two-branch convolutional encoder–decoder model architecture that uses both synthesized image–mask pairs and unannotated images, enabling effective adaptation to real images. The proposed model achieved a Dice score of 80.7% on an internal test dataset and a Dice score of 64.8% on an external test set composed of images from five countries and spanning 18 domains, indicating its potential to develop generalizable solutions that could encourage the wider adoption of advanced technologies in agriculture.