Deep Feature Learning Research Articles

Local-aware global attention network for person re-identification based on body and hand images

Learning representative, robust and discriminative information from images is essential for effective person re-identification (Re-Id). In this paper, we propose a compound approach for end-to-end discriminative deep feature learning for person Re-Id based on both body and hand images. We carefully design the Local-Aware Global Attention Network (LAGA-Net), a multi-branch deep network architecture consisting of one branch for spatial attention, one branch for channel attention, one branch for global feature representations and another branch for local feature representations. The attention branches focus on the relevant features of the image while suppressing the irrelevant backgrounds. The global and local branches intends to capture global context and fine-grained information, respectively. A set of ablation study shows that each component contributes to the increased performance of the LAGA-Net. Extensive evaluations on four popular body-based person Re-Id benchmarks and two publicly available hand datasets demonstrate that our proposed method consistently outperforms existing state-of-the-art methods.

Integration of Localized, Contextual, and Hierarchical Features in Deep Learning for Improved Skin Lesion Classification.

Skin lesion classification is vital for the early detection and diagnosis of skin diseases, facilitating timely intervention and treatment. However, existing classification methods face challenges in managing complex information and long-range dependencies in dermoscopic images. Therefore, this research aims to enhance the feature representation by incorporating local, global, and hierarchical features to improve the performance of skin lesion classification. We introduce a novel dual-track deep learning (DL) model in this research for skin lesion classification. The first track utilizes a modified Densenet-169 architecture that incorporates a Coordinate Attention Module (CoAM). The second track employs a customized convolutional neural network (CNN) comprising a Feature Pyramid Network (FPN) and Global Context Network (GCN) to capture multiscale features and global contextual information. The local features from the first track and the global features from second track are used for precise localization and modeling of the long-range dependencies. By leveraging these architectural advancements within the DenseNet framework, the proposed neural network achieved better performance compared to previous approaches. The network was trained and validated using the HAM10000 dataset, achieving a classification accuracy of 93.2%.

Explainable rumor detection based on grey clustering: Fusion of manual features and deep learning features

The importance of rumor detection on social media is self-evident. However, many existing studies have focused on exploring potential features in text content and propagation patterns, while neglecting a key aspect—the explainability of the model. The comment content can provide support for the credibility of the detection. Nevertheless, most studies that use comments encode them into specific models, rarely considering their semantic attitudes and standpoints, making it difficult for models to explain why a post is a rumor. In this study, we propose an Explainable rumor detection model based on Grey clustering called MDE-Grey, which combines Manual features and Deep learning features. In terms of manual features, we constructed a relevant vocabulary based on the specific comment environment of rumors to capture comment standpoints. In terms of deep learning features, we have designed a GCN sub network that includes two attention mechanisms to capture noteworthy content in posts and comments. Finally, we constructed a new grey clustering model to fuse the two types of features and obtain the final prediction. In the grey clustering model, we designed new whitening functions to capture the intrinsic relationship between features and rumor categories, ensuring the traceability of prediction results. The experiments on three datasets and case studies have demonstrated the effectiveness of the MDE-Grey model in detecting rumors and explaining the results.

High-Speed Motion Target Real-Time Detection Based on Lightweight Deep Feature Learning Network

High-Speed Motion Target Real-Time Detection Based on Lightweight Deep Feature Learning Network

ECG Signal Classification based on combined CNN Features and Optimised Support Vector Machine

The electrocardiogram (ECG) is a visual depiction of the electrical activity of the heart. It is utilised to detect and diagnose different cardiac conditions. Over the last decade, the categorization of cardiac conditions based on electrocardiogram signals have become very significant for both patients and medical professionals. This article presents a novel method for classifying Electrocardiogram signals into arrhythmia (ARR), congestive heart failure (CHF), or normal sinus rhythm (NSR) using combined deep learning features and optimised Support Vector Machine (Op-SVM). First, to perform classification via Deep Learning (DL)the continuous wavelet transform (CWT) was used to transform one-dimensional (1-D) ECG signals into two-dimensional (2-D) images (scalograms) which are sent then to two pre-trained convolutional neural networks (CNN) architectures (ResNet50 and DenseNet201). Next, the features extracted from both CNNs were combined and fed to the SVM classifier. To enhance the performance of the classifier, Bayesian optimisation was used to optimise its hyperparameters. The suggested method was tested using a public dataset (PhysioNet) and evaluated using performance metric techniques. It succeeded in achieving values of 99.44 % for accuracy (Acc), 99.44 % for sensitivity (Sen), 99.72 % for specificity (Sp), and 99.44 % for precision (Pr), respectively, which are exceptional compared to the values produced by models considered to be state-of-the-art. Our results showed that the suggested method is suitable for in-clinic application in diagnosing cardiac conditions using ECG signals.

DSDRec: Next POI recommendation using deep semantic extraction and diffusion model

Semantics play a crucial role in many AI tasks, yet the lack of high-quality textual data in LBSNs hampers deep semantic feature learning. Sparse user check-in records, characterized by significant spatial or temporal intervals (cut points), create challenges in understanding users' real mobile preferences. However, prior studies often rely on shallow semantic signals from independent textual properties and overlook these cut points when modeling spatio-temporal context. This work proposes a recommender model named DSDRec for the next POI recommendation. To alleviate the shortage of high-quality textual data in LBSNs, we take prompt engineering to freely obtain rich semantic texts (a.k.a., prompt sentences) from the discrete trajectory sequences and then use a pre-trained language model to acquire high-quality deep semantic features based on the functional prompt sentences. To mitigate the risk caused by the cut points, we propose an effective method, Spatio-Temporal Area Encoding, to depict the global spatial and temporal relationship of check-ins in trajectory. Besides, to study the potential of diffusion models in the next POI recommendation, we stacked an improved diffusion model to assist the recommender in learning more abstract interaction information. Comprehensive experiments conducted on two real-world datasets validate the competitive performance of DSDRec over previous SOTA approaches.

A semi-automatic deep learning model based on biparametric MRI scanning strategy to predict bone metastases in newly diagnosed prostate cancer patients.

To develop a semi-automatic model integrating radiomics, deep learning, and clinical features for Bone Metastasis (BM) prediction in prostate cancer (PCa) patients using Biparametric MRI (bpMRI) images. A retrospective study included 414 PCa patients (BM, n=136; NO-BM, n=278) from two institutions (Center 1, n=318; Center 2, n=96) between January 2016 and December 2022. MRI scans were confirmed with BM status via PET-CT or ECT pre-treatment. Tumor areas on bpMRI images were delineated as tumor's region of interest (ROI) using auto-delineation tumor models, evaluated with Dice similarity coefficient (DSC). Samples were auto-sketched, refined, and used to train the ResNet BM prediction model. Clinical, radiomics, and deep learning data were synthesized into the ResNet-C model, evaluated using receiver operating characteristic (ROC). The auto-segmentation model achieved a DSC of 0.607. Clinical BM prediction's internal validation had an accuracy (ACC) of 0.650 and area under the curve (AUC) of 0.713; external cohort had an ACC of 0.668 and AUC of 0.757. The deep learning model yielded an ACC of 0.875 and AUC of 0.907 for the internal, and ACC of 0.833 and AUC of 0.862 for the external cohort. The Radiomics model registered an ACC of 0.819 and AUC of 0.852 internally, and ACC of 0.885 and AUC of 0.903 externally. ResNet-C demonstrated the highest ACC of 0.902 and AUC of 0.934 for the internal, and ACC of 0.885 and AUC of 0.903 for the external cohort. The ResNet-C model, utilizing bpMRI scanning strategy, accurately assesses bone metastasis (BM) status in newly diagnosed prostate cancer (PCa) patients, facilitating precise treatment planning and improving patient prognoses.

Parallel CNN‐Deep Learning Clinical‐Imaging Signature for Assessing Pathologic Grade and Prognosis of Soft Tissue Sarcoma Patients

BackgroundTraditional biopsies pose risks and may not accurately reflect soft tissue sarcoma (STS) heterogeneity. MRI provides a noninvasive, comprehensive alternative.PurposeTo assess the diagnostic accuracy of histological grading and prognosis in STS patients when integrating clinical‐imaging parameters with deep learning (DL) features from preoperative MR images.Study TypeRetrospective/prospective.Population354 pathologically confirmed STS patients (226 low‐grade, 128 high‐grade) from three hospitals and the Cancer Imaging Archive (TCIA), divided into training (n = 185), external test (n = 125), and TCIA cohorts (n = 44). 12 patients (6 low‐grade, 6 high‐grade) were enrolled into prospective validation cohort.Field Strength/Sequence1.5 T and 3.0 T/Unenhanced T1‐weighted and fat‐suppressed‐T2‐weighted.AssessmentDL features were extracted from MR images using a parallel ResNet‐18 model to construct DL signature. Clinical‐imaging characteristics included age, gender, tumor‐node‐metastasis stage and MRI semantic features (depth, number, heterogeneity at T1WI/FS‐T2WI, necrosis, and peritumoral edema). Logistic regression analysis identified significant risk factors for the clinical model. A DL clinical‐imaging signature (DLCS) was constructed by incorporating DL signature with risk factors, evaluated for risk stratification, and assessed for progression‐free survival (PFS) in retrospective cohorts, with an average follow‐up of 23 ± 22 months.Statistical TestsLogistic regression, Cox regression, Kaplan–Meier curves, log‐rank test, area under the receiver operating characteristic curve (AUC)，and decision curve analysis. A P‐value <0.05 was considered significant.ResultsThe AUC values for DLCS in the external test, TCIA, and prospective test cohorts (0.834, 0.838, 0.819) were superior to clinical model (0.662, 0.685, 0.694). Decision curve analysis showed that the DLCS model provided greater clinical net benefit over the DL and clinical models. Also, the DLCS model was able to risk‐stratify patients and assess PFS.Data ConclusionThe DLCS exhibited strong capabilities in histological grading and prognosis assessment for STS patients, and may have potential to aid in the formulation of personalized treatment plans.Level of Evidence4.Technical EfficacyStage 2.

Integrating intratumoral and peritumoral radiomics with deep transfer learning for DCE-MRI breast lesion differentiation: A multicenter study comparing performance with radiologists

PurposeTo conduct the fusion of radiomics and deep transfer learning features from the intratumoral and peritumoral areas in breast DCE-MRI images to differentiate between benign and malignant breast tumors, and to compare the diagnostic accuracy of this fusion model against the assessments made by experienced radiologists. Materials and MethodsThis multi-center study conducted a retrospective analysis of DCE-MRI images from 330 women diagnosed with breast cancer, with 138 cases categorized as benign and 192 as malignant. The training and internal testing sets comprised 270 patients from center 1, while the external testing cohort consisted of 60 patients from center 2. A fusion feature set consisting of radiomics features and deep transfer learning features was constructed from both intratumoral (ITR) and peritumoral (PTR) areas. The Least absolute shrinkage and selection operator (LASSO) based support vector machine was chosen as the classifier by comparing its performance with five other machine learning models. The diagnostic performance and clinical usefulness of fusion model were verified and assessed through the area under the receiver operating characteristics (ROC) and decision curve analysis. Additionally, the performance of the fusion model was compared with the diagnostic assessments of two experienced radiologists to evaluate its relative accuracy. The study strictly adhered to CLEAR and METRICS guidelines for standardization to ensure rigorous and reproducible methods. ResultsThe findings show that the fusion model, utilizing radiomics and deep transfer learning features from the ITR and PTR, exhibited exceptional performance in classifying breast tumors, achieving AUCs of 0.950 in the internal testing set and 0.921 in the external testing set. This performance significantly surpasses that of models relying on singular regional radiomics or deep transfer learning features alone. Moreover, the fusion model demonstrated superior diagnostic accuracy compared to the evaluations conducted by two experienced radiologists, thereby highlighting its potential to support and enhance clinical decision-making in the differentiation of benign and malignant breast tumors. ConclusionThe fusion model, combining multi-regional radiomics with deep transfer learning features, not only accurately differentiates between benign and malignant breast tumors but also outperforms the diagnostic assessments made by experienced radiologists. This underscores the model’s potential as a valuable tool for improving the accuracy and reliability of breast tumor diagnosis.

Dual-branch interactive cross-frequency attention network for deep feature learning

As random noises contained in the high-frequency data could interfere with the feature learning of deep networks, low-pass filtering or wavelet transform have been integrated with deep networks to exclude the high-frequency component of input image. However, useful image details like contour and texture are also lost in such a process. In this paper, we propose Dual-branch interactive Cross-frequency attention Network (DiCaN) to separately process low-frequency and high-frequency components of input image, such that useful information is extracted from high-frequency data and included in deep learning. Our DiCaN first decomposes input image into low-frequency and high-frequency components using wavelet decomposition, and then applies two parallel residual-style branches to extract features from the two components. We further design an interactive cross-frequency attention mechanism, to highlight the useful information in the high-frequency data and interactively fuse them with the features in low-frequency branch. The features learned by our framework are then applied for both image classification and object detection and tested using ImageNet-1K and COCO datasets. The results suggest that DiCaN achieves better classification performance than different ResNet variants. Both one-stage and two-stage detectors with our DiCaN backbone also achieve better detection performance than that with ResNet backbone. The code of DiCaN will be released.

Integration of deep learning and habitat radiomics for predicting the response to immunotherapy in NSCLC patients

BackgroundThe non-invasive biomarkers for predicting immunotherapy response are urgently needed to prevent both premature cessation of treatment and ineffective extension. This study aimed to construct a non-invasive model for predicting immunotherapy response, based on the integration of deep learning and habitat radiomics in patients with advanced non-small cell lung cancer (NSCLC).MethodsIndependent patient cohorts from three medical centers were enrolled for training (n = 164) and test (n = 82). Habitat imaging radiomics features were derived from sub-regions clustered from individual’s tumor by K-means method. The deep learning features were extracted based on 3D ResNet algorithm. Pearson correlation coefficient, T test and least absolute shrinkage and selection operator regression were used to select features. Support vector machine was applied to implement deep learning and habitat radiomics, respectively. Then, a combination model was developed integrating both sources of data.ResultsThe combination model obtained a strong well-performance, achieving area under receiver operating characteristics curve of 0.865 (95% CI 0.772–0.931). The model significantly discerned high and low-risk patients, and exhibited a significant benefit in the clinical use.ConclusionThe integration of deep-leaning and habitat radiomics contributed to predicting response to immunotherapy in patients with NSCLC. The developed integration model may be used as potential tool for individual immunotherapy management.

Deep transfer learning radiomics model based on temporal bone CT for assisting in the diagnosis of inner ear malformations

Objective:To evaluate the diagnostic efficacy of traditional radiomics, deep learning, and deep learning radiomics in differentiating normal and inner ear malformations on temporal bone computed tomography（CT）. Methods:A total of 572 temporal bone CT data were retrospectively collected, including 201 cases of inner ear malformation and 371 cases of normal inner ear, and randomly divided into a training cohort（n=458） and a test cohort（n=114） in a ratio of 4∶1. Deep transfer learning features and radiomics features were extracted from the CT images and feature fusion was performed to establish the least absolute shrinkage and selection operator. The CT results interpretated by two chief otologists from the National Clinical Research Center for Otorhinolaryngological Diseases served as the gold standard for diagnosis. The model performance was evaluated using receiver operating characteristic（ROC）, and the accuracy, sensitivity, specificity, and other indicators of the models were calculated. The predictive power of each model was compared using the Delong test. Results:1 179 radiomics features were obtained from traditional radiomics, 2 048 deep learning features were obtained from deep learning, and 137 features fusion were obtained after feature screening and fusion of the two. The area under the curve（AUC） of the deep learning radiomics model on the test cohort was 0.964 0（95%CI 0.931 4-0.996 8）, with an accuracy of 0.922, sensitivity of 0.881, and specificity of 0.945. The AUC of the radiomics features alone on the test cohort was 0.929 0（95%CI 0.882 2-0.974 9）, with an accuracy of 0.878, sensitivity of 0.881, and specificity of 0.877. The AUC of the deep learning features alone on the test cohort was 0.947 0（95%CI 0.898 2-0.994 8）, with an accuracy of 0.913, sensitivity of 0.810, and specificity of 0.973. The results indicated that the prediction accuracy and AUC of the deep learning radiomics model are the highest. The Delong test showed that the differences between any two models did not reach statistical significance. Conclusion:The feature fusion model can be used for the differential diagnosis of normal and inner ear malformations, and its diagnostic performance is superior to radiomics or deep learning models alone.

Particle accelerator power system early fault diagnosis based on deep learning and multi-sensor feature fusion

Particle accelerators play a crucial role in scientific research and industrial applications, and enhancing their reliability, ensuring stable operation, and reducing downtime caused by faults are essential for achieving research goals. This paper introduces a novel particle accelerator fault diagnosis method based on deep learning and multi-sensor feature fusion. The approach employs one-dimensional convolution to extract signals from multiple sensors and achieves comprehensive feature fusion of multi-sensor data, effectively overcoming the limitations of individual sensors. It combines 1d convolutional neural networks (1D-CNN) and long short-term memory networks (LSTM) to enhance spatiotemporal feature extraction. This approach efficiently extracts features from multiple sensors while concurrently reducing data length and training time. The study uses real signals from particle accelerators and aims to achieve early detection of faults by identifying abnormal signals preceding accelerator malfunctions. Comparative analysis with other machine learning models and the use of multiple evaluation metrics validate the effectiveness and generalizability of this method. The research provides a better performance fault diagnosis model with significant implications for reducing downtime, troubleshooting faults, and improving the reliability of particle accelerators.

Satellite position and attitude estimation using an infrared earth sensor

Micro/nano-satellites are constrained by payload, size, and cost, making enhancements in design flexibility crucial, particularly for orbit and attitude determination systems. An effective strategy is the innovative utilization of existing sensors. This study investigates a novel application of Earth sensors for autonomous position and attitude determination enabling them as a viable option for autonomous navigation in low-cost micro/nano-satellites through computer vision technology. To address large resolution differences and distortions between sensed infrared images and reference maps, this paper presents a coarse-to-fine matching framework that integrates deep learning features and area-based registrations. First, CNN-based registration and optimization techniques are utilized for coarse registration. A modified VGG16 (M-VGG) and a multiscale pyramid information fusion (MPIF) module are employed to extract and describe deep abstract features, enhancing robustness. Subsequently, edge-based extended phase correlation is employed for fine registration. Finally, the ultimate position and attitude parameters are determined by integrating the results from coarse and fine registration. The effectiveness of the proposed scheme is validated using synthetic images. An analysis of 342 images demonstrates sub-pixel registration accuracy, with a root mean square error (RMSE) of 1.9322 km for position and 0.1185° for attitude.

Tiny Machine Learning for Concept Drift.

Tiny machine learning (TML) is a new research area whose goal is to design machine and deep learning (DL) techniques able to operate in embedded systems and the Internet-of-Things (IoT) units, hence satisfying the severe technological constraints on memory, computation, and energy characterizing these pervasive devices. Interestingly, the related literature mainly focused on reducing the computational and memory demand of the inference phase of machine and deep learning models. At the same time, the training is typically assumed to be carried out in cloud or edge computing systems (due to the larger memory and computational requirements). This assumption results in TML solutions that might become obsolete when the process generating the data is affected by concept drift (e.g., due to periodicity or seasonality effect, faults or malfunctioning affecting sensors or actuators, or changes in the users' behavior), a common situation in real-world application scenarios. For the first time in the literature, this article introduces a TML for concept drift (TML-CD) solution based on deep learning feature extractors and a k -nearest neighbors ( k -NNs) classifier integrating a hybrid adaptation module able to deal with concept drift affecting the data-generating process. This adaptation module continuously updates (in a passive way) the knowledge base of TML-CD and, at the same time, employs a change detection test (CDT) to inspect for changes (in an active way) to quickly adapt to concept drift by removing obsolete knowledge. Experimental results on both image and audio benchmarks show the effectiveness of the proposed solution, whilst the porting of TML-CD on three off-the-shelf micro-controller units (MCUs) shows the feasibility of what is proposed in real-world pervasive systems.

A Clinical Fusion Model Based on Radiomics Features and Deep Learning for Predicting CDKN2A/B Homozygous Deletion Status in IDH-mutant Diffuse Astrocytoma

Purpose: To construct a fusion model for predicting CDKN2A/B homozygous deletion status in patients with isocitrate dehydrogenase (IDH)-mutant diffuse astrocytoma by combining the radiomics features and deep learning (DL). Methods: A total of 200 IDH-mutant astrocytoma (103 CDKN2A/B homozygous deletion (HD) and 97 CDKN2A/B non-homozygous deletion (NHD)) patients were retrospectively enrolled in the training cohort (n = 140) and the external test cohort (n = 60) for the prediction of CDKN2A/B homozygous deletion status in patients with IDH-mutant astrocytoma. DL model was constructed by SE-Net model, radiomics features of different regions (edema, tumor and overall lesion) were extracted using Pyradiomics, and radiomics model was built by selecting 4 features in the edema region and 7 features in the tumor region by the least absolute shrinkage and selection operator (LASSO). Finally, a fusion model was jointly constructed by the DL model, radiomics model, and clinical features. The predictive performance of the 3 models was evaluated using calibration curves and decision curves, and compared with the fusion model. Results: Based on the results of the different models, we finally selected a fusion model consisting of DL model, radiomics model, and clinical features. The fusion model showed the best performance with an area under the curve (AUC) of 0.958 in the training cohort and 0.914 in the test cohort. Conclusions: The clinical fusion model based on radiomics features and DL features showed good performance in predicting CDKN2A/B homozygous deletion status in patients with IDH-mutant diffuse astrocytoma. Key Points: 1) Used DL and radiomics to non-invasively predict the CDKN2A/B homozygous deletion status. 2) The model can predict CDKN2A/B homozygous deletion status in IDH-mutant astrocytoma patients. 3) Our result improved classification accuracy and demonstrated better performance in the fusion model.

Real-world and clinical trial validation of a deep learning radiomic biomarker for PD-(L)1 immune checkpoint inhibitor response in stage IV NSCLC.

102 Background: Immune checkpoint inhibitor (ICI) therapy is standard-of-care for treatment of mutation-negative advanced non-small cell lung cancer (NSCLC). However, given known inaccuracy of PD-(L)1 markers, there is an unmet need to better identify patients most likely to derive clinical benefit from ICI. We developed and externally validated a generalizable CT imaging-based biomarker to predict response to ICI. Methods: We developed and validated a deep learning radiomic biomarker using an internally curated real-world dataset (RWD) of 2,010 stage IV NSCLC patients treated with PD-(L)1 ICIs in academic and community settings from US and Europe. Patients with missing baseline imaging, missing follow-up data, or EGFR/ALK oncogenic driver mutations were excluded, resulting in a total of 1,188 subjects. This RWD consisted of a discovery cohort (Dataset A, N=844) and a temporally distinct holdout cohort (Dataset B, N=344), which were used to generate performance metrics of the biomarker. To test generalizability, we validated our biomarker in a prospective clinical trial dataset evaluating Sasanlimab in PD-(L)1 therapy-naïve, advanced NSCLC patients (NCT02573259, Dataset C, N=54). We utilized a two-stage learning approach to model 6-month PFS. First, we used an independent multi-task deep-learning feature extractor trained on 19,184 whole chest CT scans. Second, we input the extracted features into a Cox proportional hazard (CoxPH) model along with age, sex, and baseline lesion measurements (sum of longest diameter and distant metastases counts), and the model generated a time-dependent PFS function and a response score. We performed 6-fold cross-validation on Dataset A to train and evaluate the models, which were subsequently ensembled and applied to independent Datasets B and C. To assess independence from PD-L1 status and key demographic covariates, we herein report multivariate adjusted hazard ratios (HR) for the group identified as low-risk based on the biomarker. Results: In Dataset A, the biomarker showed a cross-validation PFS adjusted HR of 0.49 (95% CI 0.38-0.63) in the all-comers cohort and 0.28 (0.17-0.46) in the first-line ICI monotherapy cohort (1LMono). In Dataset B, the PFS adjusted HRs were 0.54 (0.35-0.83) in all-comers and 0.18 (0.05-0.61) in 1LMono. In Dataset C, the adjusted HRs were 0.30 (0.14-0.68) for PFS, 0.29 (0.10-0.83) for OS, 0.31 (0.14-0.72) for TTP. Conclusions: In our validations in RWD and clinical trial cohorts, a deep-learning radiomic biomarker based on routine pre-treatment CT scans predicted response to ICI and stratified patients independently from PD-L1 status. This tool may inform clinical decision-making, such as to help guide whether concomitant chemotherapy may not be needed. In future work, we plan to further validate our approach in larger prospective datasets and expand its use to new indications.

Map-based localization for intelligent vehicles based on fusion of multiple visual features in underground parking

The ‘last mile’ problem in intelligent car localization within underground parking lots presents challenges such as localization blind spots and feature extraction difficulties. To tackle these issues, this paper proposes a mapping and multiscale matching localization method based on the fusion of multiple feature sources using a ‘binocular mapping, monocular localization’ approach. By combining traditional features with deep learning features, the proposed method enhances feature matching accuracy and addresses feature extraction challenges. A sparse three-dimensional reconstruction method based on local feature-optical flow is also proposed to mitigate localization blind spots. During the mapping process, the underground parking lot scene is segmented into several nodes to store map information, each consisting of a feature layer, scene structure layer, and trajectory layer. In the multiscale matching localization process, ORB, LoFTR, and BoW multisource features are fused to achieve feature matching, determine the nearest node, and estimate vehicle pose using the PnP model through optical flow. Finally, a localization optimization algorithm outputs the localization result. Experiments were conducted across multiple underground parking lots encompassing a total area of over 100,000 square meters. The results demonstrated that the average error of map construction using the proposed algorithm is approximately 15 mm, and the average error rate of localization is 4.45%.

Deep-learning features based on F18 fluorodeoxyglucose positron emission tomography/computed tomography (18F-FDG PET/CT) to predict preoperative colorectal cancer lymph node metastasis

The objective of this study was to create and authenticate a prognostic model for lymph node metastasis (LNM) in colorectal cancer (CRC) that integrates clinical, radiomics, and deep transfer learning features. In this study, we analyzed data from 119 CRC patients who underwent F18 fluorodeoxyglucose positron emission tomography/computed tomography (18F-FDG PET/CT) scanning. The patient cohort was divided into training and validation subsets in an 8:2 ratio, with an additional 33 external data points for testing. Initially, we conducted univariate analysis to screen clinical parameters. Radiomics features were extracted from manually drawn images using pyradiomics, and deep-learning features, radiomics features, and clinical features were selected using Least Absolute Shrinkage and Selection Operator (LASSO) regression and Spearman correlation coefficient. We then constructed a model by training a support vector machine (SVM), and evaluated the performance of the prediction model by comparing the area under the curve (AUC), sensitivity, and specificity. Finally, we developed nomograms combining clinical and radiological features for interpretation and analysis. The deep learning radiomics (DLR) nomogram model, which was developed by integrating deep learning, radiomics, and clinical features, exhibited excellent performance. The area under the curve was (AUC=0.934, 95% confidence interval [CI]: 0.884-0.983) in the training cohort, (AUC=0.902, 95% CI: 0.769-1.000) in the validation cohort, and (AUC=0.836, 95% CI: 0.673-0.998) in the test cohort. We developed a preoperative predictive machine-learning model using deep transfer learning, radiomics, and clinical features to differentiate LNM status in CRC, aiding in treatment decision-making for patients.

Deep learning nomogram for predicting neoadjuvant chemotherapy response in locally advanced gastric cancer patients

PurposeDeveloped and validated a deep learning radiomics nomogram using multi-phase contrast-enhanced computed tomography (CECT) images to predict neoadjuvant chemotherapy (NAC) response in locally advanced gastric cancer (LAGC) patients.MethodsThis multi-center study retrospectively included 322 patients diagnosed with gastric cancer from January 2013 to June 2023 at two hospitals. Handcrafted radiomics technique and the EfficientNet V2 neural network were applied to arterial, portal venous, and delayed phase CT images to extract two-dimensional handcrafted and deep learning features. A nomogram model was built by integrating the handcrafted signature, the deep learning signature, with clinical features. Discriminative ability was assessed using the receiver operating characteristics (ROC) curve and the precision-recall (P-R) curve. Model fitting was evaluated using calibration curves, and clinical utility was assessed through decision curve analysis (DCA).ResultsThe nomogram exhibited excellent performance. The area under the ROC curve (AUC) was 0.848 [95% confidence interval (CI), 0.793–0.893)], 0.802 (95% CI 0.688–0.889), and 0.751 (95% CI 0.652–0.833) for the training, internal validation, and external validation sets, respectively. The AUCs of the P-R curves were 0.838 (95% CI 0.756–0.895), 0.541 (95% CI 0.329–0.740), and 0.556 (95% CI 0.376–0.722) for the corresponding sets. The nomogram outperformed the clinical model and handcrafted signature across all sets (all P < 0.05). The nomogram model demonstrated good calibration and provided greater net benefit within the relevant threshold range compared to other models.ConclusionThis study created a deep learning nomogram using CECT images and clinical data to predict NAC response in LAGC patients undergoing surgical resection, offering personalized treatment insights.Graphical abstract