Three-dimensional Convolutional Neural Network Research Articles

Advancing EGFR mutation subtypes prediction in NSCLC by combining 3D pretrained ConvNeXt, radiomics, and clinical features

PurposeThe aim of this study was to develop a novel approach for predicting the expression status of Epidermal Growth Factor Receptor (EGFR) and its subtypes in patients with Non-Small Cell Lung Cancer (NSCLC) using a Three-Dimensional Convolutional Neural Network (3D-CNN) ConvNeXt, radiomics features and clinical features.Materials and methodsA total of 732 NSCLC patients with available CT imaging and EGFR expression data were included in this retrospective study. The region of interest (ROI) was manually segmented, and clinicopathological features were collected. Radiomic and deep learning features were extracted. The instances were randomly divided into training, validation, and test sets. Feature selection was performed, and XGBoost was used to create solo models and combined models to predict the presence of EGFR and subtypes mutations. The effectiveness of the models was assessed using ROC and PRC curves.ResultsWe established the following models: ModelCNN, Modelradiomic, Modelclinical, ModelCNN+radiomic, ModelCNN+clinical, Modelradiomic+clinical, and ModelCNN+radiomic+clinical, which were based on deep learning features, radiomic features, clinical data and combinations of these, respectively. In predicting EGFR mutations, ModelCNN+radiomic+clinical demonstrated superior performance compared to other prediction models, achieving an AUC of 0.801. For distinguishing between EGFR subtypes ex19del and L858R, ModelCNN+radiomic reached the highest AUC value of 0.775.ConclusionsBoth deep learning models and radiomic signature-based models offer reasonably accurate non-invasive predictions of EGFR status and its subtypes. Fusion models hold the potential to enhance noninvasive methods for predicting EGFR mutations and subtypes, presenting a more reliable prediction approach.

Quantum machine learning enhanced laser speckle analysis for precise speed prediction.

Laser speckle contrast imaging (LSCI) is an optical technique used to assess blood flow perfusion by modeling changes in speckle intensity, but it is generally limited to qualitative analysis due to difficulties in absolute quantification. Three-dimensional convolutional neural networks (3D CNNs) enhance the quantitative performance of LSCI by excelling at extracting spatiotemporal features from speckle data. However, excessive downsampling techniques can lead to significant information loss. To address this, we propose a hybrid quantum-classical 3D CNN framework that leverages variational quantum algorithms (VQAs) to enhance the performance of classical models. The proposed framework employs variational quantum circuits (VQCs) to replace the 3D global pooling layer, enabling the model to utilize the complete 3D information extracted by the convolutional layers for feature integration, thereby enhancing velocity prediction performance. We perform cross-validation on experimental LSCI speckle data and demonstrate the superiority of the hybrid models over their classical counterparts in terms of prediction accuracy and learning stability. Furthermore, we evaluate the models on an unseen test set and observe that the hybrid models outperform the classical models with up to 14.8% improvement in mean squared error (MSE) and up to 26.1% improvement in mean absolute percentage error (MAPE) evaluation metrics. Finally, our qualitative analysis shows that the hybrid models offer substantial improvements over classical models in predicting blood flow at both low and high velocities. These results indicate that the hybrid models possess more powerful learning and generalization capabilities.

Continuous sign language recognition algorithm based on object detection and variable-length coding sequence.

Currently, continuous sign language recognition faces challenges such as difficulty in acquiring skeletal data, long training time for Three-Dimensional convolutional neural networks, and easy occlusion and blurring of hands. To address these problems, this paper proposes a continuous sign language recognition method based on target detection and coding sequence. The algorithm uses Dual-branch Shuffle Attention Mechanis-You Only Look Once version X (DSA-YOLOX) detection network to detect the head and hands, and encodes the sign language video according to the partition to achieve the transformation from Three-Dimensional to One-Dimensional; and then uses the proposed Bi-directional Long Short-Term Memory (BiLSTM) hand coding sequence classification model with jointly weighted Fast Dynamic Time Warping (FastDTW) to extract hand coding similarity and features while reducing the number of parameters to achieve the classification and recognition of unequal-length hand coding sequences. From the results of ablation experiments and comparison experiments, all parts of the improvement perform well. The word error rate (WER) of this paper's method is reduced by 21.26% compared to Dynamic Time Warping-Hidden Markov Model (DTW-HMM) and 11.53% compared to Long Short-Term Memory-A(LSTM-A); the Giga Floating-point Operations Per Second(GFLOPs) of the algorithm are reduced dramatically, which is about 1/13 of the Visual Alignment Constraint(VAC) model and 1/57 of the Spatial-Temporal Multi-Cue(STMC) model; and the algorithm takes better account of the speed and accuracy of sign language recognition.

Advanced obstacle recognition and trajectory prediction for autonomous vehicles using 3D CNN and GRU

Abstract. Recent years have seen a rapid advancement in new energy vehicle technology, which has increased consumer demand for autonomous driving systems. However, current self-driving cars still face many technical barriers. Obstacle detection and obstacle trajectory prediction in dynamic environments is one of the main difficulties of the technique. To improve self-driving cars' capacity for route planning and decision-making in dynamic contexts, this thesis proposes an obstacle identification and trajectory prediction model that combines a gated recurrent unit (GRU) network with a three-dimensional convolutional neural network (3D CNN). The integrated framework utilises the feature recognition capability of 3D CNN and the prediction capability of GRU to accurately predict dynamic obstacles, thereby improving the performance of the model in various types of environments (e.g., visual SLAM). Experimental results based on publicly available datasets (e.g., KITTI and Argoverse) show that the proposed method has significant advantages in dynamic obstacle identification and prediction as well as overall model operation efficiency. The findings show that the integrated framework is useful and successful in advancing the creation of self-driving automobiles.

Heart Rate Estimation Algorithm Integrating Long and Short-Term Temporal Features

Non-contact heart rate monitoring from facial videos utilizing remote photoplethysmography (rPPG) has gained significant traction in remote health monitoring. Given that rPPG captures the dynamic blood flow within the human body and constitutes a time-series signal characterized by periodic properties, this study introduced a three-dimensional convolutional neural network (3D CNN) designed to simultaneously address long-term periodic and short-term temporal characteristics for effective rPPG signal extraction. Firstly, differential operations are employed to preprocess video data, enhancing the face’s dynamic features. Secondly, building upon the 3D CNN framework, multi-scale dilated convolutions and self-attention mechanisms were integrated to enhance the model’s temporal modeling capabilities further. Finally, interpolation techniques are applied to refine the heart rate calculation methodology. The experiments conducted on the UBFC-rPPG dataset indicate that, compared with the existing optimal algorithm, the average absolute error (MAE) and the root mean square error (RMSE) realized significant enhancements of approximately 28% and 35%. Additionally, through comprehensive analyses such as cross-dataset experiments and complexity analyses, the validity and stability of the proposed algorithm in the task of heart rate estimation were manifested.

Research on elastic parameter inversion method based on seismic facies-controlled deep learning network

Deep learning has been widely applied in the field of geophysics, and existing research literature has demonstrated that intelligent geophysical inversion methods have high vertical resolution but low horizontal resolution. The reason lies in the fact that existing horizontal constraint methods mainly adopt convolutional models, without fully considering other prior information of seismic data. Within the same sedimentary unit, seismic response characteristics vary gradually due to similar lithology and geological characteristics. Therefore, the seismic facies information extracted from seismic data is integrated into deep learning network to enhance the horizontal prediction stability of the network. Firstly, according to the spatial and temporal characteristics of seismic data, a fusion network of three-dimensional convolutional neural network (3D-CNN), gated recurrent unit (GRU) and attention mechanism is established to improve the vertical resolution of inversion results. Then, seismic facies classification of the target layer is achieved by applying the K-means clustering method. Subsequently, to improve the horizontal resolution of the inversion results, seismic facies classification is transformed into temporal encoding data using the position coding theory in natural language processing, to form a seismic facies-controlled deep learning network. Finally, the deep learning network is trained and tested in the thin interlayer model and practical application adopting a semi-supervised learning method. The results indicate that incorporating seismic facies-controlled technology in the deep learning network can improve the horizontal resolution of the inversion results.

Three-dimensional convolutional neural network for leak detection and localization in smart water distribution systems

Smart meters such as advanced metering infrastructure (AMI) can significantly improve identifying realistic sized leaks in water distribution networks (WDNs). However, to date, detection/localization methods for AMI systems are extremely limited. In this study, to examine the benefits of using AMIs for leak detection within distribution network, a three-dimensional (3D) convolutional neural network (CNN) deep learning (DL) model is proposed that can account for temporally and spatially distributed information of pressures. The 3D CNN is tested for a real WDN in Austin using the realistic sized leaks (e.g., 3 L/s for 150-mm pipes) that are generated from hydraulic simulations. The model's performance is evaluated using detection probability, false alarm rate, and localization pipe distance metrics. In addition, the strength of using DL for leak identification is examined by comparing the CNN results with those from an optimization-based model. The 3D CNN performed better than the optimization model indicating that DL has advantages over conventional tools such as optimization methods. However, its adaptability may limit its use in some cases. Since DL can be significantly impacted by hydraulic simulation model, a way to handle modelling error must be determined. In addition, as network changes occur, retraining is required that may be time consuming and have difficulty with the number of failure conditions.

Research on a three-dimensional radiation field reconstruction algorithm based on an improved 3D CNN

This article investigates the application of an improved three-dimensional convolutional neural network (3D CNN) for sparse data-based reconstruction of radiation fields. Sparse radiation data points are consolidated into structured three-dimensional matrices and fed into a self-attention integrated CNN, enabling the network to interpolate and produce complete radiation distribution grids. The model’s validity is assessed through experiments with randomly sourced radiation in scenarios both with and without shielding, as well as in refined grid configurations. Results indicate that in unshielded environments, a mere 5%(15 points) sampling yields an average relative error of 4%, while in shielded settings, a 7% (21 points) sampling maintains the error around 11%. In refined grid contexts, a 2% sampling rate suffices to limit the error to 6.58%. Thus, the improved 3D CNN is demonstrated to be highly effective for precise three-dimensional radiation field reconstruction in sparse data scenarios.

A one stream three-dimensional convolutional neural network for fire recognition based on spatio-temporal fire analysis

A one stream three-dimensional convolutional neural network for fire recognition based on spatio-temporal fire analysis

Influential mechanism of water occurrence states of waste-activated sludge: Specifically focusing on the pore characteristics dominated by cation-organic interactions

The solid pore characteristics are commonly considered as the important influential factors on waste-activated sludge (WAS) dewaterability, and should be related to the cohesive force of bio-flocs dominated by cation-organic interactions at solid-water interface. This study aimed to establish an approach for regulating the solid pore structure of WAS by cationic regulation. The influential mechanism of WAS dewaterability was accordingly explored from the perspective of the pore characteristics dominated by cation-organic interactions. Primarily, with the gradient removal or addition of bivalent cations, the varying pore structure of WAS flocs was tracked by in-situ synchrotron X-ray computed microtomography imaging technique (CMT). The three-dimensional visual model was established to quantify the pore structure parameters of WAS flocs. Following the visualization analysis, the artificial intelligence means, the gradient-weighted class activation mapping (Grad CAM) of three-dimensional convolutional neural network (3D-CNN), was applied for the first time to explore the linkages among solid surface properties, solid pore structure, water occurrence states and sludge dewaterability. It was found that the number and volume of isolated pores jointly determined the mobility and the fractions of vicinal water and interstitial water (p-value ≤ 0.02); also, the decreasing polar or acid-based interfacial free energy with the cationic addition was accompanied with the decreasing isolated pore mean-volume (Pearson coefficient=-0.77, p-value < 0.01). These results indicated that the pore structure characteristics determined the water occurrence states, but the solid porosity strongly depended on the interfacial properties. Accordingly, the molecular docking was applied to explore the interfacial reaction mechanism between Ca2+/Mg2+ and solid compositions in terms of complexation sites, molecular dynamics and free energy calculations. As a result, how the cation-organic interactions affected the pore characteristics through solid surface modification could be clarified, which is expected to serve as theoretical foundation for the development of novel sludge conditioning technologies, i.e., more efforts should be devoted to increasing the dense degree of sludge particles through weakening the hydration repulsion of solid surface.

Advancing Prostate Cancer Diagnosis: A Deep Learning Approach for Enhanced Detection in MRI Images.

Prostate cancer remains a leading cause of mortality among men globally, necessitating advancements in diagnostic methodologies to improve detection and treatment outcomes. Magnetic Resonance Imaging has emerged as a crucial technique for the detection of prostate cancer, with current research focusing on the integration of deep learning frameworks to refine this diagnostic process. This study employs a comprehensive approach using multiple deep learning models, including a three-dimensional (3D) Convolutional Neural Network, a Residual Network, and an Inception Network to enhance the accuracy and robustness of prostate cancer detection. By leveraging the complementary strengths of these models through an ensemble method and soft voting technique, the study aims to achieve superior diagnostic performance. The proposed methodology demonstrates state-of-the-art results, with the ensemble model achieving an overall accuracy of 91.3%, a sensitivity of 90.2%, a specificity of 92.1%, a precision of 89.8%, and an F1 score of 90.0% when applied to MRI images from the SPIE-AAPM-NCI PROSTATEx dataset. Evaluation of the models involved meticulous pre-processing, data augmentation, and the use of advanced deep-learning architectures to analyze the whole MRI slices and volumes. The findings highlight the potential of using an ensemble approach to significantly improve prostate cancer diagnostics, offering a robust and precise tool for clinical applications.

Spatial-spectral feature extraction for in-field chlorophyll content estimation using hyperspectral imaging

In-situ leaf chlorophyll content (LCC) estimation based on hyperspectral imaging (HSI) is crucial to track the growth status of crops for field management. However, spatial and spectral features of HSI data, suffering from interference of growth dynamic effect and soil, pose the challenge on accuracy and robustness of LCC estimation in several years and growth stages. Therefore, a joint spectral-spatial feature extraction method was proposed by cascade of three-dimensional convolutional neural network (3DCNN) and long short-term memory (LSTM) to reduce the interference for optimising the LCC estimation. Firstly, crop pixels were separated from soil with vegetation index segmentation method. Secondly, when raw images and segmented pixels were input, sensitive bands were selected by random frog (RF bands), and 3DCNN-LSTM was used to extract the joint spectral-spatial features. Finally, models established by RF bands, 3DCNN and 3DCNN-LSTM were compared, and robustness in individual years and stages was validated. Results showed that RF bands and 3DCNN obtained RP2 of 0.76 and 0.84 when not segmented. After segmentation, performance of 3DCNN improved (RP2 = 0.85) compared to RF bands (RP2 = 0.80). Spectral-spatial features by 3DCNN reduced the interference of soil. 3DCNN-LSTM without and with segmentation obtained good performance with RP2 of 0.95 and 0.96, and the proposed method could reduce the image segmentation process. The optimal model achieved RP2 above 0.93 in individual years (RP2 = 0.96 in 2021, RP2 = 0.94 in 2021) and RP2 in the range of 0.87–0.97 at individual stages. This paper provides a method to track growth variability between soil and crop for the LCC estimation optimisation.

Enhancing Brain Tumor Classification with a Novel Three-Dimensional Convolutional Neural Network (3D-CNN) Fusion Model

Three-dimensional convolutional neural networks (3D CNNs) have been widely applied to analyze brain tumour images (BT) to understand the disease's progress better. It is well-known that training 3D-CNN is computationally expensive and has the potential of overfitting due to the small sample size available in the medical imaging field. Here, we proposed a novel 2D-3D approach by converting a 2D brain image to a 3D fused image using a gradient of the image Learnable Weighted. By the 2D-to-3D conversion, the proposed model can easily forward the fused 3D image through a pre-trained 3D model while achieving better performance over different 3D baselines. We used VGG16 for feature extraction in the implementation as it outperformed other 3D CNN backbones. We further showed that the weights of the slices are location-dependent, and the model performance relies on the 3D-to-2D fusion view, with the best outcomes from the coronal view. With the new approach, we increased the accuracy to 0.88, compared with conventional 3D CNNs, for classifying brain tumour images. The novel 2D-3D model may have profound implications for future timely BT diagnosis in clinical settings.

Multi-resolution short-time Fourier transform providing deep features for 3D CNN to classify rolling bearing fault vibration signals

The time-frequency domain features of vibration signals provide valuable information for deep learning-based rolling bearing fault diagnosis methods, where fault signal classification aiding in the identification of nominal fault types during diagnosis. The Short-Time Fourier Transform (STFT) is a widely used time-frequency transformation method, and its window length is the key parameter that determines the trade-off between time and frequency resolution. The primary motivation of this study is to address the limitation in traditional STFT-based 2D CNN methods: the inability to adapt the window length to different types of signals. To achieve accurate classification of bearing fault types, this study proposes a method based on three-dimensional convolutional neural networks (3D CNNs) to deeply explore the time-frequency domain information of one-dimensional vibration signals from faulty bearings. This method first applies STFT with multiple window sizes to perform multi-resolution time-frequency transformations on the time-domain vibration signals, yielding three-dimensional data. Subsequently, a classifier is trained based on the proposed 3D CNN. Experimental results on public datasets show that, without any sophisticated techniques, the proposed method achieves an average classification accuracy of 99.2% for six types of bearing faults using a relatively simple CNN structure. Compared to 1D CNN and 2D CNN methods that use fixed window sizes for STFT, the proposed method significantly enhances classification performance. Furthermore, it demonstrates robust classification results even on small-scaled bearing datasets.

Advancements in Real-Time Human Activity Recognition via Innovative Fusion of 3DCNN and ConvLSTM Models

Object detection (OD) is a computer vision procedure for locating objects in digital images. Our study examines the crucial need for robust OD algorithms in human activity recognition, a vital domain spanning human-computer interaction, sports analysis, and surveillance. Nowadays, three-dimensional convolutional neural networks (3DCNNs) are a standard method for recognizing human activity. Utilizing recent advances in Deep Learning (DL), we present a novel framework designed to create a fusion model that enhances conventional methods at integrates three-dimensional convolutional neural networks (3DCNNs) with Convolutional Long-Short-Term Memory (ConvLSTM) layers. Our proposed model focuses on utilizing the spatiotemporal features innately present in video streams. An important aspect often missed in existing OD methods. We assess the efficacy of our proposed architecture employing the UCF-50 dataset, which is well-known for its different range of human activities. In addition to designing a novel deep-learning architecture, we used data augmentation techniques that expand the dataset, improve model robustness, reduce overfitting, extend dataset size, and enhance performance on imbalanced data. The proposed model demonstrated outstanding performance through comprehensive experimentation, achieving an impressive accuracy of 98.11% in classifying human activity. Furthermore, when benchmarked against state-of-the-art methods, our system provides adequate accuracy and class average for 50 activity categories.

Fault detection and identification method: 3D-CNN combined with continuous wavelet transform

This study proposes a novel fault detection and identification method using Continuous Wavelet Transform (CWT) and a three-dimensional Convolutional Neural Network (3D-CNN). In particular, multivariate time series data from chemical plants were divided by a time-shifting window and transformed into scalograms using CWT. These scalograms were fed to 3D-CNN to generate outputs indicating the faults that occurred in the process. We applied the proposed method to a Tennessee Eastman process dataset. The proposed method adequately captures the characteristics in the time–frequency domain and exhibited good fault detection and identification performances on the dataset.

Identification of Calculous Pyonephrosis by CT-Based Radiomics and Deep Learning.

Urgent detection of calculous pyonephrosis is crucial for surgical planning and preventing severe outcomes. This study aims to evaluate the performance of computed tomography (CT)-based radiomics and a three-dimensional convolutional neural network (3D-CNN) model, integrated with independent clinical factors, to identify patients with calculous pyonephrosis. We recruited 182 patients receiving either percutaneous nephrostomy tube placement or percutaneous nephrolithotomy for calculous hydronephrosis or pyonephrosis. The regions of interest were manually delineated on plain CT images and the CT attenuation value (HU) was measured. Radiomics analysis was performed using least absolute shrinkage and selection operator (LASSO). A 3D-CNN model was also developed. The better-performing machine-learning model was combined with independent clinical factors to build a comprehensive clinical machine-learning model. The performance of these models was assessed using receiver operating characteristic analysis and decision curve analysis. Fever, blood neutrophils, and urine leukocytes were independent risk factors for pyonephrosis. The radiomics model showed higher area under the curve (AUC) than the 3D-CNN model and HU (0.876 vs. 0.599, 0.578; p = 0.003, 0.002) in the testing cohort. The clinical machine-learning model surpassed the clinical model in both the training (0.975 vs. 0.904, p = 0.019) and testing (0.967 vs. 0.889, p = 0.045) cohorts.

An ensemble model for assisting early Alzheimer's disease diagnosis based on structural magnetic resonance imaging with dual-time-point fusion

Alzheimer's Disease (AD) is a progressive neurodegenerative disorder. Due to the subtlety of symptoms in the early stages of AD, rapid and accurate clinical diagnosis is challenging, leading to a high rate of misdiagnosis. Current research on early diagnosis of AD has not sufficiently focused on tracking the progression of the disease over an extended period in subjects. To address this issue, this paper proposes an ensemble model for assisting early diagnosis of AD that combines structural magnetic resonance imaging (sMRI) data from two time points with clinical information. The model employs a three-dimensional convolutional neural network (3DCNN) and twin neural network modules to extract features from the sMRI data of subjects at two time points, while a multi-layer perceptron (MLP) is used to model the clinical information of the subjects. The objective is to extract AD-related features from the multi-modal data of the subjects as much as possible, thereby enhancing the diagnostic performance of the ensemble model. Experimental results show that based on this model, the classification accuracy rate is 89% for differentiating AD patients from normal controls (NC), 88% for differentiating mild cognitive impairment converting to AD (MCIc) from NC, and 69% for distinguishing non-converting mild cognitive impairment (MCInc) from MCIc, confirming the effectiveness and efficiency of the proposed method for early diagnosis of AD, as well as its potential to play a supportive role in the clinical diagnosis of early Alzheimer's disease.

Method for measuring blood flow and depth of blood vessel based on laser speckle contrast imaging using 3D convolutional neural network: A preliminary study

Laser speckle contrast imaging (LSCI) is a technology that can acquire real-time two-dimensional perfusion images non-invasively. However, measuring perfusion velocity under a tissue is difficult due to its optical properties. Dynamic scattered light occurs by perfusion is diffused by the surrounding tissue and it depends on the depth of tissue. We aimed to develop a three-dimensional convolutional neural network (3D-CNN) model based on laser speckle contrast images to measure the flow velocity and the depth of tissue using a flow phantom. The training data was selected as a region containing both static speckles and dynamic speckles, and the model was trained on the laser speckle contrast image data. The trained model showed a measurement accuracy of more than 90 % for velocity and 99 % for depth measurement. This study has the potential of a deep-learning model based on LSCI to analyze the blood flow and depth of blood vessels in bio-applications.

Automatic detection of pulmonary embolism on computed tomography pulmonary angiogram scan using a three-dimensional convolutional neural network

ObjectiveTo propose a convolutional neural network (EmbNet) for automatic pulmonary embolism detection on computed tomography pulmonary angiogram (CTPA) scans and to assess its diagnostic performance. Methods305 consecutive CTPA scans between January 2019 and December 2021 were enrolled in this study (142 for training, 163 for internal validation), and 250 CTPA scans from a public dataset were used for external validation. The framework comprised a preprocessing step to segment the pulmonary vessels and the EmbNet to detect emboli. Emboli were divided into three location-based subgroups for detailed evaluation: central arteries, lobar branches, and peripheral regions. Ground truth was established by three radiologists. ResultsThe EmbNet’s per-scan level sensitivity, specificity, positive predictive value (PPV), and negative predictive value were 90.9%, 75.4%, 48.4%, and 97.0% (internal validation) and 88.0%, 70.5%, 42.7%, and 95.9% (external validation). At the per-embolus level, the overall sensitivity and PPV of the EmbNet were 86.0% and 61.3% (internal validation), and 83.5% and 57.5% (external validation). The sensitivity and PPV of central emboli were 89.7% and 52.0% (internal validation), and 94.4% and 43.0% (external validation); of lobar emboli were 95.2% and 76.9% (internal validation), and 93.5% and 72.5% (external validation); and of peripheral emboli were 82.6% and 61.7% (internal validation), and 80.2% and 59.4% (external validation). The average false positive rate was 0.45 false emboli per scan (internal validation) and 0.69 false emboli per scan (external validation). ConclusionThe EmbNet provides high sensitivity across embolus locations, suggesting its potential utility for initial screening in clinical practice.