Three-dimensional Convolutional Neural Network Research Articles

A one stream three-dimensional convolutional neural network for fire recognition based on spatio-temporal fire analysis

A one stream three-dimensional convolutional neural network for fire recognition based on spatio-temporal fire analysis

Influential mechanism of water occurrence states of waste-activated sludge: Specifically focusing on the pore characteristics dominated by cation-organic interactions

The solid pore characteristics are commonly considered as the important influential factors on waste-activated sludge (WAS) dewaterability, and should be related to the cohesive force of bio-flocs dominated by cation-organic interactions at solid-water interface. This study aimed to establish an approach for regulating the solid pore structure of WAS by cationic regulation. The influential mechanism of WAS dewaterability was accordingly explored from the perspective of the pore characteristics dominated by cation-organic interactions. Primarily, with the gradient removal or addition of bivalent cations, the varying pore structure of WAS flocs was tracked by in-situ synchrotron X-ray computed microtomography imaging technique (CMT). The three-dimensional visual model was established to quantify the pore structure parameters of WAS flocs. Following the visualization analysis, the artificial intelligence means, the gradient-weighted class activation mapping (Grad CAM) of three-dimensional convolutional neural network (3D-CNN), was applied for the first time to explore the linkages among solid surface properties, solid pore structure, water occurrence states and sludge dewaterability. It was found that the number and volume of isolated pores jointly determined the mobility and the fractions of vicinal water and interstitial water (p-value ≤ 0.02); also, the decreasing polar or acid-based interfacial free energy with the cationic addition was accompanied with the decreasing isolated pore mean-volume (Pearson coefficient=-0.77, p-value < 0.01). These results indicated that the pore structure characteristics determined the water occurrence states, but the solid porosity strongly depended on the interfacial properties. Accordingly, the molecular docking was applied to explore the interfacial reaction mechanism between Ca2+/Mg2+ and solid compositions in terms of complexation sites, molecular dynamics and free energy calculations. As a result, how the cation-organic interactions affected the pore characteristics through solid surface modification could be clarified, which is expected to serve as theoretical foundation for the development of novel sludge conditioning technologies, i.e., more efforts should be devoted to increasing the dense degree of sludge particles through weakening the hydration repulsion of solid surface.

Advancing Prostate Cancer Diagnosis: A Deep Learning Approach for Enhanced Detection in MRI Images.

Prostate cancer remains a leading cause of mortality among men globally, necessitating advancements in diagnostic methodologies to improve detection and treatment outcomes. Magnetic Resonance Imaging has emerged as a crucial technique for the detection of prostate cancer, with current research focusing on the integration of deep learning frameworks to refine this diagnostic process. This study employs a comprehensive approach using multiple deep learning models, including a three-dimensional (3D) Convolutional Neural Network, a Residual Network, and an Inception Network to enhance the accuracy and robustness of prostate cancer detection. By leveraging the complementary strengths of these models through an ensemble method and soft voting technique, the study aims to achieve superior diagnostic performance. The proposed methodology demonstrates state-of-the-art results, with the ensemble model achieving an overall accuracy of 91.3%, a sensitivity of 90.2%, a specificity of 92.1%, a precision of 89.8%, and an F1 score of 90.0% when applied to MRI images from the SPIE-AAPM-NCI PROSTATEx dataset. Evaluation of the models involved meticulous pre-processing, data augmentation, and the use of advanced deep-learning architectures to analyze the whole MRI slices and volumes. The findings highlight the potential of using an ensemble approach to significantly improve prostate cancer diagnostics, offering a robust and precise tool for clinical applications.

Spatial-spectral feature extraction for in-field chlorophyll content estimation using hyperspectral imaging

In-situ leaf chlorophyll content (LCC) estimation based on hyperspectral imaging (HSI) is crucial to track the growth status of crops for field management. However, spatial and spectral features of HSI data, suffering from interference of growth dynamic effect and soil, pose the challenge on accuracy and robustness of LCC estimation in several years and growth stages. Therefore, a joint spectral-spatial feature extraction method was proposed by cascade of three-dimensional convolutional neural network (3DCNN) and long short-term memory (LSTM) to reduce the interference for optimising the LCC estimation. Firstly, crop pixels were separated from soil with vegetation index segmentation method. Secondly, when raw images and segmented pixels were input, sensitive bands were selected by random frog (RF bands), and 3DCNN-LSTM was used to extract the joint spectral-spatial features. Finally, models established by RF bands, 3DCNN and 3DCNN-LSTM were compared, and robustness in individual years and stages was validated. Results showed that RF bands and 3DCNN obtained RP2 of 0.76 and 0.84 when not segmented. After segmentation, performance of 3DCNN improved (RP2 = 0.85) compared to RF bands (RP2 = 0.80). Spectral-spatial features by 3DCNN reduced the interference of soil. 3DCNN-LSTM without and with segmentation obtained good performance with RP2 of 0.95 and 0.96, and the proposed method could reduce the image segmentation process. The optimal model achieved RP2 above 0.93 in individual years (RP2 = 0.96 in 2021, RP2 = 0.94 in 2021) and RP2 in the range of 0.87–0.97 at individual stages. This paper provides a method to track growth variability between soil and crop for the LCC estimation optimisation.

Enhancing Brain Tumor Classification with a Novel Three-Dimensional Convolutional Neural Network (3D-CNN) Fusion Model

Three-dimensional convolutional neural networks (3D CNNs) have been widely applied to analyze brain tumour images (BT) to understand the disease's progress better. It is well-known that training 3D-CNN is computationally expensive and has the potential of overfitting due to the small sample size available in the medical imaging field. Here, we proposed a novel 2D-3D approach by converting a 2D brain image to a 3D fused image using a gradient of the image Learnable Weighted. By the 2D-to-3D conversion, the proposed model can easily forward the fused 3D image through a pre-trained 3D model while achieving better performance over different 3D baselines. We used VGG16 for feature extraction in the implementation as it outperformed other 3D CNN backbones. We further showed that the weights of the slices are location-dependent, and the model performance relies on the 3D-to-2D fusion view, with the best outcomes from the coronal view. With the new approach, we increased the accuracy to 0.88, compared with conventional 3D CNNs, for classifying brain tumour images. The novel 2D-3D model may have profound implications for future timely BT diagnosis in clinical settings.

Multi-resolution short-time Fourier transform providing deep features for 3D CNN to classify rolling bearing fault vibration signals

The time-frequency domain features of vibration signals provide valuable information for deep learning-based rolling bearing fault diagnosis methods, where fault signal classification aiding in the identification of nominal fault types during diagnosis. The Short-Time Fourier Transform (STFT) is a widely used time-frequency transformation method, and its window length is the key parameter that determines the trade-off between time and frequency resolution. The primary motivation of this study is to address the limitation in traditional STFT-based 2D CNN methods: the inability to adapt the window length to different types of signals. To achieve accurate classification of bearing fault types, this study proposes a method based on three-dimensional convolutional neural networks (3D CNNs) to deeply explore the time-frequency domain information of one-dimensional vibration signals from faulty bearings. This method first applies STFT with multiple window sizes to perform multi-resolution time-frequency transformations on the time-domain vibration signals, yielding three-dimensional data. Subsequently, a classifier is trained based on the proposed 3D CNN. Experimental results on public datasets show that, without any sophisticated techniques, the proposed method achieves an average classification accuracy of 99.2% for six types of bearing faults using a relatively simple CNN structure. Compared to 1D CNN and 2D CNN methods that use fixed window sizes for STFT, the proposed method significantly enhances classification performance. Furthermore, it demonstrates robust classification results even on small-scaled bearing datasets.

Advancements in Real-Time Human Activity Recognition via Innovative Fusion of 3DCNN and ConvLSTM Models

Object detection (OD) is a computer vision procedure for locating objects in digital images. Our study examines the crucial need for robust OD algorithms in human activity recognition, a vital domain spanning human-computer interaction, sports analysis, and surveillance. Nowadays, three-dimensional convolutional neural networks (3DCNNs) are a standard method for recognizing human activity. Utilizing recent advances in Deep Learning (DL), we present a novel framework designed to create a fusion model that enhances conventional methods at integrates three-dimensional convolutional neural networks (3DCNNs) with Convolutional Long-Short-Term Memory (ConvLSTM) layers. Our proposed model focuses on utilizing the spatiotemporal features innately present in video streams. An important aspect often missed in existing OD methods. We assess the efficacy of our proposed architecture employing the UCF-50 dataset, which is well-known for its different range of human activities. In addition to designing a novel deep-learning architecture, we used data augmentation techniques that expand the dataset, improve model robustness, reduce overfitting, extend dataset size, and enhance performance on imbalanced data. The proposed model demonstrated outstanding performance through comprehensive experimentation, achieving an impressive accuracy of 98.11% in classifying human activity. Furthermore, when benchmarked against state-of-the-art methods, our system provides adequate accuracy and class average for 50 activity categories.

Fault detection and identification method: 3D-CNN combined with continuous wavelet transform

This study proposes a novel fault detection and identification method using Continuous Wavelet Transform (CWT) and a three-dimensional Convolutional Neural Network (3D-CNN). In particular, multivariate time series data from chemical plants were divided by a time-shifting window and transformed into scalograms using CWT. These scalograms were fed to 3D-CNN to generate outputs indicating the faults that occurred in the process. We applied the proposed method to a Tennessee Eastman process dataset. The proposed method adequately captures the characteristics in the time–frequency domain and exhibited good fault detection and identification performances on the dataset.

Identification of Calculous Pyonephrosis by CT-Based Radiomics and Deep Learning.

Urgent detection of calculous pyonephrosis is crucial for surgical planning and preventing severe outcomes. This study aims to evaluate the performance of computed tomography (CT)-based radiomics and a three-dimensional convolutional neural network (3D-CNN) model, integrated with independent clinical factors, to identify patients with calculous pyonephrosis. We recruited 182 patients receiving either percutaneous nephrostomy tube placement or percutaneous nephrolithotomy for calculous hydronephrosis or pyonephrosis. The regions of interest were manually delineated on plain CT images and the CT attenuation value (HU) was measured. Radiomics analysis was performed using least absolute shrinkage and selection operator (LASSO). A 3D-CNN model was also developed. The better-performing machine-learning model was combined with independent clinical factors to build a comprehensive clinical machine-learning model. The performance of these models was assessed using receiver operating characteristic analysis and decision curve analysis. Fever, blood neutrophils, and urine leukocytes were independent risk factors for pyonephrosis. The radiomics model showed higher area under the curve (AUC) than the 3D-CNN model and HU (0.876 vs. 0.599, 0.578; p = 0.003, 0.002) in the testing cohort. The clinical machine-learning model surpassed the clinical model in both the training (0.975 vs. 0.904, p = 0.019) and testing (0.967 vs. 0.889, p = 0.045) cohorts.

An ensemble model for assisting early Alzheimer's disease diagnosis based on structural magnetic resonance imaging with dual-time-point fusion

Alzheimer's Disease (AD) is a progressive neurodegenerative disorder. Due to the subtlety of symptoms in the early stages of AD, rapid and accurate clinical diagnosis is challenging, leading to a high rate of misdiagnosis. Current research on early diagnosis of AD has not sufficiently focused on tracking the progression of the disease over an extended period in subjects. To address this issue, this paper proposes an ensemble model for assisting early diagnosis of AD that combines structural magnetic resonance imaging (sMRI) data from two time points with clinical information. The model employs a three-dimensional convolutional neural network (3DCNN) and twin neural network modules to extract features from the sMRI data of subjects at two time points, while a multi-layer perceptron (MLP) is used to model the clinical information of the subjects. The objective is to extract AD-related features from the multi-modal data of the subjects as much as possible, thereby enhancing the diagnostic performance of the ensemble model. Experimental results show that based on this model, the classification accuracy rate is 89% for differentiating AD patients from normal controls (NC), 88% for differentiating mild cognitive impairment converting to AD (MCIc) from NC, and 69% for distinguishing non-converting mild cognitive impairment (MCInc) from MCIc, confirming the effectiveness and efficiency of the proposed method for early diagnosis of AD, as well as its potential to play a supportive role in the clinical diagnosis of early Alzheimer's disease.

Method for measuring blood flow and depth of blood vessel based on laser speckle contrast imaging using 3D convolutional neural network: A preliminary study

Laser speckle contrast imaging (LSCI) is a technology that can acquire real-time two-dimensional perfusion images non-invasively. However, measuring perfusion velocity under a tissue is difficult due to its optical properties. Dynamic scattered light occurs by perfusion is diffused by the surrounding tissue and it depends on the depth of tissue. We aimed to develop a three-dimensional convolutional neural network (3D-CNN) model based on laser speckle contrast images to measure the flow velocity and the depth of tissue using a flow phantom. The training data was selected as a region containing both static speckles and dynamic speckles, and the model was trained on the laser speckle contrast image data. The trained model showed a measurement accuracy of more than 90 % for velocity and 99 % for depth measurement. This study has the potential of a deep-learning model based on LSCI to analyze the blood flow and depth of blood vessels in bio-applications.

Automatic detection of pulmonary embolism on computed tomography pulmonary angiogram scan using a three-dimensional convolutional neural network

ObjectiveTo propose a convolutional neural network (EmbNet) for automatic pulmonary embolism detection on computed tomography pulmonary angiogram (CTPA) scans and to assess its diagnostic performance. Methods305 consecutive CTPA scans between January 2019 and December 2021 were enrolled in this study (142 for training, 163 for internal validation), and 250 CTPA scans from a public dataset were used for external validation. The framework comprised a preprocessing step to segment the pulmonary vessels and the EmbNet to detect emboli. Emboli were divided into three location-based subgroups for detailed evaluation: central arteries, lobar branches, and peripheral regions. Ground truth was established by three radiologists. ResultsThe EmbNet’s per-scan level sensitivity, specificity, positive predictive value (PPV), and negative predictive value were 90.9%, 75.4%, 48.4%, and 97.0% (internal validation) and 88.0%, 70.5%, 42.7%, and 95.9% (external validation). At the per-embolus level, the overall sensitivity and PPV of the EmbNet were 86.0% and 61.3% (internal validation), and 83.5% and 57.5% (external validation). The sensitivity and PPV of central emboli were 89.7% and 52.0% (internal validation), and 94.4% and 43.0% (external validation); of lobar emboli were 95.2% and 76.9% (internal validation), and 93.5% and 72.5% (external validation); and of peripheral emboli were 82.6% and 61.7% (internal validation), and 80.2% and 59.4% (external validation). The average false positive rate was 0.45 false emboli per scan (internal validation) and 0.69 false emboli per scan (external validation). ConclusionThe EmbNet provides high sensitivity across embolus locations, suggesting its potential utility for initial screening in clinical practice.

Deep learning-based correction for time truncation in cerebral computed tomography perfusion.

Cerebral computed tomography perfusion (CTP) imaging requires complete acquisition of contrast bolus inflow and washout in the brain parenchyma; however, time truncation undoubtedly occurs in clinical practice. To overcome this issue, we proposed a three-dimensional (two-dimensional + time) convolutional neural network (CNN)-based approach to predict missing CTP image frames at the end of the series from earlier acquired image frames. Moreover, we evaluated three strategies for predicting multiple time points. Seventy-two CTP scans with 89 frames and eight slices from a publicly available dataset were used to train and test the CNN models capable of predicting the last 10 image frames. The prediction strategies were single-shot prediction, recursive multi-step prediction, and direct-recursive hybrid prediction.Single-shot prediction predicted all frames simultaneously, while recursive multi-step prediction used prior predictions as input for subsequent steps, and direct-recursive hybrid prediction employed separate models for each step with prior predictions as input for the next step. The accuracies of the predicted image frames were evaluated in terms of image quality, bolus shape, and clinical perfusion parameters. We found that the image quality metrics were superior when multiple CTP images were predicted simultaneously rather than recursively. The bolus shape also showed the highest correlation (r = 0.990, p < 0.001) and the lowest variance (95% confidence interval, -453.26-445.53) in the single-shot prediction. For all perfusion parameters, the single-shot prediction had the smallest absolute differences from ground truth. Our proposed approach can potentially minimize time truncation errors and support the accurate quantification of ischemic stroke.

Artificial intelligence-driven computer aided diagnosis system provides similar diagnosis value compared with doctors’ evaluation in lung cancer screening

ObjectiveTo evaluate the consistency between doctors and artificial intelligence (AI) software in analysing and diagnosing pulmonary nodules, and assess whether the characteristics of pulmonary nodules derived from the two methods are consistent for the interpretation of carcinomatous nodules.Materials and MethodsThis retrospective study analysed participants aged 40–74 in the local area from 2011 to 2013. Pulmonary nodules were examined radiologically using a low-dose chest CT scan, evaluated by an expert panel of doctors in radiology, oncology, and thoracic departments, as well as a computer-aided diagnostic(CAD) system based on the three-dimensional(3D) convolutional neural network (CNN) with DenseNet architecture(InferRead CT Lung, IRCL). Consistency tests were employed to assess the uniformity of the radiological characteristics of the pulmonary nodules. The receiver operating characteristic (ROC) curve was used to evaluate the diagnostic accuracy. Logistic regression analysis is utilized to determine whether the two methods yield the same predictive factors for cancerous nodules.ResultsA total of 570 subjects were included in this retrospective study. The AI software demonstrated high consistency with the panel's evaluation in determining the position and diameter of the pulmonary nodules (kappa = 0.883, concordance correlation coefficient (CCC) = 0.809, p = 0.000). The comparison of the solid nodules' attenuation characteristics also showed acceptable consistency (kappa = 0.503). In patients diagnosed with lung cancer, the area under the curve (AUC) for the panel and AI were 0.873 (95%CI: 0.829–0.909) and 0.921 (95%CI: 0.884–0.949), respectively. However, there was no significant difference (p = 0.0950). The maximum diameter, solid nodules, subsolid nodules were the crucial factors for interpreting carcinomatous nodules in the analysis of expert panel and IRCL pulmonary nodule characteristics.ConclusionAI software can assist doctors in diagnosing nodules and is consistent with doctors' evaluations and diagnosis of pulmonary nodules.

Basketball technique action recognition using 3D convolutional neural networks

This research investigates the recognition of basketball techniques actions through the implementation of three-dimensional (3D) Convolutional Neural Networks (CNNs), aiming to enhance the accurate and automated identification of various actions in basketball games. Initially, basketball action sequences are extracted from publicly available basketball action datasets, followed by data preprocessing, including image sampling, data augmentation, and label processing. Subsequently, a novel action recognition model is proposed, combining 3D convolutions and Long Short-Term Memory (LSTM) networks to model temporal features and capture the spatiotemporal relationships and temporal information of actions. This facilitates the facilitating automatic learning of the spatiotemporal features associated with basketball actions. The model’s performance and robustness are further improved through the adoption of optimization algorithms, such as adaptive learning rate adjustment and regularization. The efficacy of the proposed method is verified through experiments conducted on three publicly available basketball action datasets: NTURGB + D, Basketball-Action-Dataset, and B3D Dataset. The results indicate that this approach achieves outstanding performance in basketball technique action recognition tasks across different datasets compared to two common traditional methods. Specifically, when compared to the frame difference-based method, this model exhibits a significant accuracy improvement of 15.1%. When compared to the optical flow-based method, this model demonstrates a substantial accuracy improvement of 12.4%. Moreover, this method showcases strong robustness, accurately recognizing actions under diverse lighting conditions and scenes, achieving an average accuracy of 93.1%. The research demonstrates that the method reported here effectively captures the spatiotemporal relationships of basketball actions, thereby providing reliable technical assessment tools for basketball coaches and players.

Combining three-dimensional acoustic coring and a convolutional neural network to quantify species contributions to benthic ecosystems.

The seafloor is inhabited by a large number of benthic invertebrates, and their importance in mediating carbon mineralization and biogeochemical cycles is recognized. However, the majority of fauna live below the sediment surface, so most means of survey rely on destructive sampling methods that are limited to documenting species presence rather than event driven activity and functionally important aspects of species behaviour. We have developed and tested a laboratory-based three-dimensional acoustic coring system that is capable of non-invasively visualizing the presence and activity of invertebrates within the sediment matrix. Here, we present reconstructed three-dimensional acoustic images of the sediment profile, with strong backscatter revealing the presence and position of individual benthic organisms. These data were used to train a three-dimensional convolutional neural network model and, using a combination of data augmentation and data correction techniques, we were able to identify individual species with an 88% accuracy. Combining three-dimensional acoustic coring with deep learning forms an effective and non-invasive means of providing detailed mechanistic information of in situ species-sediment interactions, opening new opportunities to quantify species-specific contributions to ecosystems.

Few-shot intent detection with self-supervised pretraining and prototype-aware attention

Few-shot intent detection is a more challenging application. However, traditional prototypical networks based on averaging often suffer from issues such as missing key information, poor generalization capabilities. In previous work, using three-dimensional convolutional neural networks (3DCNN) to generate prototype representations faces challenges with long-distance dependencies. Furthermore, a pretrained encoder’s performance in a specific domain is often suboptimal because its knowledge of the specific domain is fragmented. Therefore, in this paper, we propose a simple yet effective two-stage learning strategy to address these issues. In the first stage, we propose a self-supervised multi-task pretraining (SMTP) strategy. SMTP utilizes unlabeled data from the current domain to help the pretrained encoder learn the semantic information of the text and implicitly distinguish semantically similar text representations without using any labels. SMTP aims to enhance the representation capability of the pretrained encoder in a specific domain. In the second stage, we propose a prototype-aware attention (PaAT) model to generate prototype representations of the same class. PaAT generates prototype representations by calculating the attention between class texts, which can effectively solve the long-distance dependence problem of 3DCNN. PaAT is a siamese architecture that can simultaneously generate prototype representations and sentence-level representations of unseen data. In addition, to prevent overfitting in few-shot learning, we introduce an unsupervised contrastive regularization term to constrain PaAT. Our method achieves state-of-the-art performance on four public datasets.11Our code is available: https://github.com/YS19999/SMTP-PaAT. .

Quantifying lung fissure integrity using a three-dimensional patch-based convolutional neural network on CT images for emphysema treatment planning.

Evaluation of lung fissure integrity is required to determine whether emphysema patients have complete fissures and are candidates for endobronchial valve (EBV) therapy. We propose a deep learning (DL) approach to segment fissures using a three-dimensional patch-based convolutional neural network (CNN) and quantitatively assess fissure integrity on CT to evaluate it in subjects with severe emphysema. From an anonymized image database of patients with severe emphysema, 129 CT scans were used. Lung lobe segmentations were performed to identify lobar regions, and the boundaries among these regions were used to construct approximate interlobar regions of interest (ROIs). The interlobar ROIs were annotated by expert image analysts to identify voxels where the fissure was present and create a reference ROI that excluded non-fissure voxels (where the fissure is incomplete). A CNN configured by nnU-Net was trained using 86 CT scans and their corresponding reference ROIs to segment the ROIs of left oblique fissure (LOF), right oblique fissure (ROF), and right horizontal fissure (RHF). For an independent test set of 43 cases, fissure integrity was quantified by mapping the segmented fissure ROI along the interlobar ROI. A fissure integrity score (FIS) was then calculated as the percentage of labeled fissure voxels divided by total voxels in the interlobar ROI. Predicted FIS (p-FIS) was quantified from the CNN output, and statistical analyses were performed comparing p-FIS and reference FIS (r-FIS). The absolute percent error mean (±SD) between r-FIS and p-FIS for the test set was 4.0% (), 6.0% (), and 12.2% () for the LOF, ROF, and RHF, respectively. A DL approach was developed to segment lung fissures on CT images and accurately quantify FIS. It has potential to assist in the identification of emphysema patients who would benefit from EBV treatment.

A painting authentication method based on multi-scale spatial-spectral feature fusion and convolutional neural network

The scientific authentication of paintings holds significant importance within the realm of art collection. Employing convolutional neural networks for the classification of authentic and counterfeit painting images based on color images is a viable but suboptimal choice. This study investigates the potential for authenticating paintings by incorporating high-spectral images alongside high-resolution spatial images. High-resolution and high-spectral images of genuine and counterfeit paintings were acquired using a push-broom digital scanning system. The processing methods presented in this approach for the acquired images are: 1) The study utilized the circular local binary pattern (LBP) and principal component analysis (PCA) to extract surface texture and spectral data from Chinese character images in paintings, encompassing both spatial and spectral dimensions. 2) The technique utilizing non-subsampling Shearlet transform (NSST) and pulse-coupled neural network (PCNN) was employed to integrate the spatial and spectral characteristics of the images into a pseudo-color image, producing a dataset of feature data for genuine and counterfeit paintings. 3) The experiments aimed to achieve the authenticity of artworks using EfficientNet v2-s, the hyperparameters of which were fine-tuned accordingly. The experimental findings demonstrate that this approach attained a 90.8 % accuracy on the test dataset, representing a 3.5 % enhancement over the existing top-performing three-dimensional convolutional neural network (3D-CNN).

Explainability of three-dimensional convolutional neural networks for functional magnetic resonance imaging of Alzheimer's disease classification based on gradient-weighted class activation mapping.

Currently, numerous studies focus on employing fMRI-based deep neural networks to diagnose neurological disorders such as Alzheimer's Disease (AD), yet only a handful have provided results regarding explainability. We address this gap by applying several prevalent explainability methods such as gradient-weighted class activation mapping (Grad-CAM) to an fMRI-based 3D-VGG16 network for AD diagnosis to improve the model's explainability. The aim is to explore the specific Region of Interest (ROI) of brain the model primarily focuses on when making predictions, as well as whether there are differences in these ROIs between AD and normal controls (NCs). First, we utilized multiple resting-state functional activity maps including ALFF, fALFF, ReHo, and VMHC to reduce the complexity of fMRI data, which differed from many studies that utilized raw fMRI data. Compared to methods utilizing raw fMRI data, this manual feature extraction approach may potentially alleviate the model's burden. Subsequently, 3D-VGG16 were employed for AD classification, where the final fully connected layers were replaced with a Global Average Pooling (GAP) layer, aimed at mitigating overfitting while preserving spatial information within the feature maps. The model achieved a maximum of 96.4% accuracy on the test set. Finally, several 3D CAM methods were employed to interpret the models. In the explainability results of the models with relatively high accuracy, the highlighted ROIs were primarily located in the precuneus and the hippocampus for AD subjects, while the models focused on the entire brain for NC. This supports current research on ROIs involved in AD. We believe that explaining deep learning models would not only provide support for existing research on brain disorders, but also offer important referential recommendations for the study of currently unknown etiologies.