3D Convolutional Neural Network Model Research Articles

Detection of handgun using 3D convolutional neural network model (3DCNNs)

Detection of handgun using 3D convolutional neural network model (3DCNNs)

Video Vision Transformer를 이용한 벼의 수확량 예측과 Self-attention 시각화

The government and farmers' organizations are paying much attention to the problem of predicting how much rice can be produced each year. However, it is difficult to accurately predict the yield of rice due to variable factors such as extreme climate change and various pests and diseases that change every year. In this study, images were collected several times during the growing season of rice through a multi-spectral sensor mounted on an unmanned aerial vehicle, and rice yield was predicted using a deep learning algorithm. Multispectral images can be viewed as a kind of image data taken several times at regular intervals, and rice yield was predicted using the Video Vision Transformer (ViViT) model, which applies the Transformer structure to image computer vision among deep learning algorithms. The ViViT model generates patches by dividing the input image into a certain size, and as a result of learning the model by setting the size of these patches differently, it was found that the smaller the patch size, the better the predictive power. In addition, as a result of comparing prediction performance with a 3D CNN model that receives an image as an input in a CNN (Convolutional Neural Network) structure used in the image processing field, it was found that the ViViT model using a small patch size performed better. As a result of visualizing the weight matrix of the ViViT model as a heat map, images taken in mid- to late August appear to be important in yield prediction, making it possible to predict yield about two months before rice harvest.

Classification of Pulmonary Nodules in 2-[18F]FDG PET/CT Images with a 3D Convolutional Neural Network

Purpose2-[18F]FDG PET/CT plays an important role in the management of pulmonary nodules. Convolutional neural networks (CNNs) automatically learn features from images and have the potential to improve the discrimination between malignant and benign pulmonary nodules. The purpose of this study was to develop and validate a CNN model for classification of pulmonary nodules from 2-[18F]FDG PET images.MethodsOne hundred thirteen participants were retrospectively selected. One nodule per participant. The 2-[18F]FDG PET images were preprocessed and annotated with the reference standard. The deep learning experiment entailed random data splitting in five sets. A test set was held out for evaluation of the final model. Four-fold cross-validation was performed from the remaining sets for training and evaluating a set of candidate models and for selecting the final model. Models of three types of 3D CNNs architectures were trained from random weight initialization (Stacked 3D CNN, VGG-like and Inception-v2-like models) both in original and augmented datasets. Transfer learning, from ImageNet with ResNet-50, was also used.ResultsThe final model (Stacked 3D CNN model) obtained an area under the ROC curve of 0.8385 (95% CI: 0.6455–1.0000) in the test set. The model had a sensibility of 80.00%, a specificity of 69.23% and an accuracy of 73.91%, in the test set, for an optimised decision threshold that assigns a higher cost to false negatives.ConclusionA 3D CNN model was effective at distinguishing benign from malignant pulmonary nodules in 2-[18F]FDG PET images.

Predicting gas diffusion layer flow information in proton exchange membrane fuel cells from cross-sectional data using deep learning methods

Obtaining transient flow field information of gas diffusion layers (GDLs) is a crucial issue for improving the performance of proton exchange membrane fuel cells (PEMFCs). While physically-based methods, such as effective medium theory and solving partial differential equations, can be utilized to calculate fluid flow in GDL, the computational cost associated with such methods remains substantial. In this study, a 3D multiphase flow dataset of GDLs is obtained using fluid of volume (VOF) calculation, and the 3D data is sliced. The resulting cross-sectional data are used to establish a convolutional neural network (CNN) model based on two-dimensional (2D) data for predicting fluid flow in GDLs. The reliability of predicting the saturation information of 3D GDLs using 2D cross-sectional data is verified and discussed. Subsequently, a comparison is made between the accuracy and computational cost of predicting the saturation of 3D GDLs using 2D and 3D CNN models. The results indicate that using 25 cross-sectional images provide accurate predictions of GDL saturation in 3D. When using cross-sectional images as inputs, 100 × 100 images are found to be more representative than 50 × 50 images, resulting in an average increase of 1.86% and 13.36% in R2 and RMSE, respectively. While the 3D CNN model outperforms the 2D CNN model in predicting GDL saturation in 3D by only 0.62% in terms of R2 score, its computational cost is two orders of magnitude higher. These findings suggest that 2D GDL cross-sectional images can also be used for predicting 3D GDL flow information and effectively reducing computational costs. The findings of this study provide a profound insight into the GDL flow phenomena and contribute to the development of more efficient and accurate fuel cell models.

A 3D and Explainable Artificial Intelligence Model for Evaluation of Chronic Otitis Media Based on Temporal Bone Computed Tomography: Model Development, Validation, and Clinical Application.

Temporal bone computed tomography (CT) helps diagnose chronic otitis media (COM). However, its interpretation requires training and expertise. Artificial intelligence (AI) can help clinicians evaluate COM through CT scans, but existing models lack transparency and may not fully leverage multidimensional diagnostic information. We aimed to develop an explainable AI system based on 3D convolutional neural networks (CNNs) for automatic CT-based evaluation of COM. Temporal bone CT scans were retrospectively obtained from patients operated for COM between December 2015 and July 2021 at 2 independent institutes. A region of interest encompassing the middle ear was automatically segmented, and 3D CNNs were subsequently trained to identify pathological ears and cholesteatoma. An ablation study was performed to refine model architecture. Benchmark tests were conducted against a baseline 2D model and 7 clinical experts. Model performance was measured through cross-validation and external validation. Heat maps, generated using Gradient-Weighted Class Activation Mapping, were used to highlight critical decision-making regions. Finally, the AI system was assessed with a prospective cohort to aid clinicians in preoperative COM assessment. Internal and external data sets contained 1661 and 108 patients (3153 and 211 eligible ears), respectively. The 3D model exhibited decent performance with mean areas under the receiver operating characteristic curves of 0.96 (SD 0.01) and 0.93 (SD 0.01), and mean accuracies of 0.878 (SD 0.017) and 0.843 (SD 0.015), respectively, for detecting pathological ears on the 2 data sets. Similar outcomes were observed for cholesteatoma identification (mean area under the receiver operating characteristic curve 0.85, SD 0.03 and 0.83, SD 0.05; mean accuracies 0.783, SD 0.04 and 0.813, SD 0.033, respectively). The proposed 3D model achieved a commendable balance between performance and network size relative to alternative models. It significantly outperformed the 2D approach in detecting COM (P≤.05) and exhibited a substantial gain in identifying cholesteatoma (P<.001). The model also demonstrated superior diagnostic capabilities over resident fellows and the attending otologist (P<.05), rivaling all senior clinicians in both tasks. The generated heat maps properly highlighted the middle ear and mastoid regions, aligning with human knowledge in interpreting temporal bone CT. The resulting AI system achieved an accuracy of 81.8% in generating preoperative diagnoses for 121 patients and contributed to clinical decision-making in 90.1% cases. We present a 3D CNN model trained to detect pathological changes and identify cholesteatoma via temporal bone CT scans. In both tasks, this model significantly outperforms the baseline 2D approach, achieving levels comparable with or surpassing those of human experts. The model also exhibits decent generalizability and enhanced comprehensibility. This AI system facilitates automatic COM assessment and shows promising viability in real-world clinical settings. These findings underscore AI's potential as a valuable aid for clinicians in COM evaluation. Chinese Clinical Trial Registry ChiCTR2000036300; https://www.chictr.org.cn/showprojEN.html?proj=58685.

Creating Historical Building Models by Deep Fusion of Multi-Source Heterogeneous Data Using Residual 3D Convolutional Neural Network

ABSTRACT There are a large number of historical buildings in the world, but many of them are seriously damaged. A fundamental task to revitalize them is to record their details and redraw them, but it is a highly inefficient task. This paper aimed to create three-dimensional (3D) models automatically for those damaged historical buildings while respecting the 1964 Venice Charter, the ICOMOS recommendations, and local ordinances. After reviewing the advantages and disadvantages of various measurement tools, we implemented a machine learning algorithm, the residual 3D convolutional neural network (CNN) method, to combine heterogeneous data from multiple sources including a 3D laser scanner, an unmanned aerial vehicle, a total station, and a camera, in a historical building information model (HBIM) to increase its accuracy. Those data, preprocessed by four steps including data registration, reducing noises, filtering data, and data integration, were trained and fused in a 3D CNN model, which consisted of one input layer, five 3D convolutional and pooling layers, and two connected layers. A Shanghai historical building conservation project was taken as a pilot case to verify its productivity and accuracy in developing the HBIM, on which the architectural features were thoroughly recorded, and the damaged details were recovered.

Subregional pharyngeal changes after orthognathic surgery in skeletal Class III patients analyzed by convolutional neural networks-based segmentation

ObjectivesTo evaluate the accuracy of fully automatic segmentation of pharyngeal volume of interests (VOIs) before and after orthognathic surgery in skeletal Class III patients using a convolutional neural network (CNN) model and to investigate the clinical applicability of artificial intelligence for quantitative evaluation of treatment changes in pharyngeal VOIs. Methods310 cone-beam computed tomography (CBCT) images were divided into a training set (n = 150), validation set (n = 40), and test set (n = 120). The test datasets comprised matched pairs of pre- and post-treatment images of 60 skeletal Class III patients (mean age 23.1 ± 5.0 years; ANB<-2⁰) who underwent bimaxillary orthognathic surgery with orthodontic treatment. A 3D U-Net CNNs model was applied for fully automatic segmentation and measurement of subregional pharyngeal volumes of pre-treatment (T0) and post-treatment (T1) scans. The model's accuracy was compared to semi-automatic segmentation outcomes by humans using the dice similarity coefficient (DSC) and volume similarity (VS). The correlation between surgical skeletal changes and model accuracy was obtained. ResultsThe proposed model achieved high performance of subregional pharyngeal segmentation on both T0 and T1 images, representing a significant T1–T0 difference of DSC only in the nasopharynx. Region-specific differences amongst pharyngeal VOIs, which were observed at T0, disappeared on the T1 images. The decreased DSC of nasopharyngeal segmentation after treatment was weakly correlated with the amount of maxillary advancement. There was no correlation between the mandibular setback amount and model accuracy. ConclusionsThe proposed model offers fast and accurate subregional pharyngeal segmentation on both pre-treatment and post-treatment CBCT images in skeletal Class III patients. Clinical significanceWe elucidated the clinical applicability of the CNNs model to quantitatively evaluate subregional pharyngeal changes after surgical-orthodontic treatment, which offers a basis for developing a fully integrated multiclass CNNs model to predict pharyngeal responses after dentoskeletal treatments.

RETRACTED ARTICLE: Simulation of video image detection in leisure sports tourism industry based on convolutional neural network

Due to the continuous development of information technology, emerging technologies such as 5G networks, Turing devices and deep learning are widely used in people's lives. At present, the sports and tourism industry has become a pillar industry supporting the national economy. Sports events have attracted many people's attention, and people are willing to devote themselves to sports activities. Participants and visitors are also increasing day by day. The article also conducts research on the rapid detection of human video behavior, obtains more representative features for this demand. This research is of great help to the development of the industry. This allows the network to manipulate the input video behavior at different scales and extract deeper video information. Therefore, this paper proposes another 3D convolutional neural network model that combines a multi-level pyramid network with an attention mechanism. The visual attributes in the video behavior data set are explicitly learned to refine the classification of similar features. This paper also proposes an integrated neural network based on visual attribute enhancement. It also shows good adaptability, and can complete video data processing based on time series. The extraction of time and space functions is realized by the two-stream neural network method, the two networks learn the corresponding pixel ratios, and it is not easy to lose the clues of the action recognition. This paper also proposes a spatiotemporal convolutional neural network video for human body recognition algorithm, which is applied to the leisure tourism industry according to the specific analysis of the above method, and promotes the development of the industry.

Land-cover classification with hyperspectral remote sensing image using CNN and spectral band selection

Recent developments in remote sensing technology have led to a significant rise in the demand for precise and reliable algorithms for analyzing hyperspectral remote sensing images. Hyperspectral remote sensing image analysis aims to differentiate various types of landscapes on Earth's surface, which is challenging due to the high dimensionality of the data space and the abundance of spectral bands. To address these challenges, this work proposes a novel 3D CNN-based model for classifying hyperspectral images based on both spectral and spatial features. The proposed approach comprises two stages: data pre-processing and classification. First, the raw hyperspectral image is processed, and the essential spectral bands are selected using HyperPCA to reduce the data's high dimensionality. Second, using fused multi-resolution spectral and spatial features, obtained by a 3D convolutional neural network model the landscapes of the hyperspectral remote sensing image are classified. The proposed model employs concatenation and addition operations to extract comprehensive and selective features, respectively, and uses direct max pooling to mitigate strong activations that reduce feature maps. The proposed approach enhances hyperspectral image classification even with a small number of labeled samples by enabling the automatic extraction of pertinent information while maintaining the spatial and spectral features of the data. The Indian Pines and University of Pavia datasets, two existing hyperspectral images, are used in the experiments. The results demonstrate that, in terms of classification performance, the suggested approach is competitive with the state-of-the-art.

The potential for different computed tomography-based machine learning networks to automatically segment and differentiate pelvic and sacral osteosarcoma from Ewing's sarcoma.

This study aimed to explore optimal computed tomography (CT)-based machine learning and deep learning methods for the identification of pelvic and sacral osteosarcomas (OS) and Ewing's sarcomas (ES). A total of 185 patients with pathologically confirmed pelvic and sacral OS and ES were analyzed. We first compared the performance of 9 radiomics-based machine learning models, 1 radiomics-based convolutional neural networks (CNNs) model, and 1 3-dimensional (3D) CNN model, respectively. We then proposed a 2-step no-new-Net (nnU-Net) model for the automatic segmentation and identification of OS and ES. The diagnoses by 3 radiologists were also obtained. The area under the receiver operating characteristic curve (AUC) and accuracy (ACC) were used to evaluate the different models. Age, tumor size, and tumor location showed significant differences between OS and ES (P<0.01). For the radiomics-based machine learning models, logistic regression (LR; AUC =0.716, ACC =0.660) performed best in the validation set. However, the radiomics-based CNN model had an AUC of 0.812 and ACC of 0.774 in the validation set, which were higher than those of the 3D CNN model (AUC =0.709, ACC =0.717). Among all the models, the nnU-Net model performed best, with an AUC of 0.835 and an ACC of 0.830 in the validation set, which was significantly higher than the primary physician's diagnosis (ACCs ranged from 0.757 to 0.811) (P<0.01). The proposed nnU-Net model could be an end-to-end, non-invasive, and accurate auxiliary diagnostic tool for the differentiation of pelvic and sacral OS and ES.

Generalizability of 3D CNN models for age estimation in diverse youth populations using structural MRI

Recently, several studies have investigated the neurodevelopment of psychiatric disorders using brain data acquired via structural magnetic resonance imaging (sMRI). These analyses have shown the potential of sMRI data to provide a relatively precise characterization of brain structural biomarkers. Despite these advances, a relatively unexplored question is how reliable and consistent a model is when assessing subjects from other independent datasets. In this study, we investigate the performance and generalizability of the same model architecture trained from distinct datasets comprising youths in diverse stages of neurodevelopment and with different mental health conditions. We employed models with the same 3D convolutional neural network (CNN) architecture to assess autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), brain age, and a measure of dimensional psychopathology, the Child Behavior Checklist (CBCL) total score. The investigated datasets include the Autism Brain Imaging Data Exchange II (ABIDE-II, N = 580), Attention Deficit Hyperactivity Disorder (ADHD-200, N = 922), Brazilian High-Risk Cohort Study (BHRCS, N = 737), and Adolescent Brain Cognitive Development (ABCD, N = 11,031). Models’ performance and interpretability were assessed within each dataset (for diagnosis tasks) and inter-datasets (for age estimation). Despite the demographic and phenotypic differences of the subjects, all models presented significant estimations for age (p value < 0.001) within and between datasets. In addition, most models showed a moderate to high correlation in age estimation. The results, including the models' brain regions of interest (ROI), were analyzed and discussed in light of the youth neurodevelopmental structural changes. Among other interesting discoveries, we found that less confounded training datasets produce models with higher generalization capacity.

Utility of daily 3 m Planet Fusion Surface Reflectance data for tillage practice mapping with deep learning

Tillage practices alter soil surface structure that can be potentially captured by satellite images with both high spatial and temporal resolution. This study explored tillage practice mapping using the daily Planet Fusion surface reflectance (PF-SR) gap-free 3 m data generated by fusing PlanetScope with Landsat-8, Sentinel-2 and MODIS surface reflectance data. The study area is a 220 × 220 km2 agricultural area in South Dakota, USA, and the study used 3285 PF-SR images from September 1, 2020 to August 31, 2021. The PF-SR images for the surveyed 433 fields were sliced into 10,747 training (70%) and evaluation (30%) non-overlapping time series patches. The training and evaluation patches were from different fields for evaluation data independence. The performance of four deep learning models (i.e., 2D convolutional neural networks (CNN), 3D CNN, CNN-Long short-term memory (LSTM), and attention CNN-LSTM) in tillage practice mapping, as well as their sensitivity to different spatial (i.e., 3 m, 24 m, and 96 m) and temporal resolutions (16-day, 8-day, 4-day, 2-day and 1-day) were examined. Classification accuracy continuously increased with increases in both temporal and spatial resolutions. The optimal models (3D CNN and attention CNN-LSTM) achieved ∼77% accuracy using 2-day or daily 3 m resolution data as opposed to ∼72% accuracy using 16-day 3 m resolution data or daily 24 m resolution data. This study also analyzed the feature importance of different acquisition dates for the two optimal models. The 3D CNN model feature importances were found to agree well with the tillage practice time. High feature importance was associated with observations during the fall and spring tillage period (i.e., fresh tillage signals) whereas the crop peak growing period (i.e., tillage signals weathered and confounded by dense canopy) was characterized by a relatively low feature importance. The work provides valuable insights into the utility of deep learning for tillage mapping and change event time identification based on high resolution imagery.

Clinical radiomics-based machine learning versus three-dimension convolutional neural network analysis for differentiation of thymic epithelial tumors from other prevascular mediastinal tumors on chest computed tomography scan.

To compare the diagnostic performance of radiomic analysis with machine learning (ML) model with a convolutional neural network (CNN) in differentiating thymic epithelial tumors (TETs) from other prevascular mediastinal tumors (PMTs). A retrospective study was performed in patients with PMTs and undergoing surgical resection or biopsy in National Cheng Kung University Hospital, Tainan, Taiwan, E-Da Hospital, Kaohsiung, Taiwan, and Kaohsiung Veterans General Hospital, Kaohsiung, Taiwan between January 2010 and December 2019. Clinical data including age, sex, myasthenia gravis (MG) symptoms and pathologic diagnosis were collected. The datasets were divided into UECT (unenhanced computed tomography) and CECT (enhanced computed tomography) for analysis and modelling. Radiomics model and 3D CNN model were used to differentiate TETs from non-TET PMTs (including cyst, malignant germ cell tumor, lymphoma and teratoma). The macro F1-score and receiver operating characteristic (ROC) analysis were performed to evaluate the prediction models. In the UECT dataset, there were 297 patients with TETs and 79 patients with other PMTs. The performance of radiomic analysis with machine learning model using LightGBM with Extra Tree (macro F1-Score = 83.95%, ROC-AUC = 0.9117) had better performance than the 3D CNN model (macro F1-score = 75.54%, ROC-AUC = 0.9015). In the CECT dataset, there were 296 patients with TETs and 77 patients with other PMTs. The performance of radiomic analysis with machine learning model using LightGBM with Extra Tree (macro F1-Score = 85.65%, ROC-AUC = 0.9464) had better performance than the 3D CNN model (macro F1-score = 81.01%, ROC-AUC = 0.9275). Our study revealed that the individualized prediction model integrating clinical information and radiomic features using machine learning demonstrated better predictive performance in the differentiation of TETs from other PMTs at chest CT scan than 3D CNN model.

Effects of Patchwise Sampling Strategy to Three-Dimensional Convolutional Neural Network-Based Alzheimer’s Disease Classification

In recent years, the rapid development of artificial intelligence has promoted the widespread application of convolutional neural networks (CNNs) in neuroimaging analysis. Although three-dimensional (3D) CNNs can utilize the spatial information in 3D volumes, there are still some challenges related to high-dimensional features and potential overfitting issues. To overcome these problems, patch-based CNNs have been used, which are beneficial for model generalization. However, it is unclear how the choice of a patchwise sampling strategy affects the performance of the Alzheimer's Disease (AD) classification. To this end, the present work investigates the impact of a patchwise sampling strategy for 3D CNN based AD classification. A 3D framework cascaded by two-stage subnetworks was used for AD classification. The patch-level subnetworks learned feature representations from local image patches, and the subject-level subnetwork combined discriminative feature representations from all patch-level subnetworks to generate a classification score at the subject level. Experiments were conducted to determine the effect of patch partitioning methods, the effect of patch size, and interactions between patch size and training set size for AD classification. With the same data size and identical network structure, the 3D CNN model trained with 48 × 48 × 48 cubic image patches showed the best performance in AD classification (ACC = 89.6%). The model trained with hippocampus-centered, region of interest (ROI)-based image patches showed suboptimal performance. If the pathological features are concentrated only in some regions affected by the disease, the empirically predefined ROI patches might be the right choice. The better performance of cubic image patches compared with cuboidal image patches is likely related to the pathological distribution of AD. The image patch size and training sample size together have a complex influence on the performance of the classification. The size of the image patches should be determined based on the size of the training sample to compensate for noisy labels and the problem of the curse of dimensionality. The conclusions of the present study can serve as a reference for the researchers who wish to develop a superior 3D patch-based CNN model with an appropriate patch sampling strategy.

Multislice input for 2D and 3D residual convolutional neural network noise reduction in CT.

Deep convolutional neural network (CNN)-based methods are increasingly used for reducing image noise in computed tomography (CT). Current attempts at CNN denoising are based on 2D or 3D CNN models with a single- or multiple-slice input. Our work aims to investigate if the multiple-slice input improves the denoising performance compared with the single-slice input and if a 3D network architecture is better than a 2D version at utilizing the multislice input. Two categories of network architectures can be used for the multislice input. First, multislice images can be stacked channel-wise as the multichannel input to a 2D CNN model. Second, multislice images can be employed as the 3D volumetric input to a 3D CNN model, in which the 3D convolution layers are adopted. We make performance comparisons among 2D CNN models with one, three, and seven input slices and two versions of 3D CNN models with seven input slices and one or three output slices. Evaluation was performed on liver CT images using three quantitative metrics with full-dose images as reference. Visual assessment was made by an experienced radiologist. When the input channels of the 2D CNN model increases from one to three to seven, a trend of improved performance was observed. Comparing the three models with the seven-slice input, the 3D CNN model with a one-slice output outperforms the other models in terms of noise texture and homogeneity in liver parenchyma as well as subjective visualization of vessels. We conclude the that multislice input is an effective strategy for improving performance for 2D deep CNN denoising models. The pure 3D CNN model tends to have a better performance than the other models in terms of continuity across axial slices, but the difference was not significant compared with the 2D CNN model with the same number of slices as the input.

Comparing the Performance of Accuracy Using 3D CNN Model with the Fixed Spatial Transform With 3D CNN Model for the Detection of Pulmonary Nodules

Aim: The research study aims to detect the accuracy level of the pulmonary nodule using a convolutional neural network (CNN). The comparison between the Novel 3D CNN-fixed spatial transform algorithm and Novel 3D CNN Model algorithm for accurate detection. Materials and Methods: The information for this study was gained from the Kaggle website. The samples were taken into consideration as (N=20) for 3D CNN-fixed spatial transform and (N=20) 3D CNN Model according to the clinical. com, total sample size calculation was performed. Python software is used for accurate detection. Threshold Alpha is 0.05 %, G power is 80% and the enrollment ratio is set to 1. Result: This research study found that the 3D CNN with 89.29% of accuracy is preferred over 3D CNN with fixed spatial transform which gives 78.5% accuracy with a significance value (p=0.001), (p&lt;0.05) with a 95% confidence interval. There is statistical significance between the two groups. Conclusion: The mean value of 3D CNN -fixed spatial transform is 78.5% and Novel 3D CNN is 89.29%.Novel 3D CNN appears to give better accuracy than 3D CNN-fixed spatial transform.

Enhancing Disease Classification in Abdominal CT Scans through RGB Superposition Methods and 2D Convolutional Neural Networks: A Study of Appendicitis and Diverticulitis.

The primary symptom of both appendicitis and diverticulitis is a pain in the right lower abdomen; it is almost impossible to diagnose these conditions through symptoms alone. However, there will be misdiagnoses happening when using abdominal computed tomography (CT) scans. Most previous studies have used a 3D convolutional neural network (CNN) suitable for processing sequences of images. However, 3D CNN models can be difficult to implement in typical computing systems because they require large amounts of data, GPU memory, and extensive training times. We propose a deep learning method, utilizing red, green, and blue (RGB) channel superposition images reconstructed from three slices of sequence images. Using the RGB superposition image as the input image of the model, the average accuracy was shown as 90.98% in EfficietNetB0, 91.27% in EfficietNetB2, and 91.98% in EfficietNetB4. The AUC score using the RGB superposition image was higher than the original image of the single channel for EfficientNetB4 (0.967 vs. 0.959, p = 0.0087). The comparison in performance between the model architectures using the RGB superposition method showed the highest learning performance in the EfficientNetB4 model among all indicators; accuracy was 91.98% and recall was 95.35%. EfficientNetB4 using the RGB superposition method had a 0.011 (p value = 0.0001) AUC score higher than EfficientNetB0 using the same method. The superposition of sequential slice images in CT scans was used to enhance the distinction in features like shape, size of the target, and spatial information used to classify disease. The proposed method has fewer constraints than the 3D CNN method and is suitable for an environment using 2D CNN; thus, we can achieve performance improvement with limited resources.

Development and validation of a 3D-convolutional neural network model based on chest CT for differentiating active pulmonary tuberculosis from community-acquired pneumonia.

To develop and validate a 3D-convolutional neural network (3D-CNN) model based on chest CT for differentiating active pulmonary tuberculosis (APTB) from community-acquired pneumonia (CAP). Chest CT images of APTB and CAP patients diagnosed in two imaging centers (n = 432 in center A and n = 61 in center B) were collected retrospectively. The data in center A were divided into training, validation and internal test sets, and the data in center B were used as an external test set. A 3D-CNN was built using Keras deep learning framework. After the training, the 3D-CNN selected the model with the highest accuracy in the validation set as the optimal model, which was applied to the two test sets in centers A and B. In addition, the two test sets were independently diagnosed by two radiologists. The 3D-CNN optimal model was compared with the discrimination, calibration and net benefit of the two radiologists in differentiating APTB from CAP using chest CT images. The accuracy of the 3D-CNN optimal model was 0.989 and 0.934 with the internal and external test set, respectively. The area-under-the-curve values with the 3D-CNN model in the two test sets were statistically higher than that of the two radiologists (all P < 0.05), and there was a high calibration degree. The decision curve analysis showed that the 3D-CNN optimal model had significantly higher net benefit for patients than the two radiologists. 3D-CNN has high classification performance in differentiating APTB from CAP using chest CT images. The application of 3D-CNN provides a new automatic and rapid diagnosis method for identifying patients with APTB from CAP using chest CT images.

Deep 3D Convolutional Neural Network for Facial Micro-Expression Analysis from Video Images

Micro-expression is the involuntary emotion of the human that reflects the genuine feelings that cannot be hidden. Micro-expression is exhibited by facial expressions that last for a short duration and have very low intensity. Because of these reasons, micro-expression recognition is a challenging task. Recent research on the application of 3D convolutional neural networks (CNNs) has gained much popularity for video-based micro-expression analysis. For this purpose, both spatial as well as temporal features are of great importance to achieve high accuracies. The real possibly suppressed emotions of a person are valuable information for a variety of applications, such as in security, psychology, neuroscience, medicine and many other disciplines. This paper proposes a 3D CNN model architecture which is able to extract spatial and temporal features simultaneously. Thereby, the selection of the frame sequence plays a crucial role, since the emotions are only distinctive in a subset of the frames. Thus, we employ a novel pre-processing technique to select the Apex frame sequence from the entire video, where the timestamp of the most pronounced emotion is centered within this sequence. After an extensive evaluation including many experiments, the results show that the train–test split evaluation is biased toward a particular split and cannot be recommended in case of small and imbalanced datasets. Instead, a stratified K-fold evaluation technique is utilized to evaluate the model, which proves to be much more appropriate when using the three benchmark datasets CASME II, SMIC, and SAMM. Moreover, intra-dataset as well as cross-dataset evaluations were conducted in a total of eight different scenarios. For comparison purposes, two networks from the state of the art were reimplemented and compared with the presented architecture. In stratified K-fold evaluation, our proposed model outperforms both reimplemented state-of-the-art methods in seven out of eight evaluation scenarios.

Deep Learning for Detection of Intracranial Aneurysms from Computed Tomography Angiography Images.

The accuracy of computed tomography angiography (CTA) image interpretation depends on the radiologist. This study aims to develop a new method for automatically detecting intracranial aneurysms from CTA images using deep learning, based on a convolutional neural network (CNN) implemented on the DeepMedic platform. Ninety CTA scans of patients with intracranial aneurysms are collected and divided into two datasets: training (80 subjects) and test (10 subjects) datasets. Subsequently, a deep learning architecture with a three-dimensional (3D) CNN model is implemented on the DeepMedic platform for the automatic segmentation and detection of intracranial aneurysms from the CTA images. The samples in the training dataset are used to train the CNN model, and those in the test dataset are used to assess the performance of the established system. Sensitivity, positive predictive value (PPV), and false positives are evaluated. The overall sensitivity and PPV of this system for detecting intracranial aneurysms from CTA images are 92.3% and 100%, respectively, and the segmentation sensitivity is 92.3%. The performance of the system in the detection of intracranial aneurysms is closely related to their size. The detection sensitivity for small intracranial aneurysms (≤ 3mm) is 66.7%, whereas the sensitivity of detection for large (> 10mm) and medium-sized (3-10mm) intracranial aneurysms is 100%. The deep learning architecture with a 3D CNN model on the DeepMedic platform can reliably segment and detect intracranial aneurysms from CTA images with high sensitivity.