DCNN-based Method Research Articles

MR-Class: A Python Tool for Brain MR Image Classification Utilizing One-vs-All DCNNs to Deal with the Open-Set Recognition Problem.

MR image classification in datasets collected from multiple sources is complicated by inconsistent and missing DICOM metadata. Therefore, we aimed to establish a method for the efficient automatic classification of MR brain sequences. Deep convolutional neural networks (DCNN) were trained as one-vs-all classifiers to differentiate between six classes: T1 weighted (w), contrast-enhanced T1w, T2w, T2w-FLAIR, ADC, and SWI. Each classifier yields a probability, allowing threshold-based and relative probability assignment while excluding images with low probability (label: unknown, open-set recognition problem). Data from three high-grade glioma (HGG) cohorts was assessed; C1 (320 patients, 20,101 MRI images) was used for training, while C2 (197, 11,333) and C3 (256, 3522) were for testing. Two raters manually checked images through an interactive labeling tool. Finally, MR-Class' added value was evaluated via radiomics model performance for progression-free survival (PFS) prediction in C2, utilizing the concordance index (C-I). Approximately 10% of annotation errors were observed in each cohort between the DICOM series descriptions and the derived labels. MR-Class accuracy was 96.7% [95% Cl: 95.8, 97.3] for C2 and 94.4% [93.6, 96.1] for C3. A total of 620 images were misclassified; manual assessment of those frequently showed motion artifacts or alterations of anatomy by large tumors. Implementation of MR-Class increased the PFS model C-I by 14.6% on average, compared to a model trained without MR-Class. We provide a DCNN-based method for the sequence classification of brain MR images and demonstrate its usability in two independent HGG datasets.

Integration of Multisource Data to Estimate Downward Longwave Radiation Based on Deep Neural Networks

Downward longwave radiation (DLR) at the surface is a key variable of interest in fields, such as hydrology and climate research. However, existing DLR estimation methods and DLR products are still problematic in terms of both accuracy and spatiotemporal resolution. In this article, we propose a deep convolutional neural network (DCNN)-based method to estimate hourly DLR at 5-km spatial resolution from top of atmosphere (TOA) brightness temperature (BT) of the Himawari-8/Advanced Himawari Imager (AHI) thermal channels, combined with near-surface air temperature and dew point temperature of ERA5 and elevation data. Validation results show that the DCNN-based method outperforms popular random forest and multilayer perceptron-based methods and that our proposed scheme integrating multisource data outperforms that only using remote sensing TOA observations or surface meteorological data. Compared with state-of-the-art CERES-SYN and ERA5-land DLR products, the estimated DLR by our proposed DCNN-based method with physical multisource inputs has higher spatiotemporal resolution and accuracy, with correlation coefficient (CC) of 0.95, root-mean-square error (RMSE) of 17.2 W/m², and mean bias error (MBE) of -0.8 W/m² in the testing period on the Tibetan Plateau.

Patch-Based DCNN Method for CBCT Image Enhancement

Cone-beam computed tomography (CBCT) often has suboptimal image quality compared with standard fan-beam CT due to increased sensitivity to scattering and motion artifacts. In radiation treatment of cancers, due to the convenience in image acquisition as it is fully incorporated into the treatment station, CBCT plays a significant role in treatment plan re-evaluation and adaptation in adaptive radiation therapy but is limited by the image quality. In this paper, we propose a deep convolutional neural network (DCNN) based method for head and neck (HN) CBCT enhancement, where the network learns to transform the low-quality CBCT images to high-quality images.CT and CBCT image pairs from 50 HN cancer patients who received radiotherapy treatment were used for model training, validation and testing. An additional 22 raw CBCT images from another vendor were also used for testing. Pre-processing procedures include rigid registration, center-cropping, and Otsu's thresholding to segment the anatomical region. Data quality control was applied before training a 2D U-Net to reduce the adverse impact from the unmatched pairs. A patch-based strategy was employed during training, since the randomly extracted patches could reduce the receptive field and thus reduce the probability of overfitting based on the global anatomy. We explored the impact of patch size by varying it to be 64 × 64, 128 × 128, 192 × 192 and 256 × 256. After fixing to the optimal patch size, a Pix2Pix network combining L1 loss and conditional generative adversarial network (GAN) loss was also trained.It was found that the U-Net trained with a patch size of 128 × 128 reached the optimal performance and improved the mean absolute error (MAE) from 57.91 to 34.78 Hounsfield units (HU), peak signal-to-noise ratio (PSNR) from 30. 56 to 32.84 dB, and structural similarity (SSIM) from 0.7546 to 0.8455 compared with the original CBCT. With the addition of conditional GAN loss, the trained Pix2Pix network better kept the image details, especially on the external testing set.In this work, we developed a DCNN-based method to improve the CBCT image quality. This method has the advantages of high adaptability, vendor neutrality and low cost, since it operates purely in the image domain. Our data quality control pipeline automatically removed the unmatched CT and CBCT image pairs from the training set. The proposed DCNN-based method with the patch-based training strategy showed promising results for fast CBCT image enhancement.

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Deep Convolutional Neural Networks (DCNNs) have recently shown outstanding performance in semantic image segmentation. However, state-of-the-art DCNN-based semantic segmentation methods usually suffer from high computational complexity due to the use of complex network architectures. This greatly limits their applications in the real-world scenarios that require real-time processing. In this paper, we propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes, which achieves a good trade-off between accuracy and speed. Specifically, a Lightweight Baseline Network with Atrous convolution and Attention (LBN-AA) is firstly used as our baseline network to efficiently obtain dense feature maps. Then, the Distinctive Atrous Spatial Pyramid Pooling (DASPP), which exploits the different sizes of pooling operations to encode the rich and distinctive semantic information, is developed to detect objects at multiple scales. Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate high-resolution feature maps preserving the detailed spatial information. Finally, a simple but practical Feature Fusion Network (FFN) is used to effectively combine both deep and shallow features from the semantic branch (DASPP) and the spatial branch (SPN), respectively. Extensive experimental results show that the proposed method respectively achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) at the inference speeds of 51.0 fps and 39.3 fps on the challenging Cityscapes and CamVid test datasets (by only using a single NVIDIA TITAN X card). This demonstrates that the proposed method offers excellent performance at the real-time speed for semantic segmentation of urban street scenes.

Sleep stage classification for child patients using DeConvolutional Neural Network

Studies from the literature show that the prevalence of sleep disorder in children is far higher than that in adults. Although much research effort has been made on sleep stage classification for adults, children have significantly different characteristics of sleep stages. Therefore, there is an urgent need for sleep stage classification targeting children in particular. Our method focuses on two issues: The first is timestamp-based segmentation (TSS) to deal with the fine-grained annotation of sleep stage labels for each timestamp. Compared to this, popular sliding window approaches unnecessarily aggregate such labels into coarse-grained ones. We utilize DeConvolutional Neural Network (DCNN) that inversely maps features of a hidden layer back to the input space to predict the sleep stage label at each timestamp. Thus, our DCNN can yield better classification performances by considering labels at numerous timestamps. The second issue is the necessity of multiple channels. Different clinical signs, symptoms or other auxiliary examinations could be represented by different Polysomnography (PSG) recordings, so all of them should be analyzed comprehensively. We therefor exploit multivariate time-series of PSG recordings, including 6 electroencephalograms (EEGs) channels, 2 electrooculograms (EOGs) channels (left and right), 1 electromyogram (chin EMG) channel and two leg electromyogram channels. Our DCNN-based method is tested on our SDCP dataset collected from child patients aged from 5 to 10 years old. The results show that our method yields the overall classification accuracy of 84.27% and macro F1-score of 72.51% which are higher than those of existing sliding window-based methods. One of the biggest advantages of our DCNN-based method is that it processes raw PSG recordings and internally extracts features useful for accurate sleep stage classification. We examine whether this is applicable for sleep data of adult patients by testing our method on a well-known public dataset Sleep-EDFX. Our method achieves the average overall accuracy of 90.89% which is comparable to those of state-of-the-art methods without using any hand-crafted features. This result indicates the great potential of our method because it can be generally used for timestamp-level classification on multivariate time-series in various medical fields. Additionally, we provide source codes so that researchers can reproduce the results in this paper and extend our method.

End‐to‐end cubic phase signal recovery method based on deep convolutional neural network

Cubic phase (CP) signal is widely used in radar and sonar signal processing and the CP signal received by the radar or sonar sensors may miss part of discrete samples and be corrupted by noise. In this study, a novel end-to-end CP signal recovery method is proposed to recover the CP signal that experiences data missing and noise corruption based on deep convolutional neural network (DCNN). Two key techniques are used in the structure design of the proposed network. The first is dilated convolution, which has a larger receptive field than the standard convolution operation and can extract information more efficiently in the CP signal recovery problem. The second is denseNet, which connects all layers directly and can alleviate the vanishing-gradient problem. Experiment results show that the proposed DCNN-based method achieves superior performance to the traditional compressed sensing-based method both from the perspective of recovery error and time consumption.

Computer vision-based concrete crack detection using U-net fully convolutional networks

Abstract For the first time, U-Net is adopted to detect the concrete cracks in the present study. Focal loss function is selected as the evaluation function, and the Adam algorithm is applied for optimization. The trained U-Net is able of identifying the crack locations from the input raw images under various conditions (such as illumination, messy background, width of cracks, etc.) with high effectiveness and robustness. In addition, U-Net based concrete crack detection method proposed in the present study is compared with the DCNN-based method, and U-Net is found to be more elegant than DCNN with more robustness, more effectiveness and more accurate detection. Furthermore, by examining the fundamental parameters representing the performance of the method, the present U-Net is found to reach higher accuracy with smaller training set than the previous FCNs.

Joint correction of attenuation and scatter in image space using deep convolutional neural networks for dedicated brain 18F-FDG PET

Dedicated brain positron emission tomography (PET) devices can provide higher-resolution images with much lower doses compared to conventional whole-body PET systems, which is important to support PET neuroimaging and particularly useful for the diagnosis of neurodegenerative diseases. However, when a dedicated brain PET scanner does not come with a combined CT or transmission source, there is no direct solution for accurate attenuation and scatter correction, both of which are critical for quantitative PET. To address this problem, we propose joint attenuation and scatter correction (ASC) in image space for non-corrected PET (PETNC) using deep convolutional neural networks (DCNNs). This approach is a one-step process, distinct from conventional methods that rely on generating attenuation maps first that are then applied to iterative scatter simulation in sinogram space. For training and validation, time-of-flight PET/MR scans and additional helical CTs were performed for 35 subjects (25/10 split for training and test dataset). A DCNN model was proposed and trained to convert PETNC to DCNN-based ASC PET (PETDCNN) directly in image space. For quantitative evaluation, uptake differences between PETDCNN and reference CT-based ASC PET (PETCT-ASC) were computed for 116 automated anatomical labels (AALs) across 10 test subjects (1160 regions in total). MR-based ASC PET (PETMR-ASC), a current clinical protocol in PET/MR imaging, was another reference for comparison. Statistical significance was assessed using a paired t test. The performance of PETDCNN was comparable to that of PETMR-ASC, in comparison to reference PETCT-ASC. The mean SUV differences (mean ± SD) from PETCT-ASC were 4.0% ± 15.4% (P < 0.001) and −4.2% ± 4.3% (P < 0.001) for PETDCNN and PETMR-ASC, respectively. The overall larger variation of PETDCNN (15.4%) was prone to the subject with the highest mean difference (48.5% ± 10.4%). The mean difference of PETDCNN excluding the subject was substantially improved to −0.8% ± 5.2% (P < 0.001), which was lower than that of PETMR-ASC (−5.07% ± 3.60%, P < 0.001). In conclusion, we demonstrated the feasibility of directly producing PET images corrected for attenuation and scatter using a DCNN (PETDCNN) from PETNC in image space without requiring conventional attenuation map generation and time-consuming scatter correction. Additionally, our DCNN-based method provides a possible alternative to MR-ASC for simultaneous PET/MRI.

A deep convolutional neural network approach for astrocyte detection

Astrocytes are involved in various brain pathologies including trauma, stroke, neurodegenerative disorders such as Alzheimer’s and Parkinson’s diseases, or chronic pain. Determining cell density in a complex tissue environment in microscopy images and elucidating the temporal characteristics of morphological and biochemical changes is essential to understand the role of astrocytes in physiological and pathological conditions. Nowadays, manual stereological cell counting or semi-automatic segmentation techniques are widely used for the quantitative analysis of microscopy images. Detecting astrocytes automatically is a highly challenging computational task, for which we currently lack efficient image analysis tools. We have developed a fast and fully automated software that assesses the number of astrocytes using Deep Convolutional Neural Networks (DCNN). The method highly outperforms state-of-the-art image analysis and machine learning methods and provides precision comparable to those of human experts. Additionally, the runtime of cell detection is significantly less than that of other three computational methods analysed, and it is faster than human observers by orders of magnitude. We applied our DCNN-based method to examine the number of astrocytes in different brain regions of rats with opioid-induced hyperalgesia/tolerance (OIH/OIT), as morphine tolerance is believed to activate glia. We have demonstrated a strong positive correlation between manual and DCNN-based quantification of astrocytes in rat brain.