Three-dimensional Convolutional Neural Network Model Research Articles

Three-dimensional convolutional neural network-based classification of chronic kidney disease severity using kidney MRI

A three-dimensional convolutional neural network model was developed to classify the severity of chronic kidney disease (CKD) using magnetic resonance imaging (MRI) Dixon-based T1-weighted in-phase (IP)/opposed-phase (OP)/water-only (WO) imaging. Seventy-three patients with severe renal dysfunction (estimated glomerular filtration rate [eGFR] < 30 mL/min/1.73 m2, CKD stage G4–5); 172 with moderate renal dysfunction (30 ≤ eGFR < 60 mL/min/1.73 m2, CKD stage G3a/b); and 76 with mild renal dysfunction (eGFR ≥ 60 mL/min/1.73 m2, CKD stage G1–2) participated in this study. The model was applied to the right, left, and both kidneys, as well as to each imaging method (T1-weighted IP/OP/WO images). The best performance was obtained when using bilateral kidneys and IP images, with an accuracy of 0.862 ± 0.036. The overall accuracy was better for the bilateral kidney models than for the unilateral kidney models. Our deep learning approach using kidney MRI can be applied to classify patients with CKD based on the severity of kidney disease.

Combining three-dimensional acoustic coring and a convolutional neural network to quantify species contributions to benthic ecosystems

The seafloor is inhabited by a large number of benthic invertebrates, and their importance in mediating carbon mineralization and biogeochemical cycles is recognized. However, the majority of fauna live below the sediment surface, so most means of survey rely on destructive sampling methods that are limited to documenting species presence rather than event driven activity and functionally important aspects of species behaviour. We have developed and tested a laboratory-based three-dimensional acoustic coring system that is capable of non-invasively visualizing the presence and activity of invertebrates within the sediment matrix. Here, we present reconstructed three-dimensional acoustic images of the sediment profile, with strong backscatter revealing the presence and position of individual benthic organisms. These data were used to train a three-dimensional convolutional neural network model and, using a combination of data augmentation and data correction techniques, we were able to identify individual species with an 88% accuracy. Combining three-dimensional acoustic coring with deep learning forms an effective and non-invasive means of providing detailed mechanistic information of in situ species–sediment interactions, opening new opportunities to quantify species-specific contributions to ecosystems.

Automated Radiomic Analysis of Vestibular Schwannomas and Inner Ears Using Contrast-Enhanced T1-Weighted and T2-Weighted Magnetic Resonance Imaging Sequences and Artificial Intelligence.

To objectively evaluate vestibular schwannomas (VSs) and their spatial relationships with the ipsilateral inner ear (IE) in magnetic resonance imaging (MRI) using deep learning. Cross-sectional study. A total of 490 adults with VS, high-resolution MRI scans, and no previous neurotologic surgery. MRI studies of VS patients were split into training (390 patients) and test (100 patients) sets. A three-dimensional convolutional neural network model was trained to segment VS and IE structures using contrast-enhanced T1-weighted and T2-weighted sequences, respectively. Manual segmentations were used as ground truths. Model performance was evaluated on the test set and on an external set of 100 VS patients from a public data set (Vestibular-Schwannoma-SEG). Dice score, relative volume error, average symmetric surface distance, 95th-percentile Hausdorff distance, and centroid locations. Dice scores for VS and IE volume segmentations were 0.91 and 0.90, respectively. On the public data set, the model segmented VS tumors with a Dice score of 0.89 ± 0.06 (mean ± standard deviation), relative volume error of 9.8 ± 9.6%, average symmetric surface distance of 0.31 ± 0.22 mm, and 95th-percentile Hausdorff distance of 1.26 ± 0.76 mm. Predicted VS segmentations overlapped with ground truth segmentations in all test subjects. Mean errors of predicted VS volume, VS centroid location, and IE centroid location were 0.05 cm 3 , 0.52 mm, and 0.85 mm, respectively. A deep learning system can segment VS and IE structures in high-resolution MRI scans with excellent accuracy. This technology offers promise to improve the clinical workflow for assessing VS radiomics and enhance the management of VS patients.

Three-dimensional convolutional neural network model to identify clinically significant prostate cancer in transrectal ultrasound videos: a prospective, multi-institutional, diagnostic study.

Identifying patients with clinically significant prostate cancer (csPCa) before biopsy helps reduce unnecessary biopsies and improve patient prognosis. The diagnostic performance of traditional transrectal ultrasound (TRUS) for csPCa is relatively limited. This study was aimed to develop a high-performance convolutional neural network (CNN) model (P-Net) based on a TRUS video of the entire prostate and investigate its efficacy in identifying csPCa. Between January 2021 and December 2022, this study prospectively evaluated 832 patients from four centres who underwent prostate biopsy and/or radical prostatectomy. All patients had a standardised TRUS video of the whole prostate. A two-dimensional CNN (2D P-Net) and three-dimensional CNN (3D P-Net) were constructed using the training cohort (559 patients) and tested on the internal validation cohort (140 patients) as well as on the external validation cohort (133 patients). The performance of 2D P-Net and 3D P-Net in predicting csPCa was assessed in terms of the area under the receiver operating characteristic curve (AUC), biopsy rate, and unnecessary biopsy rate, and compared with the TRUS 5-point Likert score system as well as multiparametric magnetic resonance imaging (mp-MRI) prostate imaging reporting and data system (PI-RADS) v2.1. Decision curve analyses (DCAs) were used to determine the net benefits associated with their use. The study is registered at https://www.chictr.org.cn with the unique identifier ChiCTR2200064545. The diagnostic performance of 3D P-Net (AUC: 0.85-0.89) was superior to TRUS 5-point Likert score system (AUC: 0.71-0.78, P=0.003-0.040), and similar to mp-MRI PI-RADS v2.1 score system interpreted by experienced radiologists (AUC: 0.83-0.86, P=0.460-0.732) and 2D P-Net (AUC: 0.79-0.86, P=0.066-0.678) in the internal and external validation cohorts. The biopsy rate decreased from 40.3% (TRUS 5-point Likert score system) and 47.6% (mp-MRI PI-RADS v2.1 score system) to 35.5% (2D P-Net) and 34.0% (3D P-Net). The unnecessary biopsy rate decreased from 38.1% (TRUS 5-point Likert score system) and 35.2% (mp-MRI PI-RADS v2.1 score system) to 32.0% (2D P-Net) and 25.8% (3D P-Net). 3D P-Net yielded the highest net benefit according to the DCAs. 3D P-Net based on a prostate grayscale TRUS video achieved satisfactory performance in identifying csPCa and potentially reducing unnecessary biopsies. More studies to determine how AI models better integrate into routine practice and randomized controlled trials to show the values of these models in real clinical applications are warranted. The National Natural Science Foundation of China (Grants 82202174 and 82202153), the Science and Technology Commission of Shanghai Municipality (Grants 18441905500 and 19DZ2251100), Shanghai Municipal Health Commission (Grants 2019LJ21 and SHSLCZDZK03502), Shanghai Science and Technology Innovation Action Plan (21Y11911200), and Fundamental Research Funds for the Central Universities (ZD-11-202151), Scientific Research and Development Fund of Zhongshan Hospital of Fudan University (Grant 2022ZSQD07).

Identification of metal ion-binding sites in RNA structures using deep learning method.

Metal ion is an indispensable factor for the proper folding, structural stability and functioning of RNA molecules. However, it is very difficult for experimental methods to detect them in RNAs. With the increase of experimentally resolved RNA structures, it becomes possible to identify the metal ion-binding sites in RNA structures through in-silico methods. Here, we propose an approach called Metal3DRNA to identify the binding sites of the most common metal ions (Mg2+, Na+ and K+) in RNA structures by using a three-dimensional convolutional neural network model. The negative samples, screened out based on the analysis for binding surroundings of metal ions, are more like positive ones than the randomly selected ones, which are beneficial to a powerful predictor construction. The microenvironments of the spatial distributions of C, O, N and P atoms around a sample are extracted as features. Metal3DRNA shows a promising prediction power, generally surpassing the state-of-the-art methods FEATURE and MetalionRNA. Finally, utilizing the visualization method, we inspect the contributions of nucleotide atoms to the classification in several cases, which provides a visualization that helps to comprehend the model. The method will be helpful for RNA structure prediction and dynamics simulation study. Availability and implementation: The source code is available at https://github.com/ChunhuaLiLab/Metal3DRNA.

Improving precipitation nowcasting using a three-dimensional convolutional neural network model from Multi Parameter Phased Array Weather Radar observations

In this paper, a three-dimensional convolutional neural network model (3DCNN) is proposed to nowcast a short-lived, local convective storm event by using unique 3-D observations of Multi-Parameter Phased Array Weather Radar (MP-PAWR) over Tokyo, Japan on 1 August 2019. Using statistics and forecast skill scores, nowcast skills of 3DCNN were examined with those of a three-dimensional advection nowcast model (3DNOW) which generates extrapolation-based forecasts with lead time up to 10 min. In analyzing the skill scores, two groups of a total rain area and convective rain area were made by different radar reflectivity (Z) thresholds of 10 dBZ and 37.5 dBZ, respectively. For the total rain area, it is found that 3DCNN outperformed both the 3DNOW and persistence forecast, showing the higher threat scores for all lead times. For the convective rain area, the 3DCNN and 3DNOW's performances were similar at early lead times, showing almost the same threat scores. However, the threat score of 3DCNN dropped lower than that of 3DNOW at a lead time of 10 min, indicating that 3DNOW has the better skill at relatively long lead times. Nowcasts of 3DNOW showed a limitation to yield a larger saturated Z area related to increased errors in advection vectors at longer lead times although this led to increasing the threat score.

Convolutional Neural Networks for Differential Diagnosis of Raynaud’s Phenomenon Based on Hands Thermal Patterns

Raynaud’s phenomenon (RP) is a microvessels’ disorder resulting in transient ischemia. It can be either primary or secondary to connective tissue diseases, such as systemic sclerosis. The differentiation between primary and secondary to systemic sclerosis is of paramount importance to set the proper therapeutic strategy. Thus far, thermal infrared imaging has been employed to accomplish this task by monitoring the finger temperature response to a controlled cold challenge. A completely automated methodology based on deep convolutional neural network is here introduced with the purpose of being able to differentiate systemic sclerosis from primary RP patients by relying uniquely on thermal images of the hands acquired at rest. The classification performance of such a method was compared to that of a three-dimensional convolutional neural network model implemented to classify thermal images of the hands recorded during rewarming from a cold challenge. No significant differences were found between the two procedures, thus ensuring the possibility to avoid the cold challenge. Moreover, the convolutional neural network models were compared with standard feature-based approaches and showed higher performances, thus overcoming the limitations related to the feature extraction (e.g., biases introduced by the operator). Such automated procedures can constitute promising tools for large scale screening of primary RP and secondary to systemic sclerosis in clinical practice.

Video Abnormal Event Detection Based on Optical Flow and 3DCNN

With the advancement of technology, the detection of abnormal events in videos has extensive theoretical and practical significance. Most events in videos are normal events, but a small number of abnormal events contain a lot of information. How to identify individual abnormal events from massive videos is the focus of concern. The paper made full use of the spatio-temporal continuity of video images. Firstly, a space-time cube was constructed by integrating optical flow information; secondly, a multi-layer three-dimensional convolutional neural network model was constructed; finally, the entire process of video abnormal event detection was realized based on this model. The experiment on the UCSD dataset showed that the method in this paper can effectively detect abnormal events in videos.

Convolutional Neural Network in the Evaluation of Myocardial Ischemia from CZT SPECT Myocardial Perfusion Imaging: Comparison to Automated Quantification

This study analyzes CZT SPECT myocardial perfusion images that are collected at Chang Gung Memorial Hospital, Kaohsiung Medical Center in Kaohsiung. This study focuses on the classification of myocardial perfusion images for coronary heart diseases by convolutional neural network techniques. In these gray scale images, heart blood flow distribution contains the most important features. Therefore, data-driven preprocessing is developed to extract the area of interest. After removing the surrounding noise, the three-dimensional convolutional neural network model is utilized to classify whether the patient has coronary heart diseases or not. The prediction accuracy, sensitivity, and specificity are 87.64%, 81.58%, and 92.16%. The prototype system will greatly reduce the time required for physician image interpretation and write reports. It can assist clinical experts in diagnosing coronary heart diseases accurately in practice.

Three-dimensional convolutional neural network model for tree species classification using airborne hyperspectral images

Airborne hyperspectral remote sensing data with both rich spectral and spatial features can effectively improve the classification accuracy of vegetation species. However, the spectral data of hundreds of bands brings about problems such as dimensional explosion, which poses a huge challenge for hyperspectral remote sensing classification based on classical parameters models. Deep learning methods have been used for remotely sensed images classification in recent years, but the popular HSI datasets including Kennedy Space Center, Indian Pines, Pavia University scene and Salinas scene, have low spatial resolution, significant differences between categories, and regular boundaries. When applied to the classification of forestry tree species, the accuracy often decreases because the spectral response of different plants of the same family and genus are very similar, especially under the fragmented species distribution, complex topography and the occluded canopy. So we collect new data sets, selected Gaofeng State Owned Forest Farm in Guangxi province in south China as the research area and adopted the airborne hyperspectral data obtained by the LiCHy system of the Chinese Academy of Forestry to explore an improved three-dimensional convolutional neural network(3D-CNN) model for tree species classification. The proposed model uses raw data as input without dimension reduction or feature screening, and simultaneously extracts spectral and spatial features. After the 3D convolutional layer, the captured high-level semantic concept is a joint spatial spectral feature representation, so we can turn it into a one-dimensional feature as a new input to learn a more abstract level of expression. The widely used earlystop method is also used to prevent overfitting. The proposed model is a lightweight, generalized, and fast convergence classification model, by which the short-time and large-area of multiple tree species classification with high-precision can be realized. The result shows that the 3D-1D CNN model can shorten the training time of the 3D CNN model by 60% and achieve a classification accuracy of 93.14% within 50 ha in 6.37 min, which provides a basis for the classification of tree species, the mapping of forest form and the inventory of forest resources.

Research on CT Scan Image of Lung Cancer Based on Deep Learning Method in Artificial Intelligence Field

Cancer is one of the diseases with high mortality in the 21st century, and lung cancer ranks first in all cancer morbidity and mortality. In recent years, with the rise of big data and artificial intelligence, lung cancer-assisted diagnosis based on deep learning has gradually become A popular research topic. Computer-aided lung cancer diagnosis technology is mainly the process of processing and analyzing the lung image data obtained by medical instrument imaging. The process is summarized into four steps: medical image data preprocessing, lung parenchymal segmentation, lung Nodule detection and segmentation, as well as lesion diagnosis. In order to solve the problem that the two-dimensional image model is not applicable to three-dimensional images, this paper proposes a three-dimensional convolutional neural network model suitable for lung cancer diagnosis. The model consists of two parts. The first part is a three-dimensional deep nodule detection network (FCN) model, which generates a heat map of the lung nodules. We can locate the locations of those malignant nodules through the heat map. According to the heat map generated in the first part, the second part selects those malignant nodules that are likely to be large, and then fuses the features of these selected nodules into one feature vector, showing the whole lung scan. Finally, we use this feature to classify and determine whether we have lung cancer.

4942Automated recognition of regional wall motion abnormalities by deep neural network interpretation of echocardiography

Abstract Background The deep neural network assisted in automated echocardiography interpretation joint to cardiologist final confirmation has now been gradually emerging. There were applications applied in echocardiography views classification, chamber size and myocardium mass evaluation, and certain disease detections already published. Our aim, instead of frame-by-frame “image-level” interpretation in previous studies, is to apply deep neural network in echocardiography temporal relationship analysis – “video-level” – and applied in automated left ventricle myocardium regional wall motion abnormalities recognition. Methods We collected all echocardiography performed in 2017, and preprocessed them into numeric arrays for matrix computations. Regional wall motion abnormalities were approved by authorized cardiologists, and processed into labels whether regional wall motion abnormalities presented in anterior, inferior, septal, or lateral walls of the left ventricle, as the ground truth. We then first developed a convolutional neural network (CNN) model to do view selection, and gathered parasternal long/short views, and apical four/two chamber views from each exam, as well as developing view prediction confidence for strict image quality control. Within these images, we annotated part of images to develop the second CNN model, known as U-net, for image segmentation and mark each regional wall. Finally, we developed the major three-dimensional CNN model with the inputs composed of four views of echocardiography videos and then output the final label for motion abnormalities in each wall. Results In total we collected 13,984 series of echocardiography, and gathered four main views with quality confidence level above 90%, which resulted in 9,323 series for training. Within these images, we annotated 2,736 frames for U-net model and resulted in dice score of segmentation 73%. With the join of segmentation model, the final three-dimensional CNN model predict regional wall motion with accuracy of 83%. Conclusions Deep neural network application in regional wall motion recognition is feasible and should mandate further investigation for promoting performance. Acknowledgement/Funding None