Deep Residual Network Model Research Articles

An intelligent bone age assessment model incorporating multilayer superimposed texture enhancement and the China-05 attention mechanism

In the process of intelligent bone age assessment for Chinese ethnicity, generalized convolutional network models are not well targeted in extracting specific features in medical skeletal images and lack specificity in training and predicting skeletal developmental features in different ethnicities. This study aims to propose a hybrid improved deep residual network model, ZH05-DL-ResNet50, focusing on intelligent bone age assessment for Chinese ethnicity. In the process of building the ZH05-DL-ResNet50 model. First, multiple overlapping texture enhancement layers are introduced through combinatorial optimization, which can better characterize the global features of hand bone radiographs while using less texture information, reducing the interference of redundant information and freeing up computational power; second, the China-05′s spatial focusing mechanism is designed so that the model can intelligently focus on the region of interest, efficiently locate and automatically learn key image information; Finally, the model can be used for intelligent bone age assessment of Chinese ethnicity by constructing a 50-layer residual depth network, the superposition enhancement layer and the focusing mechanism are fused to form the ZH05-DL-ResNet50 model, which mitigates the problem of gradient disappearance and explosion caused by the increase of network depth, and reduces the information loss and depletion of the multi-layer texture superposition features in the focus region of interest. The training was performed using the Chinese ethnographic dataset provided and created by Tsinghua Changgeng Hospital, where 10 % of the radiographs were used as the test set and the rest as the training and validation sets. The overall performance of the models was estimated by comparing the average accuracy and loss values of different models, and the overall performance between model training and model design was evaluated using the data distribution (Distribution) and histogram of bias weight distribution (Histogram). The results showed that the average accuracy of the bone age estimation model was 98.1 % and the average error on the test set was 0.312 years. Evaluation of the model visualization and accuracy comparison showed that the model performed well and was more appropriate and accurate than other bone age estimation models. Therefore, the first intelligent bone age assessment model for Chinese ethnicity, ZH05-DL-ResNet50, was created, which differs from the untargeted nature of other intelligent bone age assessment models, and is used to train and predict hand bone radiographs more appropriately and accurately for Chinese ethnicity.

NDMNN: A novel deep residual network based MNN method to remove batch effects from scRNA-seq data.

The rapid development of single-cell RNA sequencing (scRNA-seq) technology has generated vast amounts of data. However, these data often exhibit batch effects due to various factors such as different time points, experimental personnel, and instruments used, which can obscure the biological differences in the data itself. Based on the characteristics of scRNA-seq data, we designed a dense deep residual network model, referred to as NDnetwork. Subsequently, we combined the NDnetwork model with the MNN method to correct batch effects in scRNA-seq data, and named it the NDMNN method. Comprehensive experimental results demonstrate that the NDMNN method outperforms existing commonly used methods for correcting batch effects in scRNA-seq data. As the scale of single-cell sequencing continues to expand, we believe that NDMNN will be a valuable tool for researchers in the biological community for correcting batch effects in their studies. The source code and experimental results of the NDMNN method can be found at https://github.com/mustang-hub/NDMNN.

Application value of artificial intelligence algorithm-based magnetic resonance multi-sequence imaging in staging diagnosis of cervical cancer.

The aim of this research is to explore the application value of Deep residual network model (DRN) for deep learning-based multi-sequence magnetic resonance imaging (MRI) in the staging diagnosis of cervical cancer (CC). This research included 90 patients diagnosed with CC between August 2019 and May 2021 at the hospital. After undergoing MRI examination, the clinical staging and surgical pathological staging of patients were conducted. The research then evaluated the results of clinical staging and MRI staging to assess their diagnostic accuracy and correlation. In the staging diagnosis of CC, the feature enhancement layer was added to the DRN model, and the MRI imaging features of CC were used to enhance the image information. The precision, specificity, and sensitivity of the constructed model were analyzed, and then the accuracy of clinical diagnosis staging and MRI staging were compared. As the model constructed DRN in this research was compared with convolutional neural network (CNN) and the classic deep neural network visual geometry group (VGG), the precision was 67.7, 84.9, and 93.6%, respectively. The sensitivity was 70.4, 82.5, and 91.2%, while the specificity was 68.5, 83.8, and 92.2%, respectively. The precision, sensitivity, and specificity of the model were remarkably higher than those of CNN and VGG models (P < 0.05). As the clinical staging and MRI staging of CC were compared, the diagnostic accuracy of MRI was 100%, while that of clinical diagnosis was 83.7%, showing a significant difference between them (P < 0.05). Multi-sequence MRI under intelligent algorithm had a high diagnostic rate for CC staging, deserving a good clinical application value.

Deep learning-based automated high-accuracy location and identification of fresh vertebral compression fractures from spinal radiographs: a multicenter cohort study.

Digital radiography (DR) is a common and widely available examination. However, spinal DR cannot detect bone marrow edema, therefore, determining vertebral compression fractures (VCFs), especially fresh VCFs, remains challenging for clinicians. We trained, validated, and externally tested the deep residual network (DRN) model that automated the detection and identification of fresh VCFs from spinal DR images. A total of 1,747 participants from five institutions were enrolled in this study and divided into the training cohort, validation cohort and external test cohorts (YHDH and BMUH cohorts). We evaluated the performance of DRN model based on the area under the receiver operating characteristic curve (AUC), feature attention maps, sensitivity, specificity, and accuracy. We compared it with five other deep learning models and validated and tested the model internally and externally and explored whether it remains highly accurate for an external test cohort. In addition, the influence of old VCFs on the performance of the DRN model was assessed. The AUC was 0.99, 0.89, and 0.88 in the validation, YHDH, and BMUH cohorts, respectively, for the DRN model for detecting and discriminating fresh VCFs. The accuracies were 81.45% and 72.90%, sensitivities were 84.75% and 91.43%, and specificities were 80.25% and 63.89% in the YHDH and BMUH cohorts, respectively. The DRN model generated correct activation on the fresh VCFs and accurate peak responses on the area of the target vertebral body parts and demonstrated better feature representation learning and classification performance. The AUC was 0.90 (95% confidence interval [CI] 0.84-0.95) and 0.84 (95% CI 0.72-0.93) in the non-old VCFs and old VCFs groups, respectively, in the YHDH cohort (p = 0.067). The AUC was 0.89 (95% CI 0.84-0.94) and 0.85 (95% CI 0.72-0.95) in the non-old VCFs and old VCFs groups, respectively, in the BMUH cohort (p = 0.051). In present study, we developed the DRN model for automated diagnosis and identification of fresh VCFs from spinal DR images. The DRN model can provide interpretable attention maps to support the excellent prediction results, which is the key that most clinicians care about when using the model to assist decision-making.

WITHDRAWN: Sewing-driving training optimization-enabled feature selection and hand gesture recognition with deep residual network using sEMG signal

An innovation of an accurate model along with low latency for hand gesture identification is still a huge challenge especially for rehabilitation. This research provides a unified solution for such limitations by constructing an efficient system for hand gesture recognition by employing designed Sewing Driving Training based Optimization (SDTO)_Deep Residual Network (DRN). In this work, pre-processing is done using Gaussian filtering, whereas feature selection is carried out based on SDTO. In order to attain efficient results, appropriate features are extracted at the feature extraction phase. The last step is the hand gesture recognition phase, which is accomplished effectively using DRN and the network is efficiently trained using designed SDTO and it is achieved by the integration of STBO and DTBO. The introduced SDTO_DRN model for hand gesture recognition has attained high accuracy of 0.943, TPR of 0.929, TNR of 0.919, PPV of 0.924, and NPV of 0.924.

Low-dose CT image quality evaluation method based on radiomics and deep residual network with attention mechanism

Low-dose computed tomography (CT) produces less radiation but may degrade image quality. Research on automatic evaluation and classification of low-dose CT image quality is necessary, which is helpful to clinicians. A low-dose CT image quality evaluation method is proposed based on radiomics and deep learning. After pre-training on the ImageNet dataset, the trained deep learning model is then transferred to the target dataset of low-dose CT image. Region of interest is obtained through a deep residual network with attention mechanism. Pyradiomics is used to extract radiomics features of ROI, and T-test and LASSO algorithm are used for feature screening. Finally, support vector machine, random forest and logistic regression models are adopted to classify and predict the quality of low-dose CT image based on the selected radiomics features. Experimental results show that the trained deep residual network model with attention module can achieve 92% recognition accuracy of region of interest, and the evaluation accuracy of the three machine learning models achieve 86.6% (support vector machine), 84.5% (random forest) and 61.7% (logistic regression), respectively. Transfer learning can effectively reduce the required number of samples for the training of deep learning model and avoid overfitting. After the introduction of attention module in deep residual network model, accuracy rises faster, and the final accuracy is slightly higher than that of deep residual network model without attention module, but the improvement is slightly lower than expected. Radiomics features of region of interest reflect the quality differences of low-dose CT images which can be employed for the quality evaluation of low-dose CT image.

An autoencoder deep residual network model for multi focus image fusion

An autoencoder deep residual network model for multi focus image fusion

Segmentation and classification of medical big data on brain tumor using bacteria foraging optimization algorithm along with learning automata

AbstractDetecting the early stage of brain tumors is significant for an effective therapy that can probably minimize the death rate of patients affected from brain tumors. Magnetic resonance imaging is the benchmark standard for diagnosing brain cancers but, it was difficult to split and categorize different kinds of brain tumors due to the delicate arrangement of the brain's anatomy. To overcome these difficulties and provide an effective classification, this research introduced a hybridized optimization technique which enhance the performance of the classifier. The hybridization is done between Bacteria Foraging Optimization Algorithm (BFOA) along with Learning Automata (LA), these two techniques improve the search speed and automate the learning capability of Convolutional Neural Network (CNN) classifier. The obtained results show that proposed BFOA LN‐CNN classifier better than the existing Deep Convolutional Neural Network (DCNN) based on Improved Harris Hawks Optimization (HHO) known as G‐HHO and Improved Invasive Bat (IIB)‐based Deep Residual network model. The classification accuracy of the proposed method is 99.41% whereas the DCNN‐G‐HHO and IIB deep residual neural network provides accuracy of 97% and 91.9% respectively.

Remote Sensing Image Fusion Method Based on Progressive Cascaded Deep Residual Network

With the rapid development of deep learning in recent years, it has shown excellent performance in various image and video processing tasks. In addition, it also has a great role in promoting the spatio-temporal fusion of remote sensing images. The reconstructed image can give people a good visual experience. The invention relates to a remote sensing image fusion method based on a progressive cascade deep residual network and provides an end-to-end progressive cascade deep residual network model for remote sensing image fusion. The use of the MSE loss function may cause oversmoothing of the fused image, so a new joint loss function is defined to capture finer spatial information to improve the spatial resolution of the fused image. Resize-convolution is used to replace the transposed convolution to eliminate the checkerboard effect in the fused image caused by the transposed convolution. Through the experiments on the remote sensing image fusion simulation and real datasets of multiple satellites, the data results of the proposed algorithm are more than 5.25% better than those of the comparative algorithm in the average quantification. The calculation time and system resource occupation are also reduced, which has important theoretical significance and application value in the field of artificial intelligence and image processing. It will play a certain role in promoting the theoretical research and application of remote sensing image fusion.

A Combination of Deep Autoencoder and Multi-Scale Residual Network for Landslide Susceptibility Evaluation

Landslide susceptibility evaluation can accurately predict the spatial distribution of potential landslides, which offers great usefulness for disaster prevention, disaster reduction, and land resource management. Aiming at the problems of insufficient samples for landslide compilation, difficulty in expanding landslide samples, and insufficient expression of nonlinear relationships among evaluation factors, this paper proposes a new evaluation method of landslide susceptibility combining deep autoencoder and multi-scale residual network (DAE-MRCNN). In the first step, a deep autoencoder network was used to learn the feature expression of the original landslide data in order to acquire effective features in the data. Next, a multi-scale residual network was constructed; specifically, the model was improved into a deep residual network model by adding skip connections in the convolutional layer. In addition, the multi-scale idea was utilized to fully extract the scale characteristics of the evaluation factors. Finally, the model was used for feature training, and the results were input into the Softmax classifier to complete the prediction of landslide susceptibility. For this purpose, a machine learning method and two state-of-the-art deep learning methods, namely SVM, CPCNN-ML, and 2D-CNN, were utilized to model landslide susceptibility in Hanzhong City, Shaanxi Province. The proposed method produced the highest model performance of 0.891, followed by 0.842, 0.869, and 0.873. The experimental results show that the DAE-MRCNN method can fully express the complex nonlinear relationships among the evaluation factors, alleviate the problem of insufficient samples in convolutional neural networks (CNN) training, and significantly improve the accuracy of susceptibility prediction.

Improving ECG Classification Performance by Using an Optimized One-Dimensional Residual Network Model

Cardiovascular disease and its consequences on human health have never stopped and even show a trend of appearing in increasingly younger generations. The establishment of an excellent deep learning algorithm model to assist physicians in identifying and the early screening of ECG abnormalities can effectively improve the accuracy of diagnosis. Therefore, in this study, the deep residual network model is adapted for feature extraction and classification of ECG signals by pooling embedded into layers and double channel connection. At the same time, the wavelet adaptive threshold denoising algorithm is used to complete the high signal-to-noise filtering of ECG signals. Then, the alternate pooling residual network (APRN) is compared with the convolutional neural network (CNN), CNN with one residual unit (CNN-R), and the deep residual network (ResNet-18) using ECG datasets from the American MIT-BIH arrhythmia and ST segment abnormality database, European ST-T database, and sudden cardiac death ambulatory ECG database. The results are as follows: The average classification accuracy of the APRN on the four datasets is 97.89%, while the accuracies on CNN, CNN-R, and ResNet-18 are 97.17%, 97.53%, and 97.73%, respectively. In addition, compared with ResNet-18, the classification accuracy of our APRN on each class of data improves by 16.44% in total.

Non-invasive precise staging of liver fibrosis using deep residual network model based on plain CT images.

The aim of this study was to explore the application of five-class deep residual network models based on plain CT images and clinical features for the precise staging of liver fibrosis. This retrospective clinical study included 347 patients who underwent liver CT, with pathological staging of liver fibrosis as the gold standard. We established three ResNet models to stage liver fibrosis. The output diagnosis labels of models were 0, 1, 2, 3 and 4, which correspond to F0, F1, F2, F3, and F4 stages. Confusion matrices were used to evaluate the performances of models to precisely stage liver fibrosis. The performance for diagnosing cirrhosis (F4), advanced fibrosis (≥ F3) and significant fibrosis (≥ F2) of models was evaluated with receiver operating characteristic (ROC) analyses. The kappa coefficients of the five-class ResNet model (based on plain CT images), the five-class ResNet clinical model (based on clinical features), and the five-class mixed ResNet model (based on plain CT images and clinical features) for precise staging liver fibrosis were 0.566, 0.306, and 0.63, respectively. The recall rates and precision rates for F0, F1, F2, and F3 of three models were lower than 60%. The ROC AUC values of the five-class ResNet model, the five-class ResNet clinical model, and the five-class mixed ResNet model for diagnosing cirrhosis, advanced fibrosis, and significant fibrosis were 0.95, 0.88, and 0.82, 0.80, 0.72, and 0.70, 0.95, 0.90, and 0.83, respectively. The five-class ResNet models are of high value in the diagnosis of liver cirrhosis, advanced liver fibrosis, and significant liver fibrosis. However, for the precise staging of liver fibrosis, the models cannot accurately distinguish other liver fibrosis stages except F4. Plain CT images combined with clinical features have the potential to improve the performance of the ResNet models in diagnosing liver fibrosis.

Classification of Coarse Aggregate Particle Size Based on Deep Residual Network

Traditional aggregate particle size detection mainly relies on manual batch sieving, which is time-consuming and inefficiency. To achieve rapid automatic detection of aggregate particle sizes, a mechanical symmetric classification model of coarse aggregate particle size, based on a deep residual network, is proposed in this paper. First, aggregate images are collected by the optical vertical projection acquisition platform. The collected aggregate images are corrected, and their geometric parameters are extracted. Second, various digital image processing methods, such as size correction and morphological processing, are used to improve the image quality and enlarge the image dataset of different aggregate particle sizes. Then, the deep residual network model (ResNet50) is built to train the aggregate image dataset to achieve accurate classification of aggregate sizes. Finally, compared with the traditional single geometric particle size classification model, the comparative results show that the accuracy of the coarse aggregate classification model proposed in this paper is nearly 20% higher than that of the traditional method, reaching 0.833. The proposed model realizes the automatic classification of coarse aggregate particle size, which can significantly improve the efficiency of aggregate automatic detection.

DRN-SEAM: A deep residual network based on squeeze-and-excitation attention mechanism for motion recognition in education

In order to solve the shortcomings of the traditional motion recognition methods and obtain better motion recognition effect in education, this paper proposes a residual network based on Squeeze-and-Excitation attention mechanism. Deep residual network is widely used in various fields due to the high recognition accuracy. In this paper, the convolution layer, adjustment batch normalization layer and activation function layer in the deep residual network model are modified. Squeeze-and-Excitation (SE) attention mechanism is introduced to adjust the structure of network convolution kernel. This operation enhances the feature extraction ability of the new network model. Finally, the expansibility experiments are conducted on WISDM(Wireless Sensor Data Mining), and UCI(UC Irvine) data sets. In terms of F1, the value exceeds 90%. The results show that the proposed model is more accurate than other state-of-the-art posture recognition models. The proposed method can obtain the ideal motion recognition results.

Attention graph: Learning effective visual features for large-scale image classification

In recent years, the research of deep learning has received extensive attention, and many breakthroughs have been made in various fields. On this basis, a neural network with the attention mechanism has become a research hotspot. In this paper, we try to solve the image classification task by implementing channel and spatial attention mechanism which improve the expression ability of neural network model. Different from previous studies, we propose an attention module consisting of channel attention module (CAM) and spatial attention module (SAM). The proposed module derives attention graphs from channel dimension and spatial dimension respectively, then the input features are selectively learned according to the importance of the features. Besides, this module is lightweight and can be easily integrated into image classification algorithms. In the experiment, we combine the deep residual network model with the attention module and the experimental results show that the proposed method brings higher image classification accuracy. The channel attention module adds weight to the signals on different convolution channels to represent the correlation. For different channels, the higher the weight, the higher the correlation which required more attention. The main function of spatial attention is to capture the most informative part in the local feature graph, which is a supplement to channel attention. We evaluate our proposed module based on the ImageNet-1K and Cifar-100 respectively. Through a large number of comparative experiments, our proposed model achieved outstanding performance.

Action Recognition Based on CSI Signal Using Improved Deep Residual Network Model

In this paper, we propose an improved deep residual network model to recognize human actions. Action data is composed of channel state information signals, which are continuous fine-grained signals. We replaced the traditional identity connection with the shrinking threshold module. The module automatically adjusts the threshold of the action data signal, and filters out signals that are not related to the principal components. We use the attention mechanism to improve the memory of the network model to the action signal, so as to better recognize the action. To verify the validity of the experiment more accurately, we collected action data in two different environments. The experimental results show that the improved network model is much better than the traditional network in recognition. The accuracy of recognition in complex places can reach 92.85%, among which the recognition rate of raising hands is up to 96%. We combine the improved residual deep network model with channel state information action data, and prove the effectiveness of our model for classification through experimental data.

In-situ identification and recognition of multi-hand gestures using optimized deep residual network

The real-time perception of hand gestures in a deprived environment is a demanding machine vision task. The hand recognition operations are more strenuous with different illumination conditions and varying backgrounds. Robust recognition and classification are the vital steps to support effective human-machine interaction (HMI), virtual reality, etc. In this paper, the real-time hand action recognition is performed by using an optimized Deep Residual Network model. It incorporates a RetinaNet model for hand detection and a Depthwise Separable Convolutional (DSC) layer for precise hand gesture recognition. The proposed model overcomes the class imbalance problems encountered by the conventional single-stage hand action recognition algorithms. The integrated DSC layer reduces the computational parameters and enhances the recognition speed. The model utilizes a ResNet-101 CNN architecture as a Feature extractor. The model is trained and evaluated on the MITI-HD dataset and compared with the benchmark datasets (NUSHP-II, Senz-3D). The network achieved a higher Precision and Recall value for an IoU value of 0.5. It is realized that the RetinaNet-DSC model using ResNet-101 backbone network obtained higher Precision (99.21 %for AP0.5, 96.80%for AP0.75) for MITI-HD Dataset. Higher performance metrics are obtained for a value of γ= 2 and α= 0.25. The SGD with a momentum optimizer outperformed the other optimizers (Adam, RMSprop) for the datasets considered in the studies. The prediction time of the optimized deep residual network is 82 ms.

Research on Facial Expression Recognition Based on Improved Deep Residual Network Model

Facial expressions are the main external manifestations of human emotions. Facial expression recognition technology can be used in medical, investigation, education and other application scenarios. Aiming at the disadvantages of slow convergence speed and low recognition accuracy of traditional neural networks, in order to optimize the network more efficiently and improve the recognition rate of facial expressions, a facial expression recognition method based on improved convolutional neural networks is proposed.First, the deep residual network ResNet18 is improved, and then the improved ResNet18 network is used to extract global expression features of face images, and then the self-attention weighting module is introduced to calculate the expression features of each face image and output one The corresponding weights are used to weight the loss function. Finally, the two public expression data sets of CK+ and RAF-DB are used to experimentally verify the method in the article. The accuracy of facial expression recognition reaches 98.89% and respectively. 87.13%, compared with the original deep residual network ResNet18 model and other types of network models, the facial expression recognition rate has been significantly improved, which proves that this method can effectively improve the accuracy of facial expression recognition and has certain application value.

Media Information Dissemination Model of Wireless Networks Using Deep Residual Network

Information dissemination and its prediction in wireless networks is a challenging task. Researchers have studied the prediction process of media information dissemination in wireless networks using various methods. In this paper, we analyze information dissemination in wireless networks using a deep residual network model. In the proposed model, the relative weight of nodes and the dissemination probability of media information in wireless networks are obtained. The obtained information is the inputs into the deep residual network as features. The convolution feature extractor is used to obtain the details of the input features. Finally, the propagation information is classified according to the extracted features through the full connection layer. We have used the SELU activation function to optimize the deep residual network. In this way, a complete media information dissemination prediction of wireless networks is obtained. The simulation results show that the proposed model has fast convergence and a low bit error rate of information dissemination. It reflects the characteristics of media information dissemination in a wireless network in real-time applications. The results show accurate prediction of media information dissemination in wireless networks.

Prediction of protein secondary structure based on deep residual convolutional neural network

Proteins play a vital role in organisms, which suggests that in-depth study of the function of proteins is helpful to the application of proteins in a more accurate and effective way. Accordingly, protein structure will become the focus of discussion and research for a long time. In order to fully extract the effective information from the protein structure and improve the classification accuracy of the protein secondary sequence, a deep residual network model using different residual units was proposed to predict the secondary structure. This algorithm uses sliding window method to represent amino acid sequences and combines the powerful feature extraction ability of resent network. In this paper, the parameters of the neural network are debugged through experiments, and then the extracted features are classified and verified. The experimental results on CASP9, CASP10, CASP11 and CASP12 data sets imply that the improved deep residual network model based on different residual units can express amino acid sequences more accurately, which is more superior than existing methods.