Recurrent Residual Convolutional Neural Network Research Articles

Automatic road crack detection and analysis system based on deep feature fusion and edge structure extraction

ABSTRACT In order to solve the problem that the traditional crack image segmentation method cannot accurately extract the edge structure of road cracks, a flexible crack detection and identification system are provided. Firstly, a road crack dataset is built by processing the acquired images using the sliding window technique. A model called R2-AENet is proposed for training and testing. The model structure uses the recurrent residual convolutional neural network (RRCNN) to improve the convolutional structure and drive the accumulation of fracture features. The model decoder combines the attention-guided filtering module and the edge residuals module (ERB) better to extract the edge features of the feature map. As a result, it improves the crack segmentation performance. Finally, the research results are validated on the upgraded software platform. The experimental results demonstrate that the R2-AENet model achieves better segmentation results compared to the popular deep learning algorithm models. Evidently, the model achieves accuracy and recall of 0.982 and 0.849 on the collected dataset. Besides, the area under the curve (AUC) metric value can reach 0.992. In addition, the detection results in different data sets perform well, verifying that the model has exceptional robustness and can be applied to various environmental road inspection and maintenance projects.

Hierarchical extraction of cropland boundaries using Sentinel-2 time-series data in fragmented agricultural landscapes

Accurately extracting cropland field parcels from satellite data in fragmented agricultural landscapes presents huge challenges due to small field sizes and irregular field shapes. Here, we developed a hierarchical framework for agricultural field boundary extraction from Sentinel-2 satellites based on the concept of the degree in field fragmentation. Our framework focused on three tasks, including a core task for agricultural field extraction and two auxiliary tasks to address special situations in two different fragmented regions, namely the Plains-Basin scenes with relatively regular shapes and diverse crop types, and the Plateau-Hilly scenes, with irregular shapes and small field sizes. First, the field boundaries were delineated using a modified Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net), and post-processed using morphological filtering and the Douglas-Peucker algorithm (DPA). A rule-based decision fusion strategy assisted with a classification method was developed to address the under-segmentation problem in the Plains-Basin scenes, and an adaptive generalized buffer algorithm (AGBA) was used to address the incomplete-segmentation problem in the Plateau-Hilly scenes. We tested the proposed methods in Yangming County, Heilongjiang Province of China, using time series Sentinel-2 imageries with 10-m spatial resolution. Compared with the results from using R2U-Net or a single-temporal image alone, the fragmented field boundaries show an outstanding performance, with an overall accuracy of 86.42% and 84.15% and F1 score of 0.85 and 0.83 in Plains-Basin scenes and Plateau-Hilly scenes, respectively. Our proposed methods provide a feasible solution to finely extract field boundaries in highly fragmented and heterogeneous agricultural landscapes.

MTMC-AUR2CNet: Multi-textural multi-class attention recurrent residual convolutional neural network for COVID-19 classification using chest X-ray images

Coronavirus disease (COVID-19) has infected over 603 million confirmed cases as of September 2022, and its rapid spread has raised concerns worldwide. More than 6.4 million fatalities in confirmed patients have been reported. According to reports, the COVID-19 virus causes lung damage and rapidly mutates before the patient receives any diagnosis-specific medicine. Daily increasing COVID-19 cases and the limited number of diagnosis tool kits encourage the use of deep learning (DL) models to assist health care practitioners using chest X-ray (CXR) images. The CXR is a low radiation radiography tool available in hospitals to diagnose COVID-19 and combat this spread. We propose a Multi-Textural Multi-Class (MTMC) UNet-based Recurrent Residual Convolutional Neural Network (MTMC-UR2CNet) and MTMC-UR2CNet with attention mechanism (MTMC-AUR2CNet) for multi-class lung lobe segmentation of CXR images. The lung lobe segmentation output of MTMC-UR2CNet and MTMC-AUR2CNet are mapped individually with their input CXRs to generate the region of interest (ROI). The multi-textural features are extracted from the ROI of each proposed MTMC network. The extracted multi-textural features from ROI are fused and are trained to the Whale optimization algorithm (WOA) based DeepCNN classifier on classifying the CXR images into normal (healthy), COVID-19, viral pneumonia, and lung opacity. The experimental result shows that the MTMC-AUR2CNet has superior performance in multi-class lung lobe segmentation of CXR images with an accuracy of 99.47%, followed by MTMC-UR2CNet with an accuracy of 98.39%. Also, MTMC-AUR2CNet improves the multi-textural multi-class classification accuracy of the WOA-based DeepCNN classifier to 97.60% compared to MTMC-UR2CNet.

DRA-Net: A Dual-Branch Residual Attention Network for Pixelwise Power Line Detection

The accurate extraction of transmission lines from UAV aerial photography is a necessary but challenging task for intelligent power inspection due to some factors, such as complex backgrounds, varying illumination, and class imbalance issue. Although there have been some research achievements in transmission line extraction, especially U-Net and its variant networks, there are still some shortcomings, such as insufficiency processing of local features, limited noise denoising ability, poor edge feature representation ability, etc. In this paper, we propose a dual-branch residual attention network, called DRA-Net, which introduces a dual-branch encoder composed of residual convolutional neural network (RCNN) branch and recurrent residual convolutional neural network (RRCNN) branch to extract richer semantic information and a context fusion block (CFB) to fuse temporal and spatial information effectively. In addition, an U-shape noise de-noising block (UND) is proposed to reduce interference from complex backgrounds and an edge enhancement block (EEB) is also proposed to strength the capacity of segmentation network to extract useful edge feature information from noise. Experiments demonstrate that proposed DRA-Net has obtained an excellent segmentation manifestation on remote sensing images, with a 93.26% Dice coefficient and 93.19% mIOU value on public power line dataset (PLD), and a 96.40% Dice coefficient and 96.04% mIOU value on self-built overhead power line (OPL) dataset.

Predicting Underestimation of Invasive Cancer in Patients with Core-Needle-Biopsy-Diagnosed Ductal Carcinoma In Situ Using Deep Learning Algorithms

The prediction of an occult invasive component in ductal carcinoma in situ (DCIS) before surgery is of clinical importance because the treatment strategies are different between pure DCIS without invasive component and upgraded DCIS. We demonstrated the potential of using deep learning models for differentiating between upgraded versus pure DCIS in DCIS diagnosed by core-needle biopsy. Preoperative axial dynamic contrast-enhanced magnetic resonance imaging (MRI) data from 352 lesions were used to train, validate, and test three different types of deep learning models. The highest performance was achieved by Recurrent Residual Convolutional Neural Network using Regions of Interest (ROIs) with an accuracy of 75.0% and area under the receiver operating characteristic curve (AUC) of 0.796. Our results suggest that the deep learning approach may provide an assisting tool to predict the histologic upgrade of DCIS and provide personalized treatment strategies to patients with underestimated invasive disease.

Medical Image Segmentation Based on Recurrent Residual Convolution Neural Network

In order to solve the problem of network degradation when using deep learning for medical image segmentation, and to achieve more accurate image segmentation effect by extracting larger scale feature mapping, an image segmentation model based on recurrent residual convolutional neural network is proposed. Firstly, the recurrent convolution unit is introduced to realize the feature extraction on discrete step size, improves the utilization of image context semantic information, and realizes more extensive feature mapping extraction. Then, combined with the residual learning unit and the recurrent convolution unit, the recurrent residual convolution unit is formed to replace the feed-forward convolution unit of the ordinary convolution neural network to solve the network degradation problem faced by the deep network model. Finally, the full-scale skip connection is introduced to fuse the feature images under different scales to generate the segmented image. The experimental results of three datasets compared with the other four algorithms in PyTorch environment show that the proposed algorithm has better segmentation performance and higher image segmentation accuracy.

Visual saliency detection via a recurrent residual convolutional neural network based on densely aggregated features

Current visual saliency detection algorithms based on deep learning suffer from reduced detection effect in complex scenes owing to ineffective feature expression and poor generalization. The present study addresses this issue by proposing a recurrent residual network based on dense aggregated features. Firstly, different levels of dense convolutional features are extracted from the ResNeXt101 network. Then, the features of all layers are aggregated under an Atrous spatial pyramid pooling operation, which makes comprehensive use of all possible saliency cues. Finally, the residuals are learned recurrently under a deep supervision mechanism to achieve continuous optimization of the saliency map. Application of the proposed algorithm to publicly available datasets demonstrates that the dense aggregation of features not only enhances the aggregation of effective information within a single layer, but also enhances external interactions between information at different feature levels. As a result, the proposed algorithm provides better detection ability than that of current state-of-the-art algorithms.

SC-RoadDeepNet: A New Shape and Connectivity-Preserving Road Extraction Deep Learning-Based Network From Remote Sensing Data

Existing automated road extraction approaches concentrate on regional accuracy rather than road shape and connectivity quality. Most of these techniques produce discontinuous outputs caused by obstacles, such as shadows, buildings, and vehicles. This study proposes a shape and connectivity-preserving road identification deep learning-based architecture called SC-RoadDeepNet to overcome the discontinuous results and the quality of road shape and connectivity. The proposed model comprises a state-of-the-art deep learning-based network, namely, the recurrent residual convolutional neural network, boundary learning (BL), and a new measure based on the intersection of segmentation masks and their (morphological) skeleton called connectivity-preserving centerline Dice (CP_clDice). The recurrent residual convolutional layers accumulate low-level features for segmentation tasks, thus allowing for better feature representation. Such representation enables us to construct a UNet network with the same number of network parameters but improved segmentation effectiveness. BL also aids the model in improving the road’s boundaries by penalizing boundary misclassification and fine-tuning the road form. Furthermore, the CP_clDice method aids the model in maintaining road connectivity and obtaining accurate segmentations. We demonstrate that CP_clDice ensures connection preservation for binary segmentation, thereby allowing for efficient road network extraction at the end. The proposed model improves F1 score accuracy to 5.49%, 4.03%, 3.42%, and 2.27% compared with other comparative models, such as LinkNet, ResUNet, UNet, and VNet, respectively. Furthermore, qualitative and quantitative assessments demonstrate that the proposed SC-RoadDeepNet can improve road extraction by tackling shadow and occlusion-related interruptions. These assessments can also produce high-resolution results, particularly in the area of road network completeness.

Sentiment Analysis and Deep Learning Based Cyber Bullying Detection in Twitter Dataset

When somebody, usually a teenager, abuses or harasses individual on the internet and other digital places, mainly on social networking platforms, this is termed as cyberbullying. Cyberbullying, like all types of bullying, produces psychological, emotional, and physical distress. Every individual's reaction to being bullied is diverse, but research has discovered certain common patterns. In a recent study, we introduced a technique called Hybrid Firefly Artificial Neural Networks (HFANN) to combat cyberbullying. Nevertheless, without considering the sentiment analysis features, accuracy of cyber bullying identification is lowered in this study. The Sentiment Analysis and Deep Learning based Cyber Bullying Detection (SADL-CDD) approach is used in the suggested research approach to address this issue. The punctuations, urls, html tags, and emoticons from the input tweet comments are removed first in this study project. Sentiment feature extraction is performed after pre-processing to improve classification accuracy. The Modified Fruit Fly Algorithm (MFFA) is used to choose the best features from the extracted features. Following feature selection, cyber bullying detection is carried out using a Hybrid Recurrent Residual Convolutional Neural Network (HRecRCNN). The experimental outcome of this study indicates the efficiency of the suggested approach. In comparison to current algorithms, the SADL-CDD method delivers improved classification performance with respect to reduced time complexity, greater precision, recall, f-measure, and accuracy.

Hybrid Inception Recurrent Residual Convolutional Neural Network (HIRResCNN) with Harmony Search Optimization (HSO) for Early Breast Cancer Detection System

For women, most common cause of death is Breast tumour and in worldwide, it is the second leading reason for cancer deaths. Due the requirement of breast cancer’s early detection and false diagnosis impact on patients, made researchers to investigate Deep Learning (DL) techniques for mammograms. There are four stages in this proposed HIRResCNN framework, namely, Pre-processing, reduction of dimensionality, segmentation and classification. From images, noises are removed using two filtering algorithms called Median and mean filtering in pre-processing stage. Then canny edge detector is used for detecting edges. Gaussian filtering is used in canny edge detector to smoothen the images. In the next dimensionality reduction stage, attributes are correlated using Principal Component Analysis (PCA) inclusive of related features. So, this huge dataset is minimized and only few variables are used for expressing it. In order to detect the breast cancer accurately, foreground and background subtraction is done in the third stage called segmentation stage. At last, for detecting and classifying breast cancer, a Hybrid Inception Recurrent Residual Convolutional Neural Network (HIRResCNN) is introduced, which integrates Harmony Search Optimization (HSO) to tune bias and weight parameters and classification accuracy is enhanced using HIRResCNN-HSO model. Strength of Recurrent Convolutional Neural Network (RCNN), Residual Network (ResNet) and Inception Network (Inception-v4), are combined in a powerful Deep Convolutional Neural Network (DCNN) model called HIRResCNN. using Mammographic Image Analysis Society (MIAS) dataset, various experiments are conduced and results are compared with other available techniques. Around 92.6% accuracy rate is produced using this proposed HIRResCNN classifier in finding breast cancer.

R2AU-Net: Attention Recurrent Residual Convolutional Neural Network for Multimodal Medical Image Segmentation

In recent years, semantic segmentation method based on deep learning provides advanced performance in medical image segmentation. As one of the typical segmentation networks, U-Net is successfully applied to multimodal medical image segmentation. A recurrent residual convolutional neural network with attention gate connection (R2AU-Net) based on U-Net is proposed in this paper. It enhances the capability of integrating contextual information by replacing basic convolutional units in U-Net by recurrent residual convolutional units. Furthermore, R2AU-Net adopts attention gates instead of the original skip connection. In this paper, the experiments are performed on three multimodal datasets: ISIC 2018, DRIVE, and public dataset used in LUNA and the Kaggle Data Science Bowl 2017. Experimental results show that R2AU-Net achieves much better performance than other improved U-Net algorithms for multimodal medical image segmentation.

Side-scan sonar images segmentation for AUV with recurrent residual convolutional neural network module and self-guidance module

This paper designed an end-to-end image segmentation method for side-scan sonar mounted on Autonomous Underwater Vehicle (AUV) to obtain accurate results. Accurate segmentation results depend on precise feature extraction, and an effective segmentation network can guarantee the autonomous recognition ability of Autonomous Underwater Vehicle (AUV) when performing tasks. In this paper, the R2CNN module is used to obtain accurate sonar image features, which reduces errors and improves accuracy. Besides, the Self-guidance module was introduced to ensure the network’s stability and optimize the segmentation results. The experimental results show that the proposed method can achieve better segmentation results than UNet, SegNet and their derivative networks and has better generalization ability.

Cardiac Segmentation from MRI images using Recurrent & Residual Convolutional Neural Network based on SegNet and Level Set methods

In recent years, semantic segmentation with Deep Learning (DL) is popularly found in many real-world applications. It is also specific, mentioned that these techniques are regularly applied to various segmentation and classification techniques in the medical field. The most popular deep learning techniques, especially SegNet and U-Net, are the most used for this type of medical application. Generally, in case of U-Net architecture which can be used with a skip connection and is capable of retrieving fine data during training. However, such a network consumes a lot of computation time compared to other networks. But SegNet is another network used to retrieve the desired information with a computing efficient. Inspired by the work, the skip connection is introduced into SegNet using the residual neural network (ResNet). ResNet consists of a layer and it has taken inputs involving multiple layers of the neural network, giving precise performance. This article offers first a recurrent convolutional neural network (RecNet) based on Seg-Net called R-SegNet and also a recurrent residual convolutional neural network (R2Net) based on SegNet models called R2-SegNet respectively. The strength of SegNet, RecNet and ResNet are used and have produced the architectures to perform the segmentation of cardiac MRI images. There are a number of advantages derived from the proposed architecture. First of all, using a residual unit, proposed architectures were used to carry out training in deep architecture. Second, the use of recurrent residual convolutional layers ensures that the relevant features retrieved to perform the segmentation tasks. Third, the proposed architecture has designed a good SegNet with a limited network parameters and also produce better performance for performing the task of segmentation in cardiac images. In addition, then, applied the level definition method, to extract the contours or surfaces of the cardiac MRI images. The results proves that the hybrid proposed deep learning methods successfully segments the images and also achieves better accuracy compared to standard architectures.

Cardiac Segmentation from MRI images using Recurrent &Residual Convolutional Neural Network based on SegNet and Level Set methods

In recent years, semantic segmentation with Deep Learning (DL) is popularly foundin many real-world applications. It is also specific, mentioned that these techniques are regularly applied to various segmentation and classification techniques in the medical field. The most popular deep learning techniques, especially SegNet and U-Net, are the most used for this type of medical application.Generally, in case of U-Net architecture which can be used with a skip connection and is capable of retrieving fine data during training. However, such a network consumes a lot of computation time compared to other networks. But SegNet is another network used to retrieve the desired information with a computing efficient. Inspired by the work, the skip connection is introduced into SegNet using the residual neural network (ResNet). ResNet consists of a layer and it has taken inputs involving multiple layers of the neural network, giving precise performance. This article offers first a recurrent convolutional neural network (RecNet) based on Seg-Net called R-SegNet and also a recurrent residual convolutional neural network (R2Net) based on SegNet models called R2-SegNet respectively. The strength of SegNet, RecNet and ResNet are used and have produced the architectures to perform the segmentation of cardiac MRI images. There are a number of advantages derived from the proposed architecture. First of all, using a residual unit, proposed architectures were used to carry out training in deep architecture. Second, the use of recurrent residual convolutional layers ensures that the relevant features retrieved to perform the segmentation tasks. Third, the proposed architecture has designed a good SegNet with a limited network parameters and also produce better performance for performing the task of segmentation in cardiac images. In addition, then, applied the level definition method, to extract the contours or surfaces of the cardiac MRI images.The results proves that the hybrid proposed deep learning methods successfully segments the images and also achieves better accuracy compared to standard architectures.

Classification of Breast Cancer Using a Hybrid and Enhanced Recurrent Residual Convolutional Neural Network (ERResCNN)

Females are affected by BC (Breast Cancer) more than any other type of cancer. BC has caused more deaths than any other diseases such as tuberculosis or malaria according to WHO (World Health Organization). The mortality rates due to BC in women are high making it a candidate for early detection for prevention and cure. Diagnosing BC is a complex task as it is interleaved with normal breast tissues. Image processing methods have been proposed for detecting BC, yet better segmentation methods are required. Fuzzy based approaches provide optimal results in segmenting BC images. Hence, this work uses Fuzzy approach combined with ResCNN (Recurrent Residual Convolution Neural Network) which is the optimized by a modified GA (Genetic Algorithm). The proposed ERResCNN classifying results in detecting BC from images is accurate and efficient in comparison to other methods.

Pixel-level bridge crack detection using a deep fusion about recurrent residual convolution and context encoder network

To cope with the complex bridge crack detection environment, we developed a flexible crack identification system. Firstly, the acquired images are processed by sliding window technology to construct a bridge crack dataset. Then we propose a trainable context encoder network which uses the recurrent residual convolutional neural network (RRCNN) to improve the encoder structure to better extract low-level features. Additionally, it combines dense atrous convolution block (DAC) and residual multi-kernel pooling block (RMP) which can retain more crack information and features from the crack image, as well as improve the performance of crack segmentation. Finally, our research results are validated in the developed software platform. The experiments show that our method is more stable and accurate than other methods, its accuracy and mean intersection over union (mIoU) have reached 98.62% and 80.93%, respectively. Moreover, it performs well in cracks facing complex conditions, which can be applied to the bridge maintenance projects.

Automatic Assessment of ASPECTS Using Diffusion-Weighted Imaging in Acute Ischemic Stroke Using Recurrent Residual Convolutional Neural Network.

The early detection and rapid quantification of acute ischemic lesions play pivotal roles in stroke management. We developed a deep learning algorithm for the automatic binary classification of the Alberta Stroke Program Early Computed Tomographic Score (ASPECTS) using diffusion-weighted imaging (DWI) in acute stroke patients. Three hundred and ninety DWI datasets with acute anterior circulation stroke were included. A classifier algorithm utilizing a recurrent residual convolutional neural network (RRCNN) was developed for classification between low (1–6) and high (7–10) DWI-ASPECTS groups. The model performance was compared with a pre-trained VGG16, Inception V3, and a 3D convolutional neural network (3DCNN). The proposed RRCNN model demonstrated higher performance than the pre-trained models and 3DCNN with an accuracy of 87.3%, AUC of 0.941, and F1-score of 0.888 for classification between the low and high DWI-ASPECTS groups. These results suggest that the deep learning algorithm developed in this study can provide a rapid assessment of DWI-ASPECTS and may serve as an ancillary tool that can assist physicians in making urgent clinical decisions.

Comparison and Fusion of Deep Learning and Radiomics Features of Ground-Glass Nodules to Predict the Invasiveness Risk of Stage-I Lung Adenocarcinomas in CT Scan.

For stage-I lung adenocarcinoma, the 5-years disease-free survival (DFS) rates of non-invasive adenocarcinoma (non-IA) is different with invasive adenocarcinoma (IA). This study aims to develop CT image based artificial intelligence (AI) schemes to classify between non-IA and IA nodules, and incorporate deep learning (DL) and radiomics features to improve the classification performance. We collect 373 surgical pathological confirmed ground-glass nodules (GGNs) from 323 patients in two centers. It involves 205 non-IA (including 107 adenocarcinoma in situ and 98 minimally invasive adenocarcinoma), and 168 IA. We first propose a recurrent residual convolutional neural network based on U-Net to segment the GGNs. Then, we build two schemes to classify between non-IA and IA namely, DL scheme and radiomics scheme, respectively. Third, to improve the classification performance, we fuse the prediction scores of two schemes by applying an information fusion method. Finally, we conduct an observer study to compare our scheme performance with two radiologists by testing on an independent dataset. Comparing with DL scheme and radiomics scheme (the area under a receiver operating characteristic curve (AUC): 0.83 ± 0.05, 0.87 ± 0.04), our new fusion scheme (AUC: 0.90 ± 0.03) significant improves the risk classification performance (p < 0.05). In a comparison with two radiologists, our new model yields higher accuracy of 80.3%. The kappa value for inter-radiologist agreement is 0.6. It demonstrates that applying AI method is an effective way to improve the invasiveness risk prediction performance of GGNs. In future, fusion of DL and radiomics features may have a potential to handle the classification task with limited dataset in medical imaging.

Microaneurysms segmentation with a U-Net based on recurrent residual convolutional neural network.

Microaneurysms (MAs) play an important role in the diagnosis of clinical diabetic retinopathy at the early stage. Annotation of MAs manually by experts is laborious and so it is essential to develop automatic segmentation methods. Automatic MA segmentation remains a challenging task mainly due to the low local contrast of the image and the small size of MAs. A deep learning-based method called U-Net has become one of the most popular methods for the medical image segmentation task. We propose an architecture for U-Net, named deep recurrent U-Net (DRU-Net), obtained by combining the deep residual model and recurrent convolutional operations into U-Net. In the MA segmentation task, DRU-Net can accumulate effective features much better than the typical U-Net. The proposed method is evaluated on two publicly available datasets: E-Ophtha and IDRiD. Our results show that the proposed DRU-Net achieves the best performance with 0.9999 accuracy value and 0.9943 area under curve (AUC) value on the E-Ophtha dataset. And on the IDRiD dataset, it has achieved 0.987 AUC value (to our knowledge, this is the first result of segmenting MAs on this dataset). Compared with other methods, such as U-Net, FCNN, and ResU-Net, our architecture (DRU-Net) achieves state-of-the-art performance.

Breast Cancer Classification from Histopathological Images with Inception Recurrent Residual Convolutional Neural Network.

The Deep Convolutional Neural Network (DCNN) is one of the most powerful and successful deep learning approaches. DCNNs have already provided superior performance in different modalities of medical imaging including breast cancer classification, segmentation, and detection. Breast cancer is one of the most common and dangerous cancers impacting women worldwide. In this paper, we have proposed a method for breast cancer classification with the Inception Recurrent Residual Convolutional Neural Network (IRRCNN) model. The IRRCNN is a powerful DCNN model that combines the strength of the Inception Network (Inception-v4), the Residual Network (ResNet), and the Recurrent Convolutional Neural Network (RCNN). The IRRCNN shows superior performance against equivalent Inception Networks, Residual Networks, and RCNNs for object recognition tasks. In this paper, the IRRCNN approach is applied for breast cancer classification on two publicly available datasets including BreakHis and Breast Cancer (BC) classification challenge 2015. The experimental results are compared against the existing machine learning and deep learning-based approaches with respect to image-based, patch-based, image-level, and patient-level classification. The IRRCNN model provides superior classification performance in terms of sensitivity, area under the curve (AUC), the ROC curve, and global accuracy compared to existing approaches for both datasets.