Residual Convolutional Network Research Articles

Automatic Detection of Construction Workers’ Helmet Wear Based on Lightweight Deep Learning

To reduce the risk of head trauma to workers working in high-risk workplaces such as construction sites, we designed a new automated lightweight end-to-end convolutional neural network to identify whether all people on a construction site are wearing helmets. Firstly, we used GhostNet as the backbone feature extraction network to take advantage of its low running cost and make the model lighter overall while ensuring efficient automatic feature extraction. Secondly, we designed a multi-scale segmentation and feature fusion network (MSFFN) in the feature-processing stage to improve the algorithm’s robustness in detecting objects at different scales. In contrast, the design of the feature fusion network can enrich the diversity of helmet features and improve the accuracy of helmet detection when distance changes, viewpoint changes, and occlusion phenomena occur. Thirdly, we proposed an improved version of the attention mechanism, the lightweight residual convolutional attention network version 2 (LRCA-Netv2). The main idea of the improvement is implemented around the spatial dimension by fusing the combined features along with the horizontal and vertical directions and then weighting them separately. Such an operation allows the establishment of dependencies between the more distant features with improved accuracy compared to the original LRCA-Net. Finally, when tested on the dataset, the proposed lightweight helmet-wearing detection network has a mAP and FPS of 93.5% and 42.

The Effect of Data Augmentation Methods on Pedestrian Object Detection

Night landscapes are a key area of monitoring and security as information in pictures caught on camera is not comprehensive. Data augmentation gives these limited datasets the most value. Considering night driving and dangerous events, it is important to achieve the better detection of people at night. This paper studies the impact of different data augmentation methods on target detection. For the image data collected at night under limited conditions, three different types of enhancement methods are used to verify whether they can promote pedestrian detection. This paper mainly explores supervised and unsupervised data augmentation methods with certain improvements, including multi-sample augmentation, unsupervised Generative Adversarial Network (GAN) augmentation and single-sample augmentation. It is concluded that the dataset obtained by the heterogeneous multi-sample augmentation method can optimize the target detection model, which can allow the mean average precision (mAP) of a night image to reach 0.76, and the improved Residual Convolutional GAN network, the unsupervised training model, can generate new samples with the same style, thus greatly expanding the dataset, so that the mean average precision reaches 0.854, and the single-sample enhancement of the deillumination can greatly improve the image clarity, helping improve the precision value by 0.116.

Cutting tool wear monitoring in milling processes by integrating deep residual convolution network and gated recurrent unit with an attention mechanism

The conditions of cutting tool wear in the machining process have a great influence on the dimension accuracy and surface integrity of a workpiece. Real-time tool condition monitoring can alert a worker to changes in a tool in time before tool failure occurs to reduce downtime and improve product quality. However, traditional monitoring methods are no longer able to meet the requirements of real-time monitoring in current intelligent manufacturing. Therefore, an online monitoring method based on deep learning to monitor the cutting tool conditions is proposed In this research. In this monitoring model, one-dimensional wide-kernel convolution is used to filter the high-frequency noise in the multi-dimensional raw signals, and then a deep residual convolution layer is adopted to adaptively extract the multi-dimensional spatial features of the signals, and a gated recurrent unit (GRU) layer with an attention mechanism is employed to adaptively extract the hidden temporal features of the sequence data and adaptively assign the corresponding weights, and finally, a regression layer is used to predict the wear value of the tool according to the extracted features. The PHM 2010 challenge dataset is used to illustrate and evaluate this proposed method. The experiment results show that the proposed method can accurately predict the future flank wear values in real time based on the acquired multi-dimensional signals and avoid the complexity of the manual feature extraction.

Fully Decoupled Residual ConvNet for Real-Time Railway Scene Parsing of UAV Aerial Images

UAV-based automatic railway inspection is expected to have the potential to reform the inspection of railways. In this area, real-time railway scene parsing is quite essential. However, the limited computation resources of the UAV onboard computer pose a huge challenge for the algorithm to juggle a precise prediction with strong timeliness. Concerning this issue, this paper proposes a novel algorithm named deep fully decoupled residual convolutional network, which consists of fully decoupled residual blocks (Non-bottleneck-FDs) to deal with the dilemma between the high demand of real-time and limited resources. The residual block is constructed based on a new convolution which divides the standard convolution into three sequential convolutions to decouple the conventional operational correlations fully. Furthermore, a customized auxiliary line loss (LL) function is proposed to constrain the segmentation of railway and non-railway simultaneously without increasing the computation complexity. The proposed LL can force the predicted railway areas to concentrate in long strip areas precisely and inhibit their appearances in other impossible local areas. Subsequently, an integrated loss backpropagation strategy of the LL and cross-entropy function is presented. A comprehensive set of experiments are conducted for verification. Experiments demonstrate the superior performance of our approach with a more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2\times $ </tex-math></inline-formula> reduction in parameters and computation cost. Moreover, our approach also has a faster inference speed than the most existing lightweight architectures while providing comparable or higher accuracy. It is proven that our approach can reconcile the precise prediction with strong timeliness for railway scene parsing within the limitation of onboard computers. Besides, the results also imply its highest performance in terms of local details and edges of railway areas.

An artificial intelligence-based noninvasive solution to estimate pulmonary artery pressure.

AimsDesign to develop an artificial intelligence (AI) algorithm to accurately predict the pulmonary artery pressure (PAP) waveform using non-invasive signal inputs.Methods and resultsWe randomly sampled training, validation, and testing datasets from a waveform database containing 180 patients with pulmonary atrial catheters (PACs) placed for PAP waves collection. The waveform database consisted of six hemodynamic parameters from bedside monitoring machines, including PAP, artery blood pressure (ABP), central venous pressure (CVP), respiration waveform (RESP), photoplethysmogram (PPG), and electrocardiogram (ECG). We trained a Residual Convolutional Network using a training dataset containing 144 (80%) patients, tuned learning parameters using a validation set including 18 (10%) patients, and tested the performance of the method using 18 (10%) patients, respectively. After comparing all multi-stage algorithms on the testing cohort, the combination of the residual neural network model and wavelet scattering transform data preprocessing method attained the highest coefficient of determination R2 of 90.78% as well as the following other performance metrics and corresponding 95% confidence intervals (CIs): mean square error of 11.55 (10.22–13.5), mean absolute error of 2.42 (2.06–2.85), mean absolute percentage error of 0.91 (0.76–1.13), and explained variance score of 90.87 (85.32–93.31).ConclusionThe proposed analytical approach that combines data preprocessing, sampling method, and AI algorithm can precisely predict PAP waveform using three input signals obtained by noninvasive approaches.

Atrous residual convolutional neural network based on U-Net for retinal vessel segmentation.

Extracting features of retinal vessels from fundus images plays an essential role in computer-aided diagnosis of diseases, such as diabetes, hypertension, and cerebrovascular diseases. Although a number of deep learning-based methods have been used in this field, the accuracy of retinal vessel segmentation remains challenging due to limited densely annotated data, inter-vessel differences, and structured prediction problems, especially in areas of small blood vessels and the optic disk. In this paper, we propose an ARN model with a atrous block to address these issues, which can avoid the loss of data structure, and enlarge the receptive field, so that each convolution output contains a larger range of information. In addition, we also introduce residual convolution network to increase the network depth and improve the network performance.Some key parameters are used to measure the feasibility of the model, such as sensitivity (Se), specificity (Sp), F1-score (F1), accuracy (Acc), and area under each curve (AUC). Experimental results on two benchmark datasets demonstrate the effectiveness of the proposed methods, which accuracy are 0.9686 on the DRIVE and 0.9746 on the CHASE DB1. The segmentation structure can assist the doctor in diagnosis more effectively.

State-of-Charge Estimation for Lithium-Ion Batteries Using Residual Convolutional Neural Networks.

State-of-charge (SOC) is a relative quantity that describes the ratio of the remaining capacity to the present maximum available capacity. Accurate SOC estimation is essential for a battery-management system. In addition to informing the user of the expected usage until the next recharge, it is crucial for improving the utilization efficiency and service life of the battery. This study focuses on applying deep-learning techniques, and specifically convolutional residual networks, to estimate the SOC of lithium-ion batteries. By stacking the values of multiple measurable variables taken at many time instants as the model inputs, the process information for the voltage or current generation, and their interrelations, can be effectively extracted using the proposed convolutional residual blocks, and can simultaneously be exploited to regress for accurate SOCs. The performance of the proposed network model was evaluated using the data obtained from a lithium-ion battery (Panasonic NCR18650PF) under nine different driving schedules at five ambient temperatures. The experimental results demonstrated an average mean absolute error of 1.260%, and an average root-mean-square error of 0.998%. The number of floating-point operations required to complete one SOC estimation was 2.24 × 106. These results indicate the efficacy and performance of the proposed approach.

A multi-modal fusion framework for continuous sign language recognition based on multi-layer self-attention mechanism

Some of the existing continuous sign language recognition (CSLR) methods require alignment. However, this is time-consuming, and breaks the continuity of the frame sequence, and also affects the subsequent process of CSLR. In this paper, we propose a multi-modal network framework for CSLR based on a multi-layer self-attention mechanism. We propose a 3D convolution residual neural network (CR3D) and a multi-layer self-attention network (ML-SAN) for the feature extraction stage. The CR3D obtains the short-term spatiotemporal features of the RGB and optical flow image streams, whereas the ML-SAN uses a bi-gated recurrent unit (BGRU) to model the long-term sequence relationship and a multi-layer self-attention mechanism to learn the internal relationships between sign language sequences. For the performance optimization stage, we propose a cross-modal spatial mapping loss function, which improves the precision of CSLR by studying the spatial similarity between the video and text domains. Experiments were conducted on two test datasets: the RWTH-PHOENIX-Weather multi-signer dataset, and a Chinese SL (CSL) dataset. The results show that the proposed method can obtain state-of-the-art recognition performance on the two datasets, with word error rate (WER) value of 24.4% and accuracy value of 14.42%, respectively.

IEnhancer-DLRA: identification of enhancers and their strengths by a self-attention fusion strategy for local and global features.

Identification and classification of enhancers are highly significant because they play crucial roles in controlling gene transcription. Recently, several deep learning-based methods for identifying enhancers and their strengths have been developed. However, existing methods are usually limited because they use only local or only global features. The combination of local and global features is critical to further improve the prediction performance. In this work, we propose a novel deep learning-based method, called iEnhancer-DLRA, to identify enhancers and their strengths. iEnhancer-DLRA extracts local and multi-scale global features of sequences by using a residual convolutional network and two bidirectional long short-term memory networks. Then, a self-attention fusion strategy is proposed to deeply integrate these local and global features. The experimental results on the independent test dataset indicate that iEnhancer-DLRA performs better than nine existing state-of-the-art methods in both identification and classification of enhancers in almost all metrics. iEnhancer-DLRA achieves 13.8% (for identifying enhancers) and 12.6% (for classifying strengths) improvement in accuracy compared with the best existing state-of-the-art method. This is the first time that the accuracy of an enhancer identifier exceeds 0.9 and the accuracy of the enhancer classifier exceeds 0.8 on the independent test set. Moreover, iEnhancer-DLRA achieves superior predictive performance on the rice dataset compared with the state-of-the-art method RiceENN.

Image Segmentation of Operative Neuroanatomy Into Tissue Categories Using a Machine Learning Construct and Its Role in Neurosurgical Training.

The complexity of the relationships among the structures within the brain makes efficient mastery of neuroanatomy difficult for medical students and neurosurgical residents. Therefore, there is a need to provide real-time segmentation of neuroanatomic images taken from various perspectives to assist with training. To develop the initial foundation of a neuroanatomic image segmentation algorithm using artificial intelligence for education. A pyramidal scene-parsing network with a convolutional residual neural network backbone was assessed for its ability to accurately segment neuroanatomy images. A data set of 879 images derived from The Neurosurgical Atlas was used to train, validate, and test the network. Quantitative assessment of the segmentation was performed using pixel accuracy, intersection-over-union, the Dice similarity coefficient, precision, recall, and the boundary F1 score. The network was trained, and performance was assessed class wise. Compared with the ground truth annotations, the ensembled results for our artificial intelligence framework for the pyramidal scene-parsing network during testing generated a total pixel accuracy of 91.8%. Using the presented methods, we show that a convolutional neural network can accurately segment gross neuroanatomy images, which represents an initial foundation in artificial intelligence gross neuroanatomy that will aid future neurosurgical training. These results also suggest that our network is sufficiently robust, to an unprecedented level, for performing anatomic category recognition in a clinical setting.

A Multi-scale Dilated Residual Convolution Network for Image Denoising

A Multi-scale Dilated Residual Convolution Network for Image Denoising

Residual convolutional autoencoder combined with a non-negative matrix factorization to estimate fetal heart rate.

The fetal heart rate (fHR) plays an important role in the determination of the good health of the fetus. Beside the traditional Doppler ultrasound technique, non-invasive fetal electrocardiography (fECG) has become an interesting alternative. However, extracting clean fECG from abdominal ECG (aECG) recordings is a challenging task due to the presence of the maternal ECG component and various noise sources. In this context, we propose a deep residual convolutional autoencoder network trained on synthetic aECG simulations followed by a transfer learning phase on real aECG recordings to extract the cleanest fECG. Afterwards, we propose to use a non-negative matrix factorization based approach on the obtained fECG to estimate the fHR. Our method is evaluated on three publicly available databases demonstrating that it can provide significant performance improvement against comparative methodologies. Clinical relevance- The presented method has the advantage of estimating the fetal heart rate from a single-channel abdominal electrocardiogram without prior knowledge on the noise sources nor the maternal R-peak locations.

Deep microlocal reconstruction for limited-angle tomography

We present a deep-learning-based algorithm to jointly solve a reconstruction problem and a wavefront set extraction problem in tomographic imaging. The algorithm is based on a recently developed digital wavefront set extractor as well as the well-known microlocal canonical relation for the Radon transform. We use the wavefront set information about x-ray data to improve the reconstruction by requiring that the underlying neural networks simultaneously extract the correct ground truth wavefront set and ground truth image. As a necessary theoretical step, we identify the digital microlocal canonical relations for deep convolutional residual neural networks. We find strong numerical evidence for the effectiveness of this approach.

Facial Expression Recognition: One Attention-Modulated Contextual Spatial Information Network.

Facial expression recognition (FER) in the wild is a challenging task due to some uncontrolled factors such as occlusion, illumination, and pose variation. The current methods perform well in controlled conditions. However, there are still two issues with the in-the-wild FER task: (i) insufficient descriptions of long-range dependency of expression features in the facial information space and (ii) not finely refining subtle inter-classes distinction from multiple expressions in the wild. To overcome the above issues, an end-to-end model for FER, named attention-modulated contextual spatial information network (ACSI-Net), is presented in this paper, with the manner of embedding coordinate attention (CA) modules into a contextual convolutional residual network (CoResNet). Firstly, CoResNet is constituted by arranging contextual convolution (CoConv) blocks of different levels to integrate facial expression features with long-range dependency, which generates a holistic representation of spatial information on facial expression. Then, the CA modules are inserted into different stages of CoResNet, at each of which the subtle information about facial expression acquired from CoConv blocks is first modulated by the corresponding CA module across channels and spatial locations and then flows into the next layer. Finally, to highlight facial regions related to expression, a CA module located at the end of the whole network, which produces attentional masks to multiply by input feature maps, is utilized to focus on salient regions. Different from other models, the ACSI-Net is capable of exploring intrinsic dependencies between features and yielding a discriminative representation for facial expression classification. Extensive experimental results on AffectNet and RAF_DB datasets demonstrate its effectiveness and competitiveness compared to other FER methods.

Segmentation of ground glass pulmonary nodules using full convolution residual network based on atrous spatial pyramid pooling structure and attention mechanism

Accurate segmentation of ground glass nodule (GGN) is important in clinical. But it is a tough work to segment the GGN, as the GGN in the computed tomography images show blur boundary, irregular shape, and uneven intensity. This paper aims to segment GGN by proposing a fully convolutional residual network, i.e., residual network based on atrous spatial pyramid pooling structure and attention mechanism (ResAANet). The network uses atrous spatial pyramid pooling (ASPP) structure to expand the feature map receptive field and extract more sufficient features, and utilizes attention mechanism, residual connection, long skip connection to fully retain sensitive features, which is extracted by the convolutional layer. First, we employ 565 GGN provided by Shanghai Chest Hospital to train and validate ResAANet, so as to obtain a stable model. Then, two groups of data selected from clinical examinations (84 GGN) and lung image database consortium (LIDC) dataset (145 GGN) were employed to validate and evaluate the performance of the proposed method. Finally, we apply the best threshold method to remove false positive regions and obtain optimized results. The average dice similarity coefficient (DSC) of the proposed algorithm on the clinical dataset and LIDC dataset reached 83.46%, 83.26% respectively, the average Jaccard index (IoU) reached 72.39%, 71.56% respectively, and the speed of segmentation reached 0.1 seconds per image. Comparing with other reported methods, our new method could segment GGN accurately, quickly and robustly. It could provide doctors with important information such as nodule size or density, which assist doctors in subsequent diagnosis and treatment.

Detection of Object (Weapons) With Deep Learning Algorithms From İmages Obtained By Unmanned Aerial Vehicles

Today, the rapid development of Artificial Intelligence technologies is effective in the success of deep learning algorithms in different application areas. In addition to these applications, it detects many objects that cannot be noticed even with the human eye in object detection in video and images with deep learning algorithms. In this study, it was aimed to detect weapons by using images obtained from UAV using deep learning algorithms. Regional Based Convolutional Neural Networks and Residual Network were used and the performance evaluation of the architecture of these algorithms was presented. Performance evaluation of algorithms was made using Loss plots, Precision-Recall, and mAP curves. In this study, 200 images of different angles and heights were obtained from the unmanned aerial vehicle. Two-thirds of the images we obtained were divided into training and one-third test images. Resnet and Region Based Convolutional neural networks used in test images have been successful in object detection. Regional based neural networks and Residual Network that we use for object detection are used. Images from different angles and heights obtained from the unmanned aerial vehicle are trained using regional-based neural networks and residual network algorithms. The feature maps extracted from the images used for training were compared with the test images. By bringing these compared images closer to the desired images, the desired image detection was detected at a rate of 99%. The performance evaluation of the detected images is discussed and the success of the deep learning algorithms used in object detection is presented. Considered in the performance evaluation of deep learning algorithms, Loss Charts (Error), Precision-Recall curves show the accuracy of detection in a short time. Algorithms that will increase the possibilities and capabilities of unmanned aerial vehicles used especially in border security and internal security are considered to increase in the future.

Electroencephalograph-Based Emotion Recognition Using Brain Connectivity Feature and Domain Adaptive Residual Convolution Model.

In electroencephalograph (EEG) emotion recognition research, obtaining high-level emotional features with more discriminative information has become the key to improving the classification performance. This study proposes a new end-to-end emotion recognition method based on brain connectivity (BC) features and domain adaptive residual convolutional network (short for BC-DA-RCNN), which could effectively extract the spatial connectivity information related to the emotional state of the human brain and introduce domain adaptation to achieve accurate emotion recognition within and across the subject’s EEG signals. The BC information is represented by the global brain network connectivity matrix. The DA-RCNN is used to extract high-level emotional features between different dimensions of EEG signals, reduce the domain offset between different subjects, and strengthen the common features between different subjects. The experimental results on the large public DEAP data set show that the accuracy of the subject-dependent and subject-independent binary emotion classification in valence reaches 95.15 and 88.28%, respectively, which outperforms all the benchmark methods. The proposed method is proven to have lower complexity, better generalization ability, and domain robustness that help to lay a solid foundation for the development of high-performance affective brain-computer interface applications.

Predicting trajectory of drogue based on multi-head convolutional long-short-term memory network

Aerial refueling is an important technology of great military significance. It can effectively boost an aircraft's performance owing to the longer period of time and longer endurance of range an aircraft can maintain in the air. To solve the problem that it is hard for a receiver aircraft to track a drogue during its docking phase, a drogue trajectory prediction method based on the multi-head convolutional long-short-term memory network is proposed. First, the one-dimensional time sequence data of the drogue is extended to its high-dimensional space. Then its spatial features are extracted through the multi-head convolutional residual network and fused together. On this basis, a long-short-term memory network is adopted to reveal the underlying temporal correlations among the spatial features and predict the trajectory of the drogue. The simulation and experimental results show that the method presented in this paper has a higher prediction accuracy than the traditional prediction methods that use time sequence data.

Visual Detection Application of Lightweight Convolution and Deep Residual Networks in Wood Production

In order to improve the production capacity of traditional wood manufacturing industry, efficient wood quality and thickness detection is a challenging issue. This paper firstly carries out digital twin modeling for a drawer side panel processing line of a wood company and explores the efficiency problems existing in the links of quality inspection and thickness inspection of wood by means of value stream mapping. Therefore, we adopted a lightweight convolution neural network MobileNetV2 for wood quality detection, which realized efficient wood quality identification. In contrast, traditional convolution neural network has many weighting parameters and large scale of generating detection model, which makes it difficult to apply in situations with limited computing power and memory. Secondly, due to the stronger robustness and generalization ability of the residual network, we used ResNet to detect the wood thickness and obtain reliable performance. Finally, we reasonably embedded them in the whole wood production process and established the simulation model of production line before and after improvement in FlexSim simulation software. The experimental results show that the improved plan can simplify the workshop production process, increase the production balance rate by 29.07%, increase the product value-added rate from 0.08% to 0.11%, and shorten the production cycle by 2 hours. Performance indicators such as product inventory, number of people, and equipment utilization also improve significantly. Based on the above results, the validity of the production process improvement model proposed by US based on lightweight convolution and deep residual network is demonstrated.

DCR‐Net: Dilated convolutional residual network for fashion image retrieval

AbstractFashion image retrieval is an important branch of image retrieval technology. With the rapid development of online shopping, fashion image retrieval technology has made a breakthrough from text‐based to content‐based. But there is still not a proper deep learning method used for fashion image retrieval. This article proposes a fashion image retrieval framework based on dilated convolutional residual network which consists of two major parts, image feature extraction and feature distance measurement. For image feature extraction, we first extract the shallow features of the input image by a multi‐scale convolutional network, and then develop a novel dilated convolutional residual network to obtain the deep features of the image. Finally, the extracted features are transformed into high‐dimensional features vector by a binary retrieval vector module. For feature distance measurement, we first use PCA to reduce the dimension of the extracted high‐dimensional vectors. Then we propose a mixed distance measurement algorithm combined with cosine distance and Mahalanobis distance to calculate the spatial distance of the feature vectors for similarity ranking, which solves the problems of poor robustness in complex background fashion image retrieval and the inefficiency calculation of Mahalanobis distance. The experimental results show the superiority of our fashion image retrieval framework over existing state‐of‐the‐art methods.