Residual Convolution Research Articles

Remaining Useful Life Prediction of Bearings Using Reverse Attention Graph Convolution Network With Residual Convolution Transformer

Remaining Useful Life Prediction of Bearings Using Reverse Attention Graph Convolution Network With Residual Convolution Transformer

A method for recognizing abnormal behaviors of personnel at petroleum stations based on GTB-ResNet

PurposeThe abnormal behaviors of staff at petroleum stations pose significant safety hazards. Addressing the challenges of high parameter counts, lengthy training periods and low recognition rates in existing 3D ResNet behavior recognition models, this paper proposes GTB-ResNet, a network designed to detect abnormal behaviors in petroleum station staff.Design/methodology/approachFirstly, to mitigate the issues of excessive parameters and computational complexity in 3D ResNet, a lightweight residual convolution module called the Ghost residual module (GhostNet) is introduced in the feature extraction network. Ghost convolution replaces standard convolution, reducing model parameters while preserving multi-scale feature extraction capabilities. Secondly, to enhance the model's focus on salient features amidst wide surveillance ranges and small target objects, the triplet attention mechanism module is integrated to facilitate spatial and channel information interaction. Lastly, to address the challenge of short time-series features leading to misjudgments in similar actions, a bidirectional gated recurrent network is added to the feature extraction backbone network. This ensures the extraction of key long time-series features, thereby improving feature extraction accuracy.FindingsThe experimental setup encompasses four behavior types: illegal phone answering, smoking, falling (abnormal) and touching the face (normal), comprising a total of 892 videos. Experimental results showcase GTB-ResNet achieving a recognition accuracy of 96.7% with a model parameter count of 4.46 M and a computational complexity of 3.898 G. This represents a 4.4% improvement over 3D ResNet, with reductions of 90.4% in parameters and 61.5% in computational complexity.Originality/valueSpecifically designed for edge devices in oil stations, the 3D ResNet network is tailored for real-time action prediction. To address the challenges posed by the large number of parameters in 3D ResNet networks and the difficulties in deployment on edge devices, a lightweight residual module based on ghost convolution is developed. Additionally, to tackle the issue of low detection accuracy of behaviors amidst the noisy environment of petroleum stations, a triple attention mechanism is introduced during feature extraction to enhance focus on salient features. Moreover, to overcome the potential for misjudgments arising from the similarity of actions, a Bi-GRU model is introduced to enhance the extraction of key long-term features.

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

Judicial decision prediction using an integrated attention based bidirectional long-short term memory and dilated skip residual convolution neural network

CosGCTFormer: An end-to-end driver state recognition framework

Driver state is critical for safety in mixed traffic environments with automated and human-driven vehicles. Recognizing and monitoring the driver’s state is essential to provide early warnings of abnormal driving performance. This paper proposes a driver state recognition framework called cosGCTFormer for end-to-end recognition of the driver’s emotional state and mental load. First, we employ an inverted residual convolution module to extract local features from physiological signal data. Second, the cosGCT module is introduced to extract global features. This module uses cos attention to address the high computational complexity of the Transformer in calculating attention weights. Additionally, it employs Gated Channel Transformation (GCT) to establish relationships between different channel dimensions, something the Transformer alone cannot do effectively. Finally, a simulated driving experiment was designed to collect data for training and testing the proposed method. Results demonstrate that our framework achieves higher accuracy and a more lightweight design compared to prevalent Convolution neural network-based (CNN-based) and Transformer-based methods. Additionally, the effectiveness of both modules was verified.

Global and local feature extraction based on convolutional neural network residual learning for MR image denoising.

Given the different noise distribution information of global and local magnetic resonance (MR) images, this study aims to extend the current work on convolutional neural networks that preserve global structure and local details in MR image denoising tasks. This study proposed a parallel and serial network for denoising 3D MR images, called 3D-PSNet. We use the residual depthwise separable convolution block to learn the local information of the feature map, reduce the network parameters, and thus improve the training speed and parameter efficiency. In addition, we consider the feature extraction of the global image and utilize residual dilated convolution to process the feature map to expand the receptive field of the network and avoid the loss of global information. Finally, we combine both of them to form a parallel network. What's more, we integrate reinforced residual convolution blocks with dense connections to form serial network branches, which can remove redundant information and refine features to further obtain accurate noise information. The peak signal-to-noise ratio, structural similarity index measure, and root mean square error metrics of 3D-PSNet are as high as 47.79%, 99.81%, and 0.40%, respectively, achieving competitive denoising effect on three public datasets. The ablation experiments demonstrated the effectiveness of all the designed modules regarding all the evaluated metrics in both datasets. The proposed 3D-PSNet takes advantage of multi-scale receptive fields, local feature extraction and residual dense connections to more effectively restore the global structure and local fine features in MR images, and is expected to help doctors quickly and accurately diagnose patients' conditions.

A lightweight parallel attention residual network for tile defect recognition

In modern industrial production, permanent magnet motors are an indispensable part of industrial manufacturing. The quality of the magnetic tiles directly affects the working performance of the permanent magnet motors, making the detection of defects on the surface of magnetic tiles critically important. However, due to the small size of defects on the tile image and the reflectivity of the defective surface, the details of image characteristics are not prominently acquired.These problems bring a lot of difficulties for the recognition of magnetic tile defects. In this paper, a magnetic tile defect detection method is proposed for the probAlems of unclear image features and small defects. First, the image is processed using linear variation to enhance the image detail features. Then, by introducing the inverted bottleneck block structure in MobileNetV2, the Attention Parallel Residual Convolution Block (APR) is proposed, and the Lightweight Parallel Attention Residual Network (LPAR-Net) is built. In APR Block, 7 × 7 convolution is introduced so that the model can extract spatial features from a larger range, and weighted fusion of input images by residual structure. In addition, in this paper, CBAM is improved, split into two parts and inserted into APR Block. Finally, the mainstream image classification models and the LPAR-Net proposed in this paper are used for comparison, respectively. The experimental results show that the method achieves 93.63% accuracy on the adopted dataset, which is better than the existing mainstream image classification network models DenseNet, MobileNet, ConvNext and so on. In addition, this paper introduces a strip steel surface defect dataset and compares it with the above image classification model, which verifies that the detection method proposed in this paper still has strong recognition capability.

Dental Caries Segmentation using Deformable Dense Residual Half U-Net for Teledentistry System

Clinical practitioners’ workload and challenges are significantly reduced by classifying, predicting, and localizing lesions or dental caries. In recent research, a high-reliability diagnostic system within deep learning models has been implemented in a clinical teledentistry system. In order to construct an efficient, precise, and lightweight deep learning architecture, it is dynamically structured. In this paper, we present an efficient, accurate, and lightweight deep learning architecture for augmenting spatial locations and improving the transformation modeling abilities of fixed-structure CNNs. Deformable Dense Residual (DDR) enhances the efficacy of the residual convolution block by optimizing its structure, thereby mitigating model redundancy and ameliorating the challenge of vanishing gradients encountered during the training stages. DDR Half U-Net presents notable advancements to the simplified U-Net framework across three pivotal domains: the encoder, decoder, and loss function. Specifically, the encoder integrates deformable convolutions, thereby enhancing the model's capacity to discern features of diverse scales and configurations. In the decoder, a sophisticated arrangement of dense residual connections facilitates the fusion of low-level and high-level features, contributing to comprehensive feature extraction. Moreover, the utilization of a weight-adaptive loss function ensures equitable consideration of both caries and non-caries samples, thereby promoting balanced optimization during training.

A Novel Network for Low-Dose CT Denoising Based on Dual-Branch Structure and Multi-Scale Residual Attention.

Deep learning-based denoising of low-dose medical CT images has received great attention both from academic researchers and physicians in recent years, and has shown important application value in clinical practice. In this work, a novel two-branch and multi-scale residual attention-based network for low-dose CT image denoising is proposed. It adopts a two-branch framework structure, to extract and fuse image features at shallow and deep levels respectively, to recover image texture and structure information as much as possible. We propose the adaptive dynamic convolution block (ADCB) in the local information extraction layer. It can effectively extract the detailed information of low-dose CT denoising and enables the network to better capture the local details and texture features of the image, thereby improving the denoising effect and image quality. Multi-scale edge enhancement attention block (MEAB) is proposed in the global information extraction layer, to perform feature fusion through dilated convolution and a multi-dimensional attention mechanism. A multi-scale residual convolution block (MRCB) is proposed to integrate feature information and improve the robustness and generalization of the network. To demonstrate the effectiveness of our method, extensive comparison experiments are conducted and the performances evaluated on two publicly available datasets. Our model achieves 29.3004 PSNR, 0.8659 SSIM, and 14.0284 RMSE on the AAPM-Mayo dataset. It is evaluated by adding four different noise levels σ = 15, 30, 45, and 60 on the Qin_LUNG_CT dataset and achieves the best results. Ablation studies show that the proposed ADCB, MEAB, and MRCB modules improve the denoising performances significantly. The source code is available at https://github.com/Ye111-cmd/LDMANet .

Combining optical flow and Swin Transformer for Space-Time video super-resolution

Space–time video super-resolution is a task that aims to interpolate low frame rate, low resolution videos to high frame rate, high resolution ones. While existing Transformer-based methods have achieved results comparable to convolutional neural networks-based methods, the computational cost of Transformer limits its performance with constrained computational resources. Moreover, Swin Transformer may fail to fully exploit the spatio-temporal information of video frames due to the limitation of window size, impeding its effectiveness in handling large motions. To address these limitations, we propose an end-to-end space–time video super-resolution architecture based on optical flow alignment and Swin Transformer. The alignment module is introduced to extract spatio-temporal information from adjacent frames without significantly increasing the computational burden. Additionally, we design a residual convolution layer to enhance the translational invariance of the features extracted by Swin Transformer and introduces additional nonlinear transformations. Experimental results demonstrate that our proposed method achieves superior performance on various benchmark datasets compared to state-of-the-art methods. In terms of Peak Signal-to-Noise Ratio, our method outperforms the state-of-the-art methods by at least 0.15 dB on Vimeo-Medium dataset.

Brain Emotion Perception Inspired EEG Emotion Recognition With Deep Reinforcement Learning.

Inspired by the well-known Papez circuit theory and neuroscience knowledge of reinforcement learning, a double dueling deep Q network (DQN) is built incorporating the electroencephalogram (EEG) signals of the frontal lobe as prior information, which is named frontal lobe double dueling DQN (FLD3QN). The framework of FLD3QN is constructed in accord with the brain emotion mechanism which takes the frontal lobe and the thalamus as the core, in which the part of the Papez circuit is simulated by the bifrontal lobe residual convolution neural network (BiFRCNN). Moreover, a step penalty factor is designed to constrain the number of mistakes of the agent. The ablation studies results on the public EEG emotion dataset DEAP verified the important roles of the frontal lobe and the Papez circuit in modeling the procedure of learning rewards during the perception of emotions, with a great increase in the average accuracies by 25.24% and 23.31% in valence and arousal dimensions.

MRCNN: Multi-input residual convolution neural network for three-dimensional reconstruction of bubble flows from light field images

Accurate measurement of bubbles in air-water two-phase flows holds immense significance in the realm of thermal hydraulics assessments within nuclear reactors. Nevertheless, conventional bubble measurement techniques grapple with challenges encompassing system intricacy, limited real-time capabilities, and inaccuracies stemming from their inherent two-dimensional (2-D) nature. In response, we pioneered an innovative three-dimensional (3-D) analysis approach that leverages light field (LF) imaging diagnosis and deep learning algorithms. Unlike traditional 2-D reconstruction methods, our approach enables direct computation of bubble depth from LF images using digital refocusing technology. Following calibration, a seamless transformation is established between the camera coordinate system and the real-world coordinate system using a sharpness evaluation algorithm. This calibration process ensures precise alignment of refocused images with real-world positions. Subsequently, fully automated and highly accurate computations of bubble depth are realized from input images via the incorporation of a multi-input residual convolution neural network (MRCNN). The limitations of traditional two-dimensional imaging techniques are effectively addressed by this methodology, resulting in a reduction in measurement errors. The study confirms the feasibility of employing LF imaging diagnosis and deep learning algorithms for bubble measurements in an air-water two-phase flow, offering a significant improvement over traditional methods.

FCT-Net: A dual-encoding-path network fusing atrous spatial pyramid pooling and transformer for pavement crack detection

Cracks are a typical form of road damage, and accurate detection of cracks is of great significance for road maintenance work and ensuring traffic safety. Recently, computer vision has gradually been applied in the field of crack segmentation. However, there are still some extremely challenging problems in crack segmentation, such as complex backgrounds, information loss caused by pooling and convolution operations, and insufficient fusion of global and local semantic information. In response to the above problems, this paper proposes a dual-encoding-path network with U-Net architecture called FCT-Net, by fusing channel atrous spatial pyramid pooling (CASPP) and transformer. Specifically, CASPP obtains multi-scale receptive fields by incorporating spatial and channel attention, while refining and extracting local features. Meanwhile, we introduce long-short distance attention to construct a novel transformer with the prominent characteristic of interaction between local and global attention features. In addition, a residual convolution module is designed to enhance the local features of the transformer. Furthermore, we devise a multi-scale attention weight cross fusion module to aggregate the features of the dual encoding branch, for reducing information loss during downsampling and suppress background information. Eventually, we evaluate the performance of FCT-Net by experiments on three public datasets. Extensive experimental results show that FCT-Net achieves higher F1-score and mean intersection over union (mIoU) than state-of-the-art segmentation networks on the DeepCrack537 and CrackLS315 datasets. Meanwhile, it has excellent segmentation performance for cracks in complex scenes, with the highest recall, F1-score, and mIoU respectively as 85.64%, 81.67%, and 84.05% on the CrackTree260 dataset.

Intelligent mechanical fault diagnosis using multiscale residual network and multisensor fusion

Abstract Mechanical faults in manufacturing systems need to be diagnosed accurately to ensure safety and cost savings. With the development of sensor technologies, data from multiple sensors is frequently utilized to assess the health of intricate industrial systems. In such cases, it is necessary to study the multisensor data based intelligent mechanical fault diagnosis method. First, the multisensor data is converted into grey images and then fused into a three-channel red-green-blue (RGB) image. Then, a multiscale with residual convolution module is proposed, which can extract multiscale deep features of the complex raw signal. Additionally, an attention module for channel and spatial attention is introduced to adaptively adjust the feature response values of each scale. Two datasets and a specific engineering application are used to validate the superiority of the network. The results show that the multisensor multiscale residual network outperforms other fault diagnosis networks in terms of fault identification accuracy, diagnostic efficiency, and applicability.

Video frame interpolation based on depthwise over-parameterized recurrent residual convolution

Video frame interpolation based on depthwise over-parameterized recurrent residual convolution

A multi-scale attention mechanism for detecting defects in leather fabrics

Defect detection is critical to industrial quality control in leather production engineering. The various sizes and locations of defects in leather, as well as significant differences within the same class and indistinctive variations between different classes of defects, contribute to the complexity of the problem. To address this challenge, we propose a Multi-Layer Residual Convolutional Attention (MLRCA) approach. MLRCA enhances its ability to capture both intra-class and inter-class differences by enhancing the semantic feature representation in the backbone network. To improve multiscale fusion effects, we also incorporate the MLRCA module into the feature pyramid network (FPN) and propose a new multi-layer residual convolution attention feature pyramid network (ML-FPN). This approach enables more accurate identification of leather defects at a more detailed level by selectively capturing contextual information from different domains. We then implement the Side-Aware Boundary Localization (SABL) detection head, which accurately locates defects and helps the network distinguish between similar defect categories for more precise positioning. To validate the effectiveness of our approach, we conducted ablation experiments on the created leather dataset. Comparative experiments demonstrate the excellent capability of our model to detect minor defects. The model achieved 83.4, 89.7, and 85.6 for the AP, AP50, and AP75 evaluation metrics. In addition, the model achieves 71.3, 89.9, and 88.9 for APS, APM, and APL. Our approach has been confirmed feasible through experimentation and provides new insights for automated leather defect detection methods.

A high-speed YOLO detection model for steel surface defects with the channel residual convolution and fusion-distribution

Abstract Accurately and efficiently detecting steel surface defects is a critical step in steel manufacturing. However, the compromise between the detection speed and accuracy remains a major challenge, especially for steel surface defects with large variations in the scale. To address the issue, an improved you only look once (YOLO) based detection model is proposed through the reinforcement of its backbone and neck. Firstly, for the reduction of the redundant parameters and also the improvement of the characterization ability of the model, an effective channel residual structure is adopted to construct a channel residual convolution module and channel residual cross stage partial module as components of the backbone network, respectively. They realize the extraction of both the shallow feature and multi-scale feature simultaneously under a small number of convolutional parameters. Secondly, in the neck of YOLO, a fusion-distribution strategy is employed, which extracts and fuses multi-scale feature maps from the backbone network to provide global information, and then distributes global information into local features of different branches through an inject attention mechanism, thus enhancing the feature gap between different branches. Then, a model called CRFD-YOLO is derived for the steel surface defect detection and localization for the situations where both speed and accuracy are demanding. Finally, extensive experimental validations are conducted to evaluate the performance of CRFD-YOLO. The validation results indicate that CRFD-YOLO achieves a satisfactory detection performance with a mean average precision of 81.3% on the NEU-DET and 71.1% on the GC10-DET. Additionally, CRFD-YOLO achieves a speed of 161 frames per second, giving a great potential in real-time detection and localization tasks.

MRCM‐UCTransNet: Automatic and Accurate 3D Tooth Segmentation Network From Cone‐Beam CT Images

ABSTRACTMany scenarios in dental clinical diagnosis and treatment require the segmentation and identification of a specific tooth or the entire dentition in cone‐beam computed tomography (CBCT) images. However, traditional segmentation methods struggle to ensure accuracy. In recent years, there has been significant progress in segmentation algorithms based on deep learning, garnering considerable attention. Inspired by models from present neuro networks such as UCTransNet and DC‐Unet, this study proposes an MRCM‐UCTransNet for accurate three‐dimensional tooth segmentation from cone‐beam CT images. To enhance feature extraction while preserving the multi‐head attention mechanism, a multi‐scale residual convolution module (MRCM) is integrated into the UCTransNet architecture. This modification addresses the limitations of traditional segmentation methods and aims to improve accuracy in tooth segmentation from CBCT images. Comparative experiments indicate that, in the situation with a specific image size and small data volume, the proposed method exhibits certain advantages in segmentation accuracy and precision. Compared to traditional Unet approaches, MRCM‐UCTransNet's dice accuracy is improved by 7%, while its sensitivity is improved by about 10%. These findings highlight the efficacy of the proposed approach, particularly in scenarios with specific image size constraints and limited data availability. The proposed MRCM‐UCTransNet algorithm integrates the latest architectural advancements in the Unet model which achieves effective segmentation of six types of teeth within the tooth. It was proved to be efficient for image segmentation on small datasets, requiring less training time and fewer parameters.

Efficient Mixed-Type Wafer Defect Pattern Recognition Based on Light-Weight Neural Network.

Wafer defect pattern recognition can help engineers improve the production process of semiconductor chips. In real industrial scenarios, the recognition of mixed-type wafer defects is difficult and the production scale of semiconductor wafers is large, which requires high accuracy and speed in wafer defect pattern recognition. This study proposes a light-weight neural network model to efficiently recognize mixed-type wafer defects. The proposed model is constructed via inverted residual convolution blocks with attention mechanisms and large kernel convolution downsampling layers. The inference speed of the inverted residual convolution block is fast, and the attention mechanism can enhance feature extraction capabilities. Large kernel convolutions help the network retain more important feature information during downsampling operations. The experimental results on the real Mixed-type WM38 dataset show that the proposed model achieves a recognition accuracy of 98.69% with only 1.01 M parameters. Compared with some popular high-performance models and light-weight models, our model has advantages in both recognition accuracy and inference speed. Finally, we deploy the model as a TensorRT engine, which significantly improves the inference speed of the model, enabling it to process more than 1300 wafer maps per second.

DSTAN: A Deformable Spatial-temporal Attention Network with Bidirectional Sequence Feature Refinement for Speckle Noise Removal in Thyroid Ultrasound Video.

Thyroid ultrasound video provides significant value for thyroid diseases diagnosis, but the ultrasound imaging process is often affected by the speckle noise, resulting in poor quality of the ultrasound video. Numerous video denoising methods have been proposed to remove noise while preserving texture details. However, existing methods still suffer from the following problems: (1) relevant temporal features in the low-contrast ultrasound video cannot be accurately aligned and effectively aggregated by simple optical flow or motion estimation, resulting in the artifacts and motion blur in the video; (2) fixed receptive field in spatial features integration lacks the flexibility of aggregating features in the global region of interest and is susceptible to interference from irrelevant noisy regions. In this work, we propose a deformable spatial-temporal attention denoising network to remove speckle noise in thyroid ultrasound video. The entire network follows the bidirectional feature propagation mechanism to efficiently exploit the spatial-temporal information of the whole video sequence. In this process, two modules are proposed to address the above problems: (1) a deformable temporal attention module (DTAM) is designed after optical flow pre-alignment to further capture and aggregate relevant temporal features according to the learned offsets between frames, so that inter-frame information can be better exploited even with the imprecise flow estimation under the low contrast of ultrasound video; (2) a deformable spatial attention module (DSAM) is proposed to flexibly integrate spatial features in the global region of interest through the learned intra-frame offsets, so that irrelevant noisy information can be ignored and essential information can be precisely exploited. Finally, all these refined features are rectified and merged through residual convolution blocks to recover the clean video frames. Experimental results on our thyroid ultrasound video (US-V) dataset and the DDTI dataset demonstrate that our proposed method exceeds 1.2 1.3dB on PSNR and has clearer texture detail compared to other state-of-the-art methods. In the meantime, the proposed model can also assist thyroid nodule segmentation methods to achieve more accurate segmentation effect, which provides an important basis for thyroid diagnosis. In the future, the proposed model can be improved and extended to other medical image sequence datasets, including CT and MRI slice denoising. The code and datasets are provided at https://github.com/Meta-MJ/DSTAN .

Integration of wavelet packet transform, residual convolution neural network, and support vector machine for series arc fault detection

Series arc fault (SAF) poses a great challenge to the safe and stable operation of civil low-voltage distribution systems. For the accurate and rapid detection of SAF, this article proposes an SAF detection method using wavelet packet transform (WPT), residual convolution neural network (RCNN), and support vector machine (SVM). First, the raw current signal is decomposed into four wavelet components based on WPT. Then, the 1-D wavelet components are converted into 2-D matrices. Afterward, the matrices are input into RCNN through different channels. Finally, the detection results can be yielded by SVM. The effectiveness of the proposed method is verified based on offline experiments. The average detection accuracy of the proposed method is 99.72%, which is higher than that of the eight comparison methods. Moreover, the results of online experiments indicate that the detection time of the proposed method is less than 100 ms and can satisfy the requirement of standard 1699.