Residual Network Structure Research Articles

SSD Object Detection Algorithm Based on Feature Fusion and Channel Attention

Abstract Aiming at the problems of low object detection accuracy due to complex background and insufficient semantic information of shallow features in the object detection SSD algorithm, this paper improves the existing SSD algorithm. First, the original vgg16 network is replaced by the ResNet50 network, and the residual network structure as well as the Batch Normalization layer are added, which are used to improve the accuracy of the feature extraction network; Second, a feature fusion module is designed to fuse adjacent feature maps to improve the detection effect by integrating contextual information; Third, the SE attention mechanism is introduced to give channel weights adaptively and enhance the useful feature channels; Finally, the object detection analysis experiments are conducted on the PASCAL VOC2012 dataset. The experimental results show that the improved SSD algorithm in this paper is able to achieve an mean average precision of 72.7% in the data set, which is 2.1% better than the original SSD-VGG16 and greatly improves the object detection effect.

Research on the Estimation of Gaze Location for Head-eye Coordination Movement

Abstract Sight is the main source for humans to obtain information from the outside world. Due to the structure of the human eye [1], the range of human sight is limited. For this reason, people need to constantly move their line of sight when observing the surrounding environment and the target, and the movement of the sight is based on the coordinated movement of the head and the eye[2]. Therefore, the key issue for gaze research is how to correctly establish the relationship between head-eye movement and gaze movement. Taking the simulated flight environment as the research background, this paper collects a large number of head-eye images through the designed “three-camera and eight-light source” head-eye data acquisition platform, and proposes a gaze estimation method based on the combination of appearance and features, which effectively combines The relationship of head-eye coordination movement. Then, the ResNet-18 deep residual network structure and the traditional BP neural network structure are used to complete the effective fusion of the head pose and human eye features in the process of capturing the sight target, so as to realize the accurate estimation of the sight drop point, and its average accuracy up to 89.9%.

GraphXSS: An efficient XSS payload detection approach based on graph convolutional network

With the rapid development of the Internet age today, Web applications have become very common in modern society. Web applications are often applied to a social network, media, management, etc., and usually contain a large amount of personal privacy information, which makes Web applications a common target for hackers. The most common method for stealing private information from web applications is cross-site scripting attacks. Attackers frequently use cross-site scripting vulnerabilities to steal victims’ identity information or hijack login tokens. Therefore, we proposed a cross-site scripting payload detection model based on graph convolutional networks, which could identify the cross-site scripting payload in the content submitted by the user (We termed our implementation of this approach, GraphXSS). We preprocessed the sample, and constructed the processed data into a graph structure, and finally used the graph convolutional network and the residual network to train the cross-site scripting detection model. In experiments, the model based on graph convolutional network (GCN) could achieve AUC value of 0.997 under small sample conditions. Compared with the detection model after adding the residual network structure, the model could converge and stabilize under the multi-layer, and could make the accuracy rate reached 0.996.

Establishment of super sonic inlet flow pattern monitoring system: A workflow

To reduce the negative effects of the buzz flow state of supersonic air inlets, this paper proposes a process in which an online monitoring system for the supersonic inlet flow patterns is established with pressure sensors. The first step in this workflow is to build an experimental system based on the objective air inlet and a large number of pressure sensors. After that, for each pressure sensor, based on Sliding Window Short-Time Fourier Transform (SW-STFT) pre-processing and Deep residual network (ResNet) structure, a method to train a sub classifier (Classifier trained by only one sensor signal) with each pressure sensor is proposed to achieve efficient extraction of highly perturbed signal information and high classification accuracy of different flow patterns. However, the main limitation of this workflow is that too many pressure sensors can increase the complexity of aero engine control systems and make the installation difficult. Therefore, in the third step, a multi-objective optimization model is built to solve this problem and find the best subset of sensors in the experimental system. Compared with previous studies, the flow modal monitoring system established in this paper has the advantages of real-time monitoring, higher classification accuracy, and better performance robustness. At the same time, this workflow allows the construction of individualized flow monitoring systems for different inlets.

Imbalanced fault diagnosis of rolling bearing using improved MsR-GAN and feature enhancement-driven CapsNet

Traditional fault diagnosis approaches of rolling bearing often need abundant labeled data in advance while some certain fault data are difficult to be acquired in engineering scenarios. This imbalanced fault data problem limits the diagnostic performance. To solve it, an imbalanced fault diagnosis approach based on improved multi-scale residual generative adversarial network (GAN) and feature enhancement-driven capsule network is proposed in this paper. Firstly, frequency slicing wavelet transform is utilized to extract two-dimensional time–frequency features from original vibration signals. By designing multi-scale residual network structure and hybrid loss function, original GAN model is improved, generating high-quality fake time–frequency features to balance fault data distribution. To increase the attention of the diagnostic model to fault-sensitive features and suppress irrelevant features, a feature enhancement network is designed to dynamically weight the fault features by modeling the feature importance. On this basis, enhanced performance of imbalanced fault classification is achieved. Verification experiments demonstrate that it performs well in processing the imbalanced fault data, and has better stability and diagnostic accuracy than state-of-the-art methods.

Missing data imputation and sensor self-validation towards a sustainable operation of wastewater treatment plants via deep variational residual autoencoders

Missing data imputation and automatic fault detection of wastewater treatment plant (WWTP) sensors are crucial for energy conservation and environmental protection. Given the dynamic and non-linear characteristics of WWTP measurements, the conventional diagnosis models are inefficient and ignore potential valuable features in the offline modeling phase, leading to false alarms and inaccurate imputations. In this study, an inclusive framework for missing data imputation and sensor self-validation based on integrating variational autoencoders (VAE) with a deep residual network structure (ResNet-VAE) is proposed. This network structure can automatically extract complex features from WWTP data without the risk of vanishing gradients by learning the potential probability distribution of the input data. The proposed framework is intended to increase the reliability of faulty sensors by imputing missing data, detecting anomalies, identifying failure sources, and reconstructing faulty data to normal conditions. Several metrics were utilized to assess the performance of the suggested approach in comparison with other different methods. The VAE-ResNet approach showed superiority to detect (DRSPE = 100%), reconstruct faulty WWTP sensors (MAPE = 15.41%–5.68%) and impute the missing values (MAPE = 10.44%–3.98%). Lastly, the consequences of faulty data, missing data, reconstructed and imputed data were evaluated considering electricity consumption and resilience to demonstrate the ResNet-VAE model's superior performance for WWTP sustainability.

Remote anomaly detection and classification of solar photovoltaic modules based on deep neural network

Solar photovoltaic systems are being widely used in green energy harvesting recently. At the same rate of growth, the modules that come to the end of life are growing fast. The solar modules contain heavy metals such as lead, tin, and cadmium, which could pollute the environment. Inspection and maintenance of solar modules are important to increase the lifetime, reduce energy loss, and environmental protection. In this research, we proposed an efficient way for inspection and classification of anomaly solar modules using infrared radiation (IR) cameras and deep neural networks. The IR cameras could capture the temperature distribution on the solar modules remotely, and the deep neural networks could accurately prediction of the anomaly modules and classification of the anomaly types. We proposed a deep neural network based on a residual network structure and ensemble technique to accurately predict and classify anomaly solar modules based on the IR images. An IR images dataset on real solar farms with 20,000 images and 12 anomaly solar modules was used to verify the proposed approach. The experiment results show that the proposed model could predict an anomaly module on an average of 94% and correctly classify 12 anomaly types on an average of 86%.

Research on Airport Aircraft Target Detection Algorithm Based on YOLOv4

Aiming at the complex airport environment and the large scale transformation of aircraft, a target detection algorithm based on YOLOv4 is designed. Based on the YOLOv4 algorithm structure, a feature fusion scale is added to the original 3 detection scales, which has the ability to obtain smaller target information. Simultaneously, the residual network structure is simplified, and a certain amount of calculation is reduced. In order to optimize the loss, the regression loss function CIoU is introduced, which accelerates the convergence of the network. The test results show that the algorithm of this article can reach a prediction accuracy of 93.31% for airplane target, a recall rate of 83.92%, a mean average accuracy of 91.02%, which is 2.49% higher than YOLOv4, which improves the recognition accuracy of the model in the airport environment.

Construction of super-resolution model of remote sensing image based on deep convolutional neural network

With the continuous improvement of satellite remote sensing technology, using super-resolution image reconstruction technology to reconstruct remote sensing images has important application significance for social development. In the generator model proposed in this paper, the standard convolution layer in the residual network structure is replaced by empty convolution to improve the overall performance of the model while keeping the number of parameters unchanged and the receptive field of convolution at each stage unchanged. By analyzing the advantages of residual network, dense connection network, and cavity convolution in the field of image super resolution, an optimized super-resolution reconstruction model of GAN image with cavity convolution is constructed with dense connection block of cavity residue as a generator component. The cloud computing-based service model is introduced into the image reconstruction system, and the background management module is built through the cloud service system, which is responsible for model training, image transmission, image processing request and database reading. Through experimental analysis, it is proved that the whole automatic data processing from automatic matching data to processing data can be completed, and the performance is better than the traditional service mode, which can produce great economic benefits.

Improved Transformer Net for Hyperspectral Image Classification

In recent years, deep learning has been successfully applied to hyperspectral image classification (HSI) problems, with several convolutional neural network (CNN) based models achieving an appealing classification performance. However, due to the multi-band nature and the data redundancy of the hyperspectral data, the CNN model underperforms in such a continuous data domain. Thus, in this article, we propose an end-to-end transformer model entitled SAT Net that is appropriate for HSI classification and relies on the self-attention mechanism. The proposed model uses the spectral attention mechanism and the self-attention mechanism to extract the spectral–spatial features of the HSI image, respectively. Initially, the original HSI data are remapped into multiple vectors containing a series of planar 2D patches after passing through the spectral attention module. On each vector, we perform linear transformation compression to obtain the sequence vector length. During this process, we add the position–coding vector and the learnable–embedding vector to manage capturing the continuous spectrum relationship in the HSI at a long distance. Then, we employ several multiple multi-head self-attention modules to extract the image features and complete the proposed network with a residual network structure to solve the gradient dispersion and over-fitting problems. Finally, we employ a multilayer perceptron for the HSI classification. We evaluate SAT Net on three publicly available hyperspectral datasets and challenge our classification performance against five current classification methods employing several metrics, i.e., overall and average classification accuracy and Kappa coefficient. Our trials demonstrate that SAT Net attains a competitive classification highlighting that a Self-Attention Transformer network and is appealing for HSI classification.

A deep learning approach for fast detection and classification of concrete damage

The detection and classification of concrete damage are essential for keeping the infrastructure in good condition which have become a focused area. It is worth noting that the inference speed and accuracy of object detection algorithms are the key issues in concrete damage detection tasks. This research has made the following contributions to solving these problems. Firstly, a concrete damage data set is prepared. The data set consists of 5000 images containing the types of crack, spot, rebar exposure and spalling damage. Secondly, the object detection algorithm is optimized by depthwise separable convolution, inverse residual network and linear bottleneck structure. The experimental results show that the inference speed of the improved object detection algorithm is 24.1% and 53.5% higher than that of the original network, respectively. Compared with the original YOLO-v3 and SSD object detection algorithms, the complete detection accuracy of the network reaches 64.81% and 64.12%, which increased by 3.25% and 4.04%, respectively.

Non-Uniformity Correction of Infrared Images Based on Improved CNN With Long-Short Connections

Non-uniformity is a common phenomenon in infrared imaging system, which seriously affects imaging quality. In view of the problems of existing non-uniformity correction of infrared images, such as loss of image details and blurred edge of image, an improved non-uniformity correction method of infrared images based on convolution neural network using long-short connections (LSC-CNN) is proposed. The proposed method designs a long-short connection residual network structure suitable for non-uniformity correction of infrared image.The network depth is increased to fully learn the noise by short connections, image sizes are adjusted to reduce the number of parameters, the long connection is used to solve the problem of image information loss caused by transposed convolution, and a multiply operation is carried out to enhance the contrast of corrected images. Besides, batch normalization is utilized to improve the training speed. The experimental results show that LSC-CNN has excellent performance in non-uniformity correction of infrared images whether qualitative evaluation or quantitative evaluation. LSC-CNN is especially effective in image detail preservation and image edge protection whose average PSNR exceeds 37.5 dB and the average SSIM is greater than 0.98.

SAResNet: self-attention residual network for predicting DNA-protein binding.

Knowledge of the specificity of DNA-protein binding is crucial for understanding the mechanisms of gene expression, regulation and gene therapy. In recent years, deep-learning-based methods for predicting DNA-protein binding from sequence data have achieved significant success. Nevertheless, the current state-of-the-art computational methods have some drawbacks associated with the use of limited datasets with insufficient experimental data. To address this, we propose a novel transfer learning-based method, termed SAResNet, which combines the self-attention mechanism and residual network structure. More specifically, the attention-driven module captures the position information of the sequence, while the residual network structure guarantees that the high-level features of the binding site can be extracted. Meanwhile, the pre-training strategy used by SAResNet improves the learning ability of the network and accelerates the convergence speed of the network during transfer learning. The performance of SAResNet is extensively tested on 690 datasets from the ChIP-seq experiments with an average AUC of 92.0%, which is 4.4% higher than that of the best state-of-the-art method currently available. When tested on smaller datasets, the predictive performance is more clearly improved. Overall, we demonstrate that the superior performance of DNA-protein binding prediction on DNA sequences can be achieved by combining the attention mechanism and residual structure, and a novel pipeline is accordingly developed. The proposed methodology is generally applicable and can be used to address any other sequence classification problems.

Diagnosis and prognosis of epidemic inflammatory bowel disease under convolutional neural network algorithm and nonlinear equation model

this study was to analyze the effect of optimization algorithm based on convolutional neural network (CNN) on the image quality of magnetic resonance enterography (MRE) for small intestine, and explore the diagnosis and prognosis of epidemic inflammation bowel disease (IBD) with the MRE under the optimization algorithm and nonlinear equation model. The residual network structure (RNS) was incorporated into the CNN algorithm to establish an optimization algorithm, which was compared with other algorithms in terms of the peak signal to noise ratio (PSNR), structural similarity (SSIM), and high frequency error norm (HFEN). 392 patients suspected as IBD in the Gastroenterology Department were selected as the research objects. They were divided into a Crohn’s disease (CD) group (243 cases), an ulcerative colitis (UC) group (78 cases), an indeterminate enteritis (IDE) group (49 cases), and a non-IBD group (22 cases). The intestinal manifestations, clinical manifestations, MRE imaging manifestations, and gastrointestinal lesion distribution of patients in different groups were compared to analyze the diagnostic value of MRE for epidemic IBD. It was found that the optimized CNN algorithm showed better PSNR, SSIM, and HFEN than other algorithms. The percentages of patients with diarrhea in the CD group and UC group were 67.90% and 100%, respectively. The pediatric ulcerative colitis activity index (PUCAI) score of patients in the UC group showed that the number of patients with mild activity was at most 40 (51.28%). The Crohn’s disease activity index (CDAI) score showed that there were 197 of patients in the CD group (81.07%) with severe activity. In the CD group, the numbers of patients suffered with lesions in proximal colon, terminal ileum, and upper gastrointestinal tract were 125 (51.44%), 127 (52.26%), and 132 (54.32%), respectively. In the UC group, there were 32 (41.03%), 51 (65.38%), and 25 (32.05%) patients with lesions in the rectum, distal colon, and proximal colon, respectively. The diagnostic sensitivity and specificity of MRE for epidemic IBD was 95.18% and 46.7%, respectively. It indicated that the image quality of the optimization algorithm based on the CNN algorithm was improved greatly. Under the nonlinear equation model, the image characteristics of MRE in patients with different diseases were various, and the diseased parts were greatly different. MRE showed high sensitivity and specificity in the diagnosis of epidemic IBD. This study could provide a reference basis for the clinical diagnosis and prognosis of patients with epidemic IBD.

DR-Net: An Improved Network for Building Extraction from High Resolution Remote Sensing Image

At present, convolutional neural networks (CNN) have been widely used in building extraction from remote sensing imagery (RSI), but there are still some bottlenecks. On the one hand, there are so many parameters in the previous network with complex structure, which will occupy lots of memories and consume much time during training process. On the other hand, low-level features extracted by shallow layers and abstract features extracted by deep layers of artificial neural network cannot be fully fused, which leads to an inaccurate building extraction from RSI. To alleviate these disadvantages, a dense residual neural network (DR-Net) was proposed in this paper. DR-Net uses a deeplabv3+Net encoder/decoder backbone, in combination with densely connected convolution neural network (DCNN) and residual network (ResNet) structure. Compared with deeplabv3+net (containing about 41 million parameters) and BRRNet (containing about 17 million parameters), DR-Net contains about 9 million parameters; So, the number of parameters reduced a lot. The experimental results for both the WHU Building Dataset and Massachusetts Building Dataset, DR-Net show better performance in building extraction than other two state-of-the-art methods. Experiments on WHU building data set showed that Intersection over Union (IoU) increased by 2.4% and F1 score increased by 1.4%; in terms of Massachusetts Building Dataset, IoU increased by 3.8% and F1 score increased by 2.9%.

Image super-resolution reconstruction for secure data transmission in Internet of Things environment.

The image super-resolution reconstruction method can improve the image quality in the Internet of Things (IoT). It improves the data transmission efficiency, and is of great significance to data transmission encryption. Aiming at the problem of low image quality in image super-resolution using neural networks, a self-attention-based image reconstruction method is proposed for secure data transmission in IoT environment. The network model is improved, and the residual network structure and sub-pixel convolution are used to extract the feature of the image. The self-attention module is used extract detailed information in the image. Using generative confrontation method and image feature perception method to improve the image reconstruction effect. The experimental results on the public data set show that the improved network model improves the quality of the reconstructed image and can effectively restore the details of the image.

A Novel SAR Image Ship Small Targets Detection Method

To satisfy practical requirements of high real-time accuracy and low computational complexity of synthetic aperture radar (SAR) image ship small target detection, this paper proposes a small ship target detection method based on the improved You Only Look Once Version 3 (YOLOv3). The main contributions of this study are threefold. First, the feature extraction network of the original YOLOV3 algorithm is replaced with the VGG16 network convolution layer. Second, general convolution is transformed into depthwise separable convolution, thereby reducing the computational cost of the algorithm. Third, a residual network structure is introduced into the feature extraction network to reuse the shallow target feature information, which enhances the detailed features of the target and ensures the improvement in accuracy of small target detection performance. To evaluate the performance of the proposed method, many experiments are conducted on public SAR image datasets. For ship targets with complex backgrounds and small ship targets in the SAR image, the effectiveness of the proposed algorithm is verified. Results show that the accuracy and recall rate improved by 5.31% and 2.77%, respectively, compared with the original YOLOV3. Furthermore, the proposed model not only significantly reduces the computational effort, but also improves the detection accuracy of ship small target.

Dirt Classification of Silicon Wafers Based on Deep Learning

Deep learning technology is now widely used in the industry, and some irregular blemishes and stains can be solved by deep learning methods. In this paper, the deep learning resnet network is used to classify and judge the dirt that appears on the silicon chip. It is a residual network structure. The mapping relationship between layers is realized through jump connections to ensure that the number of network layers’ increases. In the process of gradient backpropagation, there will be no gradient disappearance and gradient explosion, and the goal of training deep networks is achieved. This article is to apply the resnet deep neural network on the silicon wafer image to make a certain area on the silicon wafer make the right judgment for the dirty problem.

Open set recognition algorithm based on Conditional Gaussian Encoder.

For the existing Closed Set Recognition (CSR) methods mistakenly identify unknown jamming signals as a known class, a Conditional Gaussian Encoder (CG-Encoder) for 1-dimensional signal Open Set Recognition (OSR) is designed. The network retains the original form of the signal as much as possible and deep neural network is used to extract useful information. CG-Encoder adopts residual network structure and a new Kullback-Leibler (KL) divergence is defined. In the training phase, the known classes are approximated to different Gaussian distributions in the latent space and the discrimination between classes is increased to improve the recognition performance of the known classes. In the testing phase, a specific and effective OSR algorithm flow is designed. Simulation experiments are carried out on 9 jamming types. The results show that the CSR and OSR performance of CG-Encoder is better than that of the other three kinds of network structures. When the openness is the maximum, the open set average accuracy of CG-Encoder is more than 70%, which is about 30% higher than the worst algorithm, and about 20% higher than the better one. When the openness is the minimum, the average accuracy of OSR is more than 95%.

Diagnosis of gastric lesions through a deep convolutional neural network

A deep convolutional neural network (CNN) was used to achieve fast and accurate artificial intelligence (AI)-assisted diagnosis of early gastric cancer (GC) and other gastric lesions based on endoscopic images. A CNN-based diagnostic system based on a ResNet34 residual network structure and a DeepLabv3 structure was constructed and trained using 21,217 gastroendoscopic images of five gastric conditions, peptic ulcer (PU), early gastric cancer (EGC) and high-grade intraepithelial neoplasia (HGIN), advanced gastric cancer (AGC), gastric submucosal tumors (SMTs), and normal gastric mucosa without lesions. The trained CNN was evaluated using a test dataset of 1091 images. The accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the CNN were calculated. The CNN diagnosis was compared with those of 10 endoscopists with over 8years of experience in endoscopic diagnosis. The diagnostic specificity and PPV of the CNN were higher than that of the endoscopists for the EGC and HGIN images (specificity: 91.2% vs. 86.7%, by 4.5%, 95% CI 2.8-7.2%; PPV: 55.4% vs. 41.7%, by 13.7%, 95% CI 11.2-16.8%) and the diagnostic accuracy of the CNN was close to those of the endoscopists for the lesion-free, EGC and HGIN, PU, AGC, and SMTs images. The CNN had image recognition time of 42s for all the test set images. The constructed CNN system could be used as a rapid auxiliary diagnostic instrument to detect EGC and HGIN, as well as other gastric lesions, to reduce the workload of endoscopists.