Residual Network Model Research Articles

Convolutional neural network-based modulation recognition technique for communication signals

The last decade has seen breakthroughs in communication technology. The increasingly complex signal transmission environment has placed higher demands on signal modulation recognition. Traditional modulation recognition approaches cannot guarantee satisfactory recognition accuracy. Fortunately, with the continuous advancement of deep learning algorithms, convolutional neural network-based communication signal modulation recognition techniques have become the mainstream of current research. Therefore, this paper first reviews the development history of signal modulation recognition techniques and introduces the concepts of signal modulation theory. It includes ASK, PSK and FSK modulation methods, which are common today. Subsequently, I analyze the principles of signal modulation recognition and the implementation method of CNN in modulation recognition. To further explore the shortcomings of CNNs, I propose two optimized models, the residual network model and the CLDNN model. After comparing the performance, the former has higher performance, but its computational complexity is higher while the latter takes into account the high recognition accuracy while still reducing the network parameters as much as possible to keep the complexity at a low level.

Automated vehicle wheelbase measurement using computer vision and view geometry

For different transportation agencies that monitor vehicle overloads, develop policies to mitigate the impact of vehicles on infrastructure, and provide the necessary data for road maintenance, they all rely on precise, detailed and real-time vehicle data. Currently, real-time collection of vehicle data (type, axle load, geometry, etc) is typically performed through weigh-in-motion (WIM) stations. In particular, the bridge WIM (BWIM) technology, which uses instrumented bridges as weighing platforms, has proven to be the most widely used inspection method. For most of the BWIM algorithms, the position of the vehicle’s axle (i.e. vehicle wheelbase) needs to be measured before calculating the axle load, and the identification of the axle load is very sensitive to the accuracy of the vehicle wheelbase. In addition, the vehicle’s wheelbase is also important data when counting stochastic traffic flow and classifying passing vehicles. When performing these statistics, the amount of data is often very large, and the statistics can take years or even decades to complete. Traditional manual inspection and recording approaches are clearly not up to the task. Therefore, to achieve automatic measurement of the on-road vehicles’ wheelbase, a framework based on computer vision and view geometry is developed. First, images of on-road vehicles are captured. From the images, the vehicle and wheel regions can be accurately detected based on the You Only Look Once version 5 (YOLOv5) architecture. Then, the residual unified network model is improved and an accurate semantic segmentation of the wheel within the bounding box is performed. Finally, a view geometry-based algorithm is developed for identifying vehicle wheelbase. The accuracy of the proposed method is verified by comparing the identified results with the true wheelbases of both two-axle vehicles and multi-axis vehicles. To further validate the effectiveness and robustness of the framework, the effects of important factors, such as camera position, vehicle angle, and camera resolution, are investigated through parametric studies. To illustrate its superiority, the developed vehicle wheelbase measurement algorithm is compared with two other advanced vehicle geometry parameter identification algorithms and the results show that the developed algorithm outperforms the other two methods in terms of the degree of automation and accuracy.

Deep learning-based pulsar candidate identification model using a variational autoencoder

As radio telescope technology continues to advance, the discovery of more pulsar candidates is anticipated. The swift and accurate identification of pulsars has emerged as a compelling research topic in recent years, garnering growing attention. Although deep learning has been expansively applied for the rapid identification of pulsar candidates, the significant imbalance between the larger number of non-pulsar signal samples and the fewer pulsar signal samples seriously impacts the performance of deep learning prediction models. To address this, our research proposes a method that combines a Vector Quantized Variational Autoencoder (VQVAE) with a Gated PixelCNN to equalize the number of pulsar and non-pulsar samples during sample generation. Subsequently, we establish a Residual Neural Network (ResNet) model that employs two feature diagnostic plots as input to classify and identify pulsar candidates. Validation on the High Time Resolution Universe (HTRU) dataset yielded a recall and accuracy of 99.60%, and an F1-score of 99.59%. Further evaluation on the Five-hundred-meter Aperture Spherical radio Telescope (FAST) dataset demonstrated a precision of 98.5%, a recall of 97.8%, and an F1-score of 98.1%, further confirming the effectiveness of our proposed model.

Improved Deep Residual Shrinkage Network for Intelligent Interference Recognition with Unknown Interference.

In complex battlefield environments, flying ad-hoc network (FANET) faces challenges in manually extracting communication interference signal features, a low recognition rate in strong noise environments, and an inability to recognize unknown interference types. To solve these problems, one simple non-local correction shrinkage (SNCS) module is constructed. The SNCS module modifies the soft threshold function in the traditional denoising method and embeds it into the neural network, so that the threshold can be adjusted adaptively. Local importance-based pooling (LIP) is introduced to enhance the useful features of interference signals and reduce noise in the downsampling process. Moreover, the joint loss function is constructed by combining the cross-entropy loss and center loss to jointly train the model. To distinguish unknown class interference signals, the acceptance factor is proposed. Meanwhile, the acceptance factor-based unknown class recognition simplified non-local residual shrinkage network (AFUCR-SNRSN) model with the capacity for both known and unknown class recognition is constructed by combining AFUCR and SNRSN. Experimental results show that the recognition accuracy of the AFUCR-SNRSN model is the highest in the scenario of a low jamming to noise ratio (JNR). The accuracy is increased by approximately 4-9% compared with other methods on known class interference signal datasets, and the recognition accuracy reaches 99% when the JNR is -6 dB. At the same time, compared with other methods, the false positive rate (FPR) in recognizing unknown class interference signals drops to 9%.

One-dimensional magnetotelluric parallel inversion using a ResNet1D-8 residual neural network

Deep Learning is an effective tool to invert the underground electrical conductivity structure. In this study, we build a new 8-layer residual neural network (ResNet1D-8) for audio magnetotelluric (AMT) data inversion based on the deep learning theory. In terms of the network structure, the degradation of the model with the increase of depth is effectively avoided by adding shortcut connections. Meanwhile, adding a batch normalization layer greatly improves the model training speed and network generalization ability. This study uses the parallel technology to quickly generate millions of samples, which effectively reduces the computational time and provides a large number of high-quality samples for deep learning model training. Compared with the simulated annealing algorithm, the neural network model in this paper has the advantages of high reliability, short inversion time, and strong model generalization ability. Moreover, we add Gaussian noise to the data of testing samples, and inversion results show that the model has good robustness. The inversion test is carried out on the field measured AMT data set collected in the Dachaidan area of Qinghai Province, China. The results show that the ResNet1D-8 residual neural network model established in this paper can effectively invert the underground electrical structure.

Clinical applicability of an artificial intelligence prediction algorithm for early prediction of non-persistent atrial fibrillation.

It is difficult to document atrial fibrillation (AF) on ECG in patients with non-persistent atrial fibrillation (non-PeAF). There is limited understanding of whether an AI prediction algorithm could predict the occurrence of non-PeAF from the information of normal sinus rhythm (SR) of a 12-lead ECG. This study aimed to derive a precise predictive AI model for screening non-PeAF using SR ECG within 4 weeks. This retrospective cohort study included patients aged 18 to 99 with SR ECG on 12-lead standard ECG (10 seconds) in Ewha Womans University Medical Center for 3 years. Data were preprocessed into three window periods (which are defined with the duration from SR to non-PeAF detection) - 1 week, 2 weeks, and 4 weeks from the AF detection prospectively. For experiments, we adopted a Residual Neural Network model based on 1D-CNN proposed in a previous study. We used 7,595 SR ECGs (extracted from 215,875 ECGs) with window periods of 1 week, 2 weeks, and 4 weeks for analysis. The prediction algorithm showed an AUC of 0.862 and an F1-score of 0.84 in the 1:4 matched group of a 1-week window period. For the 1:4 matched group of a 2-week window period, it showed an AUC of 0.864 and an F1-score of 0.85. Finally, for the 1:4 matched group of a 4-week window period, it showed an AUC of 0.842 and an F1-score of 0.83. The AI prediction algorithm showed the possibility of risk stratification for early detection of non-PeAF. Moreover, this study showed that a short window period is also sufficient to detect non-PeAF.

Solving the identification problems of Bolete origins based on multiple data processing: Take Boletus bainiugan as an example

Wild porcini mushrooms have high nutritional value and are a kind of medicinal food. It is rich in protein and micronutrients needed by the human body, which is popular among consumers. In this paper, we take the problem of origin traceability of Boletus bainiugan as an example and use multiple data processing methods (preprocessing, feature extraction, and data fusion strategies) to establish partial least squares discriminant analysis and support vector machine models. In addition, a deep learning model (3DCOS-ResNet: three-dimensional correlation spectra combined with residual convolutional neural network model) is built to compare with the above two models. It was found that 3DCOS-ResNet was the best model to solve the Boletus baniugan origin traceability problem. Compared to the chemometrics model, it does not require complex processing of the data. The ideal identification effect can be achieved by directly utilizing the raw data, and it saves a lot of time and cost.

Segmentation and classification of medical big data on brain tumor using bacteria foraging optimization algorithm along with learning automata

AbstractDetecting the early stage of brain tumors is significant for an effective therapy that can probably minimize the death rate of patients affected from brain tumors. Magnetic resonance imaging is the benchmark standard for diagnosing brain cancers but, it was difficult to split and categorize different kinds of brain tumors due to the delicate arrangement of the brain's anatomy. To overcome these difficulties and provide an effective classification, this research introduced a hybridized optimization technique which enhance the performance of the classifier. The hybridization is done between Bacteria Foraging Optimization Algorithm (BFOA) along with Learning Automata (LA), these two techniques improve the search speed and automate the learning capability of Convolutional Neural Network (CNN) classifier. The obtained results show that proposed BFOA LN‐CNN classifier better than the existing Deep Convolutional Neural Network (DCNN) based on Improved Harris Hawks Optimization (HHO) known as G‐HHO and Improved Invasive Bat (IIB)‐based Deep Residual network model. The classification accuracy of the proposed method is 99.41% whereas the DCNN‐G‐HHO and IIB deep residual neural network provides accuracy of 97% and 91.9% respectively.

Multiscale Convolutional Attention-based Residual Network Expression Recognition

<p>Expression recognition has wide application in the fields of distance education and clinical medicine. In response to the problems of insufficient feature extraction ability of expression recognition models in current research, and the deeper the depth of the model, the more serious the loss of useful information, a residual network model with multi-scale convolutional attention is proposed. This model mainly takes the residual network as the main body, adds normalization layer and channel attention mechanism, so as to extract useful image information at multiple scales, and incorporates the Inception module and channel attention module into the residual network to enhance the feature extraction ability of the model and to prevent the loss of more useful information due to too deep network, and to improve the generalization performance of the model. From results of lots of experiments we can see that the recognition accuracy of the model in FER+ and CK+ datasets reaches 87.80% and 99.32% respectively, with better recognition performance and robustness.</p> <p> </p>

Micron-sized particle discrimination and classification with deep learning and electrical sensing zone technique

This paper used deep learning algorithms based on the electrical sensing zone (ESZ) method to realize real-time online monitoring and discrimination of particle sorts and their size distribution in liquid–solid systems. Numerical simulations reveal that the characteristics of the temporal pulse signals generated by particles flowing through the ESZ provide the clue to discriminate particle sorts. Based on the above-mentioned results, a deep learning algorithm was used to successfully discriminate and classify micron-sized particles based on the differences in particle pulse signals. This study was conducted to train and validate the residual network (ResNet) model in the deep learning algorithm using four synthetic mixed datasets of different particle pulse signals in a similar size range and to evaluate the performance of the model. Furthermore, the trained ResNet model was applied to the in-house online micron-sized particle analyzer to monitor and discriminate four sets of different particle pulses in a similar size range at the micron level, and the accuracy of the results was above 89%. The results were compared with the measured particle pulse distribution to confirm the feasibility and accuracy of the method. A new method for real-time online monitoring and discrimination of micron-sized particles in industrial production was provided.

Deep learning in Transportation: Optimized driven deep residual networks for Arabic traffic sign recognition

Car manufacturers around the globe are in a race to design and build driverless cars. The concept of driverless is also being applied to any moving vehicle such as wheelchairs, golf cars, tourism carts in recreational parks, etc. To achieve this ambition, vehicles must be able to drive safely on streets stay within required lanes, sense moving objects, sense obstacles, and be able to read traffic signs that are permanent and even temporary signs. It will be a completely integrated system of the Internet of Things (IoT), Global Positioning System (GPS), Machine Learning (ML)/Deep Learning (DL), and Smart Technologies. A lot of work has been done on traffic sign recognition in the English language, but little has been done for Arabic traffic sign recognition. The concepts used for traffic sign recognition can also be applied to indoor signage, smart cities, supermarket labels, and others. In this paper, we propose two optimized Residual Network (ResNet) models (ResNet V1 and ResNet V2) for automatic traffic sign recognition using the Arabic Traffic Signs (ArTS) dataset. Additionally, the authors developed a new dataset specifically for Arabic Traffic Sign recognition consisting of 2,718 images taken from random places in the Eastern province of Saudi Arabia. The optimized proposed ResNet V1 model achieved the highest training and validation accuracies of 99.18% and 96.14%, respectively. It should be noted here that the authors accounted for both overfitting and underfitting in the proposed models. It is also important to note that the results achieved using the proposed models outperform similar methods proposed in the extant literature for the same dataset or similar-size dataset.

Variable-Structure Proportional-Integral-Derivative Laser Solder Joint Temperature Intelligent Control Method with Adjustable Power Upper Limit.

Laser soldering is a crucial soldering technique in the realm of electronic assembly. The temperature of the solder joint is intimately connected with the quality of the solder. This paper introduces an adjustable power upper limit variable-structure Proportional-Integral-Derivative (PID) intelligent control method for regulating the temperature of the solder joint during laser soldering. Distinct laser power limits are employed for workpieces with varying heat capacities. The solder joint temperature is monitored through an infrared thermometer, which enables closed-loop temperature control via a variable-structure PID algorithm. Residual neural network (ResNet) models are utilized to predict key soldering process parameters. This method has been executed and validated on a practical testing platform. Compared to other laser soldering control techniques, the proposed method demonstrates a low overshoot, rapid dynamic response, and swift adjustment capabilities, effectively enhancing the soldering quality and production efficiency.

Multi-sensor signals multi-scale fusion method for fault detection of high-speed and high-power diesel engine under variable operating conditions

Detecting faults in high-speed and high-power diesel engines under complex variable operating conditions is highly challenging. Online vibration monitoring systems have been used in such diesel engines in key fields, in which vibration sensors are installed on each cylinder to enable comprehensive monitoring. In this paper, a fault detection method for diesel engines under variable operating conditions is proposed based on multi-sensor signal multi-scale fusion. Firstly, a preprocessing framework is established for the raw vibration signals collected from each cylinder to eliminate random interference and system noise. Then, the resulting signals are phase-aligned based on the engine firing sequence and analyzed using a signal correlation algorithm to produce a multi-sensor multi-scale similarity matrix (MSMSSM). Finally, a multi-branch residual convolutional neural network (MBRCNN) model is constructed with the MSMSSM as the input to detect abnormal health states of the diesel engine. Fault simulation experiments are conducted on a 12-cylinder V-type high-speed and high-power diesel engine test rig. The comparative test results indicate that the proposed MSMSSM-MBRCNN method shows both the highest accuracy of 95.28% and the lowest standard deviation of 3.57% compared to other typical methods. The multi-sensor signals multi-scale fusion method proposed in this paper fully utilizes the key information that remains basically consistent in the synchronous acquisition signals of multiple sensors under different operating conditions. This can effectively reduce the interference of operating condition changes and improve the accuracy and robustness of fault detection.

A nonlinear solver based on residual network for seepage equation

In reservoir simulation, the linearization process after the discretization of the seepage equation will additionally introduce new errors, which can be avoided by using a nonlinear solver based on a deep neural network. In this paper, we propose a new solution method without needing linearization of the discretized seepage equation, which combines residual network and correction iteration method to solve nonlinear system of equations instead of using Newton method. The residual network is trained based on label data generated by seepage equation and approximates the solution, and the error of approximated solution is used as the new input of the trained residual network model for the next correction iteration. The single-phase seepage equation is solved to confirm the effectiveness of the proposed method. Numerical results indicate that, error caused by linearization significantly affects the solution accuracy, which reduces solution accuracy and time step. Under the same time step, the proposed solution method has higher accuracy, and under the premise of ensuring the solution accuracy, the time step of proposed method increases by more than three times.

Exclusive-Region-Map-Based Medium Access Control in Mobile Networks With Directional Antennas Through Deep Interference Learning

The medium access control (MAC) design in mobile networks with directional antennas is challenging due to the difficulty of defining the exact RF interference range between two neighboring directional links and the frequent changes of interference range due to node mobility. This research targets directional data reception (Rx) and transmission (Tx) coordination issues based on the computation of directional interference ranges from nearby directional links. An innovative MAC mechanism is designed with three features: (1) ER-map, i.e., the spatial expression of exclusive region (ER) model in the format of a heatmap. The ER-map helps to determine the directional interference range in typical communication scenarios. Different ER-map cases are analyzed based on the spatial layout differences for two nearby directional links. (2) Spatio-temporal ER-map evolution prediction: a Spatio-Temporal Residual Network (ST-ResNet+) model is used to describe the spatial correlations (for the ERs among neighboring links) and temporal correlations (for the ERs across different time instants) as well as the ER map evolution patterns. Such a Deep ST-ResNet+ model is used to predict the next-time ER map’s snapshot. (3) Optimized directional MAC protocol based on ER map predictions: The ST-ResNet+ prediction results are used to determine the MAC operations, such as Tx/Rx schedule arrangement in the one-hop area, sending rate adjustments, etc. Comprehensive simulations are conducted to validate the throughput efficiency for the proposed directional MAC scheme.

A Bearing Fault Diagnosis Method Based on a Residual Network and a Gated Recurrent Unit under Time-Varying Working Conditions.

The diagnosis of bearing faults is an important guarantee for the healthy operation of mechanical equipment. Due to the time-varying working conditions of mechanical equipment, it is necessary to achieve bearing fault diagnosis under time-varying working conditions. However, the superposition of the two-dimensional working conditions of speed and acceleration brings great difficulties to diagnosis via data-driven models. The long short-term memory (LSTM) model based on the infinitesimal method is an effective method to solve this problem, but its performance still has certain limitations. On this basis, this article proposes a model for fault diagnosis under time-varying operating conditions that combines a residual network model (ResNet) and a gate recurrent unit (model) (GRU). Firstly, the samples were segmented, and feature extraction was performed using ResNet. We then used GRU to process the information. Finally, the classification results were output through the output network. This model could ignore the influence of acceleration and achieve high fault diagnosis accuracy under time-varying working conditions. In addition, we used t-SNE to reduce the dimensionality of the features and analyzed the role of each layer in the model. Experiments showed that this method had a better performance compared with existing bearing fault diagnosis methods.

Sea Surface Chlorophyll-a Concentration Retrieval from HY-1C Satellite Data Based on Residual Network

A residual network (ResNet) model was proposed for estimating Chl-a concentrations in global oceans from the remote sensing reflectance (Rrs) observed by the Chinese ocean color and temperature scanner (COCTS) onboard the HY-1C satellite. A total of 52 images from September 2018 to September 2019 were collected, and the label data were from the multi-task Ocean Color-Climate Change Initiative (OC-CCI) daily products. The results of feature selection and sensitivity experiments show that the logarithmic values of Rrs565 and Rrs520/Rrs443, Rrs565/Rrs490, Rrs520/Rrs490, Rrs490/Rrs443, and Rrs670/Rrs565 are the optimal input parameters for the model. Compared with the classical empirical OC4 algorithm and other machine learning models, including the artificial neural network (ANN), deep neural network (DNN), and random forest (RF), the ResNet retrievals are in better agreement with the OC-CCI Chl-a products. The root-mean-square error (RMSE), unbiased percentage difference (UPD), and correlation coefficient (logarithmic, R(log)) are 0.13 mg/m3, 17.31%, and 0.97, respectively. The performance of the ResNet model was also evaluated against in situ measurements from the Aerosol Robotic Network-Ocean Color (AERONET-OC) and field survey observations in the East and South China Seas. Compared with DNN, ANN, RF, and OC4 models, the UPD is reduced by 5.9%, 0.7%, 6.8%, and 6.3%, respectively.

Deep Learning-Based Prediction of Contact Maps and Crystal Structures of Inorganic Materials.

Crystal structure prediction is one of the major unsolved problems in materials science. Traditionally, this problem is formulated as a global optimization problem for which global search algorithms are combined with first-principles free energy calculations to predict the ground-state crystal structure of a given material composition. These ab initio algorithms are currently too slow for predicting complex material structures. Inspired by the AlphaFold algorithm for protein structure prediction, herein, we propose AlphaCrystal, a crystal structure prediction algorithm that combines a deep residual neural network model for predicting the atomic contact map of a target material followed by three-dimensional (3D) structure reconstruction using genetic algorithms. Extensive experiments on 20 benchmark structures showed that our AlphaCrystal algorithm can predict structures close to the ground truth structures, which can significantly speed up the crystal structure prediction and handle relatively large systems.

FC-ResNet: A Multilingual Handwritten Signature Verification Model Using an Improved ResNet with CBAM

Offline signature verification is a widely used biometric method in finance, law, and administrative procedures. However, existing deep convolutional neural network models perform poorly on signature datasets that span different regions and ethnic people, while also suffering from problems such as large parameter counts and slow inference speeds. To address these issues, we propose an improved residual network model (FC-ResNet). This model introduces a convolutional block attention module into the classical residual network to adapt to the diversity and variability of signatures, while also compressing the model for lightweight deployment. Due to the lack of public, offline handwritten signature datasets for ethnic people, we collected a large-scale offline handwritten signature dataset, including genuine signatures and forged signatures in Chinese, Uyghur, Kazakh, and Kirgiz, totaling 38,400 images. Our FC-ResNet model achieved an accuracy of over 96% for each language in our self-built dataset, as well as accuracy rates of 96.21%, 98.42%, and 97.28% on the public datasets CEDAR, BHSig-B, and BHSig-H, respectively. Based on the above experimental results, our proposed model demonstrates great potential for both public and self-built signature datasets, while also exhibiting significant advantages in lightweight model deployment. We believe that this work can provide a feasible solution for ethnic people signature verification.

Machine learning method for 12C event classification and reconstruction in the active target time-projection chamber

Active target time projection chambers are important tools in low energy radioactive ion beams or gamma rays related researches. In this work, we present the application of machine learning methods to the analysis of data obtained from an active target time projection chamber. Specifically, we investigate the effectiveness of Visual Geometry Group (VGG) and the Residual neural Network (ResNet) models for event classification and reconstruction in decays from the excited 22+ state in 12C Hoyle rotation band. The results show that machine learning methods are effective in identifying 12C events from the background noise, with ResNet-34 achieving an impressive precision of 0.99 on simulation data, and the best performing event reconstruction model ResNet-18 providing an energy resolution of σE<77 keV and an angular reconstruction deviation of σθ<0.1 rad. The promising results suggest that the ResNet model trained on Monte Carlo samples could be used for future classifying and predicting experimental data in active target time projection chambers related experiments.