Backpropagation Phase Research Articles

Strain softening observed during nanoindentation of equimolar-ratio Co–Mn– Fe–Cr–Ni high entropy alloy

This research article presents an atomistic study on the cyclic nanoindentation of an equimolar-ratio Co–Mn–Fe–Cr–Ni high-entropy alloy (HEA) using molecular dynamics simulation. The study investigated the effects of indentation depth on the cyclic load versus the indentation depth of the HEA. The results showed that the cyclic response exhibits a pronounced shift towards plasticity with pile-up formation instead of sinking behavior at higher indentation depths. Within the realm of molecular dynamics simulations, the simulated hardness value reached up to 16 GPa for the initial indentation cycle. A steep drop in the load–displacement curve was observed during the elastic–plastic transition, signifying substantial strain softening of the substrate. It was found that the densely clustered stacking faults undergo a reverse transition during cyclic loading, contributing to the backpropagation phase responsible for elastic recovery despite subsequent strain hardening. The study provides important insights into the underlying mechanisms governing the cyclic mechanical behavior of HEAs to guide their improved micromanufacturing.

ECG Heartbeat Classification Using CONVXGB Model

ELECTROCARDIOGRAM (ECG) signals are reliable in identifying and monitoring patients with various cardiac diseases and severe cardiovascular syndromes, including arrhythmia and myocardial infarction (MI). Thus, cardiologists use ECG signals in diagnosing cardiac diseases. Machine learning (ML) has also proven its usefulness in the medical field and in signal classification. However, current ML approaches rely on hand-crafted feature extraction methods or very complicated deep learning networks. This paper presents a novel method for feature extraction from ECG signals and ECG classification using a convolutional neural network (CNN) with eXtreme Gradient Boosting (XBoost), ConvXGB. This model was established by stacking two convolutional layers for automatic feature extraction from ECG signals, followed by XGBoost as the last layer, which is used for classification. This technique simplified ECG classification in comparison to other methods by minimizing the number of required parameters and eliminating the need for weight readjustment throughout the backpropagation phase. Furthermore, experiments on two famous ECG datasets–the Massachusetts Institute of Technology–Beth Israel Hospital (MIT-BIH) and Physikalisch-Technische Bundesanstalt (PTB) datasets–demonstrated that this technique handled the ECG signal classification issue better than either CNN or XGBoost alone. In addition, a comparison showed that this model outperformed state-of-the-art models, with scores of 0.9938, 0.9839, 0.9836, 0.9837, and 0.9911 for accuracy, precision, recall, F1-score, and specificity, respectively.

Super-resolution reconstruction of biometric features recognition based on manifold learning and deep residual network

Background and objectiveIn daily life, face information has the characteristics of uniqueness and universality. However, in a real-world scene, the image information of the face acquired by the acquisition device often contains noises such as blurring and sharpening. As such, super-resolution reconstruction of face features recognition based on manifold learning is proposed in this paper. MethodsWe reconstruct low-resolution facial expression images, introduce a simplified residual block network and manifold learning, and propose joint supervision through a new hybrid loss function, which not only retains the color and characteristics of the image, but also retains the high-frequency information. The ResNet50 network uses the weight feature of information entropy to optimize the information of the pooling layer, and the esNet50 network uses the improved PSO algorithm to optimize the initial weight of the error back-propagation phase. ResultsIn the case of inputting extremely low resolution (6 × 6) facial expression images, the accuracy rate is increased by 9.091%. The accuracy of the high-resolution facial expressions after reconstruction with a size of 12×12 is 96.970%. The accuracy rate for happy expressions is 100%, the accuracy rate for anger, disgust, sadness, and surprise recognition is 97%, the accuracy rate for contempt is 94%, and the accuracy rate for fear is 88%. ConclusionsThe experimental results verify the feasibility and superiority of the system, and effectively improve the accuracy of low-resolution facial expressions.

An Inverse-Filter-Based Method to Locate Partial Discharge Sources in Power Transformers

Partial discharge (PD) occurrence in power transformers can lead to irreparable damage to the power network. In this paper, the inverse filter (IF) method to localize PDs in power transformers is proposed. To the best of the authors’ knowledge, this is the first time that the inverse filter method has been used to localize PD sources in the electromagnetic regime. The method comprises two phases: the forward phase and the backward or backpropagation phase. In the forward phase, the waveform emitted from the PD source is recorded with one or several sensors. In the backward phase, the recorded signal is transformed into the frequency domain, inverted, transformed back into the time domain, and then back injected into the medium. Finally, a suitable criterion is used to localize the PD source. The efficiency of the proposed IF method is assessed considering different scenarios. It is shown that, for the considered configurations, the proposed IF method outperforms the classical time-reversal technique.

Classification of ERP signal from amnestic mild cognitive impairment with type 2 diabetes mellitus using single-scale multi-input convolution neural network

BackgroundThe application of deep learning models to electroencephalogram (EEG) signal classification has recently become a popular research topic. Several deep learning models have been proposed to classify EEG signals in patients with various neurological diseases. However, no effective deep learning model for event-related potential (ERP) signal classification is yet available for amnestic mild cognitive impairment (aMCI) with type 2 diabetes mellitus (T2DM). MethodThis study proposed a single-scale multi-input convolutional neural network (SSMICNN) method to classify ERP signals between aMCI patients with T2DM and the control group. Firstly, the 18-electrode ERP signal on alpha, beta, and theta frequency bands was extracted by using the fast Fourier transform, and then the mean, sum of squares, and absolute value feature of each frequency band were calculated. Finally, these three features are converted into multispectral images respectively and used as the input of the SSMICNN network to realize the classification task. ResultsThe results show that the SSMICNN can fuse MSI formed by different features, SSMICNN enriches the feature quantity of the neural network input layer and has excellent robustness, and the errors of SSMICNN can be simultaneously transmitted to the three convolution channels in the back-propagation phase. Comparison with Existing Method(s): SSMICNN could more effectively identify ERP signals from aMCI with T2DM from the control group compared to existing classification methods, including convolution neural network, support vector machine, and logistic regression. ConclusionsThe combination of SSMICNN and MSI can be used as an effective biological marker to distinguish aMCI patients with T2DM from the control group.

A Correlation-Based Electromagnetic Time Reversal Technique to Locate Indoor Transient Radiation Sources

To overcome the multipath interference in locating transient electromagnetic (EM) radiation sources in an indoor environment, we propose a criterion that calculates the correlation between back-propagated signals from observation points, to be used in EM time reversal (EMTR) algorithms. The method introduced in this article has three main advantages with respect to classical methods that use full-wave simulations and other criteria, such as maximum field strength. First, compared with full-wave techniques in the back-propagation phase, the proposed correlation-based method utilizes approximated transfer functions from the ray-tracing technique, which can improve the computation efficiency. Second, an inverted-loss model is used for the back-propagation, which could reduce the localization error caused by multipath effects due to signal attenuation and time delay. Third, the proposed correlation criterion has weak correlation with the source characteristics, which makes it applicable to the localization in indoor reflective environments with only two observation points. Several numerical simulations are carried out to assess the performance of the proposed method. The results indicate that the proposed correlation-based EMTR technique is able to locate radiation sources accurately and efficiently in indoor reflective environments.

Memory Access Optimization for On-Chip Transfer Learning

Training of Deep Neural Network (DNN) at the edge faces the challenge of high energy consumption due to the requirements of a large number of memory accesses for gradient calculations. Therefore, it is necessary to minimize data fetches to perform training of a DNN model on the edge. In this paper, a novel technique has been proposed to reduce the memory access for the training of fully connected layers in transfer learning. By analyzing the memory access patterns in the backpropagation phase in fully connected layers, the memory access can be optimized. We introduce a new method to update the weights by introducing the delta term for every node of output and fully connected layer. Delta term aims to reduce memory access for the parameters which are required to access repeatedly during the training process of fully connected layers. The proposed technique shows 0.13x-13.93x energy savings for the training of fully connected layers for famous DNN architectures on multiple processor architectures. The proposed technique can be used to perform transfer learning on-chip to reduce energy consumption as well as memory access.

Endek Classification Based On GLCM Using Artificial Neural Networks with Adam Optimization

Not many people know that endek cloth itself has 4 known variances. .Nowadays. Computing and classification algorithm can be implemented to solve classification problem with respect to the features data as input. We can use this computing power to digitalize these endek pattern. The features extraction algorithm used in this research is GLCM. Where these data will act as input for the neural network model later. There is a lot of optimizer algorithm to use in back propagation phase. In this research we prefer to use adam which is one of the newest and most popular optimizer algorithm. To compare its performace we also use SGD which is older and popular optimizer algorithm. Later we find that adam algorithm generate 33% accuracy which is better than what SGD algorithm give, it is 23% accuracy. Longer epoch also give affect for overall model accuracy.

Pulsed laser-scanning laser Doppler vibrometer (PL-SLDV) phased arrays for damage detection in aluminum plates

Lamb wave phased arrays employ sensors placed close to each other in compact distributions and can quickly inspect large plate-like structures through time (or phase) delays in the way analogous to radar. Traditional Lamb wave phased arrays usually adopt large-size ultrasonic transducers bonded on (or placed close to the surface of) a structure, and hence they limit the array configurability and cannot perform inspection from a far distance to the structure. This paper presents Lamb wave phased arrays implemented with a fully noncontact pulsed laser – scanning laser Doppler vibrometer (PL-SLDV) system, which employs a PL for exciting Lamb waves at a single PL spot through the thermoelastic effect and an SLDV for acquiring signals of Lamb waves at multiple scanning points based on the Doppler effect, respectively. The fully noncontact PL-SLDV phased arrays enable inspection from a far distance, as well as easily constructible receiver phased arrays in various configurations due to the use of high-resolution scanning laser. To generate inspection images with the acquired signals of Lamb waves, an improved delay-and-sum (DAS) imaging algorithm for receiver phased arrays is developed, where the exact Lamb wave frequency-wavenumber dispersion relation is considered for both the phase delay and back-propagation phase. This improved DAS imaging algorithm addresses the dispersion effect and results in higher radial imaging resolution, compared to the conventional DAS imaging algorithm. In addition, adaptive weighting factors are implemented to improve the angular imaging resolution. Proof-of-concept experiments demonstrate that multiple simulated defects of different sizes as well as broadside and offside simulated cracks can be successfully detected using a PL-SLDV phased array with a high-dispersion A0 mode. The experimental study also shows that the improved DAS imaging algorithm can achieve high radial and angular imaging resolution.

Parallelizing and optimizing neural Encoder–Decoder models without padding on multi-core architecture

Scaling up Artificial Intelligence (AI) algorithms for massive datasets to improve their performance is becoming crucial. In Machine Translation (MT), one of most important research fields of AI, models based on Recurrent Neural Networks (RNN) show state-of-the-art performance in recent years, and many researchers keep working on improving RNN-based models to achieve better accuracy in translation tasks. Most implementations of Neural Machine Translation (NMT) models employ a padding strategy when processing a mini-batch to make all sentences in a mini-batch have the same length. This enables an efficient utilization of caches and GPU/SIMD parallelism but leads to a waste of computation time. In this paper, we implement and parallelize batch learning for a Sequence-to-Sequence (Seq2Seq) model, which is the most basic model of NMT, without using a padding strategy. More specifically, our approach forms vectors which represent the input words as well as the neural network’s states at different time steps into matrices when it processes one sentence, and as a result, the approach makes a better use of cache and optimizes the process that adjusts weights and biases during the back-propagation phase. Our experimental evaluation shows that our implementation achieves better scalability on multi-core CPUs. We also discuss our approach’s potential to be used in other implementations of RNN-based models.

Modelling of a post-combustion CO2 capture process using deep belief network

This paper presents a study on using deep learning for the modelling of a post-combustion CO2 capture process. Deep learning has emerged as a very powerful tool in machine learning. Deep learning technique includes two phases: an unsupervised pre-training phase and a supervised back-propagation phase. In the unsupervised pre-training phase, a deep belief network (DBN) is pre-trained to obtain initial weights of the subsequent supervised phase. In the supervised back-propagation phase, the network weights are fine-tuned in a supervised manner. DBN with many layers of Restricted Boltzmann Machine (RBM) can extract a deep hierarchical representation of training data. In terms of the CO2 capture process, the DBN model predicts CO2 production rate and CO2 capture level using the following variables as model inputs: inlet flue gas flow rate, CO2 concentration in inlet flue gas, pressure of flue gas, temperature of flue gas, lean solvent flow rate, MEA concentration and temperature of lean solvent. A greedy layer-wise unsupervised learning algorithm is introduced to optimize DBN, which can bring better generalization than a single hidden layer neural network. The developed deep architecture network models can then be used in the optimisation of the CO2 capture process.

Assessment of the Influence of Losses on the Performance of the Electromagnetic Time Reversal Fault Location Method

Electromagnetic time reversal (EMTR) has been shown to be an efficient method for locating faults in ac and dc power grids. In the available literature, the back-propagation medium has been considered to have identical losses as the direct-time medium. However, the telegrapher's equations describing the traveling wave propagation are time-reversal invariant if and only if inverted losses are considered in the back-propagation phase. This paper presents an analysis of the impact of losses on the performance of the EMTR-based fault location method for power networks. In this respect, three back-propagation models are proposed, analyzed, and compared. It is shown that a lossy back-propagation model, for which the wave equations are not rigorously time-reversal invariant, results in accurate fault locations. Finally, an EMTR fault location system based on the lossy back-propagation model and a fast electromagnetic transient simulation platform is developed and its performances validated.

MCTS-Minimax Hybrids

Monte Carlo tree search (MCTS) is a sampling-based search algorithm that is state of the art in a variety of games. In many domains, its Monte Carlo rollouts of entire games give it a strategic advantage over traditional depth-limited minimax search with αβ pruning. These rollouts can often detect long-term consequences of moves, freeing the programmer from having to capture these consequences in a heuristic evaluation function. But due to its highly selective tree, MCTS runs a higher risk than full-width minimax search of missing individual moves and falling into traps in tactical situations. This paper proposes MCTS-minimax hybrids that integrate shallow minimax searches into the MCTS framework. Three approaches are outlined, using minimax in the selection/expansion phase, the rollout phase, and the backpropagation phase of MCTS. Without assuming domain knowledge in the form of evaluation functions, these hybrid algorithms are a first step towards combining the strategic strength of MCTS and the tactical strength of minimax. We investigate their effectiveness in the test domains of Connect-4, Breakthrough, Othello, and Catch the Lion, and relate this performance to the tacticality of the domains.

Signal Processing Algorithms for Down-Stream Traffic in Next Generation 10 Gbit/s Fixed-Grid Passive Optical Networks

We have analyzed the impact of digital and optical signal processing algorithms, that is, Volterra equalization (VE), digital backpropagation (BP), and optical phase conjugation with nonlinearity module (OPC-NM), in next generation 10 Gbit/s (also referred to as XG) DP-QPSK long haul WDM (fixed-grid) passive optical network (PON) without midspan repeaters over 120 km standard single mode fiber (SMF) link for downstream signals. Due to the compensation of optical Kerr effects, the sensitivity penalty is improved by 2 dB by implementing BP algorithm, 1.5 dB by VE algorithm, and 2.69 dB by OPC-NM. Moreover, with the implementation of NL equalization technique, we are able to get the transmission distance of 126.6 km SMF for the 1 : 1024 split ratio at 5 GHz channel spacing in the nonlinear region.

Betweenness computation in the single graph representation of hypergraphs

Many real-world social networks are hypergraphs because they either explicitly support membership in groups or implicitly include communities. We present the HyperBC algorithm that exactly computes betweenness centrality (or BC) in hypergraphs. The forward phase of HyperBC and the backpropagation phase are specifically tailored for BC computation on hypergraphs. In addition, we present an efficient method for pruning networks through the notion of “non-bridging” vertices. We experimentally evaluate our algorithm on a variety of real and artificial networks and show that it significantly speeds up the computation of BC on both real and artificial hypergraphs, while at the same time, being very memory efficient.

An improved three-term optical backpropagation algorithm

An improved Optical Backpropagation (OBP) algorithm for training single hidden layer feedforward neural network with third term is proposed. The major limitations of backpropagation algorithm are the local minima problem and the slow rate of convergence. To solve these problems, we have proposed an algorithm by introducing a third term with optical backpropagation (OBPWT). This method has been applied to the multilayer neural network to improve the efficiency in terms of convergence speed. In the proposed algorithm, a non-linear function on the error term is introduced before applying the backpropagation phase. This error term is used along with a third term in the weight updation rule. We have shown how the new proposed algorithm drastically accelerates the training convergence at the same time maintaining the neural network’s performance. The effectiveness of the proposed algorithm has been shown by testing five benchmark problems. The simulation results show that the proposed algorithm is capable of speeding up the learning and hence the rate of convergence.

Energy current imaging method for time reversal in elastic media

An energy current imaging method is presented for use in locating sources of wave energy during the back propagation stage of the time reversal process. During the back propagation phase of an ideal time reversal experiment, wave energy coalesces from all angles of incidence to recreate the source event; after the recreation, wave energy diverges in every direction. An energy current imaging method based on this convergence/divergence behavior has been developed. The energy current imaging method yields a smaller spatial distribution for source reconstruction than is possible with traditional energy imaging methods.

Energy flux imaging method for time reversal in elastic media.

An energy flux imaging method for use in locating sources of wave energy during the back propagation stage of the time reversal process will be presented. During the back propagation phase of an ideal time reversal experiment, wave energy coalesces from all angles of incidence to recreate the source event; after the recreation, wave energy diverges in every direction. An energy flux imaging method based on this convergence/divergence behavior has been developed. The energy flux imaging method yields better spatial resolution in source reconstruction than is possible with traditional imaging methods. [This work is supported by Institutional Support (LDRD) at Los Alamos National Laboratory.]

An Algorithm of Supervised Learning for Multilayer Neural Networks

A method of supervised learning for multilayer artificial neural networks to escape local minima is proposed. The learning model has two phases: a backpropagation phase and a gradient ascent phase. The backpropagation phase performs steepest descent on a surface in weight space whose height at any point in weight space is equal to an error measure, and it finds a set of weights minimizing this error measure. When the backpropagation gets stuck in local minima, the gradient ascent phase attempts to fill up the valley by modifying gain parameters in a gradient ascent direction of the error measure. The two phases are repeated until the network gets out of local minima. The algorithm has been tested on benchmark problems, such as exclusive-or (XOR), parity, alphabetic characters learning, Arabic numerals with a noise recognition problem, and a realistic real-world problem: classification of radar returns from the ionosphere. For all of these problems, the systems are shown to be capable of escaping from the backpropagation local minima and converge faster when using the new proposed method than using the simulated annealing techniques.

A neural network learning algorithm tailored for VLSI implementation

This paper describes concepts that optimize an on-chip learning algorithm for implementation of VLSI neural networks with conventional technologies. The network considered comprises an analog feedforward network with digital weights and update circuitry, although many of the concepts are also valid for analog weights. A general, semi-parallel form of perturbation learning is used to accelerate hidden-layer update while the infinity-norm error measure greatly simplifies error detection. Dynamic gain adaption, coupled with an annealed learning rate, produces consistent convergence and maximizes the effective resolution of the bounded weights. The use of logarithmic analog-to-digital conversion, during the backpropagation phase, obviates the need for digital multipliers in the update circuitry without compromising learning quality. These concepts have been validated through network simulations of continuous mapping problems.