Delay Neural Network Research Articles

The use of MD-CUMSUM and NARX neural network for anticipating the remaining useful life of bearings

The accurate determination of remaining useful life (RUL) of bearings is of immense importance in the condition-based maintenance of any rotating machinery. In this paper, a data driven prognostic approach based on nonlinear autoregressive neural network with eXogenous Inputs (NARX-NN) in combination with wavelet-filter technique is applied to the RUL estimation of bearings. Firstly, the vibration signals generated in an experimental test rig are processed with the proposed wavelet-filter to augment the impulsive characteristics of bearing signals and improve the quality of fault feature extraction. Secondly, a variety of time-domain features are extracted from the processed bearing signals. However, these features exhibit a highly non-monotonic behavior as the bearing condition degrades. To overcome this drawback, a new health indicator (HI) based on Mahalanobis distance (MD) criterion and cumulative sum (CUMSUM) chart is proposed in this paper. Thirdly, the NARX-NN is first designed as a time delay neural network (TDNN). Then, the derived HI and the age of the bearing are used as inputs with life percentage of the bearing as output in order to train the TDNN model, which unlike the usual artificial neural networks (ANNs) performs a one-step ahead prediction of the bearing RUL. The results suggest that the proposed method can effectively predict the RUL of bearings with an acceptable degree of accuracy, and outperforms the use of self-organizing map-based indicator and the traditional FFNNs for RUL inference.

A Comparative Analysis of SVM and IDNN for Identifying Penguin Activities

ABSTRACTWhere, when and how much animals eat provide valuable insights into their ecology. In this paper, we present a comparative analysis between Support Vector Machine (SVM) and Input Delay Neural Network (IDNN) models to identify prey capture events from penguin accelerometry data. A pre-classified dataset of 3D time-series data from back-mounted accelerometers was used. We trained both the models to classify the penguins’ behavior at intervals as either ‘prey handling’ or ‘swimming’. The aim was to determine whether IDNN could achieve the same level of classification accuracy as SVM, but with reduced memory demands. This would enable the IDNN model to be embedded on the accelerometer micro-system itself, and hence reduce the magnitude of the output data to be uploaded. Based on the classification results, this paper provides an analysis of the two models from both an accuracy and applicability point of view. The experimental results show that both models achieve an equivalent accuracy of approx. 85% using the featured data, with a memory demand of 0.5 kB for IDNN and 0.7 Mb for SVM. The raw accelerometer data let us improve the generalizability of the models with a slightly lower accuracy to around 80%. This indicates that the IDNN model can embed on the accelerometer itself, reducing problems associated with raw time-series data retrieval and loss.

Novel Adaptive Control and State-Feedback Control Strategies to Finite-Time Stabilization of Discontinuous Delayed Networks

This paper is concerned with the finite-time stabilization problem for a class of delayed neural networks (DNNs) with discontinuous neuron activations. In order to stabilize the states of such DNNs in finite time, we design two classes of novel control strategies including switching adaptive control strategy and switching state-feedback control strategy. Based on the theory of differential inclusions, the famous finite-time stability theorem and the generalized Lyapunov functional method, several sufficient conditions are established to ensure the finite-time stabilization control of discontinuous DNNs. In addition, we also provide the settling time for stabilization which has an important significance in the application of neural networks for guaranteeing fast response. Finally, numerical examples are given to demonstrate the effectiveness of the proposed control strategies and main results.

Finite-Time Lag Synchronization for Memristive Mixed Delays Neural Networks with Parameter Mismatch

This paper is devoted to study the finite time lag synchronization problem of memristive mixed delays neural networks with switching jumps and parameter mismatch. The main objective of this paper is to design simple linear feedback control such as the drive response systems can realize synchronization in the finite time. Several theoretical aspects are addressed, mean Filippov differential inclusion and set-valued map. Based on the Gronwall’s inequality and properties of differential operation, a new finite time lag synchronization scheme is proposed to guarantee that coupled memristive mixed delays neural networks are in a state of lag synchronization in finite time. In addition, finite lag synchronization criteria of two identical memristive mixed delays neural networks are also considered. Finally, numerical examples are presented to illustrate the effectiveness of the our proposed theorems.

Auditory feature representation using convolutional restricted Boltzmann machine and Teager energy operator for speech recognition.

In this letter, authors propose an auditory feature representation technique with the filterbank learned using an annealing dropout convolutional restricted Boltzmann machine (ConvRBM) and noise-robust energy estimation using the Teager energy operator (TEO). TEO is applied on each subband of ConvRBM filterbank and pooled later to get the short-term spectral features. Experiments on AURORA 4 database show that the proposed features perform better than the Mel filterbank features. The relative improvement of 2.59%-11.63% and 1.26%-6.87% in word error rate is achieved using the time delay neural network and the bidirectional long short-term memory models, respectively.

A Wiener-Type Dynamic Neural Network Approach to the Modeling of Nonlinear Microwave Devices

A new Wiener-type dynamic neural network (DNN) approach for nonlinear device modeling is proposed in this paper. The proposed analytical formulation of Wiener-type DNN structure consists of a cascade of a simplified linear dynamic part and a nonlinear static part. The simplified linear dynamic equations are obtained by vector fitting enhancing the efficiency of the model. A new formulation and new sensitivity analysis technique is derived to train the proposed Wiener-type DNN with dc, small- and large-signal data. A new gradient-based training algorithm is also formulated to speed up the training of proposed Wiener-type DNN model. The proposed Wiener-type DNN model can be trained to be accurate relative to device data. Furthermore, the proposed Wiener-type DNN provides enhanced convergence properties over existing neural network approaches such as time delay neural network (TDNN) and TDNN with extrapolation. Application examples on modeling GaAs metal-semiconductor-field-effect transistor, GaAs high-electron mobility transistor (HEMT), and a real <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2 \times 50$ </tex-math></inline-formula> gatewidth GaAs pseudomorphic HEMT are presented. The use of Wiener-type DNN model in harmonic balance simulations demonstrates that the Wiener-type DNN is a robust method for developing models for different types of microwave devices. These models could be implemented into circuit simulators conveniently.

Predictive control of intersegmental tarsal movements in an insect.

In many animals intersegmental reflexes are important for postural and movement control but are still poorly undesrtood. Mathematical methods can be used to model the responses to stimulation, and thus go beyond a simple description of responses to specific inputs. Here we analyse an intersegmental reflex of the foot (tarsus) of the locust hind leg, which raises the tarsus when the tibia is flexed and depresses it when the tibia is extended. A novel method is described to measure and quantify the intersegmental responses of the tarsus to a stimulus to the femoro-tibial chordotonal organ. An Artificial Neural Network, the Time Delay Neural Network, was applied to understand the properties and dynamics of the reflex responses. The aim of this study was twofold: first to develop an accurate method to record and analyse the movement of an appendage and second, to apply methods to model the responses using Artificial Neural Networks. The results show that Artificial Neural Networks provide accurate predictions of tarsal movement when trained with an average reflex response to Gaussian White Noise stimulation compared to linear models. Furthermore, the Artificial Neural Network model can predict the individual responses of each animal and responses to others inputs such as a sinusoid. A detailed understanding of such a reflex response could be included in the design of orthoses or functional electrical stimulation treatments to improve walking in patients with neurological disorders as well as the bio/inspired design of robots.

Repeatable Folding Task by Humanoid Robot Worker Using Deep Learning

We propose a practical state-of-the-art method to develop a machine-learning-based humanoid robot that can work as a production line worker. The proposed approach provides an intuitive way to collect data and exhibits the following characteristics: task performing capability, task reiteration ability, generalizability, and easy applicability. The proposed approach utilizes a real-time user interface with a monitor and provides a first-person perspective using a head-mounted display. Through this interface, teleoperation is used for collecting task operating data, especially for tasks that are difficult to be applied with a conventional method. A two-phase deep learning model is also utilized in the proposed approach. A deep convolutional autoencoder extracts images features and reconstructs images, and a fully connected deep time delay neural network learns the dynamics of a robot task process from the extracted image features and motion angle signals. The “Nextage Open” humanoid robot is used as an experimental platform to evaluate the proposed model. The object folding task utilizing with 35 trained and 5 untrained sensory motor sequences for test. Testing the trained model with online generation demonstrates a 77.8% success rate for the object folding task.

Finite-time synchronization of master-slave neural networks with time-delays and discontinuous activations

This paper deals with the finite-time synchronization issue of time-varying delayed neural networks (DNNs) with discontinuous activations. Based on master-slave concept, several sufficient conditions are given to guarantee the finite-time synchronization of discontinuous DNNs. In order to control the synchronization error to converge zero in a finite time, we design three classes of novel switching state-feedback controllers which involve time-delays and discontinuous factors. The analysis in this paper employs the extended differential inclusion theory, the famous finite-time stability theorem, inequality techniques and generalized Lyapunov approach. Moreover, the upper bounds of the settling time of synchronization are estimated. Finally, the validity of proposed design method and theoretical results are illustrated by numerical examples.

Stability and synchronization analysis of neural networks via Halanay-type inequality

In this paper, not only some novel sufficient criteria for stability of discrete delay neural networks but also a new discriminant method for self synchronization of Hopfield neural networks are considered. Some novel stability criteria for discrete delay neural networks are obtained if the time average of the difference for two coefficients on some fixed lengths is lower bounded by some positive numbers. Moreover, a new discrete Halanay inequality is given, which extends the existing results. Based on the Halanay inequality, a novel sufficient criterion on self synchronization of Hopfield neural networks with time delay is obtained. Two numerical examples are given to demonstrate the effectiveness of proposed methods.

Split FTDNN-DFE Equalizer for Complex Non-linear Wireless Channels with NARMA Approximation

AbstractTime-dependent channel variations in wireless systems have been modeled using autoregressive (AR), moving average, and AR moving average (ARMA) approaches, which in certain conditions fail to provide the most suitable description of the scenarios. It lowers the quality of service, increases the dependence on the traditional tapped delay line model, and requires more reference symbols as fading in the channel increases. Non-linear attributes in channels are best described by non-linear AR (NAR) and non-linear ARMA (NARMA) models, which capture the channel state information (CSI) better. Learning-based systems like artificial neural networks (ANN) are better equipped to deal with NAR and NARMA approximations of channels, recover CSI, and provide proper channel estimation due to their natural association with non-linear computing. Though decision feedback equalizers (DFE) are the traditional and preferred approaches to reduce the ill effects of channel-induced variations, these are unable to mitigate noise-triggered responses that ANNs can. Here, DFEs and special forms of ANNs are combined to recover CSI from NAR and NARMA approximations of time-varying channels. We especially configure a class of fully focused time delay neural network designed in split form in unitary and composite modes with a DFE to track real and imaginary components of signals. Experimental results establish the effectiveness of such hybrid tools in tracking time-varying non-linear channels covering International Telecommunication Union pedestrian to vehicular conditions.

Object Detection Method in Traffic by On-Board Computer Vision with Time Delay Neural Network

A collision avoidance system play an important role in reducing incidents occurring on the road, which the object detection is crucial to enable obstacle avoidance for this system. The objective of this paper is to improve general object detection methods for vehicles in order to prevent a collision of the vehicles and the obstacles - of which we do not know the exact shape, size or color. A combined computer vision system with artificial neural networks can improve the performance of the vehicle has the ability to see and recognize the obstacles like human beings. In this paper, the authors present the algorithm for vehicles to detect general objects, which can classify obstacles that are real obstacles or fake obstacles, such as a painting or text on the road. The proposed method, we combined on-board computer vision system based on Histograms of Oriented Gradient (HOG) and Time Delay Neural Network (TDNN). We extract feature of the obstacles by HOG and using TDNN to recognize and classify the obstacles. The experimental results showed that this system can detect general objects, and is not restricted to vehicles, objects or pedestrians. It has provided good results along with high accuracy and reliability, which it is accurate enough to provide a warning to the driver when a collision is imminent.

Artificial Neural Network Approach for Dynamic Modelling of Heat Exchanger for Data Prediction

Objective: Artificial Neural Networks became a powerful tool for dynamic modelling of non linear physical systems and for prediction of specific parameters of complex systems. This work aims to model a heat exchanger using different neural networks and to investigate their performance in predicting its outlet temperature. Methods: In this work four different neural networks namely Elman Recurrent Neural Networks (ERNN), Time Delay Neural Networks (TDNN), Cascade Feed Forward Neural Networks (CFFNN) and Feed Forward Neural Networks (FFNN) are modelled for the prediction of the outlet liquid temperature of a saturated steam heat exchanger from its liquid flow rate. A benchmark dataset consisting of 4000 tuples is used to train, validate and test the performance of each neural network model. Findings: All the four ANN models are trained, validated and tested which predicted the outlet temperature of the Heat exchanger with acceptable accuracy and the Elman Recurrent Neural Network is found to have the best accuracy by having the lowest Mean Square Error (MSE) and best Regression due to its feedback connections. Applications: The Artificial Neural Network Models simulated, especially the Elman Neural Networks are good at prediction of operational parameters of a physical system and hence can be used in prediction and forecasting of operational parameters of engineering, medical, financial and environmental systems.Keywords: Artificial Neural Networks, Elman Recurrent Neural Network, Heat Exchanger, Prediction

Controller design for global fixed-time synchronization of delayed neural networks with discontinuous activations

This paper addresses the controller design problem for global fixed-time synchronization of delayed neural networks (DNNs) with discontinuous activations. To solve this problem, adaptive control and state feedback control laws are designed. Then based on the two controllers and two lemmas, the error system is proved to be globally asymptotically stable and even fixed-time stable. Moreover, some sufficient and easy checked conditions are derived to guarantee the global synchronization of drive and response systems in fixed time. It is noted that the settling time functional for fixed-time synchronization is independent on initial conditions. Our fixed-time synchronization results contain the finite-time results as the special cases by choosing different values of the two controllers. Finally, theoretical results are supported by numerical simulations.

Continuous Online Sequence Learning with an Unsupervised Neural Network Model.

The ability to recognize and predict temporal sequences of sensory inputs is vital for survival in natural environments. Based on many known properties of cortical neurons, hierarchical temporal memory (HTM) sequence memory recently has been proposed as a theoretical framework for sequence learning in the cortex. In this letter, we analyze properties of HTM sequence memory and apply it to sequence learning and prediction problems with streaming data. We show the model is able to continuously learn a large number of variable order temporal sequences using an unsupervised Hebbian-like learning rule. The sparse temporal codes formed by the model can robustly handle branching temporal sequences by maintaining multiple predictions until there is sufficient disambiguating evidence. We compare the HTM sequence memory with other sequence learning algorithms, including statistical methods-autoregressive integrated moving average; feedforward neural networks-time delay neural network and online sequential extreme learning machine; and recurrent neural networks-long short-term memory and echo-state networks on sequence prediction problems with both artificial and real-world data. The HTM model achieves comparable accuracy to other state-of-the-art algorithms. The model also exhibits properties that are critical for sequence learning, including continuous online learning, the ability to handle multiple predictions and branching sequences with high-order statistics, robustness to sensor noise and fault tolerance, and good performance without task-specific hyperparameter tuning. Therefore, the HTM sequence memory not only advances our understanding of how the brain may solve the sequence learning problem but is also applicable to real-world sequence learning problems from continuous data streams.

Performance comparison of artificial neural network models for daily rainfall prediction

With an aim to predict rainfall one-day in advance, this paper adopted different neural network models such as feed forward back propagation neural network (BPN), cascade-forward back propagation neural network (CBPN), distributed time delay neural network (DTDNN) and nonlinear autoregressive exogenous network (NARX), and compared their forecasting capabilities. The study deals with two data sets, one containing daily rainfall, temperature and humidity data of Nilgiris and the other containing only daily rainfall data from 14 rain gauge stations located in and around Coonoor (a taluk of Nilgiris). Based on the performance analysis, NARX network outperformed all the other networks. Though there is no major difference in the performances of BPN, CBPN and DTDNN, yet BPN performed considerably well confirming its prediction capabilities. Levenberg Marquardt proved to be the most effective weight updating technique when compared to different gradient descent approaches. Sensitivity analysis was instrumental in identifying the key predictors.

Some novel approaches on state estimation of delayed neural networks

This paper studies the issue of state estimation for a class of neural networks (NNs) with time-varying delay. A novel Lyapunov-Krasovskii functional (LKF) is constructed, where triple integral terms are used and a secondary delay-partition approach (SDPA) is employed. Compared with the existing delay-partition approaches, the proposed approach can exploit more information on the time-delay intervals. By taking full advantage of a modified Wirtinger’s integral inequality (MWII), improved delay-dependent stability criteria are derived, which guarantee the existence of desired state estimator for delayed neural networks (DNNs). A better estimator gain matrix is obtained in terms of the solution of linear matrix inequalities (LMIs). In addition, a new activation function dividing method is developed by bringing in some adjustable parameters. Three numerical examples with simulations are presented to demonstrate the effectiveness and merits of the proposed methods.

Learning Aided Behavioral Modeling and Adaptive Digital Predistortion Design for Nonlinear Power Amplifier

A new learning-based structure is defined for the identification of nonlinear attributes of a baseband power amplifier (PA) and design of an adaptive digital pre-distorter for application in wireless channels. Analytically and experimentally, generalized memory polynomial (GMP) model is derived from the transmit–receive conditions and the learning network structure. The test data are a sequences of 16-quadrature amplitude modulation (QAM) and 64-QAM complex symbols. In simulation, unknown GMP models of different order are used as a baseband PA model for identification. Minicircuits ZHL-4240 radio frequency PA modules along with NI USRP 2920 units are employed for experimental verification of the concept. The learning aspect is governed by a specially designed artificial neural network structure called real valued focused time delay neural network. Simulation and empirical results agree with analytical derivation.

Comparison of an artificial neural network and a conceptual rainfall–runoff model in the simulation of ephemeral streamflow

ABSTRACTThe rainfall–runoff process is governed by parameters that can seldom be measured directly for use with distributed models, but are rather inferred by expert judgment and calibrated against historical records. Here, a comparison is made between a conceptual model (CM) and an artificial neural network (ANN) for their ability to efficiently model complex hydrological processes. The Sacramento soil moisture accounting model (SAC-SMA) is calibrated using a scheme based on genetic algorithms and an input delay neural network (IDNN) is trained for variable delays and hidden layer neurons which are thoroughly discussed. The models are tested for 15 ephemeral catchments in Crete, Greece, using monthly rainfall, streamflow and potential evapotranspiration input. SAC-SMA performs well for most basins and acceptably for the entire sample with R2 of 0.59–0.92, while scoring better for high than low flows. For the entire dataset, the IDNN improves simulation fit to R2 of 0.70–0.96 and performs better for high flows while being outmatched in low flows. Results show that the ANN models can be superior to the conventional CMs, as parameter sensitivity is unclear, but CMs may be more robust in extrapolating beyond historical record limits and scenario building.EDITOR M.C. Acreman; ASSOCIATE EDITOR not assigned

Exponential synchronization of discrete-time mixed delay neural networks with actuator constraints and stochastic missing data

This paper investigates the problem of exponential synchronization of discrete-time neural networks with mixed time delays, actuator saturation and failures. Meanwhile, the unreliable communication links are considered between the neural networks, and such unreliable links are modeled as stochastic missing data satisfying Bernoulli distributions. In order to show the relationships between actuator constraints, unreliable communication link and mixed delay neural networks, by using Lyapunov functional approach, a missing data probability dependent exponential synchronization criterion is given. Then, based on such criterion, a reliable controller is designed to ensure that the neural networks are exponentially synchronized in the mean square. Finally, a numerical example is provided to illustrate the effectiveness of the proposed approach.