Delay Neural Network Research Articles

Estimating regional effects of climate change and altered land use on biosphere carbon fluxes using distributed time delay neural networks with Bayesian regularized learning

The ability to accurately predict changes of the carbon and energy balance on a regional scale is of great importance for assessing the effect of land use changes on carbon sequestration under future climate conditions. Here, a suite of land cover-specific Distributed Time Delay Neural Networks with a parameter adoption algorithm optimized through Bayesian regularization was used to model the statewide atmospheric exchange of CO2, water vapor, and energy in Oregon with its strong spatial gradients of climate and land cover. The network models were trained with eddy covariance data from 9 atmospheric flux towers. Compared to results derived with more common regression networks utilizing non-delayed input vectors, the performance of the DTDNN models was significantly improved with an average increase of the coefficients of determination of 64%.The optimized models were applied in combination with downscaled climate projections of the CMIP5 project to calculate future changes in the cycle of carbon, associated with a prescribed conversion of conventional grass-crops to hybrid poplar plantations for biofuel production in Oregon. The results show that under future RCP8.5 climate conditions the total statewide NEP increases by 0.87 TgC per decade until 2050 without any land use changes. With all non-forage grass completely converted to hybrid poplar the NEP averages 32.9 TgC in 2046–2050, an increase of 9%. Through comparisons with the results of a Bayesians inversion study, the results presented demonstrate that DTDNN models are a specifically well-suited approach to use the available data from flux networks to assess changes in biosphere–atmosphere exchange triggered by massive land use conversion superimposed on a changing climate.

Online remaining useful lifetime prediction of proton exchange membrane fuel cells using a novel robust methodology

This paper proposes a novel robust prognostic approach that contains three phases for degradation prediction of proton exchange membrane fuel cell (PEMFC) performance and its remaining useful lifetime (RUL) estimation. In the first detrending phase, a physical aging model (PAM) is used to remove the non-stationary trend in the original fuel cell degradation data. In the second filtering phase, the order of autoregressive and moving average (ARMA) model is determined by autocorrelation function (ACF), partial ACF and Akaike information criterion. The linear component in the stationary time series is then filtered by the identified ARMA model. In the third prediction phase, the remaining nonlinear pattern is used to train the time delay neural network (TDNN), in order to provide the final prediction result. Since the proposed prognostic approach uses appropriate methods to analyze and preprocess the original degradation data (i.e., the PAM maintains stationary trend, and then the identified ARMA filters linear component), the remaining nonlinear pattern of stationary time series can thus guarantee a good convergence performance of TDNN. In order to experimentally demonstrate the robustness and prediction accuracy of the proposed approach, degradation tests are performed using two types of PEMFC stack.

Stochastic stability for distributed delay neural networks via augmented Lyapunov–Krasovskii functionals

This paper is concerned with the analysis problem for the globally asymptotic stability of a class of stochastic neural networks with finite or infinite distributed delays. By using the delay decomposition idea, a novel augmented Lyapunov–Krasovskii functional containing double and triple integral terms is constructed, based on which and in combination with the Jensen integral inequalities, a less conservative stability condition is established for stochastic neural networks with infinite distributed delay by means of linear matrix inequalities. As for stochastic neural networks with finite distributed delay, the Wirtinger-based integral inequality is further introduced, together with the augmented Lyapunov–Krasovskii functional, to obtain a more effective stability condition. Finally, several numerical examples demonstrate that our proposed conditions improve typical existing ones.

Land cover change detection using focused time delay neural network

The development of improved satellite technology generates a huge amount of remote sensing data, these data play the crucial role in natural resource management. The land use and land cover (LULC) change intensely affects local environment, as well as the global environment. Therefore, the quantifiable knowledge about LULC changes occur in global scale is important to make effective planning for conservation and precise use of natural resources, that has motivated the scientists to develop the various land cover change detection techniques. In this paper, we have proposed neural network-based approach, i.e., focused time delay neural network (FTDNN)-based approach for land cover change detection, which is a time series prediction-based approach and detect the sudden change in the enhanced vegetation index (EVI) time series. The performance of the proposed method has been addressed by using quantitative and qualitative analysis techniques. For the quantitative evaluation, the proposed algorithm is applied to the standard synthetic data set, which are analogous to EVI time series data set. The performance result of the proposed method compares with the four previously existing data mining-based benchmark techniques. The analysis was shown that the FTDNN-based method significantly outperforms than other techniques. For qualitative analysis, the San Francisco Bay Area data set has been used, which comprises real EVI time series. The proposed FTDNN-based method is applied to the San Francisco Bay Area data set and observe the interesting land cover changes. These outcomes indicate the effectiveness of data mining techniques for the land cover change detection problem.

AN ADAPTIVE POWER SYSTEM STABILIZER BASED ON FOCUSED TIME DELAY NEURAL NETWORK

In this paper, Power System Stabilizer is designed in Single Machine Infinite Bus (SMIB) and speed control is implemented with a dynamic topology based on Focused Time Delay Neural Network (FTDNN). In case of prediction and control, two individual strategies are concerned for the current projects. The first is identification the dynamics of system. The other is an optimization unit expected for minimization disturbances. The performance of the system with FTDNN-PSS controller is compared with a Conventional PSS (C-PSS), RNN-PSS and DTDNN PSS. The results show the effectiveness of FTDNN-PSS design, and superior robust performance for enhancement power system stability compared to Conventional PSS with different cases.

Fixed-time synchronization of memristor-based fuzzy cellular neural network with time-varying delay

This paper mainly investigates the fixed-time synchronization of memristor-based fuzzy cellular neural network (MFCNN) with time-varying delay. By utilizing differential inclusion, set-valued map theory, the definitions of finite-time and fixed-time stability, we convert the fixed-time synchronization control of the drive-response MFCNN into the equivalent fixed-time stability problem of the error system between the drive-response systems. Some novel sufficient conditions are derived to guarantee the fixed-time synchronization of the drive-response MFCNN based on a simple Lyapunov function and a nonlinear feedback controller. Meanwhile, the settling time can be estimated by simple calculations. Furthermore, these fixed-time synchronization criteria here are easy to validate and extend to the MFCNN without time-varying delay and general memristor-based neural networks. Finally, three numerical examples are given to illustrate the correctness of the main results.

Power Quality Events Classification using ANN with Hilbert Transform

With the evolution of Smart Grid, Power Quality issues have become prominent. The urban development involves usage of computers, microprocessor controlled electronic loads and power electronic devices. These devices are the source of power quality disturbances. PQ problems are characterized by the variations in the magnitude and frequency in the system voltages and currents from their nominal values. To decide a control action, a proper classification mechanism is required to classify different PQ events. In this paper we propose a hybrid approach to perform this task. Different Neural topologies namely Cascade Forward Backprop Neural Network (CFBNN), Elman Backprop Neural Network (EBPNN), Feed Forward Backprop Neural Network (FFBPNN), Feed Forward Distributed Time Delay Neural Network (FFDTDNN) , Layer Recurrent Neural Network (LRNN), Nonlinear Autoregressive Exogenous Neural Network (NARX), Radial Basis Function Neural Network (RBFNN) along with the application of Hilbert Transform are employed to classify the PQ events. A meaningful comparison of these neural topologies is presented and it is found that Radial Basis Function Neural Network (RBFNN) is the most efficient topology to perform the classification task. Different levels of Additive White Gaussian Noise (AWGN) are added in the input features to present the comparison of classifiers.

Aperiodically intermittent strategy for finite-time synchronization of delayed neural networks

In this paper, the problem of the finite-time synchronization (FTS) is studied for a class of delay neural networks (DNNs) via aperiodically intermittent control. Based on the finite-time stability theory, several new conditions ensuring FTS of two DNNs are derived by establishing a very useful differential inequality and constructing a new Lyapunov function. In addition, the proposed results here are easy to verify and they also extend the previous publications. And, the upper bounds of the settling time for synchronization are estimated. Finally, numerical simulations show the effectiveness of the derived results and the developed method.

Modeling and forecasting of oilseed production of India through artificial intelligence techniques

Indian agriculture has made considerable progress in respect of principle food crops, but the performance in case of oilseed crops is so far not good as compared to food grains. Production of oilseeds and oils are not meeting the increasing demand for edible oils and this widening demand-supply gap has necessitated imports of edible oils. India is world's largest importer of oil seeds as it imports more than 50 percent of its total production. Forecasting is used to analyze the past and current behavior to forecasts the future oilseeds production which intern provide an aid to decision-making and in planning for the future effectively and efficiently. Autoregressive integrated moving average (ARIMA) model is the most widely used model for forecasting time series. One of the main drawback of this model is the presumption of linearity. To model the series which contains nonlinear patterns, the artificial intelligence techniques like time delay neural network (TDNN) and non-linear support vector regression (NLSVR) model are commonly employed. In this paper an attempt has been made to forecast the oilseed production of India using ARIMA, TDNN and NLSVR models. Empirical results clearly reveal that the artificial intelligence techniques outperformed the ARIMA model.

Auxiliary function-based integral inequality approach to robust passivity analysis of neural networks with interval time-varying delay

Abstract In this paper, we study the problem of passivity for uncertain neural networks with interval time-varying delay. Firstly, a suitable augmented Lyapunov–Krasovskii functional (LKF) containing two triple integral terms is constructed and an auxiliary function-based integral inequality (AFBI) is used to manipulate the augmented single integral terms in the derivative of LKF. Secondly, a special form of the AFBI is applied to deal with the delay-product-type term, which was used to be ignored in the time derivative of a triple integral term. As a result, less conservative delay-dependent passivity criteria are derived for normal delayed neural networks (DNNs) in the form of linear matrix inequalities (LMIs). In addition, with the same LKF, delay-dependent passivity criteria are obtained for normal DNNs without the delay-product-type term. Subsequently, these criteria are extended to DNNs with parameter uncertainties. Finally, four numerical examples and simulations are provided to illustrate the effectiveness of the proposed criteria.

Source localization using deep neural networks in a shallow water environment.

Deep neural networks (DNNs) are advantageous for representing complex nonlinear relationships. This paper applies DNNs to source localization in a shallow water environment. Two methods are proposed to estimate the range and depth of a broadband source through different neural network architectures. The first adopts the classical two-stage scheme, in which feature extraction and DNN analysis are independent steps. The eigenvectors associated with the modal signal space are extracted as the input feature. Then, the time delay neural network is exploited to model the long term feature representation and constructs the regression model. The second concerns a convolutional neural network-feed-forward neural network (CNN-FNN) architecture, which trains the network directly by taking the raw multi-channel waveforms as input. The CNNs are expected to perform spatial filtering for multi-channel signals, in an operation analogous to time domain filters. The outputs of CNNs are summed as the input to FNN. Several experiments are conducted on the simulated and experimental data to evaluate the performance of the proposed methods. The results demonstrate that DNNs are effective for source localization in complex and varied water environments, especially when there is little precise environmental information.

Approximate model predictive building control via machine learning

Many studies have proven that the building sector can significantly benefit from replacing the current practice rule-based controllers (RBC) by more advanced control strategies like model predictive control (MPC). However, the optimization-based control algorithms, like MPC, impose increasing hardware and software requirements, together with more complicated error handling capabilities required from the commissioning staff. In recent years, several studies introduced promising remedy for these problems by using machine learning algorithms. The idea is based on devising simplified control laws learned from MPC. The main advantage of the proposed methods stems from their easy implementation even on low-level hardware. However, most of the reported studies were dealing only with problems with a limited complexity of the parametric space, and devising laws only for a single control variable, which inevitably limits their applicability to more complex building control problems. In this paper, we introduce a versatile framework for synthesis of simple, yet well-performing control strategies that mimic the behavior of optimization-based controllers, also for large scale multiple-input-multiple-output (MIMO) control problems which are common in the building sector. The approach employs multivariate regression and dimensionality reduction algorithms. Particularly, deep time delay neural networks (TDNN) and regression trees (RT) are used to derive the dependency of multiple real-valued control inputs on parameters. The complexity of the problem, as well as implementation cost, are further reduced by selecting the most significant features from the set of parameters. This reduction is based on straightforward manual selection, principal component analysis (PCA) and dynamic analysis of the building model. The approach is demonstrated on a case study employing temperature control in a six-zone building, described by a linear model with 286 states and 42 disturbances, resulting in an MPC problem with more than thousand of parameters. The results show that simplified control laws retain most of the performance of the complex MPC, while significantly decreasing the complexity and implementation cost.

Target Tracking in WSN using Time Delay Neural Network

Energy efficient tracking is a challenging application of resource constrained wireless sensor network. Prediction based schemes play a vital role in energy saving by reducing an avoidable communication. Efficient tracking can be achieved only if state transition matrix used in filter closely resembles the target movement. Kalman filter has been widely used as prediction algorithm but fails in case of maneuvering target because of state transition matrix mismatch. To make tracking algorithm model free, Time delay neural network based prediction algorithm is proposed in this paper. Performance of Time Delay neural network (TDNN) is compared with Kalman filter and Interacting multiple model filter in terms of mean square error. Results shows that TDNN outperforms both the filters.

Comparative Study of Various Neural Network Architectures for MPEG-4 Video Traffic Prediction

<p>Network traffic as it is VBR in nature exhibits strong correlations which make it suitable for prediction. Real-time forecasting of network traffic load accurately and in a computationally efficient manner is the key element of proactive network management and congestion control. This paper comments on the MPEG-4 video traffic predictions evaluated by different types of neural network architectures and compares the performance of the same in terms of mean square error for the same video frames. For that three types of neural architectures are used namely Feed forward, Cascaded Feed forward and Time Delay Neural Network. The results show that cascade feed forward network produces minimum error as compared to other networks. This paper also compares the results of traditional prediction method of averaging of frames for future frame prediction with neural based methods. The experimental results show that nonlinear prediction based on NNs is better suited for traffic prediction purposes than linear forecasting models.</p>

Modeling of a pitching and plunging airfoil using experimental flow field and load measurements

The main goal of the current paper is to outline a low-order modeling procedure of a heaving airfoil in a still fluid using experimental measurements. Due to its relative simplicity, the proposed procedure is applicable for the analysis of flow fields within complex and unsteady geometries and it is suitable for analyzing the data obtained by experimentation. Currently, this procedure is used to model and predict the flow field evolution using a small number of low profile load sensors and flow field measurements. A time delay neural network is used to estimate the flow field. The neural network estimates the amplitudes of the most energetic modes using four sensory inputs. The modes are calculated using proper orthogonal decomposition of the flow field data obtained experimentally by time-resolved, phase-locked particle imaging velocimetry. To permit the use of proper orthogonal decomposition, the measured flow field is mapped onto a stationary domain using volume preserving transformation. The analysis performed by the model showed good estimation quality within the parameter range used in the training procedure. However, the performance deteriorates for cases out of this range. This situation indicates that, to improve the robustness of the model, both the decomposition and the training data sets must be diverse in terms of input parameter space. In addition, the results suggest that the property of volume preservation of the mapping does not affect the model quality as long as the model is not based on the Galerkin approximation. Thus, it may be relaxed for cases with more complex geometry and kinematics.

Audio-Facial Laughter Detection in Naturalistic Dyadic Conversations

We address the problem of continuous laughter detection over audio-facial input streams obtained from naturalistic dyadic conversations. We first present meticulous annotation of laughters, cross-talks and environmental noise in an audio-facial database with explicit 3D facial mocap data. Using this annotated database, we rigorously investigate the utility of facial information, head movement and audio features for laughter detection. We identify a set of discriminative features using mutual information-based criteria, and show how they can be used with classifiers based on support vector machines (SVMs) and time delay neural networks (TDNNs). Informed by the analysis of the individual modalities, we propose a multimodal fusion setup for laughter detection using different classifier-feature combinations. We also effectively incorporate bagging into our classification pipeline to address the class imbalance problem caused by the scarcity of positive laughter instances. Our results indicate that a combination of TDNNs and SVMs lead to superior detection performance, and bagging effectively addresses data imbalance. Our experiments show that our multimodal approach supported by bagging compares favorably to the state of the art in presence of detrimental factors such as cross-talk, environmental noise, and data imbalance.

Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation

Air pollutant concentration forecasting is an effective method of protecting public health by providing an early warning against harmful air pollutants. However, existing methods of air pollutant concentration prediction fail to effectively model long-term dependencies, and most neglect spatial correlations. In this paper, a novel long short-term memory neural network extended (LSTME) model that inherently considers spatiotemporal correlations is proposed for air pollutant concentration prediction. Long short-term memory (LSTM) layers were used to automatically extract inherent useful features from historical air pollutant data, and auxiliary data, including meteorological data and time stamp data, were merged into the proposed model to enhance the performance. Hourly PM2.5 (particulate matter with an aerodynamic diameter less than or equal to 2.5 μm) concentration data collected at 12 air quality monitoring stations in Beijing City from Jan/01/2014 to May/28/2016 were used to validate the effectiveness of the proposed LSTME model. Experiments were performed using the spatiotemporal deep learning (STDL) model, the time delay neural network (TDNN) model, the autoregressive moving average (ARMA) model, the support vector regression (SVR) model, and the traditional LSTM NN model, and a comparison of the results demonstrated that the LSTME model is superior to the other statistics-based models. Additionally, the use of auxiliary data improved model performance. For the one-hour prediction tasks, the proposed model performed well and exhibited a mean absolute percentage error (MAPE) of 11.93%. In addition, we conducted multiscale predictions over different time spans and achieved satisfactory performance, even for 13–24 h prediction tasks (MAPE = 31.47%).

A Self-Adaptive Approach for Particle Swarm Optimization Applied to Wind Speed Forecasting

Operational research has made meaningful contributions to practical forecasting in organizations. An area of substantial activity has been in nonlinear modeling. Based on Particle Swarm Optimization, we discuss a nonlinear method where a self-adaptive approach, named as Particle Swarm Optimization with aging and weakening factors, was applied to training a Focused Time Delay Neural Network. Three freely available benchmark datasets were used to demonstrate the features of the proposed approach compared to the Backpropagation algorithm, Differential Evolution and the Particle Swarm Optimization method when applied for training the artificial neural network. Even acknowledging that the effort in comparing methods across multiple empirical datasets is certainly substantial, the proposed algorithm was used to produce 30 min, 1, 3 and 6 h ahead predictions of wind speed at one site in Brazil. The use of the proposed algorithm goes further than only training the artificial neural network, but also searching the best number of hidden neurons and number of lags. The results have shown that the modified Particle Swarm Optimization algorithm obtained better results in all predictions horizons, and the use of it has remarkably reduced the training time.

On-Line Arabic Characters Recognition Using Enhanced Time Delay Neural Networks

This paper concerns with the online recognition of isolated hand writing Arabic characters, through, the interpretation of a script presented by a pen trajectory. This technique was generally used in the electronic organizers of Personal Digital Assistant type. First of all, we have built a data base with several scripters using a graphic tablet which will be used in our application. In order to have a precise recognition of the isolated characters, it is important to model their structure the most correctly possible. In this work we present the study, the implementation and the result of the test of a particular neural network which is the Time Delay Neural Networks. We have followed a two steps approach, in the first one, the character characteristics are extracted, and in the second one, a temporal multi-layered perception is developed for a future classification. Our temporal approach with the adaptive topology, responds to the nature of the Arabic script during the acquisition phase while the use of different learning algorithms can minimize the cost function and improve recognition rates. The parameterization of these two parts will allow us to analyze the impact of the neural network topology on the results of character recognition rates.

Impulsive control of unstable neural networks with unbounded time-varying delays

This paper considers the impulsive control of unstable neural networks with unbounded time-varying delays, where the time delays to be addressed include the unbounded discrete time-varying delay and unbounded distributed time-varying delay. By employing impulsive control theory and some analysis techniques, several sufficient conditions ensuring μ-stability, including uniform stability, (global) asymptotical stability, and (global) exponential stability, are derived. It is shown that an unstable delay neural network, especially for the case of unbounded time-varying delays, can be stabilized and has μ-stability via proper impulsive control strategies. Three numerical examples and their simulations are presented to demonstrate the effectiveness of the control strategy.