Application Of Echo State Networks Research Articles

On the prediction of the turbulent flow behind cylinder arrays via echo state networks

This study aims at the prediction of the turbulent flow behind cylinder arrays by the application of Echo State Networks (ESN). Three different arrangements of arrays of seven cylinders are chosen for the current study. These represent different flow regimes: single bluff body flow, transient flow, and co-shedding flow. This allows the investigation of turbulent flows that fundamentally originate from wake flows yet exhibit highly diverse dynamics. The data is reduced by Proper Orthogonal Decomposition (POD) which is optimal in terms of kinetic energy. The Time Coefficients of the POD Modes (TCPM) are predicted by the ESN. The network architecture is optimized with respect to its three main hyperparameters, Input Scaling (INS), Spectral Radius (SR), and Leaking Rate (LR), in order to produce the best predictions in terms of Weighted Prediction Score (WPS), a metric leveling statistic and deterministic prediction. In general, the ESN is capable of imitating the complex dynamics of turbulent flows even for longer periods of several vortex shedding cycles. Furthermore, the mutual interdependencies of the TCPM are well preserved. However, optimal hyperparameters depend strongly on the flow characteristics. Generally, as flow dynamics become faster and more intermittent, larger LR and INS values result in better predictions, whereas less clear trends for SR are observable.

Online learning compensation control of an electro-hydraulic shaking table using Echo State Networks

Electro-hydraulic shaking table (EHST) is widely used to simulate earthquake excitation in structural seismic tests. However, the nonlinear characteristics of the EHST, such as oil flow, friction, and dead zone, often lead to the distortion of seismic wave acceleration tracking and even cause the failure of the test. This paper describes the first application of Echo-State Networks (ESNs) to achieve high-precision seismic acceleration tracking of the EHST. The nonlinear control ability of the conventional three-variable control (TVC) algorithm in the EHST is discussed. An online learning compensation control framework using ESNs for “Transfer System” (The “Transfer System” denotes the nonlinear EHST controlled by TVC) is proposed. The least mean square (LMS) algorithm based on filtered-X is used to train the “ESN-Controller” online, and the initial transients of the “ESN-Controller” are reduced by using an estimation model of the actual “Transfer System” and a conditional switch. The offline iterative control (OIC) and nonlinear signal-based control (NSBC) are taken as the baseline algorithms of the proposed online learning compensation control. Simulation results show that the proposed online learning compensation control achieved excellent control with Pearson correlation coefficients of reference and responses above 99.6%, whereas the OIC and NSBC provided insufficient control. Moreover, the proposed online learning compensation control can also reduce the control error caused by the non-optimal TVC gains in the linear part of the EHST. This work throws new light on the study of the online learning control algorithm of the EHST.

Artificial intelligence algorithms for the recognition of Brugada type 1 pattern on standard 12-leads ECG

Abstract Funding Acknowledgements Type of funding sources: Public grant(s) – National budget only. Main funding source(s): This research project is funded by Tuscany Region Background/Introduction Electrocardiograms (ECGs) are rapidly moving from analog to digital versions. Consequently, a series of automatic analyses of standard 12-lead ECGs are attracting interest for their ability to support clinicians in the automatic recognition of specific features associated with different cardiac diseases [2]. Artificial Intelligence applications and Machine Learning (ML) algorithms have gained much attention in the last years for their ability to figure out patterns from data independently, without being explicitly taught rules. Peculiar features define the ECGs of patients with Brugada Syndrome (BrS); however, ambiguities still exist for the correct diagnosis of BrS and discrimination with respect to other pathologies. Purpose The BrAID (Brugada syndrome and Artificial Intelligence applications to Diagnosis) project aims to develop an innovative system for diagnosing Type 1 BrS based on ECG pattern recognition through the application of ML algorithms. In this work, an application of Echo State Networks (ESN), a type of Recurrent Neural Network (RNN), for the diagnosis of BrS from ECG is presented. Methods After approval from the Local Ethical Committees, 12-lead ECGs were obtained in patients enrolled in 5 Centers diagnosed with typical spontaneous Type 1 pattern (coved) (group A, 81 patients). Baseline ECG was also collected in patients undergoing the ajmaline test, classified as positive (group B, 37 patients) or negative (group C, 14 patients) according to test results. 174 patients with no clinical and familial history of arrhythmias were considered controls (group D). Data were collected from 4 beats extracted from the ECGs as input to the ESN. The datasets obtained in the different groups were used for the ESN model’s training and assessment (testing) through a double cross-validation approach. Results As shown in Table 1, the performances using three leads (V1, V2, V3) or V2 only were compared. The algorithm performance was assessed in all the datasets (group A+B+C+D) and in spontaneous BrS (group A) and controls (group D). A good accuracy (79.21%) was seen when the three leads were considered for groups A and D only; the best test set accuracy (80.20%) was obtained in the case in which V2 only was used as input in all the datasets. Conclusion(s) In this work, a novel system for diagnosing Type 1 BrS using an ESN approach was developed. Our preliminary results show that this ML model is able to detect ECG patterns associated with Type 1 BrS with good and comparable accuracy both when three leads (79.21% ) or V2 only (80.20%) were analyzed. The future availability of larger datasets could improve the model performance, increasing the ESN potentialities as a clinical support system tool to be used in everyday clinical practice. Table 1. The accuracy, specificity, and sensitivity reported for each dataset group are obtained through double cross-validation.

Robust Optimization and Validation of Echo State Networks for learning chaotic dynamics

An approach to the time-accurate prediction of chaotic solutions is by learning temporal patterns from data. Echo State Networks (ESNs), which are a class of Reservoir Computing, can accurately predict the chaotic dynamics well beyond the predictability time. Existing studies, however, also showed that small changes in the hyperparameters may markedly affect the network’s performance. The overarching aim of this paper is to improve the robustness in the selection of hyperparameters in Echo State Networks for the time-accurate prediction of chaotic solutions. We define the robustness of a validation strategy as its ability to select hyperparameters that perform consistently between validation and test sets. The goal is three-fold. First, we investigate routinely used validation strategies. Second, we propose the Recycle Validation, and the chaotic versions of existing validation strategies, to specifically tackle the forecasting of chaotic systems. Third, we compare Bayesian optimization with the traditional grid search for optimal hyperparameter selection. Numerical tests are performed on prototypical nonlinear systems that have chaotic and quasiperiodic solutions, such as the Lorenz and Lorenz-96 systems, and the Kuznetsov oscillator. Both model-free and model-informed Echo State Networks are analysed. By comparing the networks’ performance in learning chaotic (unpredictable) versus quasiperiodic (predictable) solutions, we highlight fundamental challenges in learning chaotic solutions. The proposed validation strategies, which are based on the dynamical systems properties of chaotic time series, are shown to outperform the state-of-the-art validation strategies. Because the strategies are principled – they are based on chaos theory such as the Lyapunov time – they can be applied to other Recurrent Neural Networks architectures with little modification. This work opens up new possibilities for the robust design and application of Echo State Networks, and Recurrent Neural Networks, to the time-accurate prediction of chaotic systems.

Prognostic of RUL based on Echo State Network Optimized by Artificial Bee Colony

Prognostic is an engineering technique used to predict the future health state or behavior of an equipment or system. In this work, a data-driven hybrid approach for prognostic is presented. The approach based on Echo State Network (ESN) and Artificial Bee Colony (ABC) algorithm is used to predict machine’s Remaining Useful Life (RUL). ESN is a new paradigm that establishes a large space dynamic reservoir to replace the hidden layer of Recurrent Neural Network (RNN). Through the application of ESN is possible to overcome the shortcomings of complicated computing and difficulties in determining the network topology of traditional RNN. This approach describes the ABC algorithm as a tool to set the ESN with optimal parameters. Historical data collected from sensors are used to train and test the proposed hybrid approach in order to estimate the RUL. To evaluate the proposed approach, a case study was carried out using turbofan engine signals show that the proposed method can achieve a good collected from physical sensors (temperature, pressure, speed, fuel flow, etc.). The experimental results using the engine data from NASA Ames Prognostics Data Repository RUL estimation precision. The performance of this model was compared using prognostic metrics with the approaches that use the same dataset. Therefore, the ESN-ABC approach is very promising in the field of prognostics of the RUL.

Uncertainty Quantification through Dropout in Time Series Prediction by Echo State Networks

The application of echo state networks to time series prediction has provided notable results, favored by their reduced computational cost, since the connection weights require no learning. However, there is a need for general methods that guide the choice of parameters (particularly the reservoir size and ridge regression coefficient), improve the prediction accuracy, and provide an assessment of the uncertainty of the estimates. In this paper we propose such a mechanism for uncertainty quantification based on Monte Carlo dropout, where the output of a subset of reservoir units is zeroed before the computation of the output. Dropout is only performed at the test stage, since the immediate goal is only the computation of a measure of the goodness of the prediction. Results show that the proposal is a promising method for uncertainty quantification, providing a value that is either strongly correlated with the prediction error or reflects the prediction of qualitative features of the time series. This mechanism could eventually be included into the learning algorithm in order to obtain performance enhancements and alleviate the burden of parameter choice.

A perspective on machine learning in turbulent flows

The physical complexity and the large number of degrees of freedom that can be resolved today by direct numerical simulations of turbulent flows, and by the most sophisticated experimental techniques, require new strategies to reduce and analyse the data so generated, and to model the turbulent behaviour. We discuss a few concrete examples for which the turbulence data have been analysed by machine learning tools. We also comment on work in neighbouring fields of physics, particularly astrophysical (and astronomical) work, where Big Data has been the paradigm for some time. We discuss unsupervised, semi-supervised and supervised machine learning methods to direct numerical simulations data of homogeneous isotropic turbulence, Rayleigh-Bénard convection, and the minimal flow unit of a turbulent channel flow; for the last case, we discuss in some detail the application of echo state networks, this being one implementation of reservoir computing. The paper also provides a brief perspective on machine learning applications more broadly.

Application of echo state networks for estimating voltage harmonic waveforms in power systems considering a photovoltaic system

An estimator based on echo state networks (ESNs) is presented in this study to estimate voltage harmonic distortion waveforms at non-monitored sensitive loads using current and voltage at a monitored location. Since distributed generations such as photovoltaic systems have played a special role in distribution networks so they are considered in this study and their effects on a harmonic voltage waveform estimator are evaluated. Voltage harmonics are considered as the main type of waveform distortion in a power quality approach. Voltage and current harmonics cause an increase in the malfunction of electrical equipment in power systems. To detect and analyse the voltage harmonics, installation of power quality monitors (PQMs) at all buses is not economical. Reducing the cost associated with the monitoring procedure can be achieved by optimising the number of PQMs to be installed. The presented technique is examined on an IEEE 37-bus network and results from the studies indicate an acceptably high accuracy of ESN estimator.

Research on the Application of Echo State Network in Data Prediction

Research on the Application of Echo State Network in Data Prediction

Application of echo state network for harmonic detection in distribution networks

Application of echo state network for harmonic detection in distribution networks

NGHIÊN CỨU ỨNG DỤNG MẠNG TRẠNG THÁI PHẢN HỒI ĐỂ DỰ BÁO CHẤT LƯỢNG KHÔNG KHÍ

A study on the application of Echo State Network (ESN) for the forecast of air quality in Hanoi for a period of seven days, which is based on the nonlinear relationships between the concentrations of an air pollutant to be forecasted and meteorological parameters, was conducted. Three air pollutants being SO2, NO2 and PM10 were selected for this study. Training data and testing data were extracted from the database of Lang air quality monitoring station, Hanoi, from 2003 to 2009. Values forecasted by ESN are compared with those by MLP (Multilayer Perception). Results shown that, in almost experiments, the performance of ESN is better than that of MLP in terms of the values and the correlation of concentration trends. The averages of RMSE of ESN and MLP for SO2 are 5.9 ppb and 6.9 ppb, respectively. For PM10, the accuracy of ESN is 83.8% with MAE of 53.5 μg/m3, while the accuracy of MLP is only 77.6% with MAE of 68.2 μg/m3. For NO2, the performance of ESN and MLP is similar; the accuracy of both models is in the range of 60% to 72.7%. These suggest that, ESN is a novel and feasible approach to build the air forecasting model. Keywords: Forecast, air quality, ESN, MLP, ANN, Hanoi, Vietnam.

Robust Control with Disturbance Estimation Using Echo State Networks for the Twin Rotor Aero-Dynamical System Application

This paper deals with the application of Echo State Network (ESN) model to robust control of the Twin Rotor Aero-Dynamical System (TRAS) through estimation and cancellation of disturbances. The work describes the modelling process of the plant and control scenarios in which the system is under influence of the unknown disturbances. In such control scenarios the ESN model is used to estimate the disturbances and calculate the input correction in such way to improve the control quality. All data used in experiments are collected from the TRAS through the Matlab/Simulink environment.

Application of echo state networks in short-term electric load forecasting

The paper presents the application of echo state network (ESN) to short-term load forecasting (STLF) problem in power systems for both 1-h and 24-h ahead predictions while using the least number of inputs: current-hour load, predicted target-hour temperature, and only for 24-h ahead forecasting, day-type index. The study is much attractive due to inclusion of weekends/holidays what makes STLF problem much more difficult. The main aim is to show the great capabilities of ESN as a stand-alone forecaster to learn complex dynamics of hourly electric load time series and forecast the near future loads with high accuracies. ESN as the state-of-the-art recurrent neural network (RNN) gains a reservoir of dynamics tapped by trained output units with a simple and fast single-stage training process. Furthermore, the application of ESN to predict the target-hour temperature needed by ESN-based load forecasters is examined. Since temperature prediction errors affect load forecasting accuracy, effects of such errors on ESN-based load forecasting are studied by both sensitivity analysis and applying noisy temperature series. Real hourly load and temperature data of a North-American electric utility is used as the data set. The results reflect that the ESN-based STLF method provides load forecasts with acceptable high accuracy.

Utilization of Echo State Networks for Differentiating Source and Nonlinear Load Harmonics in the Utility Network

Echo state networks (ESNs) are a special form of recurrent neural network (RNN). Complexities with existing algorithms have thus far limited supervised training techniques for RNNs from widespread use. When it comes to practical applications, multilayer perceptron neural networks (MLPNs) still dominate. This paper investigates the application of ESNs for the prediction of true current harmonics of a nonlinear load in the presence of distorted supply voltage. The determination of true harmonic current injection by individual loads is complicated by the fact that the supply voltage waveform at the point of common coupling (PCC) is distorted by other loads at the PCC or further upstream. Experimental results are presented with the proposed ESN algorithm applied to a laboratory test circuit. The difference between the measured current harmonics and the predicted ESN results are quantified in the form of a new parameter. The uniqueness of the proposed harmonic prediction method is that the results are obtained without disrupting the operation of any load and only require the acquisition of voltage and current waveforms. This method is applicable for both single- and three-phase loads.