A Recurrent Neural Network (RNN) is a specialized form of neural network that is adept at handling sequential data by retaining information from prior inputs. In contrast to conventional feedforward neural networks, RNNs incorporate loops in their architecture, allowing them to leverage data from previous time steps to affect the current output. This characteristic renders RNNs especially effective for applications that involve sequences, including time-series forecasting, natural language processing, and speech recognition. A fundamental component of RNNs is their hidden state, which acts as a dynamic memory that is refreshed with each incoming input. This allows RNNs to capture dependencies across time steps, which is crucial for understanding context in sequences. In language modeling, the interpretation of a word often relies on the words that come before it, a task that Recurrent Neural Networks (RNNs) handle well. However, RNNs struggle with issues like vanishing gradients, which hinder their ability to capture long-range dependencies. To overcome this, models such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs) were introduced. These models incorporate gates that regulate the flow of information, allowing them to better learn long-term dependencies. RNNs remain a powerful tool for working with sequential data, facilitating the modeling of temporal relationships, but their effectiveness depends on careful design and optimization. Research significance: Recurrent Neural Networks (RNNs) hold significant research value because of their capacity to simulate temporal and sequential data, which is essential in many fields. They are frequently employed in natural language processing for tasks such as sentiment analysis, language translation, and text generation. In time-series analysis, RNNs enable accurate forecasting in finance, healthcare, and climate modeling. They also are essential in speech recognition and video processing, handling dependencies across time steps. Research focuses on improving RNNs, addressing challenges like vanishing gradients, and enhancing efficiency through architectures like LSTMs and GRUs, solidifying their relevance in advancing AI and machine learning applications. Methodology: A technique for analyzing the relationships between several variables, particularly in situations when data is limited or unclear, is called gray relational analysis, or GRA. In order to comprehend the relationships between variables, it evaluates how similar or different they are. GRA aids decision-makers in identifying critical factors, prioritizing actions, and improving processes in complex fields like engineering, finance, and management. By converting both qualitative and quantitative data into gray numbers, GRA addresses uncertainty and provides valuable insights for problem-solving, decision-making, and performance improvement, leading to more informed and effective strategies. Alternative taken as Simple RNN, LSTM, GRU, Bidirectional RNN, Deep RNN, Vanilla RNN, Echo State Network, Attention-based RNN, Transformer RNN, GRU with Attention. Evaluation preference taken as Prediction Accuracy, Model Robstness, Learning Efficiency, Training Time, Complexity. Attention-based RNN has the lowest score, Deep RNN has the highest rank, according to the results.

ABSTRACT In‐sensor reservoir computing offers a promising paradigm for signal analysis by embedding sensing and computation within a single platform. However, it remains challenging to realize both dynamic temporal processing and long‐term memory using a single device. Here, we report a multi‐modal and reconfigurable oxide‐based memristive device that enables both volatile and nonvolatile switching modes in a unified architecture. By precisely tuning the crystallinity of the TiO 2 layer and adjusting the compliance current, we modulate the conductive filament dynamics to switch between volatile and nonvolatile behavior, and multi‐modal switching is verified based on nucleation theory. The volatile mode enables fading memory and nonlinearity required for high‐dimensional temporal encoding, while the nonvolatile mode provides robust analog weight storage with 5‐bit resolution and retention exceeding 10⁵ s. These dual functions are integrated into a neuromorphic in‐sensor reservoir computing system. The system accurately reconstructs ECG waveforms (NRMSE = 0.010) and achieves multi‐step prediction of pH time‐series (accuracy = 98.2%), while reducing energy consumption by over five‐fold compared to conventional echo state networks. We demonstrate a scalable and energy‐efficient approach toward intelligent biochemical sensing, highlighting how material‐level configurability in memristive devices can unlock new directions for on‐sensor neuromorphic hardware.

Characterizing the edge of chaos in echo state networks

Inferring the dynamics of a network of oscillators becomes a significant challenge in the absence of explicit system equations. We present a data-driven machine learning technique to predict different dynamical states of a network, specifically a star-structured one. The proposed method exploits a parameter-aware reservoir computing scheme based on the echo-state network (ESN) framework. Our method employs a minimal setup to learn the parameter-dependent dynamics of a large network, using only two ESN units. We utilize the topological symmetry of the network to reduce the training cost. We validate the performance of our scheme in both scenarios where the central node oscillator of the star network is identical and non-identical to the peripheral node oscillators. In both cases, the proposed scheme is able to efficiently predict various emergent multi-stable dynamics of the network with varied coupling strengths. Despite exposure to limited data during training, it shows notable performance in predicting unseen attractors, including chimera, coherent, incoherent, and cluster synchronization states present in the network dynamics. Thus, this study provides an efficient reservoir computing framework for learning the dynamics of large-scale oscillator networks.

Security Information and Event Management systems are key infrastructures for identifying and responding to threats in enterprise security operations. By centrally processing logs from multiple sources, SIEM systems improve attack traceability and response effectiveness. However, significant research and empirical studies show that SIEM systems generally present a high rate of false positives after deployment. Currently, false positives are mainly reduced using single-model approaches. These approaches can reduce the number of false positives to some extent but are often at a disadvantage regarding generalization capabilities and feature utilization. To address this challenge, this paper proposes a hybrid modeling method, which combines TF-IDF with Transformer, Echo State Network, Random Forest, and XGBoost. Through multimodal feature modeling and the combined use of model mechanisms, this approach can achieve synergistic utilization of temporal dependencies, semantic context, and structural features. Experimental validation on Hillstone Networks' enterprise-level real threat logs and the public Advanced SIEM Dataset demonstrates a 60.72% to 7.58% reduction in false positives, with stable performance and strong robustness. The research findings provide a feasible and scalable engineering pathway for intelligent SIEM false positive optimization.

Multivariate time series prediction using clustered echo state network

ABSTRACT We integrate the long short-term memory (LSTM) network into an ensemble model composed of the least squares support vector machine, echo state network, and extreme-learning machine for wind power generation forecasting. This study presents the first unified forecasting framework that combines these four machine learning techniques to evaluate their collective efficacy in improving prediction accuracy for wind power. Empirical analyses demonstrate that incorporating LSTM into the ensemble does not yield performance improvements over the three-method model. These findings indicate that the added complexity from LSTM does not enhance forecasting accuracy. Additionally, the choice of loss function is observed to have a negligible impact on the models’ predictive performance.

Abstract An echo state network ( ESN ) is a recurrent neural network that has several advantages with respect to a deep neural network, including its fast training phase and the absence of vanishing and exploding gradients. The training phase reduces to solving the least squares (LS) problem w ls = arg min v vX − y 2 2 , where X is the reservoir matrix, and X and y are functions of the training data. It is common to add regularisation to this problem because, it is claimed, it minimises the adverse effects of overfitting. Recent work in deep neural networks, physics informed neural networks and regression has shown, however, that regularisation does not solve the problem of overfitting, and thus this paper considers the application of regularisation to ESN s by analysing their predictive abilities on several time series, including a non-linear communication channel, the Hénon map and multiple superimposed oscillations. It is shown that the solution w ls of the LS problem is, for many problems, stable with respect to a perturbation in y for a wide range of parameter values of an ESN , and thus regularisation must not be applied to these problems. Each problem must, however, be considered because the need, or otherwise, to apply regularisation is dependent on many parameters of an ESN . Furthermore, regularisation is not benign because its use when a condition on the rate of decay of the singular values of X is not satisfied leads to a large error between the theoretically exact and regularised solutions of the LS problem.

Abstract Chaotic systems exhibit extreme sensitivity to initial conditions, which makes their prediction both critically important and highly challenging across various fields. Echo state network (ESN), as a classical data-driven model, has been widely applied to the prediction of chaotic systems in recent years. However, selecting hyperparameters remains a complex problem and effective mechanisms to significantly improve the prediction accuracy of ESN are still lacking. In this study, we propose a two-stage chaotic system prediction framework named one-dimensional guided (ODG). Specifically, the first stage utilizes metaheuristic optimization algorithms (MHOAs) to identify optimal hyperparameters for ESN. In the second stage, one-dimensional real data replaces predicted results of the same dimension during autonomous prediction. Experimental results on five chaotic systems show that the proposed ODG framework achieves the longest valid prediction time (VPT) with the shortest optimization time (OPT) compared to higher-order polynomial library methods. Additionally, we evaluated the framework under large-scale data prediction and noisy conditions. The results show that the ODG framework consistently maintains high prediction accuracy, demonstrating its robustness.

Enhancing echo state network with reservoir state selection for time series forecasting

An echo state network for synthesizing the standard 12-lead ECG from a two-lead ECG obtained from a single touch of a wrist-worn device

Machine learning for polycyclic aromatic hydrocarbons analysis in roasted lamb: new insights from spectral and chemical data.

Scalable mobile swarm network for reservoir computing using gaussian kernel density estimation.

Echo state networks (ESNs) are a class of recurrent neural networks in which only the readout layer is trainable, while the recurrent and input layers are fixed. This architectural constraint enables computationally efficient processing of time-series data. Traditionally, the readout layer in ESNs is trained using supervised learning with target outputs. In this study, we focus on input reconstruction (IR), where the readout layer is trained to reconstruct the input time series fed into the ESN. We show that IR can be achieved through unsupervised learning (UL), without access to supervised targets, provided that the ESN parameters are known a priori and satisfy invertibility conditions. This formulation allows applications relying on IR, such as dynamical system replication and noise filtering, to be reformulated within the UL framework via straightforward integration with existing algorithms. Our results suggest that prior knowledge of ESN parameters can reduce reliance on supervision, thereby establishing a new principle-not only by fixing part of the network parameters but also by exploiting their specific values. Furthermore, our UL-based algorithms for input reconstruction and related tasks are suitable for autonomous processing, offering insights into how analogous computational mechanisms might operate in the brain in principle. These findings contribute to a deeper understanding of the mathematical foundations of ESNs and their relevance to models in computational neuroscience.

Tourism experiences are shaped by rapidly changing conditions such as weather, local events, and visitor flows, yet most recommendation systems assume stable patterns, limiting their ability to adapt in real time. This study introduces a robust context‑aware tourism recommendation framework that integrates a Hypergraph Neural Network, an Echo State Network reservoir tuned to operate at the edge of chaos, and a transformer with Lyapunov‑adaptive attention. The hypergraph encoder models complex, multi‑entity relationships among users, destinations, and contextual factors; the reservoir captures evolving context signals with high sensitivity; and the Lyapunov‑adaptive attention mechanism adjusts focus based on online estimates of the largest Lyapunov exponent, enabling the system to detect and respond to sudden regime shifts. The framework is trained and evaluated on the publicly available Travel Recommendation Dataset from IEEE DataPort, enriched with historical weather records and local event schedules. Comparative experiments against strong context‑aware, graph‑based, and sequence‑based baselines show consistent improvements in accuracy, measured by hit rate and normalized discounted cumulative gain, and in diversity, measured by intra‑list diversity and serendipity, particularly under simulated disruptions such as abrupt weather changes. These results demonstrate that combining graph learning, recurrent dynamics, and chaos‑aware attention can substantially increase the resilience of personalization in volatile environments, paving the way for recommendation systems that remain both relevant and exploratory despite unpredictable shifts in user context.

Bayesian ARIMA models will offer a solid approach for analyzing time series data, providing more flexibility than traditional recursive models. They also effectively combine previous knowledge with current data to handle uncertainty. A particular kind of Bayesian ARIMA model with comparable considerations is called a Bayesian AR model. While Bayesian models employ prior information to estimate a wide range of possible parameter values, older methods frequently use maximum likelihood estimation to obtain single values for parameters. In order to effectively handle uncertainty, they also develop a posterior distribution. The applicability of Bayesian techniques to AR(p) models is examined in this work. It demonstrates their capacity to manage noisy, non-stationary, or incomplete data while allowing for thorough probabilistic inference, which improves uncertainty comprehension and validates probabilistic forecasts. The Bayesian AR model states that present values are linearly dependent on past values, which are further amplified by white noise. We use previous distributions to evaluate the variance and establish the model parameters. Consequently, these values are adjusted in response to observations, resulting in more complex and adaptable dimensional distributions. The Bayesian ARIMA model aids in forecasting and drawing conclusions when time series are more complicated and need variance considerations. Bayesian AR(p) models display the temporal correlations between data points regardless of how stationary they are. These models are commonly estimated using Markov Monte Carlo (MCMC) techniques like Metropolis-Hastings and Gibbs sampling. These models perform well when handling asymmetry, incomplete data, and structural changes. Even when used in a Bayesian manner, traditional models struggle to capture uncertain time series or intricate nonlinear patterns. These contemporary issues can be resolved with the appropriate use of an Echo State Network (ESN). An effective recursive neural network for forecasting evolving time series is the ESN. To identify the most effective inputs for the ESN, the hybrid Bayesian ARESN methodology utilizes the optimal configuration of the Bayesian AR model. The capacity of this approach to accurately simulate nonlinear interactions is recognized. A Bayesian AR model and an ESN model were integrated in this hybrid Bayesian AR-ESN methodology study. The results show that combining Bayesian AR and ESN significantly increases forecasting accuracy, particularly when forecasting error metrics are used. When compared to conventional techniques, the Bayesian model significantly increases predictive accuracy.

In this research, the groundwater levels (GWLs) were predicted by employing machine learning (i.e., stochastic gradient boosting (SGB), random forest (RF), generalized regression neural networks (GRNN), and group of method data handling (GMDH)) and deep learning (i.e., deep echo state network (Deep ESN) and long short-term memory (LSTM)) based on three predictive scenarios, Jeju Island, South Korea. In scenario 01, GWLs in Bongseong well was calculated utilizing rainfall, air temperature, relative humidity, wind speed, and various GWLs in different wells. Based on scenario 02, GWLs in Bongseong well was calculated using rainfall, air temperature, relative humidity, wind speed, and groundwater data (i.e., temperature, electric conductivity, and pressure). Finally, considering scenario 03, GWLs in Bongseong well were calculated by employing rainfall, air temperature, relative humidity, wind speed, and GWLs from 1-day to 15-day lead time. Five evaluation measures, including root mean squared error (RMSE), correlation coefficient (CC), Nash–Sutcliffe efficiency (NSE), relative error (RE), and root relative squared error (RRSE), were reflected for the predictive accuracy of developed models. Results showed that RF3 (RMSE = 0.053 m, CC = 1.000, NSE = 1.000, RE = 1.114, and RRSE = 0.013) based on scenario 03 performed the best predictive accuracy in GWLs of Bongseong well. Furthermore, the additional contributions of this research were achieved by the enhanced comparative evaluation through the SHapley Additive exPlanations (SHAP) strategy and one-way Analysis of Variance (ANOVA) test. The sensitivity analysis utilizing the SHAP strategy determined the significant feature indicator (i.e., GWL in 1-day lead-time) explaining its contribution to the predictive ability of developed models. The results of one-way ANOVA test provided that the predicted values were extracted from the same population as the measured values based on all models in scenario 03.

Multi-scale Regulation of Reservoir Topology in Echo State Networks

Brain-Lateralized Reservoirs: Neuro-Anatomically Constrained Echo State Networks for Chaotic and Physiological Signals

Multi-timescale prediction of lifetime and operating temperatures of PEMFC system by hierarchical echo state network