Mackey-Glass Time Series Prediction Research Articles

High-Dimensional Physical Reservoir with Back-End-of-Line-Compatible Tin Monoxide Thin-Film Transistor.

This work demonstrates a physical reservoir using a back-end-of-line compatible thin-film transistor (TFT) with tin monoxide (SnO) as the channel material for neuromorphic computing. The electron trapping and time-dependent detrapping at the channel interface induce the SnO·TFT to exhibit fading memory and nonlinearity characteristics, the critical assets for physical reservoir computing. The three-terminal configuration of the TFT allows the generation of higher-dimensional reservoir states by simultaneously adjusting the bias conditions of the gate and drain terminals, surpassing the performances of typical two-terminal-based reservoirs such as memristors. The high-dimensional SnO TFT reservoir performs exceptionally in two benchmark tests, achieving a 94.1% accuracy in Modified National Institute of Standards and Technology handwritten number recognition and a normalized root-mean-square error of 0.089 in Mackey-Glass time-series prediction. Furthermore, it is suitable for vertical integration because its fabrication temperature is <250 °C, providing the benefit of achieving a high integration density.

Scalable photonic reservoir computing based on pulse propagation in parallel passive dispersive links

We propose and demonstrate a scalable photonic reservoir computing system based on pulse propagation in parallel passive dispersive links. The approach features a simple structure and is composed of passive components, such as dispersive elements and ring resonators. The system utilizes a pulsed laser and employs a multi-channel architecture with dispersive elements for pulse broadening and ring resonators for delay stacking. It is verified for its capability to handle both digital tasks (2-bit Boolean logic operations, header recognition) and analog tasks (Mackey-Glass time-series prediction, Lorenz 63 time-series prediction). A major advantage of the approach lies in its scalability. It has been confirmed that more complex tasks can be processed by using a system with more channels, along with dispersive elements and ring resonators with larger dispersion amounts and delays, respectively.

Switching Dynamics in Anti‐Ferroelectric Transistor for Multimodal Reservoir Computing

AbstractSpatial‐temporal time series analysis and forecasting are crucial for understanding dynamic systems and making informed decisions. Recurrent neural networks (RNNs) have paved the way for reservoir computing (RC), a method enabling effective temporal information processing at low training costs. While software‐based RC performs well, physical RC systems face challenges like slow processing speed and limited state richness, leading to high hardware costs. This study introduces an innovative approach, i.e., the antiferroelectric field effect transistor‐based RC (AFeFET‐based RC) system for efficient temporal data processing. By exploiting the fading memory property inherent in hafnium oxide‐based antiferroelectric material, this system demonstrates promise for physical RC implementation. Moreover, it leverages the light sensitivity of 2D molybdenum disulfide (MoS2) channels for controllable temporal dynamics under electrical and optical stimuli. This dual‐mode modulation significantly enriches the reservoir state, boosting overall system performance. Experimental tests on standard benchmarking tasks using the AFeFET‐based RC system yielded impressive accuracy results (95.4%) in spoken‐digit recognition and a remarkable normalized root mean square error (NRMSE) of 0.015 in Mackey–Glass time series prediction.

2Memristor-1Capacitor Integrated Temporal Kernel for High-Dimensional Data Mapping.

Compact but precise feature-extracting ability is core to processing complex computational tasks in neuromorphic hardware. Physical reservoir computing (RC) offers a robust framework to map temporal data into a high-dimensional space using the time dynamics of a material system, such as a volatile memristor. However, conventional physical RC systems have limited dynamics for the given material properties, restricting the methods to increase their dimensionality. This study proposes an integrated temporal kernel composed of a 2-memristor and 1-capacitor (2M1C) using a W/HfO2/TiN memristor and TiN/ZrO2/Al2O3/ZrO2/TiN capacitor to achieve higher dimensionality and tunable dynamics. The kernel elements are carefully designed and fabricated into an integrated array, of which performances are evaluated under diverse conditions. By optimizing the time dynamics of the 2M1C kernel, each memristor simultaneously extracts complementary information from input signals. The MNIST benchmark digit classification task achieves a high accuracy of 94.3% with a (196×10) single-layer network. Analog input mapping ability is tested with a Mackey-Glass time series prediction, and the system records a normalized root mean square error of 0.04 with a 20×1 readout network, the smallest readout network ever used for Mackey-Glass prediction in RC. These performances demonstrate its high potential for efficient temporal data analysis.

Spatiotemporal Data Processing with Memristor Crossbar-Array-Based Graph Reservoir.

Memristor-based physical reservoir computing (RC) is a robust framework for processing complex spatiotemporal data parallelly. However, conventional memristor-based reservoirs cannot capture the spatial relationship between the time-varying inputs due to the specific mapping scheme assigning one input signal to one memristor conductance. Here, a physical "graph reservoir" is introduced using a metal cell at the diagonal-crossbar array (mCBA) with dynamic self-rectifying memristors. Input and inverted input signals are applied to the word and bit lines of the mCBA, respectively, storing the correlation information between input signals in the memristors. In this way, the mCBA graph reservoirs can map the spatiotemporal correlation of the input data in a high-dimensional feature space. The high-dimensional mapping characteristics of the graph reservoir achieve notable results, including a normalized root-mean-square error of 0.09 in Mackey-Glass time series prediction, a 97.21% accuracy in MNIST recognition, and an 80.0% diagnostic accuracy in human connectome classification.

Experimental demonstration of a skyrmion-enhanced strain-mediated physical reservoir computing system

Physical reservoirs holding intrinsic nonlinearity, high dimensionality, and memory effects have attracted considerable interest regarding solving complex tasks efficiently. Particularly, spintronic and strain-mediated electronic physical reservoirs are appealing due to their high speed, multi-parameter fusion and low power consumption. Here, we experimentally realize a skyrmion-enhanced strain-mediated physical reservoir in a multiferroic heterostructure of Pt/Co/Gd multilayers on (001)-oriented 0.7PbMg1/3Nb2/3O3−0.3PbTiO3 (PMN-PT). The enhancement is coming from the fusion of magnetic skyrmions and electro resistivity tuned by strain simultaneously. The functionality of the strain-mediated RC system is successfully achieved via a sequential waveform classification task with the recognition rate of 99.3% for the last waveform, and a Mackey-Glass time series prediction task with normalized root mean square error (NRMSE) of 0.2 for a 20-step prediction. Our work lays the foundations for low-power neuromorphic computing systems with magneto-electro-ferroelastic tunability, representing a further step towards developing future strain-mediated spintronic applications.

Architecture Optimization of a Non-Linear Autoregressive Neural Networks for Mackey-Glass Time Series Prediction Using Discrete Mycorrhiza Optimization Algorithm.

Recurrent Neural Networks (RNN) are basically used for applications with time series and sequential data and are currently being used in embedded devices. However, one of their drawbacks is that RNNs have a high computational cost and require the use of a significant amount of memory space. Therefore, computer equipment with a large processing capacity and memory is required. In this article, we experiment with Nonlinear Autoregressive Neural Networks (NARNN), which are a type of RNN, and we use the Discrete Mycorrhizal Optimization Algorithm (DMOA) in the optimization of the NARNN architecture. We used the Mackey-Glass chaotic time series (MG) to test the proposed approach, and very good results were obtained. In addition, some comparisons were made with other methods that used the MG and other types of Neural Networks such as Backpropagation and ANFIS, also obtaining good results. The proposed algorithm can be applied to robots, microsystems, sensors, devices, MEMS, microfluidics, piezoelectricity, motors, biosensors, 3D printing, etc.

Prediction of Time Series Empowered with a Novel SREKRLS Algorithm

For the unforced dynamical non-linear statespace model, a new Q1 and efficient square root extended kernel recursive least square estimation algorithm is developed in this article. The proposed algorithm lends itself towards the parallel implementation as in the FPGA systems. With the help of an ortho-normal triangularization method, which relies on numerically stable givens rotation, matrix inversion causes a computational burden, is reduced. Matrix computation possesses many excellent numerical properties such as singularity, symmetry, skew symmetry, and triangularity is achieved by using this algorithm. The proposed method is validated for the prediction of stationary and non-stationary MackeyGlass Time Series, along with that a component in the x-direction of the Lorenz Times Series is also predicted to illustrate its usefulness. By the learning curves regarding mean square error (MSE) are witnessed for demonstration with prediction performance of the proposed algorithm from where it’s concluded that the proposed algorithm performs better than EKRLS. This new SREKRLS based design positively offers an innovative era towards non-linear systolic arrays, which is efficient in developing very-large-scale integration (VLSI) applications with non-linear input data. Multiple experiments are carried out to validate the reliability, effectiveness, and applicability of the proposed algorithm and with different noise levels compared to the Extended kernel recursive least-squares (EKRLS) algorithm.

Fuzzy Logic Based Time Series Prediction Algorithm Using Nearest Neighborhood Clustering

This paper presents a fuzzy logic based prediction algorithm using nearest neigh-borhood clustering for the prediction of time series. The proposed algorithm hasbeen developed by using nearest neighborhood clustering scheme and optimalfuzzy system. The presented algorithm is tested by the prediction of Mackey-Glass time series, time series obtained from Logistic Map equation and an oilrefinery data series. To the best of our knowledge very few studies have at-tempted to predict Logistic Map. The results of the experimentation show thatthe presented prediction algorithm is able to predict linear, nonlinear and evenchaotic time series to a reasonable accuracy. It performes well even when themathematical model of time series does not exist.

An Improved Parallel MDBN With AVMD for Nonlinear System Modeling

Nonlinear system modeling using Deep Belief Network (DBN) is currently a research hotspot. However, the training process of DBN needs large amount of data to guarantee accuracy, and the traditional DBN may not meet the requirement of high-precision modeling. In this paper, we first improve the original DBN and Variational Mode Decomposition (VMD) algorithms, and on this basis, we then proposed a parallel Momentum Deep Belief Networks (MDBN) with Adaptive Variational Mode Decomposition (AVMD). Parallel AVMD-MDBN is an improved modeling method based on the deep learning model DBN. Firstly, a single raw dataset is decomposed into a specific number of sub-datasets using AVMD. Then these sub-datasets are distributed among a number of improved MDBNs. A single raw dataset learning model and algorithm is extended to multiple feature extraction nodes to learn the characteristics of multiple sub-datasets in parallel. Finally, the results of the multiple nodes are transmitted to the main feature extraction node to complete the regression calculation. In order to verify the effectiveness of the model, the proposed parallel AVMD-MDBN model is tested on a nonlinear dynamic system modeling, a Mackey-Glass time-series prediction and a financial stock prediction. Our experimental results show that the proposed parallel AVMD-MDBN has better performances in terms of improving feature learning ability than that of other methods.

An Adaptive Deep Belief Network With Sparse Restricted Boltzmann Machines

Deep belief network (DBN) is an efficient learning model for unknown data representation, especially nonlinear systems. However, it is extremely hard to design a satisfactory DBN with a robust structure because of traditional dense representation. In addition, backpropagation algorithm-based fine-tuning tends to yield poor performance since its ease of being trapped into local optima. In this article, we propose a novel DBN model based on adaptive sparse restricted Boltzmann machines (AS-RBM) and partial least square (PLS) regression fine-tuning, abbreviated as ARP-DBN, to obtain a more robust and accurate model than the existing ones. First, the adaptive learning step size is designed to accelerate an RBM training process, and two regularization terms are introduced into such a process to realize sparse representation. Second, initial weight derived from AS-RBM is further optimized via layer-by-layer PLS modeling starting from the output layer to input one. Third, we present the convergence and stability analysis of the proposed method. Finally, our approach is tested on Mackey-Glass time-series prediction, 2-D function approximation, and unknown system identification. Simulation results demonstrate that it has higher learning accuracy and faster learning speed. It can be used to build a more robust model than the existing ones.

Fuzzy jump wavelet neural network based on rule induction for dynamic nonlinear system identification with real data applications.

AimFuzzy wavelet neural network (FWNN) has proven to be a promising strategy in the identification of nonlinear systems. The network considers both global and local properties, deals with imprecision present in sensory data, leading to desired precisions. In this paper, we proposed a new FWNN model nominated “Fuzzy Jump Wavelet Neural Network” (FJWNN) for identifying dynamic nonlinear-linear systems, especially in practical applications.MethodsThe proposed FJWNN is a fuzzy neural network model of the Takagi-Sugeno-Kang type whose consequent part of fuzzy rules is a linear combination of input regressors and dominant wavelet neurons as a sub-jump wavelet neural network. Each fuzzy rule can locally model both linear and nonlinear properties of a system. The linear relationship between the inputs and the output is learned by neurons with linear activation functions, whereas the nonlinear relationship is locally modeled by wavelet neurons. Orthogonal least square (OLS) method and genetic algorithm (GA) are respectively used to purify the wavelets for each sub-JWNN. In this paper, fuzzy rule induction improves the structure of the proposed model leading to less fuzzy rules, inputs of each fuzzy rule and model parameters. The real-world gas furnace and the real electromyographic (EMG) signal modeling problem are employed in our study. In the same vein, piecewise single variable function approximation, nonlinear dynamic system modeling, and Mackey–Glass time series prediction, ratify this method superiority. The proposed FJWNN model is compared with the state-of-the-art models based on some performance indices such as RMSE, RRSE, Rel ERR%, and VAF%.ResultsThe proposed FJWNN model yielded the following results: RRSE (mean±std) of 10e-5±6e-5 for piecewise single-variable function approximation, RMSE (mean±std) of 2.6–4±2.6e-4 for the first nonlinear dynamic system modelling, RRSE (mean±std) of 1.59e-3±0.42e-3 for Mackey–Glass time series prediction, RMSE of 0.3421 for gas furnace modelling and VAF% (mean±std) of 98.24±0.71 for the EMG modelling of all trial signals, indicating a significant enhancement over previous methods.ConclusionsThe FJWNN demonstrated promising accuracy and generalization while moderating network complexity. This improvement is due to applying main useful wavelets in combination with linear regressors and using fuzzy rule induction. Compared to the state-of-the-art models, the proposed FJWNN yielded better performance and, therefore, can be considered a novel tool for nonlinear system identification.

An improved Elman neural network with piecewise weighted gradient for time series prediction

Time series prediction is an important tool for system analysis. Traditional forecasting methods usually achieve prediction by establishing static model or only mining the information from the sequence itself, but without considering the dynamics of the system or its triggering factors. Elman neural network (ENN) is a dynamical model that can remember historical states. In order to better establish the real multivariate time series model and improve the single-step prediction accuracy, this study proposes an improved regularized ENN method based on piecewise time weighted gradient (PWRENN). Each time the parameters of PWRENN model are updated, the weighted gradient is calculated by weighing the current gradient and the previous gradients according to a monotonically increasing temporal function. By considering both the current gradient and previous gradients, PWRENN is able to simulate the time-lag effect in the time series prediction problem. Moreover, the system environment may change with time, and the historical data of the same slot at a different time should have different effects on the predicted data. This work uses a piecewise time function with different parameters at different time periods to more accurately model the real system. In PWRENN, L2 regularization is adopted to solve the overfitting problem and enhance the generalization performance. Furthermore, the effectiveness of the proposed method is verified in an artificial Mackey-Glass time series prediction and three landslide displacement predictions.

Gray Wolf Optimization and Least Square Estimatation As A New Learning Algorithm For Interval Type-II ANFIS

Gray Wolfe Optimization (GWO) is one of the meta-heuristic method and it is a popular technique in Many engineering and economic applications. GWO and Least Square Estimatation (LSE) are used to optimize the antecedents and consequents parameters of interval type-2 ANFIS respectively. We are checking the new learning algorithm by using the interval type-2 ANFIS in prediction of Mackey-Glass time series and the results were very encouraging compared to other algorithms.

Stability of Evolving Fuzzy Systems Based on Data Clouds

Evolving fuzzy systems (EFSs) are now well developed and widely used, thanks to their ability to self-adapt both their structures and parameters online. Since the concept was first introduced two decades ago, many different types of EFSs have been successfully implemented. However, there are only very few works considering the stability of the EFSs, and these studies were limited to certain types of membership functions with specifically predefined parameters, which largely increases the complexity of the learning process. At the same time, stability analysis is of paramount importance for control applications and provides the theoretical guarantees for the convergence of the learning algorithms. In this paper, we introduce the stability proof of a class of EFSs based on data clouds, which are grounded at the AnYa type fuzzy systems and the recently introduced empirical data analytics (EDA) methodological framework. By employing data clouds, the class of EFSs of AnYa type considered in this paper avoids the traditional way of defining membership functions for each input variable in an explicit manner and its learning process is entirely data driven. The stability of the considered EFS of AnYa type is proven through the Lyapunov theory, and the proof of stability shows that the average identification error converges to a small neighborhood of zero. Although, the stability proof presented in this paper is specially elaborated for the considered EFS, it is also applicable to general EFSs. The proposed method is illustrated with Box–Jenkins gas furnace problem, one nonlinear system identification problem, Mackey–Glass time series prediction problem, eight real-world benchmark regression problems as well as a high-frequency trading prediction problem. Compared with other EFSs, the numerical examples show that the considered EFS in this paper provides guaranteed stability as well as a better approximation accuracy.

Robust Adaptive Filtering With $q$ -Gaussian Kernel Mean $p$-Power Error

In this letter, a novel information theoretic measure, namely $\boldsymbol {q}$ -Gaussian kernel mean $\boldsymbol {p}$ -power error (QKMPE), is proposed by defining the mean $\boldsymbol {p}$ -power error in the $\boldsymbol {q}$ -Gaussian kernel space, which is a generalization of the kernel mean $\boldsymbol {p}$ -power error measure. Furthermore, a recursive kernel adaptive filter algorithm, named as recursive least $\boldsymbol {q}$ -Gaussian kernel mean $\boldsymbol {p}$ -power, is derived under the least QKMPE criterion for robust learning in noisy environment. This new proposed algorithm reveals superior performance against Gaussian-type noise as well as the non-Gaussian perturbation, especially when the data contain large outliers. Experimental results in the context of Mackey–Glass time series prediction confirm the effectiveness of the proposed algorithm.

Correntropy-Based Evolving Fuzzy Neural System

In this paper, a correntropy-based evolving fuzzy neural system (CEFNS) is proposed for approximation of nonlinear systems. Different from the commonly used mean-square error criterion, correntropy has a strong outliers rejection ability through capturing the higher moments of the error distribution. Considering the merits of correntropy, this paper brings contributions to build evolving fuzzy neural system (EFNS) based on the correntropy concept to achieve a more stable evolution of the rule base and update of the rule parameters instead of the commonly used mean-square error criterion. The correntropy-EFNS (CEFNS) begins with an empty rule base, and all rules are evolved online based on the correntropy criterion. The consequent part parameters are tuned based on the maximum correntropy criterion, where the correntropy is used as the cost function so as to improve the non-Gaussian noise rejection ability. The steady-state convergence performance of the CEFNS is studied through the calculation of the steady-state excess mean square error (EMSE) in two cases: Gaussian noise; and non-Gaussian noise. Finally, the CEFNS is validated through a benchmark system identification problem, a Mackey-Glass time series prediction problem as well as five other real-world benchmark regression problems under both noise-free and noisy conditions. Compared with other EFNSs, the simulation results show that the proposed CEFNS produces better approximation accuracy using the least number of rules and training time and also owns superior non-Gaussian noise handling capability.

Sequential Outlier Criterion for Sparsification of Online Adaptive Filtering.

In this paper, we deal with the learning problem when using an adaptive filtering method. For the learning system in filtering, the knowledge is obtained and updated based on the newly acquired information that is extracted and learned from the sequential samples over time. Effective measurement on the informativeness of a sample and reasonable subsequent treatment on the sample will improve the learning performance. This paper proposes a sequential outlier criterion for sparsification of online adaptive filtering. The method is proposed to achieve effective informativeness measurement of online filtering to obtain a more accurate and more compact network in the learning process. In the proposed method, the measurement on the samples' informativeness is established based on the historical sequentially adjacent samples, and then the informative-measured samples are treated individually by the learning system based on whether the sample is informative, redundant, or abnormal. With our method, a more sensible learning process can be achieved with valid knowledge extracted, and the optimal network in the learning system can be obtained. Simulations based on static function estimation, Mackey-Glass time series prediction, and Lorenz chaotic time series prediction demonstrate that the proposed method can provide more effective classification on samples and more accurate networks in online adaptive filtering.

Prediction of chaotic time series based on the fractional-order maximum correntropy criterion algorithm

Recently, adaptive filters have been widely used to perform the prediction of chaotic time series. Generally, the Gaussian noise is considered for the system noise. However, many non-Gaussian noises, e.g., impulse noise and alpha noise, exist in real systems. Adaptive filters are therefore required to reduce such non-Gaussian noises for practical applications. For improving the robustness against non-Gaussian noise, the maximum correntropy criterion (MCC) is successfully used to derive various robust adaptive filters. In these robust adaptive filters, the steepest ascent method based on the first-order derivative is generally utilized to construct the weight update form. It is well known that the traditional derivative can be generalized by the fractional-order derivative effectively. Therefore, to further improve the performance of adaptive filters based on the MCC, the fractional-order derivative is applied to the MCC-based algorithm, generating a novel fractional-order maximum correntropy criterion (FMCC) algorithm. Under the non-Gaussian noises, the proposed FMCC algorithm can be applied to predicting the chaotic time series effectively. In the proposed FMCC algorithm, the weight update form is constructed by using a combination of the first-order derivative based term and the fractional-order derivative based term. The Riemann-Liouville definition is utilized for calculating the fractional-order derivative in the proposed FMCC algorithm. The order of the fractional-order derivative is a crucial parameter of the proposed FMCC algorithm. However, it is difficult to obtain the optimal fractional order for different nonlinear systems theoretically. Therefore, the influence of the fractional order on the prediction performance is determined by trials for different nonlinear systems. The appropriate fractional order corresponds to the optimum of prediction accuracy, and can be chosen in advance. Simulations in the context of prediction of Mackey-Glass time series and Lorenz time series demonstrate that in the case of non-Gaussian noises the proposed FMCC algorithm achieves better prediction accuracy and faster convergence rate than the least mean square (LMS) algorithm, the MCC algorithm, and the fractional-order least mean square (FLMS) algorithm. In addition, the computational complexity of different filters is compared with each other under the example of the prediction of Marckey-Glass time series by using mean consumed time. It can be found that the computational complexity of FMCC algorithm is higher than those of the MCC and the LMS algorithms, but only slightly higher than that of the FLMS algorithm. As a result, comparing with other filters, the FMCC algorithm can improve the prediction performances of chaotic time series at the cost of the increasing computational complexity.

Parallel Interval Type-2 Subsethood Neural Fuzzy Inference System

Neuro-fuzzy models are being increasingly employed in the domains like weather forecasting, stock market prediction, computational finance, control, planning, physics, economics and management, to name a few. These models enable one to predict system behavior in a more human-like manner than their crisp counterparts. In the present work, an interval type-2 neuro-fuzzy evolutionary subsethood based model has been proposed for its use in finding solutions to some well-known problems reported in the literature such as regression analysis, data mining and research problems relevant to expert and intelligent systems. A novel subsethood based interval type-2 fuzzy inference system, named as Interval Type-2 Subsethood Neural Fuzzy Inference System (IT2SuNFIS) is proposed in the present work. Mathematical modeling and empirical studies clearly bring out the efficacy of this model in a wide variety of practical problems such as Truck backer-upper control, Mackey–Glass time-series prediction, Narazaki–Ralescu and bell function approximation. The simulation results demonstrate intelligent decision making capability of the proposed system based on the available data. The major contribution of this work lies in identifying subsethood as an efficient measure for finding correlation in interval type-2 fuzzy sets and applying this concept to a wide variety of problems pertaining to expert and intelligent systems. Subsethood between two type-2 fuzzy sets is different from the commonly used sup-star methods. In the proposed model, this measure assists in providing better contrast between dissimilar objects. This method, coupled with the uncertainty handling capacity of type-2 fuzzy logic system, results in better trainability and improved performance of the system. The integration of subsethood with type-2 fuzzy logic system is a novel idea with several advantages, which is reported for the first time in this paper.