Nonlinear Vector Autoregression Research Articles

Feasibility study of using next-generation reservoir computing (NG-RC) model to estimate liver tumor motion from external breathing signals.

Respiratory motion is a challenge for accurate radiotherapy that may be mitigated by real-time tracking. Commercial tracking systems utilize a hybrid external-internal correlation model (ECM), integrating continuous external breathing monitoring with sparse X-ray imaging of the internal tumor position. This study investigates the feasibility of using the next generation reservoir computing (NG-RC) model as a hybrid ECM to transform measured external motions into estimated 3D internal motions. The NG-RC model utilizes the nonlinear vector autoregressive (NVAR) machine to account for the hysteresis or phase differences between external and internal motions. The datasets used to evaluate the efficacy of the NG-RC model include 57 motion traces from the CyberKnife system. The datasets were divided into three regions (central, lower, and upper livers) and three motion patterns. These patterns include linear and nonlinear motion patterns (Group A), hysteresis motion patterns (Group B), and all motion patterns (Group C). Moreover, various updating techniques were examined, such as continuously updating the NG-RC model using the first-in-first-out (FIFO) approach and sampling the internal tumor position every 0 s (strategy A), 60 s (strategy B), 30 s (strategy C), and 50 s (strategy D). The NG-RC model combined with strategy C resulted in better estimation accuracy than the reported CyberKnife cases (Wilcoxon signed rank p < 0.05). For linear and nonlinear motion patterns, the 3D radial estimation accuracy (mean ± SD) using the NG-RC model combined with strategy C and the CyberKnife system was 1.20 ± 0.78 and 1.1 ± 0.20 mm in the central liver, 0.66 ± 0.25 and 1.49 ± 0.50 mm in the lower liver, and 1.73 ± 0.86 and 1.61 ± 0.42 mm in the upper liver. For hysteresis motion patterns, the corresponding values were 1.13 ± 0.37 and 1.45 ± 0.33 mm, 1.43 ± 1.30 and 1.67 ± 0.42 mm, and 1.20 ± 0.68 and 1.46 ± 0.54 mm in the central, lower, and upper livers, respectively. This study proposed a new hybrid correlation model for real-time tumor tracking, which can be used to account for both linear and nonlinear motion patterns, as well as hysteresis motion patterns. Additionally, the NG-RC model required shorter training data sets (15 s) during pre-treatment and short internal motion sampling (every 30 s) during treatment compared to other ECMs.

The nonlinear impacts of macroeconomic shocks on the US regional housing markets

ABSTRACT This paper examines the asymmetrical impacts of aggregate macroeconomic shocks on the US regional housing markets by estimating nonlinear Vector Autoregressive models. We show how aggregate output, monetary, inflation, and credit shocks result in heterogenous impulse responses of the housing market volatilities across growing and declining housing market regimes in nine census divisions and 19 metropolitan cities. We find that output and inflation shocks cause qualitatively asymmetrical responses across regimes in majority of regions. A positive output shock and a negative inflation shock reduce (increase) housing price uncertainties when housing markets are in a declining (booming) regime in most regions. The responses of regional housing price volatilities to monetary and credit shocks are quantitatively asymmetrical across regimes in most regions. We also find that aggregate shocks have larger contributions on changing the probabilities of regime switching when a regional housing market is initially expanding than declining. Among aggregate shocks, inflation shock has the most prominent contributions across most regions. It also presents the strongest asymmetrical effects across positive and negative shocks. Our results can help regional decision makers utilize fiscal policies, regulatory measures, and regional planning to mitigate the destabilizing effects of macroeconomic shocks on regional housing markets.

Hardware Implementation of Next Generation Reservoir Computing with RRAM‐Based Hybrid Digital‐Analog System

Reservoir computing (RC) possesses a simple architecture and high energy efficiency for time‐series data analysis through machine learning algorithms. To date, RC has evolved into several innovative variants. The next generation reservoir computing (NGRC) variant, founded on nonlinear vector autoregression (NVAR) distinguishes itself due to its fewer hyperparameters and independence from physical random connection matrices, while yielding comparable results. However, NGRC networks struggle with massive Kronecker product calculations and matrix‐vector multiplications within the read out layer, leading to substantial efficiency challenges for traditional von Neumann architectures. In this work, a hybrid digital‐analog hardware system tailored for NGRC is developed. The digital part is a Kronecker product calculation unit with data filtering, which realizes transformation of nonlinear vector of the input linear vector. For matrix‐vector multiplication, a computing‐in‐memory architecture based on resistive random access memory array offers an energy‐efficient hardware solution, which markedly reduces data transfer and greatly improve computational parallelism and energy efficiency. The predictive capabilities of this hybrid NGRC system are validated through the Lorenz63 model, achieving a normalized root mean square error (NRMSE) of 0.00098 and an energy efficiency of 19.42TOPS W−1.

Benchmarking reservoir computing for residential energy demand forecasting

In the energy sector, accurate demand forecasts are vital but often limited by the available computational power. Reservoir computing (RC) or echo-state networks excel in chaotic time series prediction, with lower computational requirements compared to other recurrent network based methods like LSTMs. Next-generation reservoir computing (NG-RC) is a newer, more efficient variant of classical RC originating from nonlinear vector autoregression and therefore missing the randomness of classical RC. In our study, we evaluate RC and NG-RC for day-ahead energy demand predictions on four data sets and compare it to LSTMs and a naive persistence approach. We find that NG-RC outperforms all other methods when considering the root mean squared error on all data sets but struggles with very small or zero demands. Additionally, it offers a very computationally effective hyperparameter optimization and excels in replicating the inherent volatility and the erratic behavior of energy demands.

Does State Dependence Matter in Relation to Oil Price Shocks on Global Economic Conditions?

Abstract A common thread in the literature shows that an oil price shock can have a major impact on global economic conditions. We examine the global dimensions of changes to the global oil price and world economic uncertainty using three model types: ordinary least square (OLS); general additive model (GAM); and non-linear vector autoregression (VAR) model with local projections (LP). Our study highlights a positive and statistically significant effect of oil prices on economic uncertainty during non-expansionary periods, yet the impact is negative on economic uncertainty during periods of economic growth. Using a VAR-LP we analyze the global dimensions of a world oil price shock on global economic conditions and investigate whether there is consistency in how an oil price shock influences economic growth, consumer prices and economic uncertainty based on the state of economic conditions. The empirical evidence shows that during an expansionary (a non-expansionary) period, the impact of an oil price shock lowers (elevates) economic uncertainty. The empirical evidence from the three model types taken together indicate a presence of state dependence on the influence of an oil price shock.

Modeling and Causality Analysis of Human Sensorimotor Control System Based on NVAR Method

Article Modeling and Causality Analysis of Human Sensorimotor Control System Based on NVAR Method Jiyu Tan 1,2, Yurong Li 1,2,*, Qiurong Xie 3, and Xiaoling Wang 3 1 College of Electrical Engineering and Automation, Fuzhou University, Fuzhou 350108, China 2 Fujian Key Lab of Medical Instrument and Pharmaceutical Technology, Fuzhou University, Fuzhou 350116, China 3 College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou 350122, China * Correspondence: liyurong@fzu.edu.cn Received: 2 September 2023 Accepted: 19 October 2023 Published: 21 December 2023 Abstract: Neuromuscular disorders (such as stroke and spinal cord injuries) can lead to nerve damage that profoundly affects a patient's ability to control limb movements. Analyzing and modeling the human sensorimotor control system can establish a neurophysiological foundation for both fundamental research and clinical rehabilitation assessment. Electroencephalogram (EEG) signals provide insights into how the cerebral cortex regulates limb movements, while electromyogram (EMG) signals reveal how muscles respond to motor commands. Utilizing these signals, cortical-muscular models can be developed to facilitate the quantification and assessment of the human sensorimotor control system. This study proposes a method based on the nonlinear vector autoregression model and coiflets wavelet packet decomposition to perform multivariate time-frequency Granger causality analysis. The objective of this analysis is to compute the cortical-muscular causality matrix during elbow extension in stroke patients and construct a cortical-muscular causal network. The result reveals a frequency-dependent directed information flow pattern in the cortical-muscle causality matrix during elbow extension. Specifically, the GC values of EEG→EMG (down) and EMG→EEG (up) in the alpha and beta bands are significantly higher than those in the gamma band. The cortical-muscular causal network of stroke patients does not have small-world properties. The result indicates that the proposed method is able to characterize functional connections of brain myoelectric synchronization in different frequency bands within the time-frequency domain. It also uncovers the causal association that inherently exists in the human sensorimotor control system, providing a theoretical basis for further evaluation and quantification of the human sensorimotor control system.

Temporal Subsampling Diminishes Small Spatial Scales in Recurrent Neural Network Emulators of Geophysical Turbulence

AbstractThe immense computational cost of traditional numerical weather and climate models has sparked the development of machine learning (ML) based emulators. Because ML methods benefit from long records of training data, it is common to use data sets that are temporally subsampled relative to the time steps required for the numerical integration of differential equations. Here, we investigate how this often overlooked processing step affects the quality of an emulator's predictions. We implement two ML architectures from a class of methods called reservoir computing: (a) a form of Nonlinear Vector Autoregression (NVAR), and (b) an Echo State Network (ESN). Despite their simplicity, it is well documented that these architectures excel at predicting low dimensional chaotic dynamics. We are therefore motivated to test these architectures in an idealized setting of predicting high dimensional geophysical turbulence as represented by Surface Quasi‐Geostrophic dynamics. In all cases, subsampling the training data consistently leads to an increased bias at small spatial scales that resembles numerical diffusion. Interestingly, the NVAR architecture becomes unstable when the temporal resolution is increased, indicating that the polynomial based interactions are insufficient at capturing the detailed nonlinearities of the turbulent flow. The ESN architecture is found to be more robust, suggesting a benefit to the more expensive but more general structure. Spectral errors are reduced by including a penalty on the kinetic energy density spectrum during training, although the subsampling related errors persist. Future work is warranted to understand how the temporal resolution of training data affects other ML architectures.

Does risk matter more in recessions than in expansions? Implications for monetary policy

We employ a nonlinear vector autoregression and a non-recursive identification strategy to show that an equal-sized uncertainty shock generates a larger contraction in real activity when growth is low (as in recessions) than when growth is high (as in expansions). An estimated New Keynesian model with recursive preferences replicates these state-contingent responses when approximated to third order around its risky steady state due to a stronger upward nominal pricing bias in recessions than in expansions. Empirical evidence supports this state-contingent channel, and we show that it can greatly reduce the ability of systematic monetary policy to stabilize output during recessions.

Data-driven forecasting of nonequilibrium solid-state dynamics

We present a data-driven approach to efficiently approximate nonlinear transient dynamics in solid-state systems. Our proposed machine-learning model combines a dimensionality reduction stage with a nonlinear vector autoregression scheme. We report an outstanding time-series forecasting performance combined with an easy-to-deploy model and an inexpensive training routine. Our results are of great relevance as they have the potential to massively accelerate multiphysics simulation software and thereby guide the future development of solid-state-based technologies.

Reservoir computing based on solitary-like waves dynamics of liquid film flows: A proof of concept

Several theoretical works have shown that solitons —waves that self-maintain constant shape and velocity as they propagate— can be used as a physical computational reservoir, a concept where machine learning algorithms designed for digital computers are replaced by analog physical systems that exhibit nonlinear dynamical behaviour. Here we propose and experimentally validate a novel reservoir computing (RC) system that for the first time employs solitary-like (SL) waves propagating on the surface of a liquid film flowing over an inclined surface. We demonstrate the ability of the SL wave RC system (SLRC) to forecast chaotic time series, also producing experimental evidence of the possibility to combine RC with nonlinear vector autoregression techniques not only in a computer program but also in a physical system.

Assessing the impacts of financial stress on the yield spreads of Poland, Mexico and South Africa

This study employs a nonlinear vector autoregression (VAR) model and quantile-based analysis to examine the effects of the financial stress index (FSI) of developed countries and the exchange market pressure index (EMPI) on the USD-denominated yield spreads of Poland, Mexico, and South Africa. It was found by the nonlinear VAR that increases/decreases in the FSI of developed countries and in the EMPI raise/lower the yield spreads in each emerging country. Although different results are obtained among each emerging country, it was highlighted that foreign and domestic financial stress can be incorporated in the monetary policy formulation of the central banks of Poland, Mexico, and South Africa. Quantile analysis also revealed the role of different bond market pressure regimes in emerging countries, while the asymmetrical impacts of FSI and EMPI should be considered by the policymakers.

Learning stochastic filtering

We quantify the performance of approximations to stochastic filtering by the Kullback-Leibler divergence to the optimal Bayesian filter. Using a two-state Markov process that drives a Brownian measurement process as prototypical test case, we compare two stochastic filtering approximations: a static low-pass filter as baseline, and machine learning of Volterra expansions using nonlinear Vector Auto-Regression (nVAR). We highlight the crucial role of the chosen performance metric, and present two solutions to the specific challenge of predicting a likelihood bounded between 0 and 1.

The asymmetric relationship between structural oil shocks and food prices: evidence from Saudi Arabia

ABSTRACT This paper examines the impacts of structural oil shocks, namely, the index of industrial production, world production of oil, and spot crude oil prices on the food prices in Saudi Arabia between 1986 and 2020 using nonlinear autoregressive distributed lags (NARDL) and Structural Vector Autoregressive (S-VAR) models to capture both short- and long-run asymmetries between these variables. The asymmetry between industrial production and food price in the short- and long-run was revealed and the impact of negative shocks of industrial production on the food prices was dominant. This implies that lower economic activities result in declined industrial production and food shortage which exerts a hike in food prices. With respect to Saudi Arabia, a significant relationship between negative shocks in crude oil prices and food prices was found in short- as well as long-run periods. The estimations also reveal that increases in oil supply result in a decline in food prices decrease in both periods of time. Similarly, a positive association between GDP growth and food prices was found. Finally, the oil supply shocks are found as the main factor, followed by the oil demand shocks and crude oil prices, causing the food price variation in Saudi Arabia.

Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: A comparative study

In recent years, machine-learning techniques, particularly deep learning, have outperformed traditional time-series forecasting approaches in many contexts, including univariate and multivariate predictions. This study aims to investigate the capability of (i) gated recurrent neural networks, including long short-term memory (LSTM) and gated recurrent unit (GRU) networks, (ii) reservoir computing (RC) techniques, such as echo state networks (ESNs) and hybrid physics-informed ESNs, and (iii) the nonlinear vector autoregression (NVAR) approach, which has recently been introduced as the next generation RC, for the prediction of chaotic time series and to compare their performance in terms of accuracy, efficiency, and robustness. We apply the methods to predict time series obtained from two widely used chaotic benchmarks, the Mackey–Glass and Lorenz-63 models, as well as two other chaotic datasets representing a bursting neuron and the dynamics of the El Niño Southern Oscillation, and to one experimental dataset representing a time series of cardiac voltage with complex dynamics. We find that even though gated RNN techniques have been successful in forecasting time series generally, they can fall short in predicting chaotic time series for the methods, datasets, and ranges of hyperparameter values considered here. In contrast, for the chaotic datasets studied, we found that reservoir computing and NVAR techniques are more computationally efficient and offer more promise in long-term prediction of chaotic time series.

Do collective emotions drive bitcoin volatility? A triple regime-switching vector approach

In this paper, we build an empirical specification that helps to explain bitcoin volatility and to characterize phases of the bitcoin bubble using information derived from investors’ emotions and sentiment that captures investment intentions and investors’ aversion to risk. To this end, we investigated the bilateral relations between bitcoin volatility and investor emotions between 2018 and 2021, a period characterized by significant changes in bitcoin prices as well as wide disparities in investor emotions, especially in the context of the ongoing COVID-19 pandemic. The study was based on a linear and nonlinear Vector Autoregressive (VAR) model that we applied to data related to bitcoin prices and market sentiment as expressed by the Fear and Greed index. Overall, our results evince the key role played by collective emotions in the formation and collapse of the bitcoin bubble. Two findings in particular stand out. First, our model shows significant time-varying lead-lag effects between bitcoin volatility and investor sentiment that come into play bilaterally and help to characterize the dynamics of bitcoin volatility. Second, these interactions exhibit asymmetry and nonlinearity as the sign and size of collective emotions (resp. bitcoin volatility) vary with the regime and market state under consideration (calm state versus period of bubble formation, etc.). In other words, the power of sentiment has a time-varying effect on the market. Indeed, in the first regime (“calm state”), where bitcoin volatility is relatively low and the market shows evidence of stability, collective emotions have a negative impact on bitcoin volatility, prompting a stabilizing strength. However, in the second regime (“bubble formation”), the effect of emotions turns significantly positive as investors gradually become less fearful and more reassured, which can simultaneously increase volatility and destabilize the market. Finally, in the third regime (“bubble collapse”), when bitcoin reaches a high level of value and experiences impressive volatility excess, the effect of emotions again turns negative, resulting in further switching behavior that pushes investor action to provoke a bitcoin price correction, moving it toward a new state of stability. Our conclusion helps improve predictions of bitcoin price dynamics informed by the information provided by investor emotions.

Next-generation reservoir computing based on memristor array

As a kind of brain-inspired computing, reservoir computing (RC) has great potential applications in time sequence signal processing and chaotic dynamics system prediction due to its simple structure and few training parameters. Since in the RC randomly initialized network weights are used, it requires abundant data and calculation time for warm-up and parameter optimization. Recent research results show that an RC with linear activation nodes, combined with a feature vector, is mathematically equivalent to a nonlinear vector autoregression (NVAR) machine, which is named next-generation reservoir computing (NGRC). Although the NGRC can effectively alleviate the problems which traditional RC has, it still needs vast computing resources for multiplication operations. In the present work, a hardware implementation method of using computing-in memory paradigm for NGRC is proposed for the first time. We use memristor array to perform the matrix vector multiplication involved in the nonlinear vector autoregressive process for the improvement of the energy efficiency. The Lorenz63 time series prediction task is performed by simulation experiments with the memristor array, demonstrating the feasibility and robustness of this method, and the influence of the weight precision of the memristor devices on the prediction results is discussed. These results provide a promising way of implementing the hardware NGRC.

Next generation reservoir computing

Reservoir computing is a best-in-class machine learning algorithm for processing information generated by dynamical systems using observed time-series data. Importantly, it requires very small training data sets, uses linear optimization, and thus requires minimal computing resources. However, the algorithm uses randomly sampled matrices to define the underlying recurrent neural network and has a multitude of metaparameters that must be optimized. Recent results demonstrate the equivalence of reservoir computing to nonlinear vector autoregression, which requires no random matrices, fewer metaparameters, and provides interpretable results. Here, we demonstrate that nonlinear vector autoregression excels at reservoir computing benchmark tasks and requires even shorter training data sets and training time, heralding the next generation of reservoir computing.

Nonparametric Modelling of Quarterly Unemployment Rates

A seasonal additive nonlinear vector autoregression (SANVAR) model is proposed for multivariate seasonal time series to explore the possible interaction among the various univariate series. Significant lagged variables are selected and additive autoregression functions estimated based on the selected variables using spline smoothing method. Conservative confidence bands are constructed for the additive autoregression function. The model is fitted to two sets of bivariate quarterly unemployment rate data with comparisons made to the linear periodic vector autoregression model. It is found that when the data does not significantly deviate from linearity, the periodic model is preferred. In cases of strong nonlinearity, however, the additive model is more parsimonious and has much higher out-of-sample prediction power. In addition, interactions among various univariate series are automatically detected.

Uncertainty and monetary policy in the US: A journey into nonlinear territory

AbstractThis paper estimates a nonlinear vector autoregression (VAR) model to assess whether the real effects of monetary policy shocks depend on the level of uncertainty. Crucially, uncertainty is modeled endogenously in the VAR, thus allowing to take account of two unexplored channels of monetary policy transmission working through uncertainty direct reaction and uncertainty mean reversion. We find that monetary policy shocks are about 50–75% more powerful during tranquil times than during firm‐ and macro‐level uncertain times. Failing to account for endogenous uncertainty would bias responses and imply twice more effective monetary policy during tranquil times, mainly because of the non‐consideration of uncertainty mean reversion.

Uncertainty, Wages and the Business Cycle

AbstractWe show that limited wage flexibility in economic downturns generates strong and state-dependent amplification of uncertainty shocks. It also explains the cyclical behaviour of empirical measures of uncertainty. In the presence of matching frictions, an occasionally binding constraint on downward wage adjustment enhances the concavity of firms’ hiring rule, resulting in an endogenous profit risk-premium. In turn, higher uncertainty increases the profit risk-premium when the economy operates close to the wage constraint, deepening a recession. Non-linear local projections and vector autoregression estimates support the model predictions. In addition, we show that measured uncertainty rises in a recession even without uncertainty shocks.