Nonlinear Time Series Models Research Articles

Semiparametric Estimation in Time Series of Simultaneous Equations

A system of vector semiparametric nonlinear time series models is studied with possible dependence structures and nonstationarities in the parametric and nonparametric components. The parametric regressors may be endogenous while the nonparametric regressors are strictly exogenous and represent trends. The parametric regressors may be stationary or nonstationary and the nonparametric regressors are nonstationary time series. This framework allows for the nonparametric treatment of stochastic trends and subsumes many practical cases. Semiparametric least squares (SLS) estimation is considered and its asymptotic properties are derived. Due to endogeneity in the parametric regressors, SLS is generally inconsistent for the parametric component and a semiparametric instrumental variable least squares (SIVLS) method is proposed instead. Under certain regularity conditions, the SIVLS estimator of the parametric component is shown to be consistent with a limiting normal distribution that is amenable to inference. The rate of convergence in the parametric component is the usual √n rate and is explained by the fact that the common (nonlinear) trend in the system is eliminated nonparametrically by stochastic detrending.

Robust Identification of an Exponential Autoregressive Model

One of the most common nonlinear time series (random processes with discrete time) models is the exponential autoregressive model. In particular, it describes such nonlinear effects as limit cycles, resonant jumps, and dependence of the oscillation frequency on amplitude. When identifying this model, the problem arises of estimating its parameters --- the coefficients of the corresponding autoregressive equation. The most common methods for estimating the parameters of an exponential model are the least squares method and the least absolute deviation method. Both of these methods have a number of disadvantages, to eliminate which the paper proposes an estimation method based on the robust Huber approach. The obtained estimates occupy an intermediate position between the least squares and least absolute deviation estimates. It is assumed that the stochastic sequence is described by the autoregressive equation of the first order, is stationary and ergodic, and the probability distribution of the innovations process of the model is unknown. Unbiased, consistency and asymptotic normality of the proposed estimate are established by computer simulation. Its asymptotic variance was found, which allows to obtain an explicit expression for the relative efficiency of the proposed estimate with respect to the least squares estimate and the least absolute deviation estimate and to calculate this efficiency for the most widespread probability distributions of the innovations sequence of the equation of the autoregressive model

Max-linear regression models with regularization

Motivated by the newly developed max-linear competing copula factor models and max-stable nonlinear time series models, we propose a new class of max-linear regression models to take advantages of easy interpretable features embedded in linear regression models. It can be seen that linear relation is a special case of max-linear relation. We develop an EM algorithm based maximum likelihood estimation procedure. The consistency and asymptotics of the estimators for parameters are proved. To advance max-linear models to deal with high dimensional predictors, we adopt the common strategy of regularization in the high dimensional regression literature. We demonstrate the broad applicability of max-linear models using simulation examples and real applications in econometric and business modeling. The results, in terms of predictability, show a significant improvement compared with solely using regular regression models and other existing machine learning methods. The results enhance our understanding of the relationship between the response variable and the predictors, and among the predictors as well.

Traffic self-similarity analysis and application of industrial internet

Industrial internet traffic prediction is not only an academic problem, but also a concern of industry and network performance department. Efficient prediction of industrial internet traffic is helpful for protocol design, traffic scheduling, detection of network attacks, etc. This paper proposes an industrial internet traffic prediction method based on the Echo State Network. In the first place this paper proves that the industrial internet traffic data are self-similar by means of the calculation of Hurst exponent of each traffic time series. It indicates that industrial internet traffic can be predicted utilizing nonlinear time series models. Then Echo State Network is applied for industrial internet traffic forecasting. Furthermore, to avoid the weak-conditioned problem, grid search algorithm is used to optimize the reservoir parameters and coefficients. The dataset this paper perform experiments on are large-scale industrial internet traffic data at different time scale. They come from Industrial Internet in three regions and are provided by ZTE Corporation. The result shows that our approach can predict industrial internet traffic efficiently, which is also a verification of the self-similarity analysis.

Some notes on nonlinear cointegration: A partial review with some novel perspectives

Some recent work on the analysis of nonlinear and nonstationary time series models is reviewed. A couple of novel results are obtained in extending nonlinear cointegrating regression models to a time series situation. All through the paper focus is on aspects that could lead to a more well-defined concept of nonlinear cointegration.

Forecasting performance of nonlinear time-series models: an application to weather variable

Modelling the dynamic dependent data by the linear approach is the most popular among the researchers because of its simplicity in calculation and approximation, however, in real-world phenomena, most of the time-dependent data follow the nonlinearity. Moreover, most of the nonlinear modelling of time-dependent data have found in the financial applications. Besides this sector, the authors of this paper found the presence of nonlinearity in meteorological data with the help of four popular nonlinearity tests. Furthermore, there is a scarcity of the application of regime-switching threshold autoregressive nonlinear time-series model in forecasting the weather variables like temperature. Thus, this paper aims to compare the forecasting accuracy of the linear autoregressive (linear AR), self-exciting threshold autoregression (SETAR), logistic smooth transition autoregressive model (LSTAR), and feed-forward neural network (ANNs) and fitted with the determination of regime and hyperparameters. After fitting the models, twenty steps ahead forecast considered for the comparison along with the selected model selection criteria; and results depict that the LSTAR models are selected as the most appropriate fitted models for forecasting the daily Average, Maximum and Minimum temperature. Finally, it has observed that the average, as well as maximum temperature of Dhaka, Bangladesh, have an increasing trend and minimum temperature having a decreasing trend.

Statistical Causality for Multivariate Non-Linear Time Series via Gaussian Processes

The ability to test for statistical causality in linear and non-linear contexts, in stationary or non-stationary settings and to identify whether statistical causality influences trend of volatility forms a piratically important class of problems to explore in multi-modal and multivariate processes. In this paper we develop novel testing frameworks for statistical causality in general classes of multi-variate non-linear time-series models. Our framework accommodates flexible features where causality may be present in either: trend, volatility or both structural components of the general multivariate Markov processes under study. In addition, we accommodate the added possibilities of flexible structural features such as long memory and persistence in the multivariate processes when applying our semi-parametric approach to causality detection. We design a calibration procedure and formal testing procedure to detect these relationships through classes of Gaussian process models. We provide a generic framework which can be applied to a wide range of problems, including partially observed generalised diffusions or general multivariate linear or non-linear time series models. We develop several illustrative examples of features that are easily testable under our framework to study the properties of the inference procedure developed including power of the test, sensitivity and robustness. We then illustrate our method on an interesting real data example from commodity modelling.

Jump-preserving varying-coefficient models for nonlinear time series

An important and widely used class of semiparametric models is formed by the varying-coefficient models. Although the varying coefficients are traditionally assumed to be smooth functions, the varying-coefficient model is considered here with the coefficient functions containing a finite set of discontinuities. Contrary to the existing nonparametric and varying-coefficient estimation of piecewise smooth functions, the varying-coefficient models are considered here under dependence and are applicable in time series with heteroskedastic and serially correlated errors. Additionally, the conditional error variance is allowed to exhibit discontinuities at a finite set of points too. The (uniform) consistency and asymptotic normality of the proposed estimators are established and the finite-sample performance is tested via a simulation study and in a real-data example.

On temporal aggregation of some nonlinear time-series models

Recently, economic globalisation and advances in computer technologies have accelerated the development of ‘big data’ analysis. Time-series data are routinely collected from the Internet and machine transactions at mixed frequencies. The problem of temporal aggregation arises when time-series data are observed at a lower frequency than the data generation frequency of the underlying basic model. The resulting aggregated series, which contains less information, may lead to an erroneous view of the true model and flawed decisions. Therefore, it is important to study the effects of temporal aggregation to avoid making possibly improper decisions based on distorted information from the aggregated data. Temporal aggregation of linear time-series processes has been widely studied. The effects of temporal aggregation on the first two moments of some nonlinear time-series models are assessed. More specifically, four types of nonlinear time-series models are studied: the Markov switching autoregressive (MSAR) model, the bilinear (BL) model, the mixture autoregressive (MAR) model, and the self-exciting threshold autoregressive (SETAR) model.

Local Linear Quantile Regression for Time Series Under Near Epoch Dependence

This paper aims to establish asymptotic normality of the local linear kernel estimator for quantile regression under near epoch dependence, a useful concept in characterising time series dependence of extensive interests in Econometrics. In particular, near epoch dependence can cover a wide range of linear or nonlinear time series models that are even not of strong or $\alpha$-mixing property (a property usually assumed in the nonlinear time series literature). Under the mild conditions, the Bahadur representation of the quantile regression estimators is established in weak convergence sense. The method provides much richer information than mean regression and covers much more processes, which do not satisfy general mixing conditions. Simulation and application to a real data set are studied, which demonstrate the usefulness of the introduced method for analysis of time series. The theoretical results of this paper will be of widely potential interest for time series econometric semiparametric quantile regression modelling.

Models for autoregressive processes of bounded counts: How different are they?

We focus on purely autoregressive (AR)-type models defined on the bounded range {0,1,ldots , n} with a fixed upper limit n in mathbb {N}. These include the binomial AR model, binomial AR conditional heteroscedasticity (ARCH) model, binomial-variation AR model with their linear conditional mean, nonlinear max-binomial AR model, and binomial logit-ARCH model. We consider the key problem of identifying which of these AR-type models is the true data-generating process. Despite the volume of the literature on model selection, little is known about this procedure in the context of nonnested and nonlinear time series models for counts. We consider the most popular approaches used for model identification, Akaike’s information criterion and the Bayesian information criterion, and compare them using extensive Monte Carlo simulations. Furthermore, we investigate the properties of the fitted models (both the correct and wrong models) obtained using maximum likelihood estimation. A real-data example demonstrates our findings.

The control mechanisms of heart rate dynamics in a new heart rate nonlinear time series model

The control mechanisms and implications of heart rate variability (HRV) under the sympathetic (SNS) and parasympathetic nervous system (PNS) modulation remain poorly understood. Here, we establish the HR model/HRV responder using a nonlinear process derived from Newton’s second law in stochastic self-restoring systems through dynamic analysis of physiological properties. We conduct model validation by testing, predictions, simulations, and sensitivity and time-scale analysis. We confirm that the outputs of the HRV responder can be accepted as the real data-generating process. Empirical studies show that the dynamic control mechanism of heart rate is a stable fixed point, rather than a strange attractor or transitions between a fixed point and a limit cycle; HR slope (amplitude) may depend on the ratio of cardiac disturbance or metabolic demand mean (standard deviation) to myocardial electrical resistance (PNS-SNS activity). For example, when metabolic demands remain unchanged, HR amplitude depends on PNS to SNS activity; when autonomic activity remains unchanged, HR amplitude during resting reflects basal metabolism. HR parameter alterations suggest that age-related decreased HRV, ultrareduced HRV in heart failure, and ultraelevated HRV in ST segment alterations refer to age-related decreased basal metabolism, impaired myocardial metabolism, and SNS hyperactivity triggered by myocardial ischemia, respectively.

Forecasting energy consumption and wind power generation using deep echo state network

Accurate energy forecasting is of great significance for the energy sector to formulate short-term plans and long-term development strategies for meeting energy needs. This study develops a stacked hierarchy of reservoirs (DeepESN) for forecasting energy consumption and wind power generation by introducing the deep learning framework into the basic echo state network. DeepESN combines the powerful nonlinear time series modeling ability of echo state network and the efficient learning ability of the deep learning framework. Two comparative examples and an extended application are analyzed to validate the accuracy and reliability of DeepESN. These comparative examples reveal that DeepESN outperforms the existing popular models, persistence model, back-propagation neural network, and echo state network. Moreover, compared with echo state network, DeepESN shows 51.56%, 51.53%, and 35.43% improvements in terms of mean absolute error, root mean square error, and mean absolute percentage error in the extended application, respectively. Therefore, DeepESN is an appropriate tool for forecasting energy consumption and wind power generation on account of its effective and stable forecasting performance.

Nonlinear time-series modeling of feed drive system based on motion states classification

This paper proposed a novel modeling method using the running process data, i.e., the reference input positions and the actual output positions, based on the Naive Bayes method and a nonlinear autoregressive long-short term memory network (i.e., the NAR-LSTM) to address the nonlinear time-series modeling problem for increasing the prediction accuracy of the model of CNC machine feed drive system. A Naive Bayes based motion states classifier (i.e., the NB-MSC) is proposed to automatically classify the motion states with knowledge of dynamic characteristics of the feed drive system for constructing the submodels of different motion states (startup state, reverse state, etc.). In addition, a model based multi-objective optimization method is presented to extract samples for the training of the NB-MSC. Then by modifying the basic long-short term memory (LSTM) network, the NAR-LSTM network is proposed to construct those submodels. Compared to the existing modeling methods via dynamic analysis, the proposed method is better because it can achieve higher prediction accuracy on highly nonlinear motion states such as the reverse state. To validate the proposed methods, a set of experiments are conducted to prove the feasibility of the feed drive model as well as the advantages of the NB-MSC and the NAR-LSTM network in improving the modeling performance.

An autoregressive model based on the generalized hyperbolic distribution

AbstractWe define a nonlinear autoregressive time series model based on the generalized hyperbolic distribution in an attempt to model time series with non‐Gaussian features such as skewness and heavy tails. We show that the resulting process has a simple condition for stationarity and it is also ergodic. An empirical example with a forecasting experiment is presented to illustrate the features of the proposed model.

Time series central subspace with covariates and its application to forecasting pine sawtimber stumpage prices in the Southern United States

To model and forecast a monthly pine sawtimber (PST) stumpage price, $$y_t$$ , data collected across 11 southern states in the U.S., we adopt a new semiparametric approach where the first phase adopts a nonparametric method called “Time Series Central Subspace with Covariates” (TSCS-C) to extract sufficient information about $$y_t$$ through a univariate time series $$\{d_t\}$$ , which is a linear combination of a set of past values of $$y_t$$ and a high dimensional covariate vector $${{\mathbf {x}}}_t$$ of sale characteristics. Then, $$\{d_t\}$$ alone is used as the predictor series to build a parametric nonlinear time series model for $$y_t$$ . This yields a new semiparametric nonlinear time series model for $$y_t$$ . Assessment in terms of out-of-sample forecasts of monthly PST stumpage prices show that our semiparametric model with the covariate $${{\mathbf {x}}}_t$$ has the smallest average forecasting error compared to another semiparametric nonlinear time series model without $${{\mathbf {x}}}_t$$ and two other parametric counterparts based on multiplicative seasonal autoregressive integrated moving average models with and without $${{\mathbf {x}}}_t$$ . This data underscores the ability of our semiparametric approach to first reduce the dimensionality of $${{\mathbf {x}}}_t$$ and a set of past values of $$y_t$$ significantly using the TSCS-C nonparametric methodology and then to produce a superior nonlinear time series model.

Monthly streamflow prediction using a hybrid stochastic-deterministic approach for parsimonious non-linear time series modeling

Accurate streamflow prediction is essential in reservoir management, flood control, and operation of irrigation networks. In this study, the deterministic and stochastic components of modeling are considered simultaneously. Two nonlinear time series models are developed based on autoregressive conditional heteroscedasticity and self-exciting threshold autoregressive methods integrated with the gene expression programming. The data of four stations from four different rivers from 1971 to 2010 are investigated. For examining the reliability and accuracy of the proposed hybrid models, three evaluation criteria, namely the R2, RMSE, and MAE, and several visual plots were used. Performance comparison of the hybrid models revealed that the accuracy of the SETAR-type models in terms of R2 performed better than the ARCH-type models for Daryan (0.99), Germezigol (0.99), Ligvan (0.97), and Saeedabad (0.98) at the validation stage. Overall, prediction results showed that a combination of the SETAR with the GEP model performs better than ARCH-based GEP models for the prediction of the monthly streamflow. Abbreviations: ADF = Augmented Dickey-Fuller; AIC = Akaike Information Criterion; ANFIS = Adaptive Neuro-Fuzzy Inference System; ANNs = Artificial Neural Networks; AR = Autoregressive Models; ARIMA = Autoregressive Integrated Moving Average; ARCH = Autoregressive Conditional Heteroscedasticity; ATAR = Aggregation Operator Based TAR; BL = Bilinear Models; BNN = Bayesian Neural Network; CEEMD = Complete Ensemble Empirical Mode Decomposition; DDM =Data-Driven Model; GA = Genetic Algorithm; GARCH = Generalized Autoregressive Conditional Heteroscedasticity; GEP = Gene Expression Programming; KNN = K-Nearest Neighbors; KPSS = Kwiatkowski–Phillips–Schmidt–Shin; LMR = Linear and Multilinear Regressions; LR = Likelihood Ratio; LSTAR = Logistic STAR; MAE = Mean Absolute Error; PACF = Partial Autocorrelation Function; PARCH = Partial Autoregressive Conditional Heteroscedasticity; R 2 = Coefficient of Determination; RMSE = Root Mean Square Error; RNNs = Recurrent Neural Networks; SETARMA = Self-Exciting Threshold Autoregressive Moving Average; SETAR = Self-Exciting Threshold Autoregressive; STAR = Smooth Transition AR; SVR = Support Vector Regression; TAR = Threshold Autoregressive; TARMA = Threshold Autoregressive Moving Average; ULB = Urmia Lake Basin; VMD = Variational Mode Decomposition; WT = Wavelet Transforms

Investigation of the accuracy of linear and nonlinear time series models in modeling and forecasting of pan evaporation in IRAN

Water evaporation process is one of the main components of the hydrological cycle, according to which the correct estimation of this phenomenon plays an important role in irrigation management and river flow forecasting. Statistical and random models in the form of time series analysis are commonly used methods of estimation and forecasting. In this study, ARMA (autoregressive moving average), ARIMA (autoregressive integrated moving average), PARMA (periodic autoregressive moving average), and BL (bilinear) time series models (TSMs) are used to predict the annual and monthly pan evaporation values from the evaporation stations in several provinces of Iran in a 20-year statistical period. Results showed that PARMA model’s accuracy in term of correlation coefficient and Nash–Sutcliffe efficiency for almost all 31 stations had a precise forecasting among other proposed TSMs. For PARMA model, the forecasting accuracy in term of NSE indicated that PARMA model almost was the promising model among other linear and nonlinear TSMs in prediction of Epan for all stations, except Arak (NSE = 0.94) and Qom (NSE = 0.93). The ARIMA model for Khorramabad and Bandar Abbas with NSE = 0.52 had the unreliable prediction for Epan compared with other stations. In addition, Arak station in term of RMSE had the least error, 23.79 mm/day and 10.90 mm/day, respectively, for training and testing stages among the other stations.

Modeling of time series using random forests: Theoretical developments

In this paper we study asymptotic properties of random forests within the framework of nonlinear time series modeling. While random forests have been successfully applied in various fields, the theoretical justification has not been considered for their use in a time series setting. Under mild conditions, we prove a uniform concentration inequality for regression trees built on nonlinear autoregressive processes and, subsequently, we use this result to prove consistency for a large class of random forests. The results are supported by various simulations.

Study of nonlinear time series and wavelet power spectrum analysis using solar wind parameters and geomagnetic indices

ABSTRACT We investigate the time series of solar wind parameters (interplanetary magnetic field, Bz and solar wind speed, Vx) and geomagnetic indices (disturbance storm time, Dst and auroral electrojet, AE) using wavelet analysis and nonlinear dynamics time series techniques. The data were collected from the Flight Center Space Physics Data Facility (GSFC/SPDF) OMNIWEB interface between 2008 and 2017. Wavelet power spectrum (WPS) analysis assists in breaking down the time series of Bz, Vx, Dst and AE parameters into different scales. It was noted that there is a greater concentration of power between the 512 and 1024 months bands across the Bz, Vx, Dst and AE parameters. We also applied non-linear time series modelling methods to examine the Bz, Vx, Dst and AE parameters. We utilised both the time delay and embedded dimension in computing average mutual information (AMI) and false nearest neighbors (FNN), respectively. The Lyapunov exponent (LE) is used to express the complexity of the nonlinear dynamics based on embedding parameters. The Lyapunov exponents depict positive values which confirm that the complex solar wind parameters and the geomagnetic indices are deterministic chaotic systems. The results show noticeable chaotic characteristics in the Bz, Vx, Dst and AE parameters.