Nonstationary Regression Research Articles

Shrinkage Estimation and Forecasting in Dynamic Regression Models Under Structural Instability

Abstract This paper introduces a Stein-like shrinkage method for estimating slope coefficients and forecasting in first order dynamic regression models under structural breaks. The model allows for unit root and non-stationary regressors. The proposed shrinkage estimator is a weighted average of a restricted estimator that ignores the break in the slope coefficients, and an unrestricted estimator that uses the observations within each regime. The restricted estimator is the most efficient estimator but inconsistent when there is a break. However, the unrestricted estimator is consistent but not efficient. Therefore, the proposed shrinkage estimator balances the trade-off between the bias and variance efficiency of the restricted estimator. The averaging weight is proportional to the weighted distance of the restricted estimator, and the unrestricted estimator. We derive the analytical large-sample approximation of the bias, mean squared error, and risk for the shrinkage estimator, the unrestricted estimator, and the restricted estimator. We show that the risk of the shrinkage estimator is lower than the risk of the unrestricted estimator under any break size and break points. Moreover, we extend the results for the model with a unit root and non-stationary regressors. We evaluate the finite sample performance of our proposed method via extensive simulation study, and empirically in forecasting output growth.

Robust inference with stochastic local unit root regressors in predictive regressions

This paper explores predictive regression models with stochastic unit root (STUR) components and robust inference procedures that encompass a wide class of persistent and time-varying stochastically nonstationary regressors. The paper extends the mechanism of endogenously generated instrumentation known as IVX, showing that these methods remain valid for short and long-horizon predictive regressions in which the predictors have STUR and local STUR (LSTUR) generating mechanisms. Both mean regression and quantile regression methods are considered. The asymptotic distributions of the IVX estimators are new and require some new methods in their derivation. The distributions are compared to previous results and, as in earlier work, lead to pivotal limit distributions for Wald testing procedures that remain robust for both single and multiple regressors with various degrees of persistence and stochastic and fixed local departures from unit roots. Numerical experiments corroborate the asymptotic theory, and IVX testing shows good power and size control. The IVX methods are illustrated in an empirical application to evaluate the predictive capability of economic fundamentals in forecasting S&P 500 excess returns.

Estimation and Inference for the Threshold Model with Hybrid Stochastic Local Unit Root Regressors

In this paper, we study the estimation and inference of the threshold model with hybrid local stochastic unit root regressors. Our main contribution is to propose an estimator that generalizes the threshold model with various forms of nonstationary regressors and to obtain its limiting distribution theory. In particular, our proposed model generalizes the threshold model with unit root, local-to-unity, and stochastic unit root regressors. We provide the estimation strategy for the least squares estimator and derive the asymptotic results for the proposed estimator. Depending on the diminishing rate of the threshold effect, we find that the limiting distribution of the threshold estimator takes different forms. Monte Carlo simulations are used to assess our proposed estimator’s finite sample performance, which is found to perform well.

Augmented cointegrating linear models with possibly strongly correlated stationary and nonstationary regressors

This paper proposes a class of augmented cointegrating linear (ACL) models that accommodate both stationary and nonstationary variables in a single equation, where regressors may be strongly correlated defined below. Moreover, the limit theory of estimators proposed is established via joint convergence of the sample variance and covariance that eschews some existing drawback in the literature; to facilitate inference, a self-normalized central limit theorem is given as well. Furthermore, numerical simulations confirm the theoretical results, and for the US’s GDP series we show that ACL model outperforms some competing models.

Continuous Adaptation with Online Meta-Learning for Non-Stationary Target Regression Tasks

Most environments change over time. Being able to adapt to such non-stationary environments is vital for real-world applications of many machine learning algorithms. In this work, we propose CORAL, a computationally efficient regression algorithm capable of adapting to a non-stationary target. CORAL is based on Bayesian linear regression with a sliding window and offline/online meta-learning. The sliding window makes our model focus on the recently received data and ignores older observations. The meta-learning approach allows us to learn the prior distribution of the model parameters. It speeds up the model adaptation, complements the sliding window’s drawback, and enhances the performance. We evaluate CORAL on two tasks: a toy problem and a more complex blood glucose level prediction task. Our approach improves the prediction accuracy for the non-stationary target significantly while also performing well for the stationary target. We show that the two components of our method work in a complementary fashion to achieve this.

Estimation of nonstationary nonparametric regression model with multiplicative structure

Summary This paper presents a multiplicative nonstationary nonparametric regression model which allows for a broad class of nonstationary processes. We propose a three-step estimation procedure to uncover the conditional mean function and establish uniform convergence rates and asymptotic normality of our estimators. The new model can also be seen as a dimension-reduction technique for a general two-dimensional time-varying nonparametric regression model, which is especially useful in small samples and for estimating explicitly multiplicative structural models. We consider two applications: estimating a pricing equation for the US aggregate economy to model consumption growth and estimating the shape of the monthly risk premium for S&P 500 Index data.

Spurious functional-coefficient regression models and robust inference with marginal integration

Functional-coefficient cointegrating models have become popular to model nonlinear nonstationarity in econometrics (Cai et al., 2009; Xiao, 2009). However, there is rare study on testing the existence of functional-coefficient cointegration. Consequently, functional-coefficient regressions involving nonstationary regressors may be spurious. This paper investigates the effect that spurious functional-coefficient regression has on the model diagnostics. We find that common characteristics of spurious regressions are manifest, including divergent local significance tests, random local goodness-of-fit, and local Durbin–Watson ratio converging to zero, complementing those discovered in spurious linear and nonparametric regressions (Phillips, 1986, 2009). In addition, spuriousness causes the divergences of the global significance tests proposed by Xiao (2009) and Sun et al. (2016), which are likely to produce misleading conclusions for practitioners when cointegration tests fail to reject a spurious regression. To resolve the problems, we propose a simple-to-implement inference procedure based on a semiparametric balanced regression, by augmenting regressors of the original spurious regression with lagged dependent variable and independent variables, with the aid of the marginal integration. This procedure achieves spurious regression detection via standard nonparametric inferential asymptotics, and is found robust to the true relationship between the integrated processes. The theoretical results are also corroborated by simulations.

Unifying inference for semiparametric regression

Summary In the literature, a discrepancy in the limiting distributions of least square estimators between the stationary and nonstationary cases exists in various regression models with different persistence level regressors. This hinders further statistical inference since one has to decide which distribution should be used next. In this paper, we develop a semiparametric partially linear regression model with stationary and nonstationary regressors to attenuate this difficulty, and propose a unifying inference procedure for the coefficients. To be specific, we propose a profile weighted estimation equation method that facilitates the unifying inference. The proposed method is applied to the predictive regressions of stock returns, and an empirical likelihood procedure is developed to test the predictability. It is shown that the Wilks theorem holds for the empirical likelihood ratio regardless of predictors being stationary or not, which provides a unifying method for constructing confidence regions of the coefficients of state variables. Simulations show that the proposed method works well and has favourable finite sample performance over some existing approaches. An empirical application examining the predictability of equity returns highlights the value of our methodology.

Testing heteroskedasticity for predictive regressions with nonstationary regressors

This paper proposes the Cramér–von Mises type test statistic for testing heteroskedasticity in predictive regression when regressors are nonstationary. A Monte Carlo simulation study is conducted to illustrate the finite sample performance and a real empirical example is examined.

INSTRUMENTAL VARIABLE ESTIMATION OF STRUCTURAL VAR MODELS ROBUST TO POSSIBLE NONSTATIONARITY

This paper considers the estimation of dynamic causal effects using a proxy structural vector-autoregressive model with possibly nonstationary regressors. We provide general conditions under which the asymptotic normal approximation remains valid. In this case, the asymptotic variance depends on the persistence property of each series. We further provide a consistent asymptotic covariance matrix estimator that requires neither knowledge of the presistence properties of the variables nor pretests for nonstationarity. The proposed consistent covariance matrix estimator is robust and is easy to implement in practice. When all regressors are indeed stationary, the method becomes the same as the standard procedure.

Augmented cointegrating linear models with possibly strongly correlated stationary and nonstationary regressors regressors

Augmented cointegrating linear models with possibly strongly correlated stationary and nonstationary regressors regressors

Corporate bonds, exchange rates and business strategy

We examine the relationship between corporate bonds and exchange rates in the USA and Greece highlighting asymmetries and volatility among markets. A theoretical framework is constructed to treat the issue of corporate bonds. The employed methodology concerns the QARDL-ECM that is applied to non-stationary regressors. In Greece, our key findings point that changes in corporate bonds are mainly driven by changes in exchange rates. The quantile estimates show that the strength of long-run relationship increases as the risk of default for the country diminishes. In the USA, both long-run and short-run relationships between variables are defined. The changes in exchange rates are mainly driven by changes in corporate bonds. The quantile estimates exhibit a non-symmetrical pattern of the relationship which is a clear evidence of possible financing problems in enterprises. A key contribution concerns the suggestion of a sound policy for the enterprises for attaining stable growth in an environment of financial stress and asymmetry.

OPTIMAL BANDWIDTH SELECTION IN NONLINEAR COINTEGRATING REGRESSION

We study optimal bandwidth selection in nonparametric cointegrating regression where the regressor is a stochastic trend process driven by short or long memory innovations. Unlike stationary regression, the optimal bandwidth is found to be a random sequence which depends on the sojourn time of the process. All random sequences $h_{n}$ that lie within a wide band of rates as the sample size $n\rightarrow \infty $ have the property that local level and local linear kernel estimates are asymptotically normal, which enables inference and conveniently corresponds to limit theory in the stationary regression case. This finding reinforces the distinctive flexibility of data-based nonparametric regression procedures for nonstationary nonparametric regression. The present results are obtained under exogenous regressor conditions, which are restrictive but which enable flexible data-based methods of practical implementation in nonparametric predictive regressions within that environment.

Hierarchical Gaussian Process Models for Improved Metamodeling

Simulations are often used for the design of complex systems as they allow one to explore the design space without the need to build several prototypes. Over the years, the simulation accuracy, as well as the associated computational cost, has increased significantly, limiting the overall number of simulations during the design process. Therefore, metamodeling aims to approximate the simulation response with a cheap to evaluate mathematical approximation, learned from a limited set of simulator evaluations. Kernel-based methods using stationary kernels are nowadays widely used. In many problems, the smoothness of the function varies in space, which we call nonstationary behavior [20]. However, using stationary kernels for nonstationary responses can be inappropriate and result in poor models when combined with sequential design. We present the application of two recent techniques: Deep Gaussian Processes and Gaussian Processes with nonstationary kernel, which are better able to cope with these difficulties. We evaluate the method for nonstationary regression on a series of real-world problems, showing that these recent approaches outperform the standard Gaussian Processes with stationary kernels. Results show that these techniques are suitable for the simulation community, and we outline the variational inference method for the Gaussian Process with nonstationary kernel.

Nonstationary panel models with latent group structures and cross-section dependence

This paper proposes a novel Lasso-based approach to handle unobserved parameter heterogeneity and cross-section dependence in nonstationary panel models. In particular, a penalized principal component (PPC) method is developed to estimate group-specific long-run relationships and unobserved common factors and jointly to identify the unknown group membership. The PPC estimators are shown to be consistent under weakly dependent innovation processes. But they suffer an asymptotically non-negligible bias from correlations between the nonstationary regressors and unobserved stationary common factors and/or the equation errors. To remedy these shortcomings we provide three bias-correction procedures under which the estimators are re-centered about zero as both dimensions (N and T) of the panel tend to infinity. We establish a mixed normal limit theory for the estimators of the group-specific long-run coefficients, which permits inference using standard test statistics. Simulations suggest good finite sample performance. An empirical application applies the methodology to study international R&D spillovers and the results offer a convincing explanation for the growth convergence puzzle through the heterogeneous impact of R&D spillovers.

Small-sample tests for stock return predictability with possibly non-stationary regressors and GARCH-type effects

We develop a simulation-based procedure to test for stock return predictability with multiple regressors. The process governing the regressors is left completely free and the test procedure remains valid in small samples even in the presence of non-normalities and GARCH-type effects in the stock returns. The usefulness of the new procedure is demonstrated in a simulation study and by examining the ability of a group of financial variables to predict excess stock returns. We find some evidence of predictability during the period 1948–2014, driven entirely by the term spread. This empirical evidence, however, is much weaker over subsamples.

LATENT VARIABLE NONPARAMETRIC COINTEGRATING REGRESSION

This article studies the asymptotic properties of empirical nonparametric regressions that partially misspecify the relationships between nonstationary variables. In particular, we analyze nonparametric kernel regressions in which a potential nonlinear cointegrating regression is misspecified through the use of a proxy regressor in place of the true regressor. Such models occur in linear and nonlinear regressions where the regressor suffers from measurement error or where the true regressor is a latent or filtered variable as in mixed-data-sampling. The treatment allows for endogenous regressors as the latent variable and proxy variables that cointegrate asymptotically with the true latent variable, including correctly specified as well as misspecified systems, and is therefore intermediate between nonlinear nonparametric cointegrating regression and completely spurious nonparametric nonstationary regression. The results relate to recent work on dynamic misspecification in nonparametric nonstationary systems and the limit theory accommodates regressor variables with autoregressive roots that are local to unity and whose errors are driven by long memory and short memory innovations, thereby encompassing applications with a wide range of economic and financial time series. Some implications for forecasting under misspecification are also examined.

Estimation for double-nonlinear cointegration

In recent years statistical inference for nonlinear cointegration has attracted attention from both academics and practitioners. This paper proposes a new type of cointegration in the sense that two univariate time series yt and xt are cointegrated via two (unknown) smooth nonlinear transformations, further generalizing the notion of cointegration initially revealed by Box and Tiao (1977), and more systematically studied by Engle and Granger (1987). More precisely, it holds that G(yt,β0)=g(xt)+ut, where G(⋅,β0) is strictly increasing and known up to an unknown parameter β0, g(⋅) is unknown and smooth, xt is I(1), and ut is the stationary disturbance. This setting nests the nonlinear cointegration model of Wang and Phillips (2009b) as a special case with G(y,β0)=y. It extends the model of Linton et al. (2008) to the cases with a unit-root nonstationary regressor. Sieve approximations to the smooth nonparametric function g are applied, leading to an extremum estimator for β and a plugging-in estimator for g(⋅). Asymptotic properties of the estimators are established, revealing that both the convergence rates and the limiting distributions depend intimately on the properties of the two nonlinear transformation functions. Simulation studies demonstrate that the estimators perform well even with small samples. A real data example on the environmental Kuznets curve portraying the nonlinear impact of per-capita GDP on air-pollution illustrates the practical relevance of the proposed double-nonlinear cointegration.

Inference Under Random Limit Bootstrap Measures

Asymptotic bootstrap validity is usually understood as consistency of the distribution of a bootstrap statistic, conditional on the data, for the unconditional limit distribution of a statistic of interest. From this perspective, randomness of the limit bootstrap measure is regarded as a failure of the bootstrap. We show that such limiting randomness does not necessarily invalidate bootstrap inference if validity is understood as control over the frequency of correct inferences in large samples. We first establish sufficient conditions for asymptotic bootstrap validity in cases where the unconditional limit distribution of a statistic can be obtained by averaging a (random) limiting bootstrap distribution. Further, we provide results ensuring the asymptotic validity of the bootstrap as a tool for conditional inference, the leading case being that where a bootstrap distribution estimates consistently a conditional (and thus, random) limit distribution of a statistic. We apply our framework to several inference problems in econometrics, including linear models with possibly nonstationary regressors, CUSUM statistics, conditional Kolmogorov–Smirnov specification tests and tests for constancy of parameters in dynamic econometric models.

Kernel-based Inference in Time-Varying Coefficient Cointegrating Regression

This paper studies nonlinear cointegrating models with time-varying coefficients and multiple nonstationary regressors using classic kernel smoothing methods to estimate the coefficient functions. Extending earlier work on nonstationary kernel regression to take account of practical features of the data, we allow the regressors to be cointegrated and to embody a mixture of stochastic and deterministic trends, complications which result in asymptotic degeneracy of the kernel-weighted signal matrix. To address these complications new local and global rotation techniques are introduced to transform the covariate space to accommodate multiple scenarios of induced degeneracy. Under regularity conditions we derive asymptotic results that differ substantially from existing kernel regression asymptotics, leading to new limit theory under multiple convergence rates. For the practically important case of endogenous nonstationary regressors we propose a fully-modified kernel estimator whose limit distribution theory corresponds to the prototypical pure cointegration case (i.e., with exogenous covariates), thereby facilitating inference using a generalized Wald-type test statistic. These results substantially generalize econometric estimation and testing techniques in the cointegration literature to accommodate time variation and complications of co-moving regressors. Finally, Monte-Carlo simulation studies as well as an empirical illustration to aggregate US data on consumption, income, and interest rates are provided to illustrate the methodology and evaluate the numerical performance of the proposed methods in finite samples.