Asymptotic Stationarity Research Articles

Epicasting: An Ensemble Wavelet Neural Network for forecasting epidemics

Infectious diseases remain among the top contributors to human illness and death worldwide, among which many diseases produce epidemic waves of infection. The lack of specific drugs and ready-to-use vaccines to prevent most of these epidemics worsens the situation. These force public health officials and policymakers to rely on early warning systems generated by accurate and reliable epidemic forecasters. Accurate forecasts of epidemics can assist stakeholders in tailoring countermeasures, such as vaccination campaigns, staff scheduling, and resource allocation, to the situation at hand, which could translate to reductions in the impact of a disease. Unfortunately, most of these past epidemics exhibit nonlinear and non-stationary characteristics due to their spreading fluctuations based on seasonal-dependent variability and the nature of these epidemics. We analyze various epidemic time series datasets using a maximal overlap discrete wavelet transform (MODWT) based autoregressive neural network and call it Ensemble Wavelet Neural Network (EWNet) model. MODWT techniques effectively characterize non-stationary behavior and seasonal dependencies in the epidemic time series and improve the nonlinear forecasting scheme of the autoregressive neural network in the proposed ensemble wavelet network framework. From a nonlinear time series viewpoint, we explore the asymptotic stationarity of the proposed EWNet model to show the asymptotic behavior of the associated Markov Chain. We also theoretically investigate the effect of learning stability and the choice of hidden neurons in the proposal. From a practical perspective, we compare our proposed EWNet framework with twenty-two statistical, machine learning, and deep learning models for fifteen real-world epidemic datasets with three test horizons using four key performance indicators. Experimental results show that the proposed EWNet is highly competitive compared to the state-of-the-art epidemic forecasting methods.

Asymptotic regularity for Lipschitzian nonlinear optimization problems with applications to complementarity constrained and bilevel programming

Asymptotic stationarity and regularity conditions turned out to be quite useful to study the qualitative properties of numerical solution methods for standard nonlinear and complementarity constrained programs. In this paper, we first extend these notions to nonlinear optimization problems with nonsmooth but Lipschitzian data functions in order to find reasonable notions of asymptotic stationarity and regularity in terms of Clarke's and Mordukhovich's subdifferential construction. Particularly, we compare the associated novel asymptotic constraint qualifications with already existing ones. The second part of the paper presents two applications of the obtained theory. On the one hand, we specify our findings for complementarity constrained optimization problems and recover recent results from the literature which demonstrates the power of the approach. Furthermore, we hint at potential extensions to or- and vanishing constrained optimization. On the other hand, we demonstrate the usefulness of asymptotic regularity in the context of bilevel optimization. More precisely, we justify a well-known stationarity system for affinely constrained bilevel optimization problems in a novel way. Afterwards, we suggest a solution algorithm for this class of bilevel optimization problems which combines a penalty method with ideas from DC-programming. After a brief convergence analysis, we present the results of some numerical experiments.

Binary optimal control by trust-region steepest descent

We present a trust-region steepest descent method for dynamic optimal control problems with binary-valued integrable control functions. Our method interprets the control function as an indicator function of a measurable set and makes set-valued adjustments derived from the sublevel sets of a topological gradient function. By combining this type of update with a trust-region framework, we are able to show by theoretical argument that our method achieves asymptotic stationarity despite possible discretization errors and truncation errors during step determination. To demonstrate the practical applicability of our method, we solve two optimal control problems constrained by ordinary and partial differential equations, respectively, and one topological optimization problem.

Invariance Properties of Controlled Stochastic Nonlinear Systems Under Information Constraints

Given a stochastic nonlinear system controlled over a possibly noisy communication channel, the article studies the largest class of channels for which there exist coding and control policies so that the closed-loop system is stochastically stable. The stability criterion considered is asymptotic mean stationarity (AMS). In this article, we develop a general method based on ergodic theory and probability to derive fundamental bounds on information transmission requirements leading to stabilization. Through this method, we develop a new notion of entropy which is tailored to derive lower bounds for AMS for both noise-free and noisy channels. The bounds obtained through probabilistic and ergodic-theoretic analysis are more refined in comparison with the bounds obtained earlier via information-theoretic methods. Moreover, our approach is more versatile in view of the models considered and allows for finer lower bounds when the AMS measure is known to admit further properties, such as moment bounds.

Consistent online Gaussian process regression without the sample complexity bottleneck

Gaussian processes provide a framework for nonlinear nonparametric Bayesian inference widely applicable across science and engineering. Unfortunately, their computational burden scales cubically with the training sample size, which in the case that samples arrive in perpetuity, approaches infinity. This issue necessitates approximations for use with streaming data, which to date mostly lack convergence guarantees. Thus, we develop the first online Gaussian process approximation that preserves convergence to the population posterior, i.e., asymptotic posterior consistency, while ameliorating its intractable complexity growth with the sample size. We propose an online compression scheme that, following each a posteriori update, fixes an error neighborhood with respect to the Hellinger metric centered at the current posterior, and greedily tosses out past kernel dictionary elements until its boundary is hit. We call the resulting method Parsimonious Online Gaussian Processes (POG). For diminishing error radius, asymptotic statistical stationarity is achieved (Theorem 1ii) at the cost of unbounded memory in the limit. On the other hand, for constant error radius, POG converges to a neighborhood of stationarity (Theorem 1ii) but with finite memory at-worst determined by the metric entropy of the feature space (Theorem 2). Here stationarity refers to the distributional distance between sequential marginal posteriors approaching null with the time index. Experimental results are presented on several nonlinear regression problems which illuminates the merits of this approach as compared with alternatives that fix the subspace dimension defining the history of past points.

Memory-dependent noise-induced resonance and diffusion in non-Markovian systems.

We study random processes with nonlocal memory and obtain solutions of the Mori-Zwanzig equation describing non-Markovian systems. We analyze the system dynamics depending on the amplitudes ν and μ_{0} of the local and nonlocal memory and pay attention to the line in the (ν,μ_{0}) plane separating the regions with asymptotically stationary and nonstationary behavior. We obtain general equations for such boundaries and consider them for three examples of nonlocal memory functions. We show that there exist two types of boundaries with fundamentally different system dynamics. On the boundaries of the first type, diffusion with memory takes place, whereas on borderlines of the second type the phenomenon of noise-induced resonance can be observed. A distinctive feature of noise-induced resonance in the systems under consideration is that it occurs in the absence of an external regular periodic force. It takes place due to the presence of frequencies in the noise spectrum, which are close to the self-frequency of the system. We analyze also the variance of the process and compare its behavior for regions of asymptotic stationarity and nonstationarity, as well as for diffusive and noise-induced-resonance borderlines between them.

Asymptotic stationarity and regularity for nonsmooth optimization problems

Based on the tools of limiting variational analysis, we derive a sequential necessary optimality condition for nonsmooth mathematical programs which holds without any additional assumptions. In order to ensure that stationary points in this new sense are already Mordukhovich-stationary, the presence of a constraint qualification which we call AM-regularity is necessary. We investigate the relationship between AM-regularity and other constraint qualifications from nonsmooth optimization like metric (sub-)regularity of the underlying feasibility mapping. Our findings are applied to optimization problems with geometric and, particularly, disjunctive constraints. This way, it is shown that AM-regularity recovers recently introduced cone-continuity-type constraint qualifications, sometimes referred to as AKKT-regularity, from standard nonlinear and complementarity-constrained optimization. Finally, we discuss some consequences of AM-regularity for the limiting variational calculus.

Constrained Variable Projection Optimization for Stationary RBF-AR Models

Stationarity is fundamental for time-series modeling and prediction. In this article, we focus on the radial basis function network-based autoregressive (RBF-AR) models which have been widely used in practical applications. Compared to previous work, we give a less-restrictive sufficient condition for the asymptotic stationarity of the RBF-AR model. The parameter estimation of the RBF-AR model is converted to the optimization of a variable projection functional with constraints of stationarity to always derive a stationary model. The constrained evolutionary algorithm is used to solve the optimization problem. Numerical results demonstrate the effectiveness of the proposed method.

Unemployment Rate Forecasting: A Hybrid Approach

Unemployment has always been a very focused issue causing a nation as a whole to lose its economic and financial contribution. Unemployment rate prediction of a country is a crucial factor for the country’s economic and financial growth planning and a challenging job for policymakers. Traditional stochastic time series models, as well as modern nonlinear time series techniques, were employed for unemployment rate forecasting previously. These macroeconomic data sets are mostly nonstationary and nonlinear in nature. Thus, it is atypical to assume that an individual time series forecasting model can generate a white noise error. This paper proposes an integrated approach based on linear and nonlinear models that can predict the unemployment rates more accurately. The proposed hybrid model of the unemployment rate can improve their forecasts by reflecting the unemployment rate’s asymmetry. The model’s applications are shown using seven unemployment rate data sets from various countries, namely, Canada, Germany, Japan, Netherlands, New Zealand, Sweden, and Switzerland. The results of computational tests are very promising in comparison with other conventional methods. The results for asymptotic stationarity of the proposed hybrid approach using Markov chains and nonlinear time series analysis techniques are given in this paper which guarantees that the proposed model cannot show ‘explosive’ behavior or growing variance over time.

A Hybrid Model for European Unemployment Rate Forecasting and Its Asymptotic Behavior

Unemployment has always been a very focused issue causing the nation as a whole to lose its economic and financial contribution. Unemployment rate prediction of a country is a crucial factor for the country's economic and financial growth planning and a challenging job for policymakers. Traditional stochastic time series models, as well as modern nonlinear time series techniques, were employed for unemployment rate forecasting previously. These macroeconomic data sets are mostly nonstationary and nonlinear in nature. Thus, it is atypical to assume that an individual time series forecasting model can generate a white noise error. This paper proposes an integrated approach based on linear and nonlinear models that can predict the unemployment rates more accurately. The proposed hybrid model of the unemployment rate can improve their forecasts by refecting the unemployment rate's asymmetry. The model's applications are shown using seven unemployment rate data sets from various countries, namely, Canada, Germany, Japan, Netherlands, New Zealand, Sweden, and Switzerland. The results of computational tests are very promising in comparison with other conventional methods. The results for asymptotic stationarity of the proposed hybrid approach using Markov chains and nonlinear time series analysis techniques are given in this paper which guarantees that the proposed model cannot show `explosive' behavior or growing variance over time.

Capacity Achieving Distributions and Separation Principle for Feedback Gaussian Channels With Memory: the LQG Theory of Directed Information

A method is developed to realize optimal channel input conditional distributions, which maximize the finite transmission feedback information (FTFI) capacity, often called $n$ -block length feedback capacity, by information lossless randomized strategies. The method is applied to compute closed form expressions for the FTFI capacity and feedback capacity, of nonstationary, nonergodic, unstable, multiple input multiple output Gaussian channels with memory on past channel outputs, subject to average transmission cost constraints of quadratic form in the channel inputs and outputs. It is shown that randomized strategies decompose into two orthogonal parts-an deterministic part, which controls the channel output process, and an innovation part, which transmits new information over the channel. Then a separation principle is shown between the computation of the optimal deterministic part and the random part of the optimal randomized strategies. Finally, the ergodic theory of linear-quadratic-Gaussian stochastic optimal control theory, is applied to identify sufficient conditions, expressed in terms of solutions to matrix difference and algebraic Riccati equations, so that the optimal control part of randomized strategies induces asymptotic stationarity and ergodicity, and feedback capacity is characterized by the per unit time limit of the FTFI capacity. The method reveals an interaction of the control and the information transmission parts of the optimal randomized strategies, and that whether feedback increases capacity, is directly related to the channel parameters and the transmission cost function, through the solutions of the matrix Riccati equations. For unstable channels, it is shown that feedback capacity exists and it is strictly positive, provided the power exceeds a critical threshold.

On Stochastic Stability of a Class of non-Markovian Processes and Applications in Quantization

In many applications, the common assumption that a driving noise process affecting a system is independent or Markovian may not be realistic, but the noise process may be assumed to be stationary. To study such problems, this paper investigates stochastic stability properties of a class of non-Markovian processes, where the existence of a stationary measure, asymptotic mean stationarity and ergodicity conditions are studied. Applications in feedback quantization and stochastic control are presented.

Univariate Conditional Distributions of an Open-Loop TAR Stochastic Process

<p>Clusters of large values are observed in sample paths of certain open-loop threshold autoregressive (TAR) stochastic processes. In order to characterize the stochastic mechanism that generates this empirical stylized fact, three types of marginal conditional distributions of the underlying stochastic process are analyzed in this paper. One allows us to find the conditional variance function that explains the aforementioned stylized fact. As a by-product, we are able to derive a sufficient condition to have asymptotic weak stationarity in an open-loop TAR stochastic process.</p>

Stationary and Ergodic Properties of Stochastic NonLinear Systems Controlled over Communication Channels

This paper is concerned with the following problem: Given a stochastic nonlinear system controlled over a noisy channel, what is the largest class of channels for which there exist coding and control policies so that the closed-loop system is stochastically stable? Stochastic stability notions considered are stationarity, ergodicity or asymptotic mean stationarity. We do not restrict the state space to be compact, for example, systems considered can be driven by unbounded noise. Necessary and sufficient conditions are obtained for a large class of systems and channels. A generalization of Bode's integral formula for a large class of nonlinear systems and information channels is obtained. The findings generalize existing results for linear systems.

On the tightness of linear policies for stabilization of linear systems over Gaussian networks

In this paper, we consider stabilization of multi-dimensional linear systems driven by Gaussian noise controlled over parallel Gaussian channels. For such systems, it has been recognized that for stabilization in the sense of asymptotic stationarity or stability in probability, Shannon capacity of a channel is an appropriate measure on characterizing whether a system can be made stable when controlled over the channel. However, this is in general not the case for quadratic stabilization. On a related problem of joint-source channel coding, in the information theory literature, the source-channel matching principle has been shown to lead to optimality of uncoded or analog transmission and when such matching conditions occur, it has been shown that capacity is also a relevant figure of merit for quadratic stabilization. A special case of this result is applicable to a scalar LQG system controlled over a scalar Gaussian channel. In this paper, we show that even in the absence of source-channel matching, to achieve quadratic stability, it may suffice that information capacity (in Shannon’s sense) is greater than the sum of the logarithm of unstable eigenvalue magnitudes. In particular, we show that periodic linear time varying coding policies are optimal in the sense of obtaining a finite second moment for the state of the system with minimum transmit power requirements for a large class of vector Gaussian channels. Our findings also extend the literature which has considered noise-free systems.

Asymptotic mean stationarity and absolute continuity of point process distributions

This paper relates - for point processes $\Phi$ on $\mathbb{R}$ - two types of asymptotic mean stationarity (AMS) properties and several absolute continuity results for the common probability measures emerging from point process theory. It is proven that $\Phi$ is AMS under the time-shifts if and only if it is AMS under the event-shifts. The consequences for the accompanying two types of ergodic theorem are considered. Furthermore, the AMS properties are equivalent or closely related to several absolute continuity results. Thus, the class of AMS point processes is characterized in several ways. Many results from stationary point process theory are generalized for AMS point processes. To obtain these results, we first use Campbell's equation to rewrite the well-known Palm relationship for general nonstationary point processes into expressions which resemble results from stationary point process theory.

Weak consistency of estimators in linear regression model

ABSTRACTA linear regression model and M-estimator of its regression coefficients are considered. We present a derivation of a weak consistency of the M-estimator together with a rate. Derivation is made under general conditions set on the error term, say “asymptotic stationarity” property. The results are proved by means of L2-convergence and cover the cases as the error term is ARMA, ARCH, GARCH process or it is attracted by an ARMA, ARCH, GARCH process. We do not separate random and deterministic covariates. Both cases are treated in one general setting.

Characterization of Information Channels for Asymptotic Mean Stationarity and Stochastic Stability of Nonstationary/Unstable Linear Systems

Stabilization of nonstationary linear systems over noisy communication channels is considered. Stochastically stable sources, and unstable but noise-free or bounded-noise systems have been extensively studied in the information theory and control theory literature since the 1970s, with a renewed interest in the past decade. There have also been studies on noncausal and causal coding of unstable/nonstationary linear Gaussian sources. In this paper, tight necessary and sufficient conditions for stochastic stabilizability of unstable (nonstationary) possibly multidimensional linear systems driven by Gaussian noise over discrete channels (possibly with memory and feedback) are presented. Stochastic stability notions include recurrence, asymptotic mean stationarity and sample path ergodicity, and the existence of finite second moments. Our constructive proof uses random-time state-dependent stochastic drift criteria for stabilization of Markov chains. For asymptotic mean stationarity (and thus sample path ergodicity), it is sufficient that the capacity of a channel is (strictly) greater than the sum of the logarithms of the unstable pole magnitudes for memoryless channels and a class of channels with memory. This condition is also necessary under a mild technical condition. Sufficient conditions for the existence of finite average second moments for such systems driven by unbounded noise are provided.

Equilibration implies asymptotic stationarity for open quantum systems

We use the exact Nakajima–Zwanzig form of the master equation to show that open quantum systems which exhibit equilibration (or thermalization) by evolving to a time independent asymptotic state, have under certain conditions a reduced density matrix for the system which commutes with the effective system Hamiltonian. We also show that if the initial system–bath density matrix is of product form then the asymptotic reduced density matrix of the system depends only on the diagonal elements of the initial system density matrix in the eigenbasis of the effective Hamiltonian.

Equilibration and asymptotic stationarity for non-Markovian master equations

We show that certain positivity-preserving non-Markovian generalizations of the Kossakowski–Lindblad master equation can exhibit equilibration to an asymptotic state which is stationary with respect to the shifted system Hamiltonian for general system–bath coupling. This is in sharp contrast to results for Markovian forms which require strong relations between these operators (e.g. commutation of isolated system Hamiltonian and coupling operator). We also expand the list of sufficient conditions for positivity of non-Markovian master equations.