General State Space Research Articles

Ancestral Lineages and Limit Theorems for Branching Markov Chains in Varying Environment

We consider branching processes in discrete time for structured population in varying environment. Each individual has a trait which belongs to some general state space and both its reproduction law and the trait inherited by its offsprings may depend on its trait and the environment. We study the long-time behavior of the population and the ancestral lineage of typical individuals under general assumptions. We focus on the mean growth rate and the trait distribution among the population. The approach relies on many-to-one formulae and the analysis of the genealogy, and a key role is played by well-chosen (possibly non-homogeneous) Markov chains. The applications use large deviations principles and the Harris ergodicity for these auxiliary Markov chains.

Approximation of Discounted Minimax Markov Control Problems and Zero-Sum Markov Games Using Hausdorff and Wasserstein Distances

This paper is concerned with a minimax control problem (also known as a robust Markov decision process (MDP) or a game against nature) with general state and action spaces under the discounted cost optimality criterion. We are interested in approximating numerically the value function and an optimal strategy of this general discounted minimax control problem. To this end, we derive structural Lipschitz continuity properties of the solution of this robust MDP by imposing suitable conditions on the model, including Lipschitz continuity of the elements of the model and absolute continuity of the Markov transition kernel with respect to some probability measure $$\mu $$ . Then, we are able to provide an approximating minimax control model with finite state and action spaces, and hence computationally tractable, by combining these structural properties with a suitable discretization procedure of the state space (related to a probabilistic criterion) and the action spaces (associated to a geometric criterion). Finally, it is shown that the corresponding approximation errors for the value function and the optimal strategy can be controlled in terms of the discretization parameters. These results are also extended to a two-player zero-sum Markov game.

MEXIT: Maximal un-coupling times for stochastic processes

Classical coupling constructions arrange for copies of the same Markov process started at two different initial states to become equal as soon as possible. In this paper, we consider an alternative coupling framework in which one seeks to arrange for two different Markov (or other stochastic) processes to remain equal for as long as possible, when started in the same state. We refer to this “un-coupling” or “maximal agreement” construction as MEXIT, standing for “maximal exit”. After highlighting the importance of un-coupling arguments in a few key statistical and probabilistic settings, we develop an explicit MEXIT construction for stochastic processes in discrete time with countable state-space. This construction is generalized to random processes on general state-space running in continuous time, and then exemplified by discussion of MEXIT for Brownian motions with two different constant drifts.

Dynamic modeling, control design and stability analysis of railway active power quality conditioner

Two-phase three-wire converter has been proposed to compensate the power quality issues in AC electrified railway systems. In this paper, a step-by-step modeling procedure is presented and applied to the compensator to obtain a linear model. The model is then used in a linear control system design and stability analysis of the compensator. The first step is to obtain the bi-linear form of the exact (switched) model of the compensator representing the high and low frequency behavior of the converter. Then, the generalized state space averaging is applied to the bi-linear model to obtain the continuous time model of the compensator. The linear small signal model is derived by differentiating the averaged model around its equilibrium point. Linear control system design is established based on the linear small-signal model to improve DC-link voltage performance. The simulated model of compensator is verified by quantitatively comparing to the previously reported experimental results. The verified simulation model is used to validate the obtained models. Finally, the open loop stability analysis is performed by studying the locus of system eigenvalues with respect to the circuit parameters variations.

Combinatorial Lévy processes

Combinatorial Lévy processes evolve on general state spaces of combinatorial structures, of which standard examples include processes on sets, graphs and $n$-ary relations and more general possibilities are given by processes on graphs with community structure and multilayer networks. In this setting, the usual Lévy process properties of stationary, independent increments are defined in an unconventional way in terms of the symmetric difference operation on sets. The main theorems characterize both finite and infinite state space combinatorial Lévy processes by a unique $\sigma$-finite measure. Under the additional assumption of exchangeability, I prove a more explicit characterization by which every exchangeable combinatorial Lévy process corresponds to a Poisson point process on the same state space. Associated behavior of the projection into a space of limiting objects reflects certain structural features of the covering process.

Reduction of total-cost and average-cost MDPs with weakly continuous transition probabilities to discounted MDPs

This note describes sufficient conditions under which total-cost and average-cost Markov decision processes (MDPs) with general state and action spaces, and with weakly continuous transition probabilities, can be reduced to discounted MDPs. For undiscounted problems, these reductions imply the validity of optimality equations and the existence of stationary optimal policies. The reductions also provide methods for computing optimal policies. The results are applied to a capacitated inventory control problem with fixed costs and lost sales.

Empirical evolution equations

Evolution equations comprise a broad framework for describing the dynamics of a system in a general state space: when the state space is finite-dimensional, they give rise to systems of ordinary differential equations; for infinite-dimensional state spaces, they give rise to partial differential equations. Several modern statistical and machine learning methods concern the estimation of objects that can be formalized as solutions to evolution equations, in some appropriate state space, even if not stated as such. The corresponding equations, however, are seldom known exactly, and are empirically derived from data, often by means of non-parametric estimation. This induces uncertainties on the equations and their solutions that are challenging to quantify, and moreover the diversity and the specifics of each particular setting may obscure the path for a general approach. In this paper, we address the problem of constructing general yet tractable methods for quantifying such uncertainties, by means of asymptotic theory combined with bootstrap methodology. We demonstrates these procedures in important examples including gradient line estimation, diffusion tensor imaging tractography, and local principal component analysis. The bootstrap perspective is particularly appealing as it circumvents the need to simulate from stochastic (partial) differential equations that depend on (infinite-dimensional) unknowns. We assess the performance of the bootstrap procedure via simulations and find that it demonstrates good finite-sample coverage.

Large Signal Modeling Method for AC/DC Independent Power System in dq-Coordinates

Large signal stability means that a system can be a transition to a new stable state without losing synchronism after a large disturbance. The large signal stability problem brought by large capacity electrical loads in ac/dc independent power system is prominent, which seriously affects the dynamic security of the system. In view of modeling characteristics of ac/dc system, a generalized state space average large signal modeling method for the ac/dc independent power system in dq-coordinates is proposed. A large signal model of the ac/dc independent power system is constructed, which includes generators, inverter units, rectifying units, and dc/dc converters. The simulation results indicate that the large signal model constructed in this paper can accurately reflect the dynamic characteristics of the system under large disturbances, such as startup and sudden change in load, which is helpful to understand and improve the dynamic performance of the system under large disturbance. Furthermore, with feasibility and effectiveness, the large signal model provides convenience and possibility for the theoretical analysis of large signal stability of ac/dc independent power systems in the future.

Particle smoothing via Markov chain Monte Carlo in general state space models

Sequential Monte Carlo (SMC) methods (also known as particle filter) provide a way to solve the state estimation problem in nonlinear non-Gaussian state space models (SSM) through numerical approximation. Particle smoothing is one retrospective state estimation method based on particle filtering. In this paper, we propose a new particle smoother. The basic idea is easy and leads to a forward-backward procedure, where the Metropolis-Hastings algorithm is used to resample the filtering particles. The goodness of the new scheme is assessed using a nonlinear SSM. It is concluded that this new particle smoother is suitable for state estimation in complicated dynamical systems.

Particle smoothing via Markov chain Monte Carlo in general state space models

Particle smoothing via Markov chain Monte Carlo in general state space models

A benchmark study of the Signed-particle Monte Carlo algorithm for the Wigner equation

Abstract The Wigner equation represents a promising model for the simulation of electronic nanodevices, which allows the comprehension and prediction of quantum mechanical phenomena in terms of quasi-distribution functions. During these years, a Monte Carlo technique for the solution of this kinetic equation has been developed, based on the generation and annihilation of signed particles. This technique can be deeply understood in terms of the theory of pure jump processes with a general state space, producing a class of stochastic algorithms. One of these algorithms has been validated successfully by numerical experiments on a benchmark test case.

Perturbation Bounds for Markov Chains with General State Space

The aim of this paper is to investigate the stability of Markov chains with general state space. We present new conditions for the strong stability of Markov chains after a small perturbation of their transition kernels. Also, we obtain perturbation bounds with respect to different quantities.

Automatic selection of unobserved components models for supply chain forecasting

For many companies, automatic forecasting has come to be an essential part of business analytics applications. The large amounts of data available, the short life-cycle of the analysis and the acceleration of business operations make traditional manual data analysis unfeasible in such environments. In this paper, an automatic forecasting support system that comprises several methods and models is developed in a general state space framework built in the SSpace toolbox written for Matlab. Some of the models included are well-known, such as exponential smoothing and ARIMA, but we also propose a new model family that has been used only very rarely in this context, namely unobserved components models. Additional novelties include the use of unobserved components models in an automatic identification environment and the comparison of their forecasting performances with those of exponential smoothing and ARIMA models estimated using different software packages. The new system is tested empirically on a daily dataset of all of the products sold by a franchise chain in Spain (166 products over a period of 517 days). The system works well in practice and the proposed automatic unobserved components models compare very favorably with other methods and other well-known software packages in forecasting terms.

Verifiable conditions for the irreducibility and aperiodicity of Markov chains by analyzing underlying deterministic models

We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1}=F(\Phi_{k},\alpha(\Phi_{k},U_{k+1}))$ where the function $F$ is $C^{1}$ while $\alpha$ is typically discontinuous and $\{U_{k}:k\in\mathbb{Z}_{>0}\}$ is an independent and identically distributed process. We assume that for all $x$, the random variable $\alpha(x,U_{1})$ admits a density $p_{x}$ such that $(x,w)\mapsto p_{x}(w)$ is lower semi-continuous. We generalize and extend previous results that connect properties of the underlying deterministic control model to provide conditions for the chain to be $\varphi$-irreducible and aperiodic. By building on those results, we show that if a rank condition on the controllability matrix is satisfied for all $x$, there is equivalence between the existence of a globally attracting state for the control model and $\varphi$-irreducibility of the Markov chain. Additionally, under the same rank condition on the controllability matrix, we prove that there is equivalence between the existence of a steadily attracting state and the $\varphi$-irreducibility and aperiodicity of the chain. The notion of steadily attracting state is new. We additionally derive practical conditions by showing that the rank condition on the controllability matrix needs to be verified only at a globally attracting state (resp. steadily attracting state) for the chain to be a $\varphi$-irreducible $T$-chain (resp. $\varphi$-irreducible aperiodic $T$-chain). Those results hold under considerably weaker assumptions on the model than previous ones that would require $(x,u)\mapsto F(x,\alpha(x,u))$ to be $C^{\infty}$ (while it can be discontinuous here). Additionally the establishment of a necessary and sufficient condition on the control model for the $\varphi$-irreducibility and aperiodicity without a structural assumption on the control set is novel – even for Markov chains where $(x,u)\mapsto F(x,\alpha(x,u))$ is $C^{\infty}$. We illustrate that the conditions are easy to verify on a non-trivial and non-artificial example of Markov chain arising in the context of adaptive stochastic search algorithms to optimize continuous functions in a black-box scenario.

Modeling and Analysis of 12-Pulse Inverter in Shipboard or Aircraft

Abstract With the development of DC distribution system within the isolated power system of a ship or an aircraft, more constant frequency loads will be supplied by inverters connected to DC main bus. In the operating mode conversion process of an isolated power system, inverters will inevitably suffer from serious disturbance and affect the stability of the system. Therefore, it is important to establish a model of the inverter that reflects its dynamic characteristics and based on which to conduct the stability analysis. This paper proposes a 12-pulse inverter model based on the generalized state space averaging (GSSA) method. This model can overcome the limitations of 12-pulse inverter state space averaging (SSA) model in transient analysis with good accuracy and fast analysis ability effectively. Three kinds of models for a 12-pulse aircraft inverter are built in MATLAB, namely GSSA model, SSA model and detail device model. The simulation results show the high accuracy of GSSA model in stability analysis. This study provides an effective analytical tool for stability analysis of 12-pulse inverter and also provides a reference for inverter modeling research of isolated power system such as in aircraft or ship.

On uniform closeness of local times of Markov chains and i.i.d. sequences

In this paper we consider the field of local times of a discrete-time Markov chain on a general state space, and obtain uniform (in time) upper bounds on the total variation distance between this field and the one of a sequence of n i.i.d. random variables with law given by the invariant measure of that Markov chain. The proof of this result uses a refinement of the soft local time method of Popov and Teixeira (2015).

Efficient Monte Carlo-based algorithms for the Wigner transport equation

The Wigner transport equation is solved by Direct Simulation Monte Carlo, based on the generation and annihilation of signed particles. In this framework, stochastic algorithms are derived using the theory of pure jump processes with a general state space. Numerical experiments on benchmark test cases are shown.

Algebraic structure of the anti-causal system

This paper presents a behavioral framework, as developed by J.C. Willems, of a discrete time anticausal pure predictor system. This study is deduced by identifying that the generalized state space of the system has a natural polynomial module structure over the formal series ring.

A Note on Foster–Lyapunov Drift Condition for Recurrence of Markov Chains on General State Spaces

In this paper, we prove that the Foster–Lyapunov drift condition is necessary and sufficient for recurrence of a Markov chain on a general state space. And then an answer in the affirmative for the question presented in the monograph of Meyn and Tweedie (Markov chains and stochastic stability. Springer, Berlin, 1993, P175) is obtained.

Recurrence Criteria for Generalized Dirichlet Forms

We develop sufficient analytic conditions for recurrence and transience of non-sectorial perturbations of possibly non-symmetric Dirichlet forms on a general state space. These form an important subclass of generalized Dirichlet forms which were introduced in \cite{St1}. In case there exists an associated process, we show how the analytic conditions imply recurrence and transience in the classical probabilistic sense. As an application, we consider a generalized Dirichlet form given on a closed or open subset of $\mathbb{R}^d$ which is given as a divergence free first order perturbation of a non-symmetric energy form. Then using volume growth conditions of the sectorial and non-sectorial first order part, we derive an explicit criterion for recurrence. Moreover, we present concrete examples with applications to Muckenhoupt weights and counterexamples. The counterexamples show that the non-sectorial case differs qualitatively from the symmetric or non-symmetric sectorial case. Namely, we make the observation that one of the main criteria for recurrence in these cases fails to be true for generalized Dirichlet forms.