Stochastic Discrete-time Research Articles

Subgame-consistent cooperative solutions in randomly furcating stochastic dynamic games

In the analysis of cooperative stochastic dynamic games a stringent condition–subgame consistency–is required for a dynamically stable solution. A cooperative solution is subgame consistent if an extension of the solution policy to a subgame starting at a later time with a feasible state brought about by prior optimal behavior would remain optimal. This paper considers subgame consistent cooperative solutions in randomly furcating stochastic discrete-time dynamic games. Analytically tractable payoff distribution procedures contingent upon specific random realizations of the state and payoff structures are derived. In computer modeling and operations research discrete-time analysis often proved to be more applicable and compatible with actual data than continuous-time analysis. This is the first time that a subgame consistent solution for randomly-furcating stochastic dynamic games has been obtained. It widens the application of cooperative dynamic game theory to discrete-time problems where the evolution of the state and future payoff structures are not known with certainty.

Optimal Annuity Purchase Decisions Under Uncertain Lifetime

This paper developed a wealth allocation framework for longevity risk protection under stochastic lifetime. By combining the dynamics of wealth evolution and health evolution in stochastic multi-period discrete-time models, an optimization problem was formulated with the objective of maximizing lifetime utility of consumption and bequest. We implemented the framework in different scenarios and provided results to illustrate the practical implications of the framework. Computational results suggested that it is optimal for most people to purchase an annuity at some point in their lives. However, an individual’s health status, risk aversion, retirement objective, and the insurance charge associated with annuities could significantly influence the choice of optimal annuitization time, consumption plans, and trade off between lifetime versus fixed-term annuity.

A KICKED EPIDEMIC SIRS MODEL

A discrete-time deterministic epidemic model is proposed with the aim of reproducing the behavior observed in the incidence of real infectious diseases, such as oscillations and irregularities. For this purpose, we introduce, in a naïve discrete-time SIRS model, seasonal variability (i) in the loss of immunity and (ii) in the infection probability, modeled by sequences of kicks. Effects of a variable population size (assumed as logistic) are also analyzed. Restrictive assumptions are made on the parameters of the models, in order to guarantee that the transitions are determined by true probabilities, so that comparisons with stochastic discrete-time previsions can be also provided. Numerical simulations show that the characteristics of real infectious diseases can be adequately modeled.

Robust stochastic tracking for discrete-time models: designing of ellipsoid where random trajectories converge with probability one

We study the behaviour of stochastic discrete-time models controlled by an output linear feedback during a tracking process. The controlled system is assumed to be nonlinear satisfying the global ‘quasi-Lipschitz’ condition and subjected to stochastic input and output disturbances. Two gain matrices (in a feedback and in an observer) define an ‘averaged’ ellipsoid in the tracking-error space where all system's trajectories arrive ‘with probability one’. The selection of the ‘best’ gain matrices is realised numerically by application of the robust attractive ellipsoid method with the linear matrix inequality technique application. The suggested approach is illustrated by designing of a robust tracking controller for a benchmark example in the presence of stochastic noises both in the state dynamics and in the output observations.

Three-stage Kalman filter for state and fault estimation of linear stochastic systems with unknown inputs

The paper studies the problem of simultaneously estimating the state and the fault of linear stochastic discrete-time varying systems with unknown inputs. The fault and the unknown inputs affect both the state and the output. However, if the dynamical evolution models of the fault and the unknown inputs are available the filtering problem will be solved by the Optimal three-stage Kalman Filter (OThSKF). The OThSKF is obtained after decoupling the covariance matrices of the Augmented state Kalman Filter (ASKF) using a three-stage U–V transformation. Nevertheless, if the fault and the unknown inputs models are not perfectly known the Robust three-stage Kalman Filter (RThSKF) will be applied to give an unbiased minimum-variance estimation. Finally, a numerical example is given in order to illustrate the proposed filters.

H<sub>∞</sub> Dynamic Output Feedback Control for a Class of Discrete-Time Stochastic Systems with Sector Nonlinearities and Mixed Time-Delays

This paper deals with the dynamic output feedback control problem for a class of discrete-time stochastic systems involving sector nonlinearities and mixed time-delays. Based on the control theory and Lyapunov-Krasovskii functional method, an effective approach is proposed to design a dynamic output feedback controller such that the resultant closed-loop system is stochastically stable with a prescribed disturbance attention in the mean square sense and has a performance level. A numerical example demonstrates the effectiveness of the method.

Optimal Control of Gene Mutation in DNA Replication

We propose a molecular-level control system view of the gene mutations in DNA replication from the finite field concept. By treating DNA sequences as state variables, chemical mutagens and radiation as control inputs, one cell cycle as a step increment, and the measurements of the resulting DNA sequence as outputs, we derive system equations for both deterministic and stochastic discrete-time, finite-state systems of different scales. Defining the cost function as a summation of the costs of applying mutagens and the off-trajectory penalty, we solve the deterministic and stochastic optimal control problems by dynamic programming algorithm. In addition, given that the system is completely controllable, we find that the global optimum of both base-to-base and codon-to-codon deterministic mutations can always be achieved within a finite number of steps.

A hybrid model for real time simulation of urban traffic

In this paper a hybrid model is presented to predict the behavior of traffic in urban area. Intersections and on/off-ramps are modeled by Colored Timed Petri Nets while road links are modeled by a stochastic discrete time model. It is assumed that vehicle routings at each intersection are stochastic. A Particle Filter algorithm based on the hybrid model is developed to estimate the system state. The validation of the model by using real data is discussed. The limited computational effort required makes the model suitable to be used in simulation-based control schemes for urban traffic.

Stochastic Receding Horizon Control With Bounded Control Inputs: A Vector Space Approach

We design receding horizon control strategies for stochastic discrete-time linear systems with additive (possibly) unbounded disturbances while satisfying hard bounds on the control actions. We pose the problem of selecting an appropriate optimal controller on vector spaces of functions and show that the resulting optimization problem has a tractable convex solution. Under marginal stability of the zero-control and zero-noise system we synthesize receding horizon polices that ensure bounded variance of the states while enforcing hard bounds on the controls. We provide examples that illustrate the effectiveness of our control strategies, and how quantities needed in the formulation of the resulting optimization problems can be calculated off-line.

Stochastic receding horizon control with output feedback and bounded controls

We study the problem of receding horizon control for stochastic discrete-time systems with bounded control inputs and incomplete state information. Given a suitable choice of causal control policies, we first present a slight extension of the Kalman filter to estimate the state optimally in mean-square sense. We then show how to augment the underlying optimization problem with a negative drift-like constraint, yielding a second-order cone program to be solved periodically online. We prove that the receding horizon implementation of the resulting control policies renders the state of the overall system mean-square bounded under mild assumptions. We also discuss how some quantities required by the finite-horizon optimization problem can be computed off-line, thus reducing the on-line computation.

A recursive state estimator in the presence of state inequality constraints

This paper proposes an optimal recursive estimator to estimate the states of a stochastic discrete time linear dynamic system when the states of the system are constrained with inequality constraints. The case when the constraints are strictly satisfied is treated independently from the case when some of the constraints are violated. For the first case, the well known Kalman filter estimator is used. In the second case, an algorithm which uses a series of successive orthogonalizations on the measurement subspaces is employed to obtain the optimal estimate. It is shown that the proposed estimator has several attractive properties such that it is an unbiased estimator. More importantly, compared to other estimator found in the literature, the proposed estimator needs less computational efforts, is numerically more stable and it leads to a smaller variance. To show the effectiveness of the proposed estimator, several simulation results are presented and discussed.

Information fusion strategies and performance bounds in packet-drop networks

In this paper, we discuss suboptimal distributed estimation schemes for stable stochastic discrete time linear systems under the assumptions that (i) distributed sensors have computation capabilities, (ii) the communication between the sensors and the estimation center is subject to random packet loss, and (iii) there is no communication between sensors. We consider strategies which are based on raw measurement fusion (MF) as well as on fusing local estimates, such as local Kalman filters or other pre-processing rules. We show that the optimal mean square estimation error that can be achieved under packet loss, referred as the infinite bandwidth filter (IBF), cannot be reached using a limited bandwidth channel; we also compare these strategies under specific noise regimes. We also propose novel mathematical tools to derive analytical upper and lower bounds for the expected estimation error covariance of the MF and the IBF strategies assuming identical sensors. The theoretical findings are complemented with simulation results.

Exponential Stability of Discrete-Time Delayed Hopfield Neural Networks with Stochastic Perturbations and Impulses

This paper derives some sufficient conditions for exponential stability in the mean square of stochastic discrete-time delayed Hopfield neural networks (DHNN) with impulse effects. The Lyapunov–Krasovskii stability theory, Halanay inequality, and linear matrix inequality (LMI) are employed to investigate the problem. It is shown that the impulses in certain regions might preserve the stability property of the DHNN when the impulses-free part converges to its equilibrium point. Moreover, the feasible interval of the jump operator is also derived.

A Discrete SIRS Model with Kicked Loss of Immunity and Infection Probability

A discrete-time deterministic epidemic model is proposed with the aim of reproducing the behaviour observed in the incidence of real infectious diseases, such as oscillations and irregularities. For this purpose we introduce, in a naïve discrete-time SIRS model, seasonal variability in the loss of immunity and in the infection probability, modelled by sequences of kicks. Restrictive assumptions are made on the parameters of the models, in order to guarantee that the transitions are determined by true probabilities, so that comparisons with stochastic discrete-time previsions can be also provided. Numerical simulations show that the characteristics of real infectious diseases can be adequately modeled.

Control of Discrete-Time Stochastic Systems with State Equality Constraints

AbstractDesign conditions for existence of state feedback control stabilizing discrete-time stochastic systems with multiplicative noise where closed-loop state variables satisfy equality constraints in the mean are presented in the paper. Using classical feedback control principle the LMI-based procedure is provided for computation of the gain matrix of state control laws, and influence of equality constraints is explained. The approach is successfully illustrated on example demonstrating validity of the proposed method using an equality constraint tying together all state variables of the system.

Optimal and suboptimal minimum-variance filtering in networked systems with mixed uncertainties of random sensor delays, packet dropouts and missing measurements

In this paper the Kalman filtering problem for networked stochastic linear discrete-time systems with random measurement delays, packet dropouts and missing measurements is studied. Based on a quasi Markov-chain approach, a unified/combined model is developed to accommodate random delay, packet dropout and missing measurement. Two approaches for constructing a filter via the linear matrix inequality approach are proposed. Simulation studies are then conducted to evaluate the effectiveness of the constructed estimators.

Linear Generalized Stochastic Systems for Insurance Portfolios

We consider a typical portfolio of different insurance products and investigate the pricing process using the framework of a linear time invariant generalized stochastic discrete-time model. Moreover, we assume that, due to regulatory constraints, the resulting system is (regular) descriptor and calculate the solution using the tools of matrix pencil theory. Finally, we present a numerical application for two different portfolios.

A self-stabilizing MSA algorithm in high-dimension data stream

Minor subspace analysis (MSA) is a statistical method for extracting the subspace spanned by all the eigenvectors associated with the minor eigenvalues of the autocorrelation matrix of a high-dimension vector sequence. In this paper, we propose a self-stabilizing neural network learning algorithm for tracking minor subspace in high-dimension data stream. Dynamics of the proposed algorithm are analyzed via a corresponding deterministic continuous time (DCT) system and stochastic discrete time (SDT) system methods. The proposed algorithm provides an efficient online learning for tracking the MS and can track an orthonormal basis of the MS. Computer simulations are carried out to confirm the theoretical results.

A Stochastic Version of the Eigen Model

We exhibit a stochastic discrete time model that produces the Eigen model (Naturwissenschaften 58:465-523, 1971) in the deterministic and continuous time limits. The model is based on the competition among individuals differing in terms of fecundity but with the same viability. We explicitly write down the Markov matrix of the discrete time stochastic model in the two species case and compute the master sequence concentration numerically for various values of the total population. We also obtain the master equation of the model and perform a Van Kampen expansion. The results obtained in the two species case are compared with those coming from the Eigen model. Finally, we comment on the range of applicability of the various approaches described, when the number of species is larger than two.

Impulsive pinning synchronization of stochastic discrete-time networks

The pinning synchronization problem of stochastic impulsive networks (SIN) is investigated. Using Lyapunov stability theory and pinning method, some sufficient criteria are derived for asymptotical synchronization and exponential synchronization of such dynamical networks in mean square. Illustrative examples are provided to verify the effectiveness of the proposed approach.