Compact State Space Research Articles

The stability of storage models with shot noise input

We examine the existence of limiting behavior, or stability, for storage models with shot noise input and general release rules. The shot noise feature of the input process allows the individual inputs to gradually enter the store. We first show that a store under the unit release rule is stable if and only if the traffic intensity is less than one; this extends the classic result of Prabhu (1980) to the case of shot noise input. The stability of the unit release rule store and various stochastic orderings are then used to derive a sufficient condition for a store with a general release rule to be stable. Finally, we show that when restricted to a compact state space, our storage model is always stable. An important component of the paper is the methodology employed: coupling and stochastic monotonicity play a key role in analyzing the non-Markov processes encountered.

The stability of storage models with shot noise input

We examine the existence of limiting behavior, or stability, for storage models with shot noise input and general release rules. The shot noise feature of the input process allows the individual inputs to gradually enter the store.We first show that a store under the unit release rule is stable if and only if the traffic intensity is less than one; this extends the classic result of Prabhu (1980) to the case of shot noise input. The stability of the unit release rule store and various stochastic orderings are then used to derive a sufficient condition for a store with a general release rule to be stable. Finally, we show that when restricted to a compact state space, our storage model is always stable.An important component of the paper is the methodology employed: coupling and stochastic monotonicity play a key role in analyzing the non-Markov processes encountered.

Average Reward Markov Decision Processes with Multiple Cost Constraints

Abstract We consider constrained Markov decision processes (MDP’s) with compact state and action spaces under long-run average reward or cost criteria, and give the characterization of an optimal pair of initial state distribution and policy, which maximize over all policies the essential infimum of the sample-path average reward subject to multiple average cost constraints. First, applying the idea of occupation measures constrained MDP’s considered here are equivalently transfered to the infinite linear programming and the existence of an optimal pair associated with a randomized stationary policy is shown by compact analysis. Secondly, we introduce the notion of a state-wise mixed stationary policy (s.w.m. policy) to get further results. And for any e>0, the existence of an e-optimal pair associated with a s.w.m. policy is inductively proved by using a Lagrange multiplier. In case of countable state space, the existence of an optimal pair associated with a s.w.m. policy is shown.

Bayesian ergodic adaptive control of discrete time markov processes

A general Markov model for discrete time adaptive control problems is considered, with compact state and action spaces. The unknown parameter is modeled as a random variable taking values also on a compact space. Under continuity assumptions on the transition operator of the Markov process, a nearly self optimizing control strategy is given. Two versions of such strategy are studied, with forced and randomized controls respectively

Logarithmic Sobolev inequalities and Langevin algorithms inRn

We study the Langevin algorithm in R n by means of the theory of ultracontractive semigroups and of the corresponding logarithmic Sobolev inequalities. Our results generalize to diffusions in R n the estimates obtained by Holley and Stroock (Commun. Math. Phys. 115 (1988), 553-569) for the case of simulated annealing on a finite (or at least compact) state space

Discrete-Time Transitivity and Accessibility: Analytic Systems

A basic open question for discrete-time nonlinear systems is that of determining when, in analogy with the classical continuous-time “positive form of Chow’s Lemma,” accessibility follows from transitivity of a natural group action.This paper studies the problem and establishes the desired implication for analytic systems in several cases: (i) compact state space, (ii) under a Poisson stability condition, and (iii) in a generic sense. In addition, the paper studies accessibility properties of the “controllability sets” recently introduced in the context of dynamical systems studies. Finally, various examples and counterexamples are provided relating the various Lie algebras introduced in past work.

A limit theorem in some dynamic fuzzy systems

Abstract In this paper, using a fuzzy relation we define a dynamic fuzzy system with a compact state space. By introducing a contraction property, we study the limit of a sequence of fuzzy states and obtain a theorem for the existence and uniqueness of the solution of a fuzzy relational equation. As an application, we deal with a dynamic fuzzy system with a terminal fuzzy gain and give some characterizations for the fuzzy expected gain. A numerical example is given to comprehend our idea in this paper.

Average cost Markov decision processes under the hypothesis of Doeblin

Average cost Markov decision processes (MDPs) with compact state and action spaces and bounded lower semicontinuous cost functions are considered. Kurano [7] has treated the general case in which several ergodic classes and a transient set are permitted for the Markov process induced by any randomized stationary policy under the hypothesis of Doeblin and showed the existence of a minimum pair of state and policy. This paper considers the same case as that discussed in Kurano [7] and proves some new results which give the existence theorem of an optimal stationary policy under some reasonable conditions.

On the Consistency of Posterior Mixtures and Its Applications

Consider i.i.d. pairs $(\theta_i, X_i), i \geq 1$, where $\theta_1$ has an unknown prior distribution $\omega$ and given $\theta_1, X_1$ has distribution $P_{\theta_1}$. This setup arises naturally in the empirical Bayes problems. We put a probability (a hyperprior) on the space of all possible $\omega$ and consider the posterior mean $\hat{\omega}$ of $\omega$. We show that, under reasonable conditions, $P_{\hat{\omega}} = \int P_\theta d\hat{\omega}$ is consistent in $L_1$. Under a identifiability assumption, this result implies that $\hat{\omega}$ is consistent in probability. As another application of the $L_1$ consistency, we consider a general empirical Bayes problem with compact state space. We prove that the Bayes empirical Bayes rules are asymptotically optimal.

Functional characterization for average cost Markov decision processes with Doeblin's conditions

This paper is a sequel to Kurano [1,2], in which average cost Markov decision process (MDPs) with compact state and action spaces have been considered under the hypothesis of Doeblin and the existence of optimal stationary policies has been shown. The objective of this paper is to give the functional characterization. That is, we derive optimality inequalities for MDPs under the hypothesis of Doeblin and prove the validity of them under some reasonable conditions.

$\alpha$-Congruence for Markov Processes

We prove infinite-time extensions of invariance principles for certain random walks with essentially compact state spaces. The extensions are uniform-like in time since they use the $\bar{d}$-metric of the Bernoulli theory and imply the classical results. These are then generalized to couplings involving an isomorphism between the processes. In general a Doeblin-type condition is needed to hold for the walks but relaxation of this is indicated.

On the compactness method in general ergodic impulsive control of markov processes

In this paper the impulsive control of Feller Markov processes on compact state space with long run average cost criterion is studied. Under the assumption of compactness of the resolvent operator the optimal strategies corresponding to general cost for impulses are constructed. Also the case of purely jump Markov processes is considered

Nonparametric adaptive control of discounted stochastic systems with compact state space

We are concerned with discrete-time stochastic control models for which the random disturbances are independent with a common unknown distribution. When the state space is compact, we prove that mild continuity conditions are sufficient to obtain adaptive policies which are asymptotically optimal with respect to the discounted reward criterion.

The Structure of the Spectrum of Fourth-Order Differential Operators with Random Coefficients

In this paper it is proved that under certain conditions on the coefficients the random operator H = being a stationary, ergodic Markov process with compact State space K, has almost surely pure point spectrum and exponentially decreasing eigenfunctions. The method used here can be extended to operators corresponding to certain matrix Sturm-Liouville problems.

The Existence of a Minimum Pair of State and Policy for Markov Decision Processes under the Hypothesis of Doeblin

This paper studies the average-cost Markov decision process with compact state and action spaces and bounded lower semicontinuous cost functions. Following the idea of Borkar’s excellent papers [SIAMJ. Control Optim., 21 (1983), pp. 652–666; 22 (1984), pp. 965–978], the general case where irreducibility is not assumed is considered under the hypothesis of Doeblin and the existence of a minimum pair of state and policy, which attains the infimum of the average expected cost over all initial states and policies, is established. Further, it is proved that under additional weak conditions there exists an optimal stationary policy in the usual sense.

On Impulse Control with Partial Observation

This paper presents an existence result for an impulse control problem with partial observation. The unobserved process evolves between any two successive impulse times as a Feller Markov process on a locally compact separable state space, and the observation process is of a signal + white noise type.

De Finetti-type Theorems: An Analytical Approach

A famous theorem of De Finetti (1931) shows that an exchangeable sequence of $\{0, 1\}$-valued random variables is a unique mixture of coin tossing processes. Many generalizations of this result have been found; Hewitt and Savage (1955) for example extended De Finetti's theorem to arbitrary compact state spaces (instead of just $\{0, 1\}$). Another type of question arises naturally in this context. How can mixtures of independent and identically distributed random sequences with certain specified (say normal, Poisson, or exponential) distributions be characterized among all exchangeable sequences? We present a general theorem from which the "abstract" theorem of Hewitt and Savage as well as many "concrete" results--as just mentioned--can be easily deduced. Our main tools are some rather recent results from harmonic analysis on abelian semigroups.

A sufficient condition for the transitivity of pseudo-semigroups: Application to system theory

In his very interesting paper [8], Sussmann introduces, among other important concepts, a property he calls (P) and which is equivalent to the openness of accessibility sets of a system on a manifold. He states that property (P) is not stable under small perturbation of the system. To sustain his claim he constructs an example. As the second author has noticed, the example is wrong. It turns out that property (P) is stable: as a consequence of the main result of this paper, it follows that the openness of the accessibility sets implies the complete controllability of the system at least in the case of a compact state space. This last property is stable by the main result of Sussmann’s paper just quoted. Our main theorem is the generalization of the well-known and trivial fact that if the orbits of a connected topological group acting on a connected topological space are open, then the action is transitive. We extend this result to the action on a manifold of pseudo-semigroups satisfying a connectedness property. The extension is not as trivial as the group case since in the semi group case the orbits of the action are not disjoint any more.

Fuzzy dynamical systems

A fuzzy dynamical system on an underlying complete, locally compact metric state space X is defined axiomatically in terms of a fuzzy attainability set mapping on X. This definition includes as special cases crisp single and multivalued dynamical systems on X. It is shown that the support of such a fuzzy dynamical system on X is a crisp multivalued dynamical system on X, and that such a fuzzy dynamical system can be considered as a crisp dynamical system on a state space of nonempty compact fuzzy subsets of X. In addition fuzzy trajectories are defined, their existence established and various properties investigated.

Existence of Value and Randomized Strategies in Zero-Sum Discrete-Time Stochastic Dynamic Games

Two players with conflicting objectives are simultaneously controlling a discrete-time stochastic system. The goal of this paper is to analyze such zero-sum, discrete-time, stochastic systems when the two players are allowed to use randomized strategies. Previous results have been restricted to systems with finite or compact state spaces. Such restrictions are usually untenable from the point of view of applications, since many applications frequently use either the integers or $\mathbb{R}^n $ as a state space. Our results are proved for complete, separable, metric spaces which are very useful for applications. All previously known results emerge as special cases of our results. In addition, a variety of conjectures and open problems are resolved regarding the existence of a value function, its properties such as Borel measurability or continuity, and the existence for either or both players of optimal or $\varepsilon $-optimal stationary strategies.