Partition Of State Space Research Articles

State Predictability and Information Flow in Simple Chaotic Systems

The possibility of state prediction in deterministic chaotic systems, which are described by 1-D maps, is discussed in the light o f information theory. A quantity h(l) is defined which represents the production of uncertainty on a future state by the chaotic dynamics (intrinsic noise) after / time steps have passed. h(l) is related to the Lyapunov characteristic exponent. Moreover, the influence of the measuring process (overlappings o f mapped boxes o f state space partition) and external noise on the state predictability are investigated quantitatively.

Markov decision processes and strongly excessive functions

Strongly excessive functions play an important role in the theory of Markov decision processes and Markov games. In this paper the following question is investigated: What are the properties of Markov decision processes which possess a strongly excessive function? A probabilistic characterization is presented in the form of a random drift through a partitioned state space. For strongly excessive functions which have a positive lower bound a characterization is given in terms of the lifetime distribution of the process. Finally we give a characterization in terms of the spectral radius.

Markov Decisions on a Partitioned State Space

An important practical constraint on admissible control policies is defined for the Markov decision process. The framework of an algorithm based on the infinite return optimization algorithms of Howard and Jewell is suggested to compute the optimal policy under this constraint. Iterative convergence to the optimal policy cannot be guaranteed, but techniques proposed for state-space reduction and rapid resolution of undetermined policies should render many problems tractable.