Nonhomogeneous markov chains with desired properties - the nearest markovian model

Abstract

A new class of operators performing an optimization (optimization operators or, simply, optimators) which generate transition matrices with required properties such as ergodicity, recurrence etc., is considered and their fundamental features are described. Some criteria for comparing such operators by taking into account their strenght are given and sufficient conditions for both weak and strong ergodicity are derived. The nearest Markovian model with respect to a given set of observed probability vectors is then defined as a sequence of transition matrices satisfying certain constraints that express our prior knowledge about the system. Finally, sufficient conditions for the existence of such a model are given and the related algorithm is illustrated by an example.