Optimal decision strategy for discrete-time Markovian jump linear systems

Abstract

This paper investigates the discrete-time Markovian jump linear systems (MJLSs) whose mode transition probability matrix (MTPM) can be adjusted by decisions. Motivated by switching law design in switched systems, the optimal decision strategy is proposed for stabilisation and optimisation of such MJLSs where decision cost is taken into account. First, aiming at system stability, the feasible domain of decision is given for stable and unstable MJLSs with initial MTPM. Second, a generalised performance index is put forward which contains both state cost and decision cost, and we obtain the quantitative relationship between the index and decision via stochastic dynamic programming. Finally, for the optimisation of the performance index, a value iteration algorithm is proposed with its convergence proof. This algorithm, searching for the optimal decision within the feasible domain, achieves superior performance on the basis of ensuring stability. Simulation results illustrate the effectiveness of the proposed optimal decision strategy.