Constructing the Pareto front for multi-objective Markov chains handling a strong Pareto policy approach

Abstract

In this paper, we present a multi-objective solution of the Pareto front for a certain class of discrete-time ergodic controllable Markov chains. For solving the problem, we transform the original multi-objective problem into an equivalent nonlinear programming problem implementing the Lagrange principle. We then propose to adopt the Tikhonov’s regularization method to ensure the convergence of the cost functions to a unique point into the Pareto front. We show that Pareto policies are characterized as optimal policies. One of the fundamental problems related to the construction of the Pareto front is the existence and characterization of both, Pareto and strong Pareto policies. By introducing the Tikhonov’s regularizator, we ensure the existence of strong Pareto policies. This paper proposes a real solution to this problem. We formulate the original problem considering several constraints: (a) we employ the c-variable method for the introduction of linear constraints over the nonlinear problem and (b) restrict the cost functions, allowing points in the Pareto front to have a small distance from one another. The constraints imposed by the c-variable method make the problem computationally tractable and the restriction imposed by the small distance change ensures the continuation of the Pareto front. The resulting equation in this nonlinear system is an optimization problem for which the necessary and efficient condition of a minimum is solved using the projected gradient method. Moreover, we also prove the convergence conditions and compute the estimate rate of convergence of variables corresponding to the Lagrange principle and the Tikhonov’s regularization, respectively. We provide all the details needed to implement the proposed method in an efficient way. The usefulness of the method is successfully demonstrated by a numerical example.