Multiple-gradient Descent Algorithm for Pareto-Front Identification

Abstract

This article compounds and extends several publications in which a Multiple-Gradient Descent Algorithm (MGDA), has been proposed and tested for the treatment of multi-objective differentiable optimization. Originally introduced in [3], the method has been tested and reformulated in [8]. Its efficacy to identify the Pareto front [18] has been demonstrated in [22], in comparison with an evolutionary strategy. Recently, a variant, MGDA-II, has been proposed in which the descent direction is calculated by a direct procedure [6] based on a Gram-Schmidt orthogonalization process (GSP) with special normalization. This algorithm was tested in the context of a simulation by domain partitioning, as a technique to match the different interface components concurrently [4]. The experimentation revealed the importance of scaling, and a slightly modified normalization procedure was proposed (“MGDA-IIb”). Two novel variants have been proposed since. The first, MGDA-III, realizes two enhancements. Firstly, the GSP is conducted incompletely whenever a test reveals that the current estimate of the direction of search is adequate also w.r.t. the gradients not yet taken into account; this improvement simplifies the identification of the search direction when the gradients point roughly in the same direction, and makes the directional derivative common to several objective-functions larger. Secondly, the order in which the different gradients are considered in the GSP is defined in a unique way devised to favor an incomplete GSP. In the second variant, MGDA-IV, the question of scaling is addressed when the Hessians are known. A variant is also proposed in which the Hessians are estimated by the Broyden-Fletcher-Goldfarb-Shanno (BFGS) formula. Lastly, a solution is proposed to adjust the step-size optimally in the descent step.