Improving the performance of weighted Lagrange-multiplier methods for nonlinear constrained optimization

Abstract

Nonlinear constrained optimization problems can be solved by a Lagrange-multiplier method in a continuous space or by its extended discrete version in a discrete space. These methods rely on gradient descents in the objective space to find high-quality solutions, and gradient ascents in the Lagrangian space to satisfy the constraints. The balance between descents and ascents depends on the relative weights between the objective and the constraints that indirectly control the convergence speed and solution quality of the method. To improve convergence speed without degrading solution quality, the authors propose an algorithm to dynamically control these relative weights. Starting from an initial weight, the algorithm automatically adjusts the weights based on the behaviour of the search progress. With this strategy, one is able to eliminate divergence, reduce oscillations, and speed up convergence. They show improved convergence behaviour of the proposed algorithm on both nonlinear continuous and discrete problems.