Penalty Methods, Barrier Methods and Augmented Lagrangians

Abstract

Abstract We now turn to study some general optimization methods upon which we can later base the design of model decomposition algorithms (see Chapter 1). These general methods for constrained nonlinear optimization are the penalty methods that employ penalty or barrier functions giving rise to so-called exterior penalty methods or interior penalty methods, respectively. These methods are closely related to the primal-dual Lagrangian algorithmic scheme and to the method of augmented Lagrangians (sometimes referred to as the multiplier method), which we also discuss here. Such algorithms are not only considered most efficacious in solving large-scale constrained optimization problems, but are also suitable problem modifiers in our model decomposition algorithms (Chapter 7). Barrier function methods are also helpful when studying the primal-dual path following algorithm for linear programming (see Chapter 8). This algorithm is developed using a logarithmic barrier function. Finally, there is a close re lationship between the method of augmented Lagrangians and the proximal minimization algorithms (see Chapter 3), which should not be overlooked. Section 4.1 examines penalty functions and their use in penalty methods, wherein the optimal solution of the problem is approached from outside the feasible set (exterior penalty). In Section 4.2 we describe barrier functions and their use in approaching the optimal solution from within the feasible set. Section 4.3 explains the basics of duality theory and the primal-dual algorithmic framework. Section 4.4 gives an account of augmented Lagrangian methods and how they are related to the proximal minimization approach developed earlier (see Chapter 3), which should not be overlooked.