Numerical Stability and Efficiency of Penalty Algorithms

Abstract

Penalty algorithms have been somewhat forgotten due to numerical instabilities once believed to be inherent to those methods. One usually has to solve a sequence of such problems, and when the penalty factor decreases too fast, the subproblems may become intractable. Moreover, as the penalty factor decreases, the unconstrained subproblem becomes ill conditioned, and thus difficult to solve. Also, in several intermediate computations, numerical instability may show up. The author proposes remedies to such problems and presents a wide class of numerically stable penalty algorithms. The work is done in the more general context of variational inequality problems, which encompasses optimization problems. The author’s results yield a family of globally convergent, two-step superlinearly convergent, numerically stable algorithms for variational inequality problems. Finally, issues in the numerically stable implementation of intermediate computations within those algorithms are discussed.