An Adaptive Lagrangian-Based Scheme for Nonconvex Composite Optimization

Abstract

This paper develops a novel adaptive, augmented, Lagrangian-based method to address the comprehensive class of nonsmooth, nonconvex models with a nonlinear, functional composite structure in the objective. The proposed method uses an adaptive mechanism for the update of the feasibility penalizing elements, essentially turning our multiplier type method into a simple alternating minimization procedure based on the augmented Lagrangian function from some iteration onward. This allows us to avoid the restrictive and, until now, mandatory surjectivity-type assumptions on the model. We establish the iteration complexity of the proposed scheme to reach an ε-critical point. Moreover, we prove that the limit point of every bounded sequence generated by a procedure that employs the method with strictly decreasing levels of precision is a critical point of the problem. Our approach provides novel results even in the simpler composite linear model, in which the surjectivity of the linear operator is a baseline assumption. Funding: N. Hallak’s research was partially supported by the Israel Science Foundation [Grant 637/21]. M. Teboulle’s research was partially supported by the Israel Science Foundation [Grants 1844-16 and 2619-20].