Abstract
Motivated by recent developments of using cubic regularization to escape saddle points of unconstrained optimization, in this paper we explore its potential in pursuing second-order stationary points of nonconvex constrained optimization whose exact objective function information may be hard to obtain. We first propose an algorithmic framework of inexact cubic-regularized primal-dual methods for equality constrained optimization, named as ICPD. To update the primal variable at each iteration, we construct a cubic regularized model relying on inexact first- and second-order derivatives of the objective function together with information of constraint functions. By allowing an inexact solution to each subproblem under certain conditions, we establish the iteration complexity of ICPD to find an ϵ \epsilon -approximate first- and second-order stationary point, respectively. We then consider a stochastic variant of algorithm, stochastic cubic-regularized primal-dual algorithm (SCPD) for equality constrained optimization whose objective takes an expectation form. Through a proper sampling strategy to calculate stochastic gradients and Hessians, we address the oracle complexity of SCPD to reach approximate stationary points with high probability. We also investigate the behavior of the standard gradient descent when solving each subproblem with a random perturbation. We provide a detailed analysis on how to fulfill the required conditions on an inexact subproblem solution with high probability at each iteration. Additionally, we present an analysis of an adaptive variant of ICPD which updates penalty parameters dynamically and discuss the applicability of adaptive cubic regularization parameters. Finally, preliminary numerical results are reported to showcase the performances of our proposed algorithms.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.