Abstract
We present a framework for generalizing the primal-dual gradient method, also known as the gradient descent ascent method, for solving convex-concave minimax problems. The framework is based on the observation that the primal-dual gradient method can be viewed as an inexact gradient method applied to the primal problem. Unlike the setting of traditional inexact gradient methods, the inexact gradient is computed by a dynamic inexact oracle, which is a discrete-time dynamical system whose output asymptotically approaches the exact gradient. For minimax problems, dynamic inexact oracles are capable of modeling a range of first-order methods for computing the gradient of the primal objective, which relies on solving the inner maximization problem. We provide a unified convergence analysis of gradient methods with dynamic inexact oracles and demonstrate its use in creating new accelerated primal-dual algorithms.
Highlights
We present a framework for generalizing the primal-dual gradient method, known as the gradient descent ascent method, for solving convex-concave minimax problems
This letter attempts to address this question based on an alternative view of the primal-dual gradient method (PDGM): We show that the PDGM is equivalent to applying an inexact gradient method to the primal problem (2), where the gradient ∇p is computed approximately by a dynamic inexact oracle (Definition 1), whose output approaches the exact gradient asymptotically
We begin by considering another way to solve the primal problem (2) by directly applying the gradient method, which reveals that the PDGM can be viewed alternatively as an inexact gradient method applied to the primal problem
Summary
Abstract—We present a framework for generalizing the primal-dual gradient method, known as the gradient descent ascent method, for solving convex-concave minimax problems. Dynamic inexact oracles are capable of modeling a range of first-order methods for computing the gradient of the primal objective, which relies on solving the inner maximization problem. We provide a unified convergence analysis of gradient methods with dynamic inexact oracles and demonstrate its use in creating new accelerated primal-dual algorithms. This letter attempts to address this question based on an alternative view of the PDGM: We show that the PDGM is equivalent to applying an inexact gradient method to the primal problem (2), where the gradient ∇p is computed approximately by a dynamic inexact oracle (Definition 1), whose output approaches the exact gradient asymptotically. The convergence analysis enables us to build new primal-dual algorithms by changing the realization of the inexact oracle used in PDGM to other first-order methods in a “plug-and-play” manner
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
