Proximal Minimization Method Research Articles

Principled analyses and design of first-order methods with inexact proximal operators

Proximal operations are among the most common primitives appearing in both practical and theoretical (or high-level) optimization methods. This basic operation typically consists in solving an intermediary (hopefully simpler) optimization problem. In this work, we survey notions of inaccuracies that can be used when solving those intermediary optimization problems. Then, we show that worst-case guarantees for algorithms relying on such inexact proximal operations can be systematically obtained through a generic procedure based on semidefinite programming. This methodology is primarily based on the approach introduced by Drori and Teboulle (Math Program 145(1–2):451–482, 2014) and on convex interpolation results, and allows producing non-improvable worst-case analyses. In other words, for a given algorithm, the methodology generates both worst-case certificates (i.e., proofs) and problem instances on which they are achieved. Relying on this methodology, we study numerical worst-case performances of a few basic methods relying on inexact proximal operations including accelerated variants, and design a variant with optimized worst-case behavior. We further illustrate how to extend the approach to support strongly convex objectives by studying a simple relatively inexact proximal minimization method.

Distributed decision-coupled constrained optimization via Proximal-Tracking

In this paper we deal with decision-coupled problems involving multiple agents over a network. Each agent has its own local objective function and local constraints, and all agents aim at finding the value of a common decision vector that minimizes the sum of all agents’ cost functions and satisfies all local constraints. To this purpose, we introduce a Proximal-Tracking distributed optimization algorithm that integrates dynamic average consensus within the proximal minimization method. Convergence to an optimal consensus solution is guaranteed for any value of a constant penalty parameter, under a convexity assumption only, without requiring differentiability, Lipschitz continuity, or smoothness of the local objective functions. Numerical simulations show the effectiveness of the proposed scheme.

SpicyMKL: a fast algorithm for Multiple Kernel Learning with thousands of kernels

We propose a new optimization algorithm for Multiple Kernel Learning (MKL) called SpicyMKL, which is applicable to general convex loss functions and general types of regularization. The proposed SpicyMKL iteratively solves smooth minimization problems. Thus, there is no need of solving SVM, LP, or QP internally. SpicyMKL can be viewed as a proximal minimization method and converges super-linearly. The cost of inner minimization is roughly proportional to the number of active kernels. Therefore, when we aim for a sparse kernel combination, our algorithm scales well against increasing number of kernels. Moreover, we give a general block-norm formulation of MKL that includes non-sparse regularizations, such as elastic-net and ℓ p -norm regularizations. Extending SpicyMKL, we propose an efficient optimization method for the general regularization framework. Experimental results show that our algorithm is faster than existing methods especially when the number of kernels is large (>1000).

Continuous regularized variable-metric proximal minimization method

The article considers the variable-metric continuous proximal method for the convex programming problem. The convergence of the method is proved. The regularized version of the method is proposed for the case where the objective function and the feasible set are defined inexactly. A regularizing operator is constructed.

Continuous regularized proximal minimization method

A continuous regularization method based on the proximal method is proposed for minimization problems with an inexact objective function. Sufficient convergence conditions are given, and the regularizing operator is constructed.