Nondifferentiable Convex Functions Research Articles

A distributed proximal gradient method with time-varying delays for solving additive convex optimizations

We consider the problem of minimizing a finite sum of differentiable and nondifferentiable convex functions in the setting of finite-dimensional Euclidean space. We propose and analyze a distributed proximal gradient method with computational delays. The occurrence of local delays when computing local gradient of each differentiable cost function allows the use of out-of-date iterates when generating the next estimates, which benefits a situation where the cost of gradient computation is expensive so that it cannot be done within a limited time constraints. We provide a condition on control parameter to guarantee that the sequences generated by the proposed method converge to the unique solution. We finally illustrate the presented theoretical results by performing some numerical experiments on binary image classification.

“Relative Continuity” for Non-Lipschitz Nonsmooth Convex Optimization Using Stochastic (or Deterministic) Mirror Descent

The usual approach to developing and analyzing first-order methods for nonsmooth (stochastic or deterministic) convex optimization assumes that the objective function is uniformly Lipschitz continuous with parameter Mf. However, in many settings, the nondifferentiable convex function f is not uniformly Lipschitz continuous—for example, (i) the classical support vector machine problem, (ii) the problem of minimizing the maximum of convex quadratic functions, and even (iii) the univariate setting with [Formula: see text]. Herein, we develop a notion of “relative continuity” that is determined relative to a user-specified “reference function” h (that should be computationally tractable for algorithms), and we show that many nondifferentiable convex functions are relatively continuous with respect to a correspondingly fairly simple reference function h. We also similarly develop a notion of “relative stochastic continuity” for the stochastic setting. We analyze two standard algorithms—the (deterministic) mirror descent algorithm and the stochastic mirror descent algorithm—for solving optimization problems in these new settings, providing the first computational guarantees for instances where the objective function is not uniformly Lipschitz continuous. This paper is a companion paper for nondifferentiable convex optimization to the recent paper by Lu et al. [Lu H, Freund RM, Nesterov Y (2018) Relatively smooth convex optimization by first-order methods, and applications. SIAM J. Optim. 28(1): 333–354.], which developed analogous results for differentiable convex optimization.

Stochastic Source Seeking for Mobile Robots in Obstacle Environments Via the SPSA Method

This paper considers a class of stochastic source-seeking problems to drive a mobile robot to the minimizer of a source signal. Our approach is first analyzed in an obstacle-free scenario, where measurements of the signal at the robot location and information of a contact sensor are required. We extend our results to environments with obstacles under mild assumptions on the step size. Our approach builds on the simultaneous perturbation stochastic approximation idea to obtain information of the signal field. We prove the practical convergence of the algorithms to a ball whose size depends on the step size that contains the location of the source. The novelty relies in that we consider nondifferentiable convex functions, a fixed step size, and the environment may contain obstacles. Our proof methods employ nonsmooth Lyapunov function theory, tools from convex analysis, and stochastic difference inclusions. Finally, we illustrate the applicability of the proposed algorithms in a two-dimensional scenarios.

Fixed point algorithm based on adapted metric method for convex minimization problem with application to image deblurring

Recently, optimization algorithms for solving a minimization problem whose objective function is a sum of two convex functions have been widely investigated in the field of image processing. In particular, the scenario when a non-differentiable convex function such as the total variation (TV) norm is included in the objective function has received considerable interests since many variational models encountered in image processing have this nature. In this paper, we propose a fast fixed point algorithm based on the adapted metric method, and apply it in the field of TV-based image deblurring. The novel method is derived from the idea of establishing a general fixed point algorithm framework based on an adequate quadratic approximation of one convex function in the objective function, in a way reminiscent of Quasi-Newton methods. Utilizing the non-expansion property of the proximity operator we further investigate the global convergence of the proposed algorithm. Numerical experiments on image deblurring problem demonstrate that the proposed algorithm is very competitive with the current state-of-the-art algorithms in terms of computational efficiency.

On Proximal Subgradient Splitting Method for Minimizing the sum of two Nonsmooth Convex Functions

In this paper we present a variant of the proximal forward-backward splitting iteration for solving nonsmooth optimization problems in Hilbert spaces, when the objective function is the sum of two nondifferentiable convex functions. The proposed iteration, which will be called Proximal Subgradient Splitting Method, extends the classical subgradient iteration for important classes of problems, exploiting the additive structure of the objective function. The weak convergence of the generated sequence was established using different stepsizes and under suitable assumptions. Moreover, we analyze the complexity of the iterates.

Proximity algorithms for the L1/TV image denoising model

This paper introduces a proximity operator framework for studying the L1/TV image denoising model which minimizes the sum of a data fidelity term measured in the ?1-norm and the total-variation regularization term. Both terms in the model are non-differentiable. This causes algorithmic difficulties for its numerical treatment. To overcome the difficulties, we formulate the total-variation as a composition of a convex function (the ?1-norm or the ?2-norm) and the first order difference operator, and then express the solution of the model in terms of the proximity operator of the composition. By developing a "chain rule" for the proximity operator of the composition, we identify the solution as fixed point of a nonlinear mapping expressed in terms of the proximity operator of the ?1-norm or the ?2-norm, each of which is explicitly given. This formulation naturally leads to fixed-point algorithms for the numerical treatment of the model. We propose an alternative model by replacing the non-differentiable convex function in the formulation of the total variation with its differentiable Moreau envelope and develop corresponding fixed-point algorithms for solving the new model. When partial information of the underlying image is available, we modify the model by adding an indicator function to the minimization functional and derive its corresponding fixed-point algorithms. Numerical experiments are conducted to test the approximation accuracy and computational efficiency of the proposed algorithms. Also, we provide a comparison of our approach to two state-of-the-art algorithms available in the literature. Numerical results confirm that our algorithms perform favorably, in terms of PSNR-values and CPU-time, in comparison to the two algorithms.

Minimization of A Nondifferentiable Convex Function Defined not Everywhere

We examine an oracle-type method to minimize a convex function f over a convex polyhedron G . The method is an extension of the level-method to the case, when f is a not everywhere finite function, i.e., it may equal to + X at some points of G . An estimate of its efficiency is given, and some modifications of the method are mentioned. Finally, we indicate possible ways of its employment and report results of a numerical experiment.

A method for minimizing convex functions based on continuous approximations to the subdifferential

A numerical method for unconstrained minization of nondifferentiable convex functions is proposed and studied. This method is based on a continuous approximations to the subdifferential. An algorithm for the construction of the continuous approximations is described. Results of numerical experiments are presented

A minimizing algorithm for complex nonconvex nondifferentiable functions

The minimization of nonconvex, nondifferentiable functions that are compositions of max-type functions formed by nondifferentiable convex functions is discussed in this paper. It is closely related to practical engineering problems. By utilizing the globality of e-subdifferential and the theory of quasidifferential, and by introducing a new scheme which selects several search directions and consider them simultaneously at each iteration, a minimizing algorithm is derived. It is simple in structure, implementable, numerically efficient and has global convergence. The shortcomings of the existing algorithms are thus overcome both in theory and in application.

Sensitivity analysis of nondifferentiable sums of singular values of rectangular matrices

Let K(x) be an s-by-r rectangular matrix depending on a parameter x e E and denote by g(x) the sum of its m largest singular values (1 ≤ m ≤ Min{s,r}). If K(x) depends affinely on x, then g is a nondifferentiable convex function. In this paper we consider first the affine case and give some formulas for the conjugate, subdifferential, and e-subdifferential of g. These formulas are then used to obtain perturbation bounds for g(x). We study next the nonaffine case and discuss some questions related with the regularity, generalized subdifferentiability, and directional differentiability of g.

Extremal properties of nondifferentiable convex functions on euclidean sets of combinations with repetitions

A general approach is suggested for studying extremal properties of nondifferentiable convex functions on Euclidean combinatorial sets. On the basis of this approach, by solving the linear optimization problem on a set of combinations with repetitions, we obtain estimates of minimum values of convex and strongly convex objective functions in optimization problems on sets of combinations with repetitions and establish sufficient conditions for the existence of the corresponding minima.

The Second-Order Subdifferential and the Dupin Indicatrices of a Non-Differentiable Convex Function

The Second-Order Subdifferential and the Dupin Indicatrices of a Non-Differentiable Convex Function

The Le Chatelier Principle in Convex Programming

The Le Chatelier Principle, first applied in economics by Samuelson [11] many years ago, can be stated in the following way: if one alters one of the parameters (pressure, temperature, concentration of any one compound, etc.) of a system of physical or chemical equilibrium, the remaining parameters will adjust so as to counteract the disturbance. The interpretation of the Lagrange multipliers (or dual variables) as a price system for available resources is well known [1], [3], [8]. If we suppose that the quantity of one of the available resources changes, one would expect, by the Le Chatelier Principle, that its price changes in the opposite direction. We can then show that the marginal value of a resource increases if its amount is reduced, and vice versa. This property, shown in Proposition 1, will be called the Le Chatelier Principle I. We can also state, as a corollary to this principle, an equivalent property for the levels of an activity. Using a recent generalization, by Rockafellar [10], of the Kuhn-Tucker conditions for non-differentiable convex functions (but admitting subgradients), we demonstrate that if the marginal cost (in terms of resource inputs) of an activity decreases, the level of this activity is stepped up, and vice versa. This property will be called the Le Chatelier Principle II. In the last section we present a particular case to these two principles. Restricting ourselves to the class of homogeneous programming problems [8], we first show that globally, i.e. without any differentiability assumption, the traditional equality between the total factor payment and the total profit holds, and secondly that the perturbation function [4], [10] is a positively linear homogeneous function. As a consequence to these properties, it follows that the dual variables remain unchanged for proportional variations of the parameters.

A Gradient Inequality for a Class of Nondifferentiable Functions

A well known gradient inequality for differentiable convex minimands and their minima over convex sets is discussed. An analogous type of result is obtained for the class of non-differentiable convex functions of the form f(x) = atx + (xtCx)1/2 over convex polyhedral sets K = {x∣Ax ≤ b} in Euclidean n-space.