Approximate Subgradients Research Articles

Local minimizers of the Crouzeix ratio: a nonsmooth optimization case study

Given a square matrix A and a polynomial p, the Crouzeix ratio is the norm of the polynomial on the field of values of A divided by the 2-norm of the matrix p(A). Crouzeix’s conjecture states that the globally minimal value of the Crouzeix ratio is 0.5, regardless of the matrix order and polynomial degree, and it is known that 1 is a frequently occurring locally minimal value. Making use of a heavy-tailed distribution to initialize our optimization computations, we demonstrate for the first time that the Crouzeix ratio has many other locally minimal values between 0.5 and 1. Besides showing that the same function values are repeatedly obtained for many different starting points, we also verify that an approximate nonsmooth stationarity condition holds at computed candidate local minimizers. We also find that the same locally minimal values are often obtained both when optimizing over real matrices and polynomials, and over complex matrices and polynomials. We argue that minimization of the Crouzeix ratio makes a very interesting nonsmooth optimization case study, illustrating among other things how effective the BFGS method is for nonsmooth, nonconvex optimization. Our method for verifying approximate nonsmooth stationarity is based on what may be a novel approach to finding approximate subgradients of max functions on an interval. Our extensive computations strongly support Crouzeix’s conjecture: in all cases, we find that the smallest locally minimal value is 0.5.

Convex optimization using quantum oracles

We study to what extent quantum algorithms can speed up solving convex optimization problems. Following the classical literature we assume access to a convex set via various oracles, and we examine the efficiency of reductions between the different oracles. In particular, we show how a separation oracle can be implemented using O~(1) quantum queries to a membership oracle, which is an exponential quantum speed-up over the Ω(n) membership queries that are needed classically. We show that a quantum computer can very efficiently compute an approximate subgradient of a convex Lipschitz function. Combining this with a simplification of recent classical work of Lee, Sidford, and Vempala gives our efficient separation oracle. This in turn implies, via a known algorithm, that O~(n) quantum queries to a membership oracle suffice to implement an optimization oracle (the best known classical upper bound on the number of membership queries is quadratic). We also prove several lower bounds: Ω(n) quantum separation (or membership) queries are needed for optimization if the algorithm knows an interior point of the convex set, and Ω(n) quantum separation queries are needed if it does not.

Inexact proximal $$\epsilon $$-subgradient methods for composite convex optimization problems

We present two approximate versions of the proximal subgradient method for minimizing the sum of two convex functions (not necessarily differentiable). At each iteration, the algorithms require inexact evaluations of the proximal operator, as well as, approximate subgradients of the functions (namely: the$$\epsilon $$-subgradients). The methods use different error criteria for approximating the proximal operators. We provide an analysis of the convergence and rate of convergence properties of these methods, considering various stepsize rules, including both, diminishing and constant stepsizes. For the case where one of the functions is smooth, we propose an inexact accelerated version of the proximal gradient method, and prove that the optimal convergence rate for the function values can be achieved. Moreover, we provide some numerical experiments comparing our algorithm with similar recent ones.

Proximal bundle methods based on approximate subgradients for solving Lagrangian duals of minimax fractional programs

We are interested in this work to solve the Lagrangian dual of a generalized fractional program (GFP), which gives the minimal value of the primal problem, thanks to some duality results. With the help of a general minimax equality assumption, we give duality results under minimal assumptions. Since the associated parametric programs of the dual of GFP are always concave, we use a general approximating proximal scheme to these subproblems and construct bundle methods by means of approximate values and approximate subgradients of the dual parametric function. As for all dual algorithms, the proposed methods generate a sequence of values that converges from below to the minimal value of the GFP and a sequence of approximate solutions that converges to a solution of the Lagrangian dual of the GFP. For certain classes of problems, the convergence is at least linear.

Computing Proximal Points of Convex Functions with Inexact Subgradients

Locating proximal points is a component of numerous minimization algorithms. This work focuses on developing a method to find the proximal point of a convex function at a point, given an inexact oracle. Our method assumes that exact function values are at hand, but exact subgradients are either not available or not useful. We use approximate subgradients to build a model of the objective function, and prove that the method converges to the true prox-point within acceptable tolerance. The subgradient g k used at each step k is such that the distance from g k to the true subdifferential of the objective function at the current iteration point is bounded by some fixed e > 0. The algorithm includes a novel tilt-correct step applied to the approximate subgradient.

An Approximate Redistributed Proximal Bundle Method with Inexact Data for Minimizing Nonsmooth Nonconvex Functions

We describe an extension of the redistributed technique form classical proximal bundle method to the inexact situation for minimizing nonsmooth nonconvex functions. The cutting-planes model we construct is not the approximation to the whole nonconvex function, but to the local convexification of the approximate objective function, and this kind of local convexification is modified dynamically in order to always yield nonnegative linearization errors. Since we only employ the approximate function values and approximate subgradients, theoretical convergence analysis shows that an approximate stationary point or some double approximate stationary point can be obtained under some mild conditions.

An Approximate Proximal Bundle Method to Minimize a Class of Maximum Eigenvalue Functions

We present an approximate nonsmooth algorithm to solve a minimization problem, in which the objective function is the sum of a maximum eigenvalue function of matrices and a convex function. The essential idea to solve the optimization problem in this paper is similar to the thought of proximal bundle method, but the difference is that we choose approximate subgradient and function value to construct approximate cutting-plane model to solve the above mentioned problem. An important advantage of the approximate cutting-plane model for objective function is that it is more stable than cutting-plane model. In addition, the approximate proximal bundle method algorithm can be given. Furthermore, the sequences generated by the algorithm converge to the optimal solution of the original problem.

A Decomposition Algorithm for Convex Nondifferentiable Minimization with Errors

A decomposition algorithm based on proximal bundle‐type method with inexact data is presented for minimizing an unconstrained nonsmooth convex function f. At each iteration, only the approximate evaluation of f and its approximate subgradients are required which make the algorithm easier to implement. It is shown that every cluster of the sequence of iterates generated by the proposed algorithm is an exact solution of the unconstrained minimization problem. Numerical tests emphasize the theoretical findings.

Numerical methods for stochastic programs with second order dominance constraints with applications to portfolio optimization

Inspired by the successful applications of the stochastic optimization with second order stochastic dominance (SSD) model in portfolio optimization, we study new numerical methods for a general SSD model where the underlying functions are not necessarily linear. Specifically, we penalize the SSD constraints to the objective under Slater’s constraint qualification and then apply the well known stochastic approximation (SA) method and the level function method to solve the penalized problem. Both methods are iterative: the former requires to calculate an approximate subgradient of the objective function of the penalized problem at each iterate while the latter requires to calculate a subgradient. Under some moderate conditions, we show that w.p.1 the sequence of approximated solutions generated by the SA method converges to an optimal solution of the true problem. As for the level function method, the convergence is deterministic and in some cases we are able to estimate the number of iterations for a given precision. Both methods are applied to portfolio optimization problem where the return functions are not necessarily linear and some numerical test results are reported.

On the subdifferential of the supremum of an arbitrary family of extended real-valued functions

In this paper, we derive a new formula for the subdifferential of the supre- mum of an arbitrary family of extended real-valued functions, in terms of the approximate subgradients of well chosen convex combinations of the data functions. The data functions are neither convex nor lower semicontinuous, but in this paper we assume that the supre- mum of the second conjugates of the data functions is proper and coincides with the second conjugate of the supremum function. Some applications of the main formula are provided. In particular, new formulas are given for the subdifferential of the closed convex hull of an extended real-valued function, as well as for the corresponding set of minimizers. We also get a generalization of the formula in Hiriart-Urruty et al. (J Funct Anal 118, 154-166 1993, Theorem 2.1) for the subdifferential of the sum. The paper finishes with the presentation of some open problems.

An approximate decomposition algorithm for convex minimization

For nonsmooth convex optimization, Robert Mifflin and Claudia Sagastizábal introduce a V U -space decomposition algorithm in Mifflin and Sagastizábal (2005) [11]. An attractive property of this algorithm is that if a primal–dual track exists, this algorithm uses a bundle subroutine. With the inclusion of a simple line search, it is proved to be globally and superlinearly convergent. However, a drawback is that it needs the exact subgradients of the objective function, which is expensive to compute. In this paper an approximate decomposition algorithm based on proximal bundle-type method is introduced that is capable to deal with approximate subgradients. It is shown that the sequence of iterates generated by the resulting algorithm converges to the optimal solutions of the problem. Numerical tests emphasize the theoretical findings.

Convergence Analysis of Deflected Conditional Approximate Subgradient Methods

Subgradient methods for nondifferentiable optimization benefit from deflection, i.e., defining the search direction as a combination of the previous direction and the current subgradient. In the constrained case they also benefit from projection of the search direction onto the feasible set prior to computing the steplength, that is, from the use of conditional subgradient techniques. However, combining the two techniques is not straightforward, especially if an inexact oracle is available which can only compute approximate function values and subgradients. We present a convergence analysis of several different variants, both conceptual and implementable, of approximate conditional deflected subgradient methods. Our analysis extends the available results in the literature by using the main stepsize rules presented so far, while allowing deflection in a more flexible way. Furthermore, to allow for (diminishing/square summable) rules where the stepsize is tightly controlled a priori, we propose a new class of deflection-restricted approaches where it is the deflection parameter, rather than the stepsize, which is dynamically adjusted using the “target value” of the optimization sequence. For both Polyak-type and diminishing/square summable stepsizes, we propose a “correction” of the standard formula which shows that, in the inexact case, knowledge about the error computed by the oracle (which is available in several practical applications) can be exploited in order to strengthen the convergence properties of the method. The analysis allows for several variants of the algorithm; at least one of them is likely to show numerical performances similar to these of “heavy ball” subgradient methods, popular within backpropagation approaches to train neural networks, while possessing stronger convergence properties.

A Method of Centers with Approximate Subgradient Linearizations for Nonsmooth Convex Optimization

We give a proximal bundle method for constrained convex optimization. It requires only evaluating the problem functions and their subgradients with an unknown accuracy $\epsilon$. Employing a combination of the classic method of centers' improvement function with an exact penalty function, it does not need a feasible starting point. It asymptotically finds points with at least $\epsilon$-optimal objective values that are $\epsilon$-feasible. When applied to the solution of linear programming problems via column generation, it allows for $\epsilon$-accurate solutions of column generation subproblems.

An approximate bundle-type auxiliary problem method for solving generalized variational inequalities

By combining the bundle idea with the auxiliary problem method we present an algorithm for solving a generalized variational inequality problem (GVIP). The proposed algorithm only requires the approximate function values and the approximate subgradients of the involved function instead of the exact ones at some points, which makes the algorithm easier to implement. In addition to that, the conditions imposed on the auxiliary functions are weaker than the preexisting results: convex is enough, not necessarily strongly convex. We also prove the weak convergence of the proposed algorithm under some conditions.

A quasi-Newton bundle method based on approximate subgradients

In this paper we propose an implementable method for solving a nonsmooth convex optimization problem by combining Moreau-Yosida regularization, bundle and quasi-Newton ideas. The method we propose makes use of approximate subgradients of the objective function, which makes the method easier to implement. We also prove the convergence of the proposed method under some additional assumptions.

A Proximal Bundle Method with Approximate Subgradient Linearizations

We give a proximal bundle method for minimizing a convex function f over a closed convex set. It only requires evaluating f and its subgradients with an accuracy $\epsilon>0$, which is fixed but possibly unknown. It asymptotically finds points that are $\epsilon$-optimal. When applied to Lagrangian relaxation, it allows for $\epsilon$-accurate solutions of Lagrangian subproblems and finds $\epsilon$-optimal solutions of convex programs.

Approximate Subgradient Methods for Nonlinearly Constrained Network Flow Problems

The minimization of nonlinearly constrained network flow problems can be performed by using approximate subgradient methods. The idea is to solve this kind of problem by means of primal-dual methods, given that the minimization of nonlinear network flow problems can be done efficiently exploiting the network structure. In this work, it is proposed to solve the dual problem by using e-subgradient methods, as the dual function is estimated by minimizing approximately a Lagrangian function, which includes the side constraints (nonnetwork constraints) and is subject only to the network constraints. Some well-known subgradient methods are modified in order to be used as e-subgradient methods and the convergence properties of these new methods are analyzed. Numerical results appear very promising and effective for this kind of problems

A Proximal Bundle Method Based on Approximate Subgradients

In this paper a proximal bundle method is introduced that is capable to deal with approximate subgradients. No further knowledge of the approximation quality (like explicit knowledge or controllabi...

Approximate subgradients and coderivatives in R n

We show that in two dimensions or higher, the Mordukhovich-Ioffe approximate subdifferential and Clarke subdifferential may differ almost everywhere for real-valued Lipschitz functions. Uncountably many Frechet differentiable vector-valued Lipschitz functions differing by more than constants can share the same Mordukhovich-Ioffe coderivatives. Moreover, the approximate Jacobian associated with the Mordukhovich-Ioffe coderivative can be nonconvex almost everywhere for Frechet differentiable vector-valued Lipschitz functions. Finally we show that for vector-valued Lipschitz functions the approximate Jacobian associated with the Mordukhovich-Ioffe coderivative can be almost everywhere disconnected.

Maximization of Signal-to-Noise Ratio in an Optical Filter

Previous article Next article Maximization of Signal-to-Noise Ratio in an Optical FilterJames E. FalkJames E. Falkhttps://doi.org/10.1137/0117055PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] E. M. L. Beale, On quadratic programming, Naval Res. Logist. Quart., 6 (1959), 227–243 MR0113722 CrossrefGoogle Scholar[2] G. Debreu, The continuity of multi-valued functions in economicsCowles Commission Discussion Paper, Cowles Commission. Yale University, New Haven, Connecticut, 1953 Google Scholar[3] W. Dinkelbach, Die Maximierung eines Quotienten zweier linearen Funktionen unter linearen Nebenbedingungen, Z. Wahrscheinlichkeitstheorie Verw. Gebiete, 1 (1962), 141–145 10.1007/BF01844416 MR0198967 0124.36402 CrossrefGoogle Scholar[4] J. F. Falk, Masters Thesis, A modified Lagrangian approach to nonlinear programming, Doctoral thesis, University of Michigan, Ann Arbor, 1965 Google Scholar[5] H. S. Houthakker, The capacity method of quadratic programming, Econometrica, 28 (1960), 62–87 MR0113723 0089.16001 CrossrefISIGoogle Scholar[6] Samuel Karlin, Mathematical methods and theory in games, programming and economics. Vol. I: Matrix games, programming, and mathematical economics. Vol. II: The theory of infinite games, Addison-Wesley Publishing Co., Inc., Reading, Mass.-London, 1959Vol. I, x+433 pp. Vol. II, xi+386 MR0111634 Google Scholar[7] R. Legault, Optical spectral filtering, to appear Google Scholar[8] D. J. Wilds, Optimum Seeking Methods, Prentice-Hall, Englewood Cliffs, New Jersey, 1964 Google Scholar[9] Philip Wolfe, The simplex method for quadratic programming, Econometrica, 27 (1959), 382–398 MR0106783 0103.37603 CrossrefISIGoogle Scholar Previous article Next article FiguresRelatedReferencesCited ByDetails Sufficient optimality condition and duality of nondifferentiable minimax ratio constraint problems under ( p , r )- ρ -( η , θ )-invexityControl and Cybernetics, Vol. 51, No. 1 | 12 August 2022 Cross Ref Augmented Lagrangian dual for nonconvex minimax fractional programs and proximal bundle algorithms for its resolutionJournal of Industrial and Management Optimization, Vol. 0, No. 0 | 1 Jan 2022 Cross Ref Proximal bundle methods based on approximate subgradients for solving Lagrangian duals of minimax fractional programsJournal of Global Optimization, Vol. 74, No. 2 | 11 March 2019 Cross Ref Duality Results and Dual Bundle Methods Based on the Dual Method of Centers for Minimax Fractional ProgramsKarima Boufi and Ahmed RoubiSIAM Journal on Optimization, Vol. 29, No. 2 | 4 June 2019AbstractPDF (468 KB)Proximal Bundle Algorithms for Nonlinearly Constrained Convex Minimax Fractional ProgramsJournal of Optimization Theory and Applications, Vol. 179, No. 1 | 10 July 2018 Cross Ref A nonlinear fractional programming approach for environmental–economic power dispatchInternational Journal of Electrical Power & Energy Systems, Vol. 78 | 1 Jun 2016 Cross Ref Bilevel fractional programming; Farkas lemma; Farkas lemma: Generalizations; Fractional combinatorial optimization; Quadratic fractional programming: Dinkelbach methodBilevel fractional programming; Fractional combinatorial optimization; Quadratic fractional programming: Dinkelbach methodFRACTIONAL PROGRAMMINGEncyclopedia of Optimization | 1 Jan 2001 Cross Ref Fractional ProgrammingHandbook of Global Optimization | 1 Jan 1995 Cross Ref Bibliography in fractional programmingZeitschrift für Operations Research, Vol. 26, No. 1 | 1 Dec 1982 Cross Ref Linear Restoration of Incoherently Radiating ObjectsJournal of the Optical Society of America, Vol. 62, No. 3 | 1 March 1972 Cross Ref Fractional ProgrammingHandbook of Generalized Convexity and Generalized Monotonicity Cross Ref Fractional ProgrammingEncyclopedia of Optimization Cross Ref Volume 17, Issue 3| 1969SIAM Journal on Applied Mathematics495-613 History Submitted:17 May 1967Accepted:24 June 1968Published online:12 July 2006 InformationCopyright © 1969 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/0117055Article page range:pp. 582-592ISSN (print):0036-1399ISSN (online):1095-712XPublisher:Society for Industrial and Applied Mathematics