Proximal Bundle Method Research Articles

A transformation-proximal bundle algorithm for multistage adaptive robust optimization and application to constrained robust optimal control

This paper presents a novel transformation-proximal bundle algorithm for multistage adaptive robust optimization problems. By partitioning recourse decisions into state and control decisions, the proposed algorithm applies affine control policy only to state decisions and allows control decisions to be fully adaptive, thus transforming the original problem into an equivalent two-stage Adaptive Robust Optimization (ARO) problem. Importantly, this multi-to-two transformation is general enough to be employed with any two-stage ARO solution algorithms, thus opening a new avenue for a variety of multistage ARO algorithms. The proximal bundle method is developed for the resulting two-stage problem along with convergence analysis. In an inventory control application, the affine disturbance-feedback control policy suffers from a severe suboptimality with an average gap of 34.88%, while the proposed algorithm generates an average gap of merely 1.68%.

A new infeasible proximal bundle algorithm for nonsmooth nonconvex constrained optimization

Proximal bundle method has usually been presented for unconstrained convex optimization problems. In this paper, we develop an infeasible proximal bundle method for nonsmooth nonconvex constrained optimization problems. Using the improvement function we transform the problem into an unconstrained one and then we build a cutting plane model. The resulting algorithm allows effective control of the size of quadratic programming subproblems via the aggregation techniques. The novelty in our approach is that the objective and constraint functions can be any arbitrary (regular) locally Lipschitz functions. In addition the global convergence, starting from any point, is proved in the sense that every accumulation point of the iterative sequence is stationary for the improvement function. At the end, some encouraging numerical results with a MATLAB implementation are also reported.

Proximal bundle methods based on approximate subgradients for solving Lagrangian duals of minimax fractional programs

We are interested in this work to solve the Lagrangian dual of a generalized fractional program (GFP), which gives the minimal value of the primal problem, thanks to some duality results. With the help of a general minimax equality assumption, we give duality results under minimal assumptions. Since the associated parametric programs of the dual of GFP are always concave, we use a general approximating proximal scheme to these subproblems and construct bundle methods by means of approximate values and approximate subgradients of the dual parametric function. As for all dual algorithms, the proposed methods generate a sequence of values that converges from below to the minimal value of the GFP and a sequence of approximate solutions that converges to a solution of the Lagrangian dual of the GFP. For certain classes of problems, the convergence is at least linear.

Proximal bundle methods for nonsmooth DC programming

We consider the problem of minimizing the difference of two nonsmooth convex functions over a simple convex set. To deal with this class of nonsmooth and nonconvex optimization problems, we propose new proximal bundle algorithms and show that the given approaches generate subsequences of iterates that converge to critical points. Trial points are obtained by solving strictly convex master programs defined by the sum of a convex cutting-plane model and a freely-chosen Bregman function. In the unconstrained case with the Bregman function being the Euclidean distance, new iterates are solutions of strictly convex quadratic programs of limited sizes. Stronger convergence results (d-stationarity) can be achieved depending on (a) further assumptions on the second DC component of the objective function and (b) solving possibly more than one master program at certain iterations. The given approaches are validated by encouraging numerical results on some academic DC programs.

Dual algorithms based on the proximal bundle method for solving convex minimax fractional programs

In this work, we propose an approximating scheme based on the proximal point algorithm, for solving generalized fractional programs (GFP) by their continuous reformulation, also known to as partial dual counterparts of GFP. Bundle dual algorithms are then derived from this scheme. We prove the convergence and the rate of convergence of these algorithms. As for dual algorithms, the proposed methods generate a sequence of values that converges from below to the minimal value of \begin{document}$(P)$\end{document} , and a sequence of approximate solutions that converges to a solution of the dual problem. For certain classes of problems, the convergence is at least linear.

A parallelizable augmented Lagrangian method applied to large-scale non-convex-constrained optimization problems

We contribute improvements to a Lagrangian dual solution approach applied to large-scale optimization problems whose objective functions are convex, continuously differentiable and possibly nonlinear, while the non-relaxed constraint set is compact but not necessarily convex. Such problems arise, for example, in the split-variable deterministic reformulation of stochastic mixed-integer optimization problems. We adapt the augmented Lagrangian method framework to address the presence of nonconvexity in the non-relaxed constraint set and to enable efficient parallelization. The development of our approach is most naturally compared with the development of proximal bundle methods and especially with their use of serious step conditions. However, deviations from these developments allow for an improvement in efficiency with which parallelization can be utilized. Pivotal in our modification to the augmented Lagrangian method is an integration of the simplicial decomposition method and the nonlinear block Gauss–Seidel method. An adaptation of a serious step condition associated with proximal bundle methods allows for the approximation tolerance to be automatically adjusted. Under mild conditions optimal dual convergence is proven, and we report computational results on test instances from the stochastic optimization literature. We demonstrate improvement in parallel speedup over a baseline parallel approach.

Incremental Bundle Methods using Upper Models

We propose a family of proximal bundle methods for minimizing sum-structured convex nondifferentiable functions which require two slightly uncommon assumptions that are satisfied in many relevant applications: Lipschitz continuity of the functions and oracles which also produce upper estimates on the function values. In exchange, the methods: (i) use upper models of the functions that allow one to estimate function values at points where the oracle has not been called; (ii) provide the oracles with more information about when the function computation can be interrupted, possibly diminishing their cost; (iii) allow one to skip oracle calls entirely for some of the component functions, not only at “null steps” but also at “serious steps''; (iv) provide explicit and reliable a posteriori estimates of the quality of the obtained solutions; (v) work with all possible combinations of different assumptions on how the oracles deal with not being able to compute the function with arbitrary accuracy. We also discus...

A proximal bundle method for constrained nonsmooth nonconvex optimization with inexact information

We propose an inexact proximal bundle method for constrained nonsmooth nonconvex optimization problems whose objective and constraint functions are known through oracles which provide inexact information. The errors in function and subgradient evaluations might be unknown, but are merely bounded. To handle the nonconvexity, we first use the redistributed idea, and consider even more difficulties by introducing inexactness in the available information. We further examine the modified improvement function for a series of difficulties caused by the constrained functions. The numerical results show the good performance of our inexact method for a large class of nonconvex optimization problems. The approach is also assessed on semi-infinite programming problems, and some encouraging numerical experiences are provided.

Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations

We introduce a proximal bundle method for the numerical minimization of a nonsmooth difference-of-convex (DC) function. Exploiting some classic ideas coming from cutting-plane approaches for the convex case, we iteratively build two separate piecewise-affine approximations of the component functions, grouping the corresponding information in two separate bundles. In the bundle of the first component, only information related to points close to the current iterate are maintained, while the second bundle only refers to a global model of the corresponding component function. We combine the two convex piecewise-affine approximations, and generate a DC piecewise-affine model, which can also be seen as the pointwise maximum of several concave piecewise-affine functions. Such a nonconvex model is locally approximated by means of an auxiliary quadratic program, whose solution is used to certify approximate criticality or to generate a descent search-direction, along with a predicted reduction, that is next explored in a line-search setting. To improve the approximation properties at points that are far from the current iterate a supplementary quadratic program is also introduced to generate an alternative more promising search-direction. We discuss the main convergence issues of the line-search based proximal bundle method, and provide computational results on a set of academic benchmark test problems.

Proximal Bundle Method for Contact Shape Optimization Problem

From the mathematical point of view, the contact shape optimization is a problem of nonlinear optimization with a specific structure, which can be exploited in its solution. In this paper, we show how to overcome the difficulties related to the nonsmooth cost function by using the proximal bundle methods. We describe all steps of the solution, including linearization, construction of a descent direction, line search, stopping criterion, etc. To illustrate the performance of the presented algorithm, we solve a shape optimization problem associated with the discretized two-dimensional contact problem with Coulomb's friction.

Optimized Color Filter Arrays for Sparse Representation-Based Demosaicking

Demosaicking is the problem of reconstructing a color image from the raw image captured by a digital color camera that covers its only imaging sensor with a color filter array (CFA). Sparse representation-based demosaicking has been shown to produce superior reconstruction quality. However, almost all existing algorithms in this category use the CFAs, which are not specifically optimized for the algorithms. In this paper, we consider optimally designing CFAs for sparse representation-based demosaicking, where the dictionary is well-chosen. The fact that CFAs correspond to the projection matrices used in compressed sensing inspires us to optimize CFAs via minimizing the mutual coherence. This is more challenging than that for traditional projection matrices because CFAs have physical realizability constraints. However, most of the existing methods for minimizing the mutual coherence require that the projection matrices should be unconstrained, making them inapplicable for designing CFAs. We consider directly minimizing the mutual coherence with the CFA's physical realizability constraints as a generalized fractional programming problem, which needs to find sufficiently accurate solutions to a sequence of nonconvex nonsmooth minimization problems. We adapt the redistributed proximal bundle method to address this issue. Experiments on benchmark images testify to the superiority of the proposed method. In particular, we show that a simple sparse representation-based demosaicking algorithm with our specifically optimized CFA can outperform LSSC [1]. To the best of our knowledge, it is the first sparse representation-based demosaicking algorithm that beats LSSC in terms of CPSNR.

A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes

In this paper, we develop a version of the bundle method to solve unconstrained difference of convex (DC) programming problems. It is assumed that a DC representation of the objective function is available. Our main idea is to utilize subgradients of both the first and second components in the DC representation. This subgradient information is gathered from some neighborhood of the current iteration point and it is used to build separately an approximation for each component in the DC representation. By combining these approximations we obtain a new nonconvex cutting plane model of the original objective function, which takes into account explicitly both the convex and the concave behavior of the objective function. We design the proximal bundle method for DC programming based on this new approach and prove the convergence of the method to an $$\varepsilon $$ź-critical point. The algorithm is tested using some academic test problems and the preliminary numerical results have shown the good performance of the new bundle method. An interesting fact is that the new algorithm finds nearly always the global solution in our test problems.

A Second-Order Bundle Method Based on -Decomposition Strategy for a Special Class of Eigenvalue Optimizations

ABSTRACTIn the past decade, eigenvalue optimization has gained remarkable attention in various engineering applications. One of the main difficulties with numerical analysis of such problems is that the eigenvalues, considered as functions of a symmetric matrix, are not smooth at those points where they are multiple. We propose a new explicit nonsmooth second-order bundle algorithm based on the idea of the proximal bundle method on minimizing the arbitrary eigenvalue over an affine family of symmetric matrices, which is a special class of eigenvalue function–D.C. function. To the best of our knowledge, few methods currently exist for minimizing arbitrary eigenvalue function. In this work, we apply the -Lagrangian theory to this class of D.C. functions: the arbitrary eigenvalue function λi with affine matrix-valued mappings, where λi is usually not convex. We prove the global convergence of our method in the sense that every accumulation point of the sequence of iterates is stationary. Moreover, under mild conditions we show that, if started close enough to the minimizer x*, the proposed algorithm converges to x* quadratically. The method is tested on some constrained optimization problems, and some encouraging preliminary numerical results show the efficiency of our method.

Special backtracking proximal bundle method for nonconvex maximum eigenvalue optimization

We present a proximal bundle method for minimizing the nonconvex maximum eigenvalue function based on a real time control system. The oracle used in our proximal bundle method is able to compute separately the value and subgradient of the outer convex function. Besides, it can also calculate the value and derivatives of the smooth inner mapping. In each iteration, we solve a certain quadratic programming problem in which the smooth inner mapping is replaced by its Taylor-series linearization around the current serious step. By using the backtracking test, we can make a better approximation of the objective function. With no additional assumption, we prove the global convergence of our special bundle method. We present numerical examples demonstrating the efficiency of our algorithm on several feedback control syntheses.

A doubly stabilized bundle method for nonsmooth convex optimization

We propose a bundle method for minimizing nonsmooth convex functions that combines both the level and the proximal stabilizations. Most bundle algorithms use a cutting-plane model of the objective function to formulate a subproblem whose solution gives the next iterate. Proximal bundle methods employ the model in the objective function of the subproblem, while level methods put the model in the subproblem’s constraints. The proposed algorithm defines new iterates by solving a subproblem that employs the model in both the objective function and in the constraints. One advantage when compared to the proximal approach is that the level set constraint provides a certain Lagrange multiplier, which is used to update the proximal parameter in a novel manner. We also show that in the case of inexact function and subgradient evaluations, no additional procedure needs to be performed by our variant to deal with inexactness (as opposed to the proximal bundle methods that require special modifications). Numerical experiments on almost 1,000 instances of different types of problems are presented. Our experiments show that the doubly stabilized bundle method inherits useful features of the level and the proximal versions, and compares favorably to both of them.

A strongly convergent proximal bundle method for convex minimization in Hilbert spaces

A key procedure in proximal bundle methods for convex minimization problems is the definition of stability centres, which are points generated by the iterative process that successfully decrease the objective function. In this paper we study a different stability-centre classification rule for proximal bundle methods. We show that the proposed bundle variant has at least two particularly interesting features: (i) the sequence of stability centres generated by the method converges strongly to the solution that lies closest to the initial point; (ii) if the sequence of stability centres is finite, being its last element, then the sequence of non-stability centres (null steps) converges strongly to . Property (i) is useful in some practical applications in which a minimal norm solution is required. We show the interest of this property on several instances of a full sized unit-commitment problem.

An Approximate Redistributed Proximal Bundle Method with Inexact Data for Minimizing Nonsmooth Nonconvex Functions

We describe an extension of the redistributed technique form classical proximal bundle method to the inexact situation for minimizing nonsmooth nonconvex functions. The cutting-planes model we construct is not the approximation to the whole nonconvex function, but to the local convexification of the approximate objective function, and this kind of local convexification is modified dynamically in order to always yield nonnegative linearization errors. Since we only employ the approximate function values and approximate subgradients, theoretical convergence analysis shows that an approximate stationary point or some double approximate stationary point can be obtained under some mild conditions.

Convex proximal bundle methods in depth: a unified analysis for inexact oracles

The last few years have seen the advent of a new generation of bundle methods, capable to handle inexact oracles, polluted by "noise". Proving convergence of a bundle method is never simple and coping with inexact oracles substantially increases the technicalities. Besides, several variants exist to deal with noise, each one needing an ad hoc proof to show convergence. We state a synthetic convergence theory, in which we highlight the main arguments and specify which assumption is used to establish each intermediate result. The framework is comprehensive and generalizes in various ways a number of algorithms proposed in the literature. Based on the ingredients of our synthetic theory, we consider various bundle methods adapted to oracles for which high accuracy is possible, yet it is preferable not to make exact calculations often, because they are too time consuming.

SOLVING A MINIMIZATION PROBLEM FOR A CLASS OF CONSTRAINED MAXIMUM EIGENVALUE FUNCTION

Nonsmooth convex optimization problem is a class of important problems in operational research. Bundle methods are considered as one of the most efficient methods for solving nonsmooth optimization problems. The methods have already been applied to many practical problems. In this paper, using bundle method, the optimization problem that the sum of maximum eigenvalue function and general non-smooth convex function can be solved. Through approximation to the objective function, the proximal bundle method based on approximate model is given. We prove that the sequences generated by the algorithm converge to the optimal solution of the original problem. Finally, the algorithm is used to solve a class of constrained maximum eigenvalue function.

Constrained Nonconvex Nonsmooth Optimization via Proximal Bundle Method

In this paper, we consider a constrained nonconvex nonsmooth optimization, in which both objective and constraint functions may not be convex or smooth. With the help of the penalty function, we transform the problem into an unconstrained one and design an algorithm in proximal bundle method in which local convexification of the penalty function is utilized to deal with it. We show that, if adding a special constraint qualification, the penalty function can be an exact one, and the sequence generated by our algorithm converges to the KKT points of the problem under a moderate assumption. Finally, some illustrative examples are given to show the good performance of our algorithm.