Proximal Bundle Method Research Articles

Entry trajectory optimization of lifting-body vehicle by successive difference-of-convex programming

The complexity of the three-dimensional entry trajectory optimization problem has escalated due to the need to liberalize the angle of attack and bank angle as control variables, thereby enhancing the inherent maneuverability and control capabilities of lifting-body vehicles. The difference-of-convex (DC) properties inherent in the constraints of the problem are exploited in this paper. A DC decomposition approach is utilized to address the nonlinear auxiliary control equations, and the DC relaxation technique is applied to resolve iteration infeasibilities arising from Taylor expansion. The dependence on the initial trajectory is diminished by the implementation of an exact penalty method, thus improving the applicability of the methods. Furthermore, a control variable oscillation suppression mechanism has been constructed to tackle the control variable oscillation issues arising from the relaxation of the angle of attack and bank angle. This mechanism effectively suppresses large jumps in the angle of attack and high-frequency oscillations in the bank angle. Two novel successive DC programming methods are proposed: the successive concave-convex procedure and the successive proximal bundle method, functioning independently of trust-region constraints. Numerical experiments have demonstrated that the two proposed successive DC optimization methods exhibit exceptional performance in accuracy, feasibility, adaptability, and low sensitivity to initial values when applied to solving the three-dimensional entry trajectory optimization problem.

An asynchronous proximal bundle method

AbstractWe develop a fully asynchronous proximal bundle method for solving non-smooth, convex optimization problems. The algorithm can be used as a drop-in replacement for classic bundle methods, i.e., the function must be given by a first-order oracle for computing function values and subgradients. The algorithm allows for an arbitrary number of master problem processes computing new candidate points and oracle processes evaluating functions at those candidate points. These processes share information by communication with a single supervisor process that resembles the main loop of a classic bundle method. All processes run in parallel and no explicit synchronization step is required. Instead, the asynchronous and possibly outdated results of the oracle computations can be seen as an inexact function oracle. Hence, we show the convergence of our method under weak assumptions very similar to inexact and incremental bundle methods. In particular, we show how the algorithm learns important structural properties of the functions to control the inaccuracy induced by the asynchronicity automatically such that overall convergence can be guaranteed.

Linear Convergence of the Derivative-Free Proximal Bundle Method on Convex Nonsmooth Functions, with Application to the Derivative-Free VU\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{VU}$\\end{document}-Algorithm

Proximal bundle methods are a class of optimisation algorithms that leverage the proximal operator to address nonsmoothness in the objective function efficiently. This study focuses on a derivative-free (DFO) proximal bundle method and one of its applications called the DFO VU\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{VU}$\\end{document}-algorithm. These algorithms incorporate approximate proximal points as subprocedures in order to optimise convex nonsmooth functions based on approximated subdifferential information. Interestingly, the classical VU\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{VU}$\\end{document}-algorithm, which operates on true subgradient values, achieves superlinear convergence. At each iteration, the algorithm divides the whole space into two: the smooth U\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{U}$\\end{document}-space and the nonsmooth V\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{V}$\\end{document}-space. It takes a Newton-like step on the U\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{U}$\\end{document}-space and a proximal-point step on the V\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{V}$\\end{document}-space, enabling it to handle both smooth and nonsmooth parts effectively and converge faster. In this work, we reveal the worst possible convergence rate for the DFO VU\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\mathcal{VU}$\\end{document}-method by showing the linear convergence of the DFO proximal bundle method. This will be done by presenting a suitable framework and using the subdifferential-based error bound on the distance to critical points.

A single cut proximal bundle method for stochastic convex composite optimization

A single cut proximal bundle method for stochastic convex composite optimization

Difference-of-Convex approach to chance-constrained Optimal Power Flow modelling the DSO power modulation lever for distribution networks

The increasing expansion of renewable energy sources leads to the growth of uncertainty in the distribution network operation. Short-term operational planning performed by distribution system operators should evolve to address those new operating conditions, in particular to allow the efficient utilization of different flexibility levers. In this work, the use of a chance-constrained Alternating Current Optimal Power Flow (AC-OPF) is proposed to model the operational planning problem, considering the activation of several levers such as power modulation and power curtailment. The correlation between the renewable generation profiles and loads is considered via a joint probability constraint approximated with scenarios. The main novelty of the present manuscript is the adoption of a Difference-of-Convex approach that allows to solve the obtained optimization problem without convexification or linearization of the core OPF equations. Furthermore, the approach yields a natural and embarrassingly parallelizable scenario decomposition. The method starts with a reformulation of the model as a Difference-of-Convex optimization problem, and then a proximal bundle method algorithm is applied to solve it. The proposed methodology is tested in a 33 bus distribution network with 11 different values for the safety level defining the probability constraint, ranging from 0.75 to 1.

A Unified Analysis of a Class of Proximal Bundle Methods for Solving Hybrid Convex Composite Optimization Problems

This paper presents a proximal bundle (PB) framework based on a generic bundle update scheme for solving the hybrid convex composite optimization (HCCO) problem and establishes a common iteration-complexity bound for any variant belonging to it. As a consequence, iteration-complexity bounds for three PB variants based on different bundle update schemes are obtained in the HCCO context for the first time and in a unified manner. Although two of the PB variants are universal (i.e., their implementations do not require parameters associated with the HCCO instance), the other newly (as far as the authors are aware) proposed one is not, but has the advantage that it generates simple—namely, one-cut—bundle models. The paper also presents a universal adaptive PB variant (which is not necessarily an instance of the framework) based on one-cut models and shows that its iteration-complexity is the same as the two aforementioned universal PB variants. Funding: Financial support from the Office of Naval Research [N00014-18-1-2077] and the Air Force Office of Scientific Research [Grant FA9550-22-1-0088] is gratefully acknowledged.

A redistributed proximal bundle method for nonsmooth nonconvex functions with inexact information

In this paper, we propose a redistributed proximal bundle method for a class of nonconvex nonsmooth optimization problems with inexact information, i.e., we consider the problem of computing the approximate critical points when only the inexact information about the function values and subgradients are available and show that reasonable convergence properties are obtained. We assume that the errors in the computation of functions and subgradients are only bounded and in principle do not have to vanish within the limits. For the nonconvex functions, we design the convexification technique, which ensures that the linearization error of its augmentation function is nonnegative. Meanwhile, for the inexact information, we utilize noise management strategies and update approximate parameters to reduce the impact of inexact information. Based on this method, we can obtain the approximate solution.

A proximal bundle method-based algorithm with penalty strategy and inexact oracles for constrained nonsmooth nonconvex optimization

In this paper, we consider a class of nonconvex nonsmooth constrained problems with inexact data. To deal with the constraints, the penalty strategy is adopted during the process to transfer the primal problem into an unconstrained problem. For inexact data, we add quadratic term with variable coefficient to each term in the unconstrained problem to keep the corresponding linearization errors nonnegative. For the modified unconstrained problem, we utilize the disaggregate technique and design corresponding cutting planes for each function. Meanwhile, the sum of the corresponding cutting planes is regarded as the cutting plane for the modified unconstrained problem and proximal bundle method is adopted to deal with the model. The preliminary numerical results show our algorithm is effective and attractive.

A class of infeasible proximal bundle methods for nonsmooth nonconvex multi-objective optimization problems

A class of infeasible proximal bundle methods for nonsmooth nonconvex multi-objective optimization problems

A proximal bundle algorithm for nonsmooth optimization on Riemannian manifolds

Abstract Proximal bundle methods are among the most successful approaches for convex and nonconvex optimization problems in linear spaces and it is natural to extend these methods to the manifold setting. In this paper we propose a proximal bundle method for solving nonsmooth, nonconvex optimization problems on Riemannian manifolds. At every iteration, by using the proximal bundle method a candidate descent direction is computed and by employing a restricted-step procedure and a retraction the next iterate is built. The global convergence, starting from any point, is proved in the sense that if the number of serious iterates is finite then the last serious iterate is stationary and otherwise every accumulation point of the serious iterate sequence is stationary. At the end, some numerical experiments are provided to illustrate the effectiveness of the proposed method and clarify the theoretical results.

Lagrangian Relaxation Based on Improved Proximal Bundle Method for Short-Term Hydrothermal Scheduling

Short-term hydrothermal scheduling (STHS) can improve water use efficiency, reduce carbon emissions, and increase economic benefits by optimizing the commitment and dispatch of hydro and thermal generating units together. However, limited by the large system scale and complex hydraulic and electrical constraints, STHS poses great challenges in modeling for operators. This paper presents an improved proximal bundle method (IPBM) within the framework of Lagrangian relaxation for STHS, which incorporates the expert system (ES) technique into the proximal bundle method (PBM). In IPBM, initial values of Lagrange multipliers are firstly determined using the linear combination of optimal solutions in the ES. Then, each time PBM declares a null step in the iterations, the solution space is inferred from the ES, and an orthogonal design is performed in the solution space to derive new updates of the Lagrange multipliers. A case study in a large-scale hydrothermal system in China is implemented to demonstrate the effectiveness of the proposed method. Results in different cases indicate that IPBM is superior to standard PBM in global search ability and computational efficiency, providing an alternative for STHS.

A Redistributed Bundle Algorithm Based on Local Convexification Models for Nonlinear Nonsmooth DC Programming

Abstract For nonlinear nonsmooth DC programming (difference of convex functions), we introduce a new redistributed proximal bundle method. The subgradient information of both the DC components is gathered from some neighbourhood of the current stability center and it is used to build separately an approximation for each component in the DC representation. Especially we employ the nonlinear redistributed technique to model the second component of DC function by constructing a local convexification cutting plane. The corresponding convexification parameter is adjusted dynamically and is taken sufficiently large to make the ”augmented” linearization errors nonnegative. Based on above techniques we obtain a new convex cutting plane model of the original objective function. Based on this new approximation the redistributed proximal bundle method is designed and the convergence of the proposed algorithm to a Clarke stationary point is proved. A simple numerical experiment is given to show the validity of the presented algorithm.

A Proximal Bundle Method with Exact Penalty Technique and Bundle Modification Strategy for Nonconvex Nonsmooth Constrained Optimization

The bundle modification strategy for the convex unconstrained problems was proposed by Alexey et al. [[2007] European Journal of Operation Research, 180(1), 38–47.] whose most interesting feature was the reduction of the calls for the quadratic programming solver. In this paper, we extend the bundle modification strategy to a class of nonconvex nonsmooth constraint problems. Concretely, we adopt the convexification technique to the objective function and constraint function, take the penalty strategy to transfer the modified model into an unconstrained optimization and focus on the unconstrained problem with proximal bundle method and the bundle modification strategies. The global convergence of the corresponding algorithm is proved. The primal numerical results show that the proposed algorithms are promising and effective.

An Adaptive Proximal Bundle Method with Inexact Oracles for a Class of Nonconvex and Nonsmooth Composite Optimization

In this paper, an adaptive proximal bundle method is proposed for a class of nonconvex and nonsmooth composite problems with inexact information. The composite problems are the sum of a finite convex function with inexact information and a nonconvex function. For the nonconvex function, we design the convexification technique and ensure the linearization errors of its augment function to be nonnegative. Then, the sum of the convex function and the augment function is regarded as an approximate function to the primal problem. For the approximate function, we adopt a disaggregate strategy and regard the sum of cutting plane models of the convex function and the augment function as a cutting plane model for the approximate function. Then, we give the adaptive nonconvex proximal bundle method. Meanwhile, for the convex function with inexact information, we utilize the noise management strategy and update the proximal parameter to reduce the influence of inexact information. The method can obtain an approximate solution. Two polynomial functions and six DC problems are referred to in the numerical experiment. The preliminary numerical results show that our algorithm is effective and reliable.

An Efficient Descent Method for Locally Lipschitz Multiobjective Optimization Problems

We present an efficient descent method for unconstrained, locally Lipschitz multiobjective optimization problems. The method is realized by combining a theoretical result regarding the computation of descent directions for nonsmooth multiobjective optimization problems with a practical method to approximate the subdifferentials of the objective functions. We show convergence to points which satisfy a necessary condition for Pareto optimality. Using a set of test problems, we compare our method with the multiobjective proximal bundle method by Mäkelä. The results indicate that our method is competitive while being easier to implement. Although the number of objective function evaluations is larger, the overall number of subgradient evaluations is smaller. Our method can be combined with a subdivision algorithm to compute entire Pareto sets of nonsmooth problems. Finally, we demonstrate how our method can be used for solving sparse optimization problems, which are present in many real-life applications.

A Proximal Bundle Variant with Optimal Iteration-Complexity for a Large Range of Prox Stepsizes

This paper presents a proximal bundle variant, namely, the relaxed proximal bundle (RPB) method, for solving convex nonsmooth composite optimization problems. Like other proximal bundle variants, R...

An inexact multiple proximal bundle algorithm for nonsmooth nonconvex multiobjective optimization problems

For a class of nonsmooth nonconvex multiobjective problems, we develop an inexact multiple proximal bundle method. In our approach instead of scalarization, we find descent direction for every objective function separately by utilizing the inexact proximal bundle method. Then we attempt to find a common descent direction for all objective functions. We study the effect of the inexactness of the objective and subgradient values on the new proposed method and obtain the reasonable convergence properties. We further consider a class of difficult nonsmooth nonconvex problems, made even more difficult by inserting the inexactness in the available information. At the end, to demonstrate the efficiency of the proposed algorithm, some encouraging numerical experiments are provided.

Resource allocation for contingency planning: An inexact proximal bundle method for stochastic optimization

Resource contingency planning aims to mitigate the effects of unexpected disruptions in supply chains. While these failures occur infrequently, they often have disastrous consequences. This paper formulates the resource allocation problem in contingency planning as a two-stage stochastic optimization problem with a risk-averse recourse function. The solution method proposed relies on an inexact proximal bundle method with subgradient approximations through a scenario reduction mechanism. The paper extends the inexact oracle to a more general risk-averse setting, and proves that it meets the requirements of the oracle in the inexact bundle method, ensuring convergence to an optimal solution. The practical performance of the developed inexact bundle method under risk aversion is investigated for our resource allocation problem. We create a library of test problems and obtain their optimal values by applying the exact bundle method. The computed solutions from the developed inexact bundle method are compared against these optimal values, under different coherent risk measures. Our analyses indicate that our inexact bundle method significantly reduces the computational time of solving the resource allocation problem in comparison to the exact bundle method, and is capable of achieving a high percentage of optimality within a much shorter time.

A filter proximal bundle method for nonsmooth nonconvex constrained optimization

A filter proximal bundle algorithm is presented for nonsmooth nonconvex constrained optimization problems. The new algorithm is based on the proximal bundle method and utilizes the improvement function to regularize the constraint. At every iteration by solving a convex piecewise-linear subproblem a trial point is obtained. The process of the filter technique is employed either to accept the trial point as a serious iterate or to reject it as a null iterate. Under some mild and standard assumptions and for every possible choice of a starting point, it is shown that every accumulation point of the sequence of serious iterates is feasible. In addition, there exists at least one accumulation point which is stationary for the improvement function. Finally, some encouraging numerical results show that the proposed algorithm is effective.

A Long-Term Evaluation on Transmission Line Expansion Planning with Multistage Stochastic Programming

The purpose of this paper is to apply multistage stochastic programming to the transmission line expansion planning problem, especially when uncertain demand scenarios exist. Since the problem of transmission line expansion planning requires an intensive computational load, dual decomposition is used to decompose the problem into smaller problems. Following this, progressive hedging and proximal bundle methods are used to restore the decomposed solutions to the original problems. Mixed-integer linear programming is involved in the problem to decide where new transmission lines should be constructed or reinforced. However, integer variables in multistage stochastic programming (MSSP) are intractable since integer variables are not restored. Therefore, the branch-and-bound algorithm is applied to multistage stochastic programming methods to force convergence of integer variables.In addition, this paper suggests combining progressive hedging and dual decomposition in stochastic integer programming by sharing penalty parameters. The simulation results tested on the IEEE 30-bus system verify that our combined model sped up the computation and achieved higher accuracy by achieving the minimised cost.