Class Of Nonconvex Optimization Problems Research Articles

Efficiency of quantum vs. classical annealing in nonconvex learning problems

Quantum annealers aim at solving nonconvex optimization problems by exploiting cooperative tunneling effects to escape local minima. The underlying idea consists of designing a classical energy function whose ground states are the sought optimal solutions of the original optimization problem and add a controllable quantum transverse field to generate tunneling processes. A key challenge is to identify classes of nonconvex optimization problems for which quantum annealing remains efficient while thermal annealing fails. We show that this happens for a wide class of problems which are central to machine learning. Their energy landscapes are dominated by local minima that cause exponential slowdown of classical thermal annealers while simulated quantum annealing converges efficiently to rare dense regions of optimal solutions.

DC approach to weakly convex optimization and nonconvex quadratic optimization problems

Motivated by weakly convex optimization and quadratic optimization problems, we first show that there is no duality gap between a difference of convex (DC) program over DC constraints and its associated dual problem. We then provide certificates of global optimality for a class of nonconvex optimization problems. As an application, we derive characterizations of robust solutions for uncertain general nonconvex quadratic optimization problems over nonconvex quadratic constraints.

A proximal bundle method for constrained nonsmooth nonconvex optimization with inexact information

We propose an inexact proximal bundle method for constrained nonsmooth nonconvex optimization problems whose objective and constraint functions are known through oracles which provide inexact information. The errors in function and subgradient evaluations might be unknown, but are merely bounded. To handle the nonconvexity, we first use the redistributed idea, and consider even more difficulties by introducing inexactness in the available information. We further examine the modified improvement function for a series of difficulties caused by the constrained functions. The numerical results show the good performance of our inexact method for a large class of nonconvex optimization problems. The approach is also assessed on semi-infinite programming problems, and some encouraging numerical experiences are provided.

Peaceman–Rachford splitting for a class of nonconvex optimization problems

We study the applicability of the Peaceman–Rachford (PR) splitting method for solving nonconvex optimization problems. When applied to minimizing the sum of a strongly convex Lipschitz differentiable function and a proper closed function, we show that if the strongly convex function has a large enough strong convexity modulus and the step-size parameter is chosen below a threshold that is computable, then any cluster point of the sequence generated, if exists, will give a stationary point of the optimization problem. We also give sufficient conditions guaranteeing boundedness of the sequence generated. We then discuss one way to split the objective so that the proposed method can be suitably applied to solving optimization problems with a coercive objective that is the sum of a (not necessarily strongly) convex Lipschitz differentiable function and a proper closed function; this setting covers a large class of nonconvex feasibility problems and constrained least squares problems. Finally, we illustrate the proposed algorithm numerically.

Solving DC programs with a polyhedral component utilizing a multiple objective linear programming solver

A class of non-convex optimization problems with DC objective function is studied, where DC stands for being representable as the difference $f=g-h$ of two convex functions $g$ and $h$. In particular, we deal with the special case where one of the two convex functions $g$ or $h$ is polyhedral. In case $g$ is polyhedral, we show that a solution of the DC program can be obtained from a solution of an associated polyhedral projection problem. In case $h$ is polyhedral, we prove that a solution of the DC program can be obtained by solving a polyhedral projection problem and finitely many convex programs. Since polyhedral projection is equivalent to multiple objective linear programming (MOLP), a MOLP solver (in the second case together with a convex programming solver) can be used to solve instances of DC programs with polyhedral component. Numerical examples are provided, among them an application to locational analysis.

The Local Convexity of Solving Systems of Quadratic Equations

This paper considers the recovery of a rank r positive semidefinite matrix \({X X^T \in \mathbb{R}^{n\times n}}\) from m scalar measurements of the form \({y_i := a_i^T X X^T a_i}\) (i.e., quadratic measurements of X). Such problems arise in a variety of applications, including covariance sketching of high-dimensional data streams, quadratic regression, quantum state tomography, among others. A natural approach to this problem is to minimize the loss function \({f(U) = \sum_i (y_i - a_{i}^{T}UU^{T}a_i)^2}\) which has an entire manifold of solutions given by \({\{XO\}_{O\in\mathcal{O}_r}}\) where \({\mathcal{O}_r}\) is the orthogonal group of \({r\times r}\) orthogonal matrices; this is non-convex in the \({n\times r}\) matrix U, but methods like gradient descent are simple and easy to implement (as compared to semidefinite relaxation approaches). In this paper we show that once we have \({m \geq Cnr\,{\rm log}^2(n)}\) samples from isotropic gaussian \({a_i}\), with high probability (a) this function admits a dimension-independent region of local strong convexity on lines perpendicular to the solution manifold, and (b) with an additional polynomial factor of r samples, a simple spectral initialization will land within the region of convexity with high probability. Together, this implies that gradient descent with initialization (but no re-sampling) will converge linearly to the correct X, up to an orthogonal transformation. We believe that this general technique (local convexity reachable by spectral initialization) should prove applicable to a broader class of nonconvex optimization problems.

Robust static and fixed‐order dynamic output feedback control of discrete‐time parametric uncertain Luré systems: Sequential SDP relaxation approaches

SummaryDesign methods are proposed for static and fixed‐order dynamic output feedback controllers for discrete‐time Luré systems with sector‐bounded nonlinearities in the presence of parametric uncertainties described by polytopes. The derived design conditions are represented in terms of bilinear matrix inequalities, which are nonconvex. By using convex relaxation methods, controller design equations are derived for systems with multiple states, outputs, and nonlinearities in terms of linear matrix inequalities (LMIs) and iterative LMIs, which are associated with semidefinite programs. The proposed design methods are developed from stability conditions using parameter‐dependent Lyapunov functions, and existing iterative numerical methods are adapted to solve certain classes of nonconvex optimization problems for controller design. Several numerical examples are provided to illustrate and verify the proposed design methods. Copyright © 2016 John Wiley & Sons, Ltd.

A neurodynamic optimization technique based on overestimator and underestimator functions for solving a class of non-convex optimization problems

In this paper, a novel neural network model for solving non-convex quadratic optimization problem is proposed. It is proved that the equilibrium points of the neural network model coincides with the local and global optimal solutions of the constrained non-convex optimization problem. Furthermore, it is shown that under suitable assumptions this model is globally convergent and stable in the sense of Lyapunov at each equilibrium points. Moreover a sufficient global optimality for non-convex non-quadratic objective function subject to a quadratic constraint is presented based on the corresponding underestimator function. Then for solving a class of non-convex optimization problems a novel neurodynamic optimization technique based on the necessary and sufficient global optimality is proposed. Both theoretical and numerical approaches are considered. Numerical simulations for several different non-convex quadratic optimization problems are discussed to illustrate great agreement between the theoretical and numerical results. The efficiency of the proposed neural network model is illustrated by numerical simulations and comparisons with available literature models.

Strong duality in optimization: shifted power reformulation

For a general class of non-convex optimization problems, a class of power reformulation closes the duality gap between the primal problem and its Lagrangian dual, when the order of the power is sufficiently large. In this paper, we first estimate a lower bound of the power above which the attainment of the zero duality gap can be ensured. After introducing a suitable shifting, we further show, surprisingly, that order three is always sufficient to guarantee the zero duality gap. We then extend the proposed shifted power reformulation to discrete optimization.

On conic QPCCs, conic QCQPs and completely positive programs

This paper studies several classes of nonconvex optimization problems defined over convex cones, establishing connections between them and demonstrating that they can be equivalently formulated as convex completely positive programs. The problems being studied include: a conic quadratically constrained quadratic program (QCQP), a conic quadratic program with complementarity constraints (QPCC), and rank constrained semidefinite programs. Our results do not make any boundedness assumptions on the feasible regions of the various problems considered. The first stage in the reformulation is to cast the problem as a conic QCQP with just one nonconvex constraint $$q(x) \le 0$$q(x)≤0, where q(x) is nonnegative over the (convex) conic and linear constraints, so the problem fails the Slater constraint qualification. A conic QPCC has such a structure; we prove the converse, namely that any conic QCQP satisfying a constraint qualification can be expressed as an equivalent conic QPCC. The second stage of the reformulation lifts the problem to a completely positive program, and exploits and generalizes a result of Burer. We also show that a Frank---Wolfe type result holds for a subclass of this class of conic QCQPs. Further, we derive necessary and sufficient optimality conditions for nonlinear programs where the only nonconvex constraint is a quadratic constraint of the structure considered elsewhere in the paper.

Data Processing with Combined Homotopy Methods for a Class of Nonconvex Optimization Problems

On the existing theoretical results, this paper studies the realization of combined homotopy methods on optimization problems in a specific class of nonconvex constrained region. Contraposing to this nonconvex constrained region, we give the structure method of the quasi-normal, prove that the chosen mappings on constrained grads are positive independent and the feasible region on SLM satisfies the quasi-normal cone condition. And we construct combined homotopy equation under the quasi-normal cone condition with numerical value and examples, and get a preferable result by data processing.

Knapsack problems with sigmoid utilities: Approximation algorithms via hybrid optimization

We study a class of non-convex optimization problems involving sigmoid functions. We show that sigmoid functions impart a combinatorial element to the optimization variables and make the global optimization computationally hard. We formulate versions of the knapsack problem, the generalized assignment problem and the bin-packing problem with sigmoid utilities. We merge approximation algorithms from discrete optimization with algorithms from continuous optimization to develop approximation algorithms for these NP-hard problems with sigmoid utilities.

Direct Optimization of the Dictionary Learning Problem

A novel way of solving the dictionary learning problem is proposed in this paper. It is based on a so-called direct optimization as it avoids the usual technique which consists in alternatively optimizing the coefficients of a sparse decomposition and in optimizing dictionary atoms. The algorithm we advocate simply performs a joint proximal gradient descent step over the dictionary atoms and the coefficient matrix. After having derived the algorithm, we also provided in-depth discussions on how the stepsizes of the proximal gradient descent have been chosen. In addition, we uncover the connection between our direct approach and the alternating optimization method for dictionary learning. We have shown that it can be applied to a broader class of non-convex optimization problems than the dictionary learning one. As such, we have denoted the algorithm as a one-step block-coordinate proximal gradient descent. The main advantage of our novel algorithm is that, as suggested by our simulation study, it is more efficient than alternating optimization algorithms.

Duality and Optimization for Generalized Multi-Hop MIMO Amplify-and-Forward Relay Networks With Linear Constraints

We consider a generalized multi-hop MIMO amplify-and-forward (AF) relay network with multiple sources/destinations and arbitrarily number of relays. We establish two dualities and the corresponding dual transformations between such a network and its dual, respectively under single network linear constraint and per-hop linear constraint. The result is a generalization of the previous dualities under different special cases and is proved using new techniques which reveal more insight on the duality structure that can be exploited to optimize MIMO precoders. A unified optimization framework is proposed to find a stationary point for an important class of non-convex optimization problems of AF relay networks based on a local Lagrange dual method, where the primal algorithm only finds a stationary point for the inner loop problem of maximizing the Lagrangian w.r.t. the primal variables. The input covariance matrices are shown to satisfy a polite water-filling structure at a stationary point of the inner loop problem. The duality and polite water-filling are exploited to design fast primal algorithms. Compared to the existing algorithms, the proposed optimization framework with duality-based primal algorithms can be used to solve more general problems with lower computation cost.

The KKT optimality conditions in a class of generalized convex optimization problems with an interval-valued objective function

In this paper, we study the Karush–Kuhn–Tucker optimality conditions in a class of nonconvex optimization problems with an interval-valued objective function. Firstly, the concepts of preinvexity and invexity are extended to interval-valued functions. Secondly, several properties of interval-valued preinvex and invex functions are investigated. Thirdly, the KKT optimality conditions are derived for LU-preinvex and invex optimization problems with an interval-valued objective function under the conditions of weakly continuous differentiablity and Hukuhara differentiablity. Finally, the relationships between a class of variational-like inequalities and the interval-valued optimization problems are established.

Hidden Convexity in Partially Separable Optimization

The paper identifies classes of nonconvex optimization problems whose convex relaxations have optimal solutions which at the same time are global optimal solutions of the original nonconvex problems. Such a hidden convexity property was so far limited to quadratically constrained quadratic problems with one or two constraints. We extend it here to problems with some partial separable structure. Among other things, the new hidden convexity results open up the possibility to solve multi-stage robust optimization problems using certain nonlinear decision rules.

A nonlinear programming technique to compute a tight lower bound for the real structured singular value

The real structured singular value (RSSV, or real μ) is a useful measure to analyze the robustness of linear systems subject to structured real parametric uncertainty, and surely a valuable design tool for the control systems engineers. We formulate the RSSV problem as a nonlinear programming problem and use a new computation technique, F-modified subgradient (F-MSG) algorithm, for its lower bound computation. The F-MSG algorithm can handle a large class of nonconvex optimization problems and requires no differentiability. The RSSV computation is a well known NP hard problem. There are several approaches that propose lower and upper bounds for the RSSV. However, with the existing approaches, the gap between the lower and upper bounds is large for many problems so that the benefit arising from usage of RSSV is reduced significantly. Although the F-MSG algorithm aims to solve the nonconvex programming problems exactly, its performance depends on the quality of the standard solvers used for solving subproblems arising at each iteration of the algorithm. In the case it does not find the optimal solution of the problem, due to its high performance, it practically produces a very tight lower bound. Considering that the RSSV problem can be discontinuous, it is found to provide a good fit to the problem. We also provide examples for demonstrating the validity of our approach.

The modified subgradient algorithm based on feasible values

In this article, we continue to study the modified subgradient (MSG) algorithm previously suggested by Gasimov for solving the sharp augmented Lagrangian dual problems. The most important features of this algorithm are those that guarantees a global optimum for a wide class of non-convex optimization problems, generates a strictly increasing sequence of dual values, a property which is not shared by the other subgradient methods and guarantees convergence. The main drawbacks of MSG algorithm, which are typical for many subgradient algorithms, are those that uses an unconstrained global minimum of the augmented Lagrangian function and requires knowing an approximate upper bound of the initial problem to update stepsize parameters. In this study we introduce a new algorithm based on the so-called feasible values and give convergence theorems. The new algorithm does not require to know the optimal value initially and seeks it iteratively beginning with an arbitrary number. It is not necessary to find a global minimum of the augmented Lagrangian for updating the stepsize parameters in the new algorithm. A collection of test problems are used to demonstrate the performance of the new algorithm.

A novel approach to Bilevel nonlinear programming

Recently developed methods of monotonic optimization have been applied successfully for studying a wide class of nonconvex optimization problems, that includes, among others, generalized polynomial programming, generalized multiplicative and fractional programming, discrete programming, optimization over the efficient set, complementarity problems. In the present paper the monotonic approach is extended to the General Bilevel Programming GBP Problem. It is shown that (GBP) can be transformed into a monotonic optimization problem which can then be solved by "polyblock" approximation or, more efficiently, by a branch-reduce-and-bound method using monotonicity cuts. The method is particularly suitable for Bilevel Convex Programming and Bilevel Linear Programming.

A note on a paper by E. Khorram and A. Ghodousian

The aim of this short note is to show on a numerical example that one of the two optimization algorithms proposed in [E. Khorram, A. Ghodousian, Linear objective function optimization with fuzzy relation equation constraints regarding max-av composition, Appl. Math. Comput. 173 (2006) 872–886] may not lead to the optimal solution in some cases. Besides it will be pointed out that the other algorithm (correct, but unfortunately ineffective for larger problems) proposed in [E. Khorram, A. Ghodousian, Linear objective function optimization with fuzzy relation equation constraints regarding max-av composition, Appl. Math. Comput. 173 (2006) 872–886] can be extended to a wider class of non-convex optimization problems.