Nonconvex Quadratic Problems Research Articles

Accelerate rotation invariant sliced Gromov-Wasserstein distance by an alternative optimization method

The traditional Wasserstein distance is inadequate for comparing two distributions in different metric spaces. The Gromov-Wasserstein distance (GWD), which evolves from the Gromov-Hausdorff distance, constructs intraspace pairs to ensure comparability. However, solving the GWD requires tackling a complex nonconvex quadratic problem, a process that typically demands significant time and space complexity. The sliced Gromov-Wasserstein distance (SGW) addresses the efficiency issues of GWD by employing a projection-slice method. Unfortunately, slicing destroys the crucial rotation invariance property of GWD. The rotation invariant sliced Gromov-Wasserstein distance (RISGW) restores it by enumerating the alignments of the distributions. Inspired by our observations, we introduce BRISGW, a novel batch rotation invariant sliced Gromov-Wasserstein distance that alternately iterates and approximates the underlying best alignment both effectively and efficiently. We theoretically prove the feasibility the proposed our approach. Moreover, significant experimental results show that the proposed method outperforms the state-of-the-art methods.

Solving a real world non-convex Quadratic Assignment Problem

The India-Japan Lighting company operates three plants in India. These three plants manufacture headlamps and taillights for the automotive industry. This study examines the facility location problem in one of these plants where 12 facilities must be placed in a two-column multi-row cellular layout. The machining sequences for the 20 parts conveyed among the 12 facilities were specified. The Quadratic Assignment Problem (QAP) is classified as an NP-hard problem in large instances. We modelled the specific instance as a QAP and reported the solution obtained by an easily available generalised reduced gradient (GRG) nonlinear solver and the solution obtained from the Gurobi optimiser. The Gurobi optimiser provides an excellent incumbent solution in quick time, but takes exponential time to reduce the duality gap.

Denoising Multi-Similarity Formulation: A Self-Paced Curriculum-Driven Approach for Robust Metric Learning

Deep Metric Learning (DML) is a group of techniques that aim to measure the similarity between objects through the neural network. Although the number of DML methods has rapidly increased in recent years, most previous studies cannot effectively handle noisy data, which commonly exists in practical applications and often leads to serious performance deterioration. To overcome this limitation, in this paper, we build a connection between noisy samples and hard samples in the framework of self-paced learning, and propose a Balanced Self-Paced Metric Learning (BSPML) algorithm with a denoising multi-similarity formulation, where noisy samples are treated as extremely hard samples and adaptively excluded from the model training by sample weighting. Especially, due to the pairwise relationship and a new balance regularization term, the sub-problem w.r.t. sample weights is a nonconvex quadratic function. To efficiently solve this nonconvex quadratic problem, we propose a doubly stochastic projection coordinate gradient algorithm. Importantly, we theoretically prove the convergence not only for the doubly stochastic projection coordinate gradient algorithm, but also for our BSPML algorithm. Experimental results on several standard data sets demonstrate that our BSPML algorithm has better generalization ability and robustness than the state-of-the-art robust DML approaches.

NEW BRANCH AND BOUND METHOD OVER A BOXED SET OF $\mathbb{R}^{n}$

We present in this paper the new Branch and Bound method with new quadratic approach over a boxed set (a rectangle) of $\mathbb{R}^{n}$. We construct an approximate convex quadratics functions of the objective function to fined a lower bound of the global optimal value of the original non convex quadratic problem (NQP) over each subset of this boxed set. We applied a partition and technical reducing on the domain of (NQP) to accelerate the convergence of the proposed algorithm. Finally,we study the convergence of the proposed algorithm and we give a simple comparison between this method and another methods wish have the same principle.

Support solving method for box-constrained indefinite quadratic minimisation problems

This paper provides a new support solving method (SSM) of global optimisation for the box-constrained non-convex quadratic minimisation problem, in particular with one negative eigenvalue. We investigate the support of the objective function and exploit properties of the indefinite associated matrix for establishing global optimality criterion (necessary and sufficient conditions). Furthermore, using these conditions and computational techniques of abstract convexity, we suggest an SSM that can effectively solve an indefinite quadratic minimisation problem, providing thus a global minimiser. We present numerical examples and generate some test problems with known global minimiser, solving them by the proposed SSM. Finally, we provide the comparative effectiveness of the SSM with active set method (ASM) and interior point method (IPM) implemented under MATLAB optimisation toolbox.

A note on completely positive relaxations of quadratic problems in a multiobjective framework

In a single-objective setting, nonconvex quadratic problems can equivalently be reformulated as convex problems over the cone of completely positive matrices. In small dimensions this cone equals the cone of matrices which are entrywise nonnegative and positive semidefinite, so the convex reformulation can be solved via SDP solvers. Considering multiobjective nonconvex quadratic problems, naturally the question arises, whether the advantage of convex reformulations extends to the multicriteria framework. In this note, we show that this approach only finds the supported nondominated points, which can already be found by using the weighted sum scalarization of the multiobjective quadratic problem, i.e. it is not suitable for multiobjective nonconvex problems.

Conic optimization: A survey with special focus on copositive optimization and binary quadratic problems

• We survey recent progress in conic optimization. • We discuss second order cone programs (SOCP), semidefinite problems (SDP), and copositive problems. • Special emphasis is given to modelling issues. • The survey contains more than 110 references. A conic optimization problem is a problem involving a constraint that the optimization variable be in some closed convex cone. Prominent examples are linear programs (LP), second order cone programs (SOCP), semidefinite problems (SDP), and copositive problems. We survey recent progress made in this area. In particular, we highlight the connections between nonconvex quadratic problems, binary quadratic problems, and copositive optimization. We review how tight bounds can be obtained by relaxing the copositivity constraint to semidefiniteness, and we discuss the effect that different modelling techniques have on the quality of the bounds. We also provide some new techniques for lifting linear constraints and show how these can be used for stable set and coloring relaxations.

Adaptive Ensemble-Based Optimisation for Petrophysical Inversion

This article introduces a new optimisation algorithm designed for sampling the solution space of non-linear, non-convex quadratic problems. This method has been specifically designed for inversion problems where multiple distinct scenarios must be explored, which would not be achievable with a standard ensemble-based optimisation method (EnOpt). A prior ensemble is used to sample the uncertainty of the parameters before updating each member of the ensemble independently in order to find the minimum of an objective function. This update is given by a Gauss–Newton-like approach, where the first-derivative matrix is adaptively estimated from sub-ensembles of parameters and their corresponding forward model responses. As the first-derivative matrix is statistically computed from an ensemble of realisations, the forward model does not need to be known and the algorithm is independent of it. The final ensemble provides an estimation of the uncertainty after the inversion process. The efficiency of this adaptive ensemble optimisation (A-EnOpt) method is first tested on simple two-dimensional problems where known mathematical functions are used as forward models. The results show that the method minimises the objective functions and samples the final uncertainties of the problems to a better degree than a standard EnOpt. The A-EnOpt method is then applied to a real heavy-oil field petrophysical inversion. Porosity, shale fraction and fluid saturations are inverted under continuity and Lagrange constraints, conditioned by P-impedance models from stochastic seismic inversion. The updated properties are incorporated in a fluid flow model of the field. The simulation results produce a better match to historical production data at one well than equivalent flow simulations using the properties before inversion. This shows that conditioning by seismic data improves the quality of the geological models.

Attributed Network Alignment: Problem Definitions and Fast Solutions

Networks are prevalent and often collected from multiple sources in many high-impact domains, which facilitate many emerging applications that require the connections across multiple networks. Network alignment (i.e., to find the node correspondence between different networks) has become the very first step in many applications and thus has been studied in decades. Although some existing works can use the attribute information as part of the alignment process, they still have certain limitations. For example, some existing network alignment methods can use node attribute similarities as part of the prior alignment information, whereas most of them solely explore the topology consistency without the consistency among attributes of the underlying networks. On the other hand, traditional graph matching methods encode both the node and edge attributes (and possibly the topology) into an affinity matrix and formulate it as a constrained nonconvex quadratic maximization problem. However, these methods cannot scale well to the large-scale networks. In this paper, we propose a family of network alignment algorithms (FINAL) to efficiently align the attributed networks. The key idea is to leverage the node/edge attribute information to guide the (topology-based) alignment process. We formulate this problem as a convex quadratic optimization problem, and develop effective and efficient algorithms to solve it. Moreover, we derive FINAL On-Query, an online variant of FINAL that can find similar nodes for the query nodes across networks. We perform extensive evaluations on real networks to substantiate the superiority of our proposed approaches.

Solving Quadratic Programming by Cutting Planes

We propose new cutting planes for strengthening the linear relaxations that appear in the solution of nonconvex quadratic problems with linear constraints. By a famous result of Motzkin and Straus,...

Branch and Bound Method to Resolve Non-Convex Quadratic Problems Over a Boxed Set

We present in this paper the technique Branch and Bound with new quadratic approch over a boxed arrangement of Rn. We develop an inexact arched quadratic capacity of the target capacity to decide a lower bound of the worldwide ideal estimation of the first non raised issue (NQP) over every subset of this boxed set. We connected a segment and specialized lessening on the feasable area of (NQP)to quicken the intermingling of the proposed calculation. Finally,we think about the assembly of the proposed calculation and we give a straightforward examination between this strategy and another technique wish have a similar guideline.

Approximate Global Minimizers to Pairwise Interaction Problems via Convex Relaxation

We present a new approach for computing approximate global minimizers to a large class of nonlocal pairwise interaction problems defined over probability distributions. The approach predicts candidate global minimizers, with a recovery guarantee, that are sometimes exact, and often within a few percent of the optimum energy (under appropriate normalization of the energy). The procedure relies on a convex relaxation of the pairwise energy that exploits translational symmetry, followed by a recovery procedure that minimizes a relative entropy. Numerical discretizations of the convex relaxation yield a linear programming problem over convex cones that can be solved using well-known methods. One advantage of the approach is that it provides sufficient conditions for global minimizers to a nonconvex quadratic variational problem, in the form of a linear, convex, optimization problem for the autocorrelation of the probability density. We demonstrate the approach in a periodic domain for examples arising from models in materials, social phenomena, and flocking. The approach also exactly recovers the global minimizer when a lattice of Dirac masses solves the convex relaxation. An important by-product of the relaxation is a decomposition of the pairwise energy functional into the sum of a convex functional and nonconvex functional. We observe that in some cases, the nonconvex component of the decomposition can be used to characterize the support of the recovered minimizers.

The min-cut and vertex separator problem

We consider graph three-partitions with the objective of minimizing the number of edges between the first two partition sets while keeping the size of the third block small. We review most of the existing relaxations for this min-cut problem and focus on a new class of semidefinite relaxations, based on matrices of order 2n+1 which provide a good compromise between quality of the bound and computational effort to actually compute it. Here, n is the order of the graph. Our numerical results indicate that the new bounds are quite strong and can be computed for graphs of medium size (n approx 300) with reasonable effort of a few minutes of computation time. Further, we exploit those bounds to obtain bounds on the size of the vertex separators. A vertex separator is a subset of the vertex set of a graph whose removal splits the graph into two disconnected subsets. We also present an elegant way of convexifying non-convex quadratic problems by using semidefinite programming. This approach results with bounds that can be computed with any standard convex quadratic programming solver.

A weighted linear discriminant analysis framework for multi-label feature extraction

Abstract Linear discriminant analysis (LDA) is one of the most popular single-label (multi-class) feature extraction techniques. For multi-label case, two slightly different generalized versions have been introduced independently. We argue whether there exists a framework to unify such two multi-label LDA methods and to derive more well-performed multi-label LDA techniques further. In this paper, we build a weighted multi-label LDA framework (wMLDA) to consolidate two existing multi-label LDA-type methods with binary and correlation-based weight forms, and further collect two additional weight forms with entropy and fuzzy principles. To exploit both label and feature information more sufficiently, via maximizing dependence based on Hilbert–Schmidt independence criterion, a novel dependence-based weight form is proposed, which is formulated as a non-convex quadratic programing problem with l 1 -norm and non-negative constraints and then is solved by random block coordinate descent method with a linear convergence rate. Experiments on ten data sets illustrate that our dependence-based wMLDA works the best, and five wMLDA-type algorithms are superior to canonical correlation analysis and multi-label dimensionality reduction via dependency maximization, according to five multi-label classification performance measures and Wilcoxon statistical test.

A range division and contraction approach for nonconvex quadratic program with quadratic constraints.

This paper presents a novel range division and contraction approach for globally solving nonconvex quadratic program with quadratic constraints. By constructing new underestimating linear relaxation functions, we can transform the initial nonconvex quadratic program problem into a linear program relaxation problem. By employing a branch and bound scheme with a range contraction approach, we describe a novel global optimization algorithm for effectively solving nonconvex quadratic program with quadratic constraints. Finally, the global convergence of the proposed algorithm is proved and numerical experimental results demonstrate the effectiveness of the proposed approach.

On the complex fractional quadratic optimization with a quadratic constraint

In this paper, we study the problem of minimizing the ratio of two complex indefinite quadratic functions subject to a strictly convex quadratic constraint. First, using the known method due to Dinkelbach, we reformulate the fractional problem as a univariate equation. To find the root of the univariate equation, the generalized Newton method is utilized that requires solving a nonconvex quadratic optimization problem at each iteration. To solve these nonconvex quadratic problems, we present an efficient algorithm by a diagonalization scheme that requires solving a univariate minimization problem at each iteration. Moreover, for the homogeneous case, it requires solving a simple linear optimization problem. Our preliminary numerical experiments on several randomly generated test problems show that, the new approach is much faster in finding the global optimal solution than the known semidefinite relaxation approach, especially when solving large scale problems.

Relaxing Nonconvex Quadratic Functions by Multiple Adaptive Diagonal Perturbations

The current bottleneck of globally solving mixed-integer (non-convex) quadratically constrained problem (MIQCP) is still to construct strong but computationally cheap convex relaxations, especially when dense quadratic functions are present. We propose a cutting surface procedure based on multiple diagonal perturbations to derive strong convex quadratic relaxations for nonconvex quadratic problem with separable constraints. Our resulting relaxation does not use significantly more variables than the original problem, in contrast to many other relaxations based on lifting. The corresponding separation problem is a highly structured semidefinite program (SDP) with convex but non-smooth objective. We propose to solve this separation problem with a specialized primal-barrier coordinate minimization algorithm. Computational results show that our approach is very promising. First, our separation algorithm is at least an order of magnitude faster than interior point methods for SDPs on problems up to a few hundred variables. Secondly, on nonconvex quadratic integer problems, our cutting surface procedure provides lower bounds of almost the same strength with the diagonal SDP bounds used by (Buchheim and Wiegele, 2013) in their branch-and-bound code Q-MIST, while our procedure is at least an order of magnitude faster on problems with dimension greater than 70. Finally, combined with (linear) projected RLT cutting planes proposed by (Saxena, Bonami and Lee, 2011), our procedure provides slightly weaker bounds than their projected SDP+RLT cutting surface procedure, but in several order of magnitude shorter time. Finally we discuss various avenues to extend our work to design more efficient branch-and-bound algorithms for MIQCPs.

Solving a class of non-convex quadratic problems based on generalized KKT conditions and neurodynamic optimization technique

In this paper, based on a generalized Karush-Kuhn-Tucker (KKT) method a modified recurrent neural network model for a class of non-convex quadratic programming problems involving a so-called $Z$-matrix is proposed. The basic idea is to express the optimality condition as a mixed nonlinear complementarity problem. Then one may specify conditions for guaranteeing the global solutions of the original problem by using results from the S-lemma. This process is proved by building up a dynamic system from the optimality condition whose equilibrium point is exactly the solution of the mixed nonlinear complementarity problem. By the study of the resulting dynamic system it is shown that under given assumptions, steady states of the dynamic system are stable. Numerical simulations and comparisons with the other methods are presented to illustrate the efficiency of the practical technique that is proposed in this paper.

A gentle, geometric introduction to copositive optimization

This paper illustrates the fundamental connection between nonconvex quadratic optimization and copositive optimization--a connection that allows the reformulation of nonconvex quadratic problems as...

Parallel Selective Algorithms for Nonconvex Big Data Optimization

We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all possibilities “in between” with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.