Quadratic Unconstrained Optimization Research Articles

On the emerging potential of quantum annealing hardware for combinatorial optimization

Over the past decade, the usefulness of quantum annealing hardware for combinatorial optimization has been the subject of much debate. Thus far, experimental benchmarking studies have indicated that quantum annealing hardware does not provide an irrefutable performance gain over state-of-the-art optimization methods. However, as this hardware continues to evolve, each new iteration brings improved performance and warrants further benchmarking. To that end, this work conducts an optimization performance assessment of D-Wave Systems’ Advantage Performance Update computer, which can natively solve sparse unconstrained quadratic optimization problems with over 5,000 binary decision variables and 40,000 quadratic terms. We demonstrate that classes of contrived problems exist where this quantum annealer can provide run time benefits over a collection of established classical solution methods that represent the current state-of-the-art for benchmarking quantum annealing hardware. Although this work does not present strong evidence of an irrefutable performance benefit for this emerging optimization technology, it does exhibit encouraging progress, signaling the potential impacts on practical optimization tasks in the future.

A gradient method exploiting the two dimensional quadratic termination property

The quadratic termination property is important to the efficiency of gradient methods. We consider equipping a family of gradient methods, where the stepsize is given by the ratio of two norms, with two dimensional quadratic termination. Such a desired property is achieved by cooperating with a new stepsize which is derived by maximizing the stepsize of the considered family in the next iteration. It is proved that each method in the family will asymptotically alternate in a two dimensional subspace spanned by the eigenvectors corresponding to the largest and smallest eigenvalues. Based on this asymptotic behavior, we show that the new stepsize converges to the reciprocal of the largest eigenvalue of the Hessian. Furthermore, by adaptively taking the long Barzilai–Borwein stepsize and reusing the new stepsize with retard, we propose an efficient gradient method for unconstrained quadratic optimization. We prove that the new method is R-linearly convergent with a rate of \(1-1/\kappa\), where \(\kappa\) is the condition number of Hessian. Numerical experiments show the efficiency of our proposed method.

Alternating Minimization-Based Sparse Least-Squares Classifier for Accuracy and Interpretability Improvement of Credit Risk Assessment

When dealing with complex and redundant data classification problems, many classifiers cannot provide high predictive accuracy and interpretability. We also find that the least-squares support vector classifiers (LSSVCs) hardly identify important instances and features from data, so they cannot give an interpretable prediction. Although the LSSVC has the properties of low bias and high robustness, its high variance often gives a poor predictive performance. In this paper, we propose an alternating minimization-based sparse least-squares classifier (AMSLC) approach in the framework of LSSVCs to address the aforementioned problems. Based on the reconstructed row- and column-wise kernel matrices, the sparsity-induced [Formula: see text]-norm approximation function is introduced to the LSSVC model. By alternately solving two unconstrained quadratic optimization problems or two systems of linear equations, AMSLC can predict the class labels of given instances and extract the least number of important instances and features to obtain the interpretable classification. Compared with SVC, LSSVC, [Formula: see text]-norm SVC (L1SVC), [Formula: see text]-norm SVC (L0SVC), the least absolute shrinkage and selection operator classifier (LASSOC), and multiple kernel learning SVC (MKLSVC) on four real credit datasets, the experimental results show that the proposed AMSLC method generally obtains the best predictive accuracy and the interpretable classification with the minimum number of important instances and features.

Exact and Approximate Solving of the Aircraft Collision Resolution Problem via Turn Changes

The aircraft conflict detection and resolution problem in air traffic management consists of deciding the best strategy for an arbitrary aircraft configuration such that all conflicts in the airspace are avoided. A conflict situation occurs if two or more aircraft do not maintain the minimum safety distance during their flight plans. A two-step approach is presented. The first step consists of a nonconvex mixed integer nonlinear optimization (MINLO) model based on geometric constructions. The objective is to minimize the weighted aircraft angle variations to obtain the new flight configuration. The second step consists of a set of unconstrained quadratic optimization models where aircraft are forced to return to their original flight plan as soon as possible once there is no aircraft in conflict with any other. The main results of extensive computation are reported by comparing the performance of state-of-the-art nonconvex MINLO solvers and an approximation by discretizing the possible angles of motion for solving a sequence of integer linear optimization (SILO) models in an iterative way. Minotaur, one of the nonconvex MINLO solvers experimented with, gives better solutions but requires more computation time than the SILO approach, which requires only a short time to obtain a good, feasible solution. Its value in the objective function has a reasonable goodness gap compared with the Minotaur solution. Given the need to solve the problem in almost real time, the approximate SILO approach is favored because of its short computation time and solution quality for the testbeds used in the experiment, which include both small- and real-sized instances. However, Minotaur is useful in this particular case for simulation purposes and for calibrating the SILO approach.

On the Complexity of Local Search in Unconstrained Quadratic Binary Optimization

We consider the problem of finding a local minimum of a binary quadratic function and show by an elementary construction that every descending local search algorithm takes exponential time in the worst case.

An elastic approach for developing non-developable sheets

Surface development origins from the cloth-making and computer graphics without consideration of the thickness, involving nonlinear optimization and constraints. Moreover the research of surface development mainly focuses on the planar development. In this paper, the development of the non-developable sheet to planar, singly-curved and doubly-curved surface patterns is investigated. An optimal developing algorithm is formulated to minimize the strain energy required for the deformation of the sheet, in which an orthogonal curvilinear coordinate system is used for three target patterns to simplify the constraints for the developing process, resulting in an unconstrained quadratic optimization. Both shell element and solid element are utilized in the finite element analysis. Similar developed results are obtained for planar and spherical patterns by using these two types of elements. But for the cylindrical pattern, the solid element model gives more accurate result due to no contribution of the shell element to the translation of the rotational freedom.

From Angular Manifolds to the Integer Lattice: Guaranteed Orientation Estimation With Application to Pose Graph Optimization

Pose graph optimization from relative measurements is challenging because of the angular component of the poses: the variables live on a manifold product with nontrivial topology and the likelihood function is nonconvex and has many local minima. Because of these issues, iterative solvers are not robust to large amounts of noise. This paper describes a global estimation method, called multi-hypothesis orientation-from-lattice estimation in 2-D (MOIE2D), for the estimation of the nodes' orientation in a pose graph. We demonstrate that the original nonlinear optimization problem on the manifold product is equivalent to an unconstrained quadratic optimization problem on the integer lattice. Exploiting this insight, we show that, in general, the maximum likelihood estimate alone cannot be considered a reliable estimator. Therefore, MOIE2D returns a set of point estimates, for which we can derive precise probabilistic guarantees. Experiments show that the method is able to tolerate extreme amounts of noise, far above all noise levels of sensors used in applications. Using MOIE2D's output to bootstrap the initial guess of iterative pose graph optimization methods improves their robustness and makes them avoid local minima even for high levels of noise.

A non-monotonic method for large-scale non-negative least squares

We present a new algorithm for solving the non-negative least-squares (NNLS) problem. Our algorithm extends the unconstrained quadratic optimization algorithm of Barzilai and Borwein (BB) [J. Barzilai and J. M. Borwein; Two-Point Step Size Gradient Methods. IMA J. Numer. Anal. 1988.] to handle nonnegativity constraints. Our extension differs from other constrained BB variants in simple but crucial aspects, the most notable being our modification to the BB stepsize itself. Our stepsize computation takes into account the nonnegativity constraints, and is further refined by a stepsize scaling strategy. These changes, in combination with orthogonal projections onto the nonnegative orthant, yield an effective NNLS algorithm. We compare our algorithm with several competing approaches, including established bound-constrained solvers, popular BB-based methods, and also a specialised NNLS algorithm. On several synthetic and real-world datasets our method displays highly competitive empirical performance.

Interactive elastic motion editing through space–time position constraints

ABSTRACTWe present an intuitive and interactive approach for motion editing through space–time constraints on positions. Given an input motion of an elastic body, our approach enables the user to interactively edit node positions in order to alter and fine‐tune the motion. We formulate our motion editing as an optimization problem with dynamics constraints to enforce a physically plausible result. Through linearization of the editing around the input trajectory, we simplify this constrained optimal control problem into an unconstrained quadratic optimization. The optimal motion thus becomes the solution of a dense linear system, which we solve efficiently by applying the adjoint method in each iteration of a conjugate gradient solver. We demonstrate the efficiency and quality of our motion editing technique on a series of examples. Copyright © 2013 John Wiley & Sons, Ltd.

Elementary landscape decomposition of the 0-1 unconstrained quadratic optimization

Landscapes' theory provides a formal framework in which combinatorial optimization problems can be theoretically characterized as a sum of an especial kind of landscape called elementary landscape. The elementary landscape decomposition of a combinatorial optimization problem is a useful tool for understanding the problem. Such decomposition provides an additional knowledge on the problem that can be exploited to explain the behavior of some existing algorithms when they are applied to the problem or to create new search methods for the problem. In this paper we analyze the 0-1 Unconstrained Quadratic Optimization from the point of view of landscapes' theory. We prove that the problem can be written as the sum of two elementary components and we give the exact expressions for these components. We use the landscape decomposition to compute autocorrelation measures of the problem, and show some practical applications of the decomposition.

Spectral bounds for unconstrained (−1,1)-quadratic optimization problems

Given an unconstrained quadratic optimization problem in the following form: ( QP ) min { x t Qx | x ∈ { - 1 , 1 } n } , with Q ∈ R n × n , we present different methods for computing bounds on its optimal objective value. Some of the lower bounds introduced are shown to generally improve over the one given by a classical semidefinite relaxation. We report on theoretical results on these new bounds and provide preliminary computational experiments on small instances of the maximum cut problem illustrating their performance.

A polynomial-time recursive algorithm for some unconstrained quadratic optimization problems

Determining the cells of an arrangement of hyperplanes is a classical problem in combinatorial geometry. In this paper we present an efficient recursive procedure to solve it. In fact, by considering a connection between the latter problem and optimality conditions of unconstrained quadratic optimization problems in the following form: ( Q P ) min { x t Q x ∣ x ∈ { − 1 , 1 } n } , with Q ∈ R n × n , we can show the proposed algorithm solves problems of the form ( Q P ) in polynomial time when the rank of the matrix Q is fixed and the number of its positive diagonal entries is O ( log ( n ) ) .

Distributed Asynchronous Algorithms for Solving Positive Definite Linear Equations over Networks—Part I: Agent Networks

—This two-part paper presents and analyzes a family of distributed asynchronous algorithms for solving symmetric positive definite systems of linear equations over networks. In this Part I, we develop Subset Equalizing (SE), a Lyapunov-based algorithm for solving such equations over networks of agents with arbitrary asynchronous interactions and spontaneous membership dynamics, both of which may be exogenously driven and completely unpredictable. To analyze the behavior of SE, we introduce several notions of network connectivity, capable of handling such interactions and membership dynamics, and a time-varying quadratic Lyapunov-like function, defined on a state space with changing dimension. Based on them, we derive sufficient conditions for ensuring the boundedness, asymptotic convergence, and exponential convergence of SE, and show that these conditions are mild. Finally, we illustrate the effectiveness of SE through an example, using it to perform unconstrained quadratic optimization over a volatile multi-agent system.

A Fast 2D Shape Recovery Approach by Fusing Features and Appearance

In this paper, we present a fusion approach to solve the nonrigid shape recovery problem, which takes advantage of both the appearance information and the local features. We have two major contributions. First, we propose a novel progressive finite Newton optimization scheme for the feature-based nonrigid surface detection problem, which is reduced to only solving a set of linear equations. The key is to formulate the nonrigid surface detection as an unconstrained quadratic optimization problem that has a closed-form solution for a given set of observations. Second, we propose a deformable Lucas-Kanade algorithm that triangulates the template image into small patches and constrains the deformation through the second-order derivatives of the mesh vertices. We formulate it into a sparse regularized least squares problem, which is able to reduce the computational cost and the memory requirement. The inverse compositional algorithm is applied to efficiently solve the optimization problem. We have conducted extensive experiments for performance evaluation on various environments, whose promising results show that the proposed algorithm is both efficient and effective.

Efficient Reduction of Polynomial Zero-One Optimization to the Quadratic Case

We address the problem of optimizing a polynomial with real coefficients over binary variables. We show that a complete polyhedral description of the linearization of such a problem can be derived in a simple way from the polyhedral description of the linearization of some quadratic optimization problem. The number of variables in the latter linearization is only slightly larger than in the former. If polynomial constraints are present in the original problem, then their linearized counterparts carry over to the linearized quadratic problem unchanged. If the original problem formulation does not contain any constraints, we obtain a reduction to unconstrained quadratic zero-one optimization, which is equivalent to the well-studied max-cut problem. The separation problem for general unconstrained polynomial zero-one optimization thus reduces to the separation problem for the cut polytope. This allows us to transfer the entire knowledge gained for the latter polytope by intensive research and, in particular, the sophisticated separation techniques that have been developed. We report preliminary experimental results obtained with a straightforward implementation of this approach.

Recursive subspace identification based on instrumental variable unconstrained quadratic optimization

AbstractThe problem of the recursive formulation of the MOESP class of subspace identification algorithms is considered and two novel instrumental variable approaches are introduced. The first one leads to an RLS‐like implementation, the second to a gradient type iteration. The relative merits of both approaches are analysed and discussed, while simulation results are used to compare their performance with one of the existing techniques. Copyright © 2004 John Wiley & Sons, Ltd.

New suboptimal multiuser detectors for synchronous CDMA systems

In code-division multiple access (CDMA) systems, the optimal multiuser detection is equivalent to an unconstrained quadratic bivalent optimization problem, and its computational complexity increases exponentially with the number of users. In this paper, we propose several new suboptimal detectors with a greatly reduced computational complexity based on a local minimization algorithm for a synchronous CDMA system. The performances of these detectors are compared with those of the conventional, optimal, and several other suboptimal detectors.