Semidefinite Programming Solvers Research Articles

Theory and applications of the sum-of-squares technique

Abstract The sum-of-squares (SOS) approximation method is a technique used in optimization problems to derive lower bounds on the optimal value of an objective function. By representing the objective function as a sum of squares in a feature space, the SOS method transforms non-convex global optimization problems into solvable semidefinite programs. This note presents an overview of the SOS method. We start with its application in finite-dimensional feature spaces and, subsequently, we extend it to infinite-dimensional feature spaces using reproducing kernels (k-SOS). Additionally, we highlight the utilization of SOS for estimating some relevant quantities in information theory, including the log-partition function.

Solving Max-cut Problem with a mixed penalization method for Semidefinite Programming

This paper is an attempt to exploit the opportunity of Semi Definite Programming (SDP), which is an area of convex and conic optimization. Indeed, Numerous NP-hard problems can be solveby using this approach. Hence, we intend to investigate the strength of SDP to model and provide tight relaxations of combinatorial and quadratic problems in order to present a new polynomial time algorithm for solving this robust model. This algorithm was firstly use to solve the nonlinear programs, the reason for which we search to extend it to the SDP programs. Actually, this algorithm designs the combination of two penalization methods. The first one is a primal-dual interior point (PDIM) method while the second one is a primal-dual exterior point (PDEM) method. Unlike the first method, which converges globally, the second one, also called the primal dual nonlinear rescaling method, has local super linear/quadratic convergence. Therefore, it seems appropriate to use a mixed algorithm based on the interior-exterior point method (IEPM). In fact, this resolution starts from the interior method, and at a certain level of execution, it proceeds to exterior method. Hence, a convergence evaluation function is use to know the level of permutation. Through evaluation, it has been approve that our approach is use to solve some instances of max-cut problem. This problem is a central graph theory model that occurs in many real problems and it is one of many NP-hard problems, which has attracted many researchers over the years. Then, we have used a semi definite programming solver SDPA (Semi Definite Programming Algorithm) that is modify to include the exterior point method subroutine. From the computational performance, we conclude that as the problem size increases, interior-exterior point algorithm gets relatively faster. The numerical results obtained are promising.

Revealing Vulnerability of N-1 Secure Power Systems to Coordinated Cyber-Physical Attacks

Coordinated cyber-physical (CCP) attacks have attracted wide attention in power systems because of their potential to cause severe disturbance including cascading failures. However, as realistic power systems operate within the N-1 security criterion, existing CCP attacks may be in part mitigated. As such, this paper analyzes impacts of CCP attacks on the vulnerability of the power systems that employ the N-1 security constrained optimal power flow (SCOPF). Specifically, a tri-level model is proposed to analyze the CCP attack impacts, whereby the adversary coordinates a physical attack with cyber attacks to initiate and propagate post-contingency overload based on N-1 SCOPF. A methodology utilizing semidefinite programming (SDP) relaxation and primal-dual formulation is proposed to transform the tri-level model into a conic optimization, such that the model can be easily solved by SDP solvers. Case studies on the IEEE 14, 57, and 118 bus test systems demonstrate that the CCP attacks in power systems with N-1 security criterion are able to cause N-1-1 contingency and even trigger cascading failures.

A structure-protecting kernelized semi-supervised space adjustment for classification

We introduce a semi-supervised space adjustment framework in this paper. In the introduced framework, the dataset contains two subsets: (a) training data subset (space-one data (SOD)) and (b) testing data subset (space-two data (STD)). Our semi-supervised space adjustment framework learns under three assumptions: (I) it is assumed that all data points in the SOD are labeled, and only a minority of the data points in the STD are labeled (we call the labeled space-two data as LSTD), (II) the size of LSTD is very small comparing to the size of SOD, and (III) it is also assumed that the data of SOD and the data of STD have different distributions. We denote the unlabeled space-two data by ULSTD, which is equal to STD - LSTD. The aim is to map the training data, i.e., the data from the training labeled data subset and those from LSTD (note that all labeled data are considered to be training data, i.e., SOD ∪ LSTD) into a shared space (ShS). The mapped SOD, ULSTD, and LSTD into ShS are named MSOD, MULSTD, and MLSTD, respectively. The proposed method does the mentioned mapping in such a way that structures of the data points in SOD and MSOD, in STD and MSTD, in ULSTD and MULSTD, and in LSTD and MLSTD are the same. In the proposed method, the mapping is proposed to be done by a principal component analysis transformation on kernelized data. In the proposed method, it is tried to find a mapping that (a) can maintain the neighbors of data points after the mapping and (b) can take advantage of the class labels that are known in STD during transformation. After that, we represent and formulate the problem of finding the optimal mapping into a non-linear objective function. To solve it, we transform it into a semidefinite programming (SDP) problem. We solve the optimization problem with an SDP solver. The examinations indicate the superiority of the learners trained in the data mapped by the proposed approach to the learners trained in the data mapped by the state of the art methods.

Gridless DOA Estimation With Multiple Frequencies

Direction-of-arrival (DOA) estimation is widely applied in acoustic source localization. A multi-frequency model is suitable for characterizing the broadband structure in acoustic signals. In this paper, the continuous (gridless) DOA estimation problem with multiple frequencies is considered. This problem is formulated as an atomic norm minimization (ANM) problem. The ANM problem is equivalent to a semi-definite program (SDP) which can be solved by an off-the-shelf SDP solver. The dual certificate condition is provided to certify the optimality of the SDP solution so that the sources can be localized by finding the roots of a polynomial. We also construct the dual polynomial to satisfy the dual certificate condition and show that such a construction exists when the source amplitude has a uniform magnitude. In multi-frequency ANM, spatial aliasing of DOAs at higher frequencies can cause challenges. We discuss this issue extensively and propose a robust solution to combat aliasing. Numerical results support our theoretical findings and demonstrate the effectiveness of the proposed method.

Secrecy rate optimization for dual‐functional radar‐communication system with energy receiver

AbstractThis article studies the secrecy rate optimization in the downlink transmission of a dual‐functional radar‐communication system, under the simultaneous wireless information and power transfer (SWIPT) setting. Specifically, the base station (BS) simultaneously transmits information and energy signals while detecting the target, where the energy receiver (ER) also behaves as a potential eavesdropper. In order to ensure secure information transmission, we design the beamformer to avoid information leakage by maximizing the secrecy rate under the total transmit power budget, the received signal‐to‐noise ratio (SNR) target at the BS, and the received energy requirement of the ER. To deal with the non‐convexity of the cost function and constraints in the original problem, we first transform the problem into an equivalent semidefinite relaxation (SDR) version by linearizing the cost function with the introduced auxiliary variables. Thus, the off‐the‐shelf semidefinite programming (SDP) solver can be leveraged to obtain desired beamforming matrices, which is guaranteed to be rank‐1. To alleviate the computational costs of SDP, we further propose a sub‐optimal solution via the block coordinate descent method and the Taylor series expansion. Numerical results validate the effectiveness and superiority of the proposed algorithms.

Robust Optimization and Data Classification for Characterization of Huntington Disease Onset via Duality Methods

The features that characterize the onset of Huntington disease (HD) are poorly understood yet have significant implications for research and clinical practice. Motivated by the need to address this issue, and the fact that there may be inaccuracies in clinical HD data, we apply robust optimization and duality techniques to study support vector machine (SVM) classifiers in the face of uncertainty in feature data. We present readily numerically solvable semi-definite program reformulations via conic duality for a broad class of robust SVM classification problems under a general spectrahedron uncertainty set that covers the most commonly used uncertainty sets of robust optimization models, such as boxes, balls, and ellipsoids. In the case of the box-uncertainty model, we also provide a new simple quadratic program reformulation, via Lagrangian duality, leading to a very efficient iterative scheme for robust classifiers. Computational results on a range of datasets indicate that these robust classification methods allow for greater classification accuracies than conventional support vector machines in addition to selecting groups of highly correlated features. The conic duality-based robust SVMs were also successfully applied to a new, large HD dataset, achieving classification accuracies of over 95% and providing important information about the features that characterize HD onset.

On exact Reznick, Hilbert-Artin and Putinar's representations

We consider the problem of computing exact sums of squares (SOS) decompositions for certain classes of non-negative multivariate polynomials, relying on semidefinite programming (SDP) solvers.We provide a hybrid numeric-symbolic algorithm computing exact rational SOS decompositions with rational coefficients for polynomials lying in the interior of the SOS cone. The first step of this algorithm computes an approximate SOS decomposition for a perturbation of the input polynomial with an arbitrary-precision SDP solver. Next, an exact SOS decomposition is obtained thanks to the perturbation terms and a compensation phenomenon. We prove that bit complexity estimates on output size and runtime are both polynomial in the degree of the input polynomial and singly exponential in the number of variables. Next, we apply this algorithm to compute exact Reznick, Hilbert-Artin's representation and Putinar's representations respectively for positive definite forms and positive polynomials over basic compact semi-algebraic sets. We also report on practical experiments done with the implementation of these algorithms and existing alternatives such as the critical point method and cylindrical algebraic decomposition.

Second-Order Cone Representation for Convex Sets in the Plane

Semidefinite programming (SDP) is the task of optimizing a linear function over the common solution set of finitely many linear matrix inequalities (LMIs). For the running time of SDP solvers, the maximal matrix size of these LMIs is usually more critical than their number. The semidefinite extension degree ${sxdeg}(K)$ of a convex set $K\subseteq\mathbb{R}^n$ is the smallest number $d$ such that $K$ is a linear image of a finite intersection $S_1\cap\dots\cap S_N$, where each $S_i$ is a spectrahedron defined by a linear matrix inequality of size $\le d$. Thus ${sxdeg}(K)$ can be seen as a measure for the complexity of performing semidefinite programs over the set $K$. We give several equivalent characterizations of ${sxdeg}(K)$ and use them to prove our main result: ${sxdeg}(K)\le2$ holds for any closed convex semialgebraic set $K\subseteq\mathbb{R}^2$. In other words, such $K$ can be represented using the second-order cone.

Exact Semidefinite Programming Bounds for Packing Problems

In this paper we give an algorithm to round the floating point output of a semidefinite programming solver to a solution over the rationals or a quadratic extension of the rationals. This algorithm...

Exploiting low-rank structure in semidefinite programming by approximate operator splitting

In contrast to many other convex optimization classes, state-of-the-art semidefinite programming solvers are still unable to efficiently solve large-scale instances. This work aims to reduce this scalability gap by proposing a novel proximal algorithm for solving general semidefinite programming problems. The key characteristic of the proposed algorithm is to be able to exploit the low-rank property inherent to several semidefinite programming problems. Exploiting the low-rank structure provides a substantial speedup and allows the operator splitting method to efficiently scale to larger instances. As opposed to other low-rank based methods, the proposed algorithm has convergence guarantees for general semidefinite programming problems. Additionally, an open-source semidefinite programming solver called ProxSDP is made available and its implementation details are discussed. Case studies are presented in order to evaluate the performance of the proposed methodology.

Non-unique games over compact groups and orientation estimation in cryo-EM

Let be a compact group and let . We define the non-unique games (NUG) problem as finding to minimize . We introduce a convex relaxation of the NUG problem to a semidefinite program (SDP) by taking the Fourier transform of fij over . The NUG framework can be seen as a generalization of the little Grothendieck problem over the orthogonal group and the unique games problem and includes many practically relevant problems, such as the maximum likelihood estimator to registering bandlimited functions over the unit sphere in d-dimensions and orientation estimation of noisy cryo-electron microscopy (cryo-EM) projection images. We implement an SDP solver for the NUG cryo-EM problem using the alternating direction method of multipliers (ADMM). Numerical study with synthetic datasets indicate that while our ADMM solver is slower than existing methods, it can estimate the rotations more accurately, especially at low signal-to-noise ratio (SNR).

Sums of squares in Macaulay2

The Macaulay2 package SumsOfSquares decomposes polynomials as sums of squares. It is based on methods to rationalize sum-of-squares decompositions due to Parrilo and Peyrl. The package features a data type for sums-of-squares polynomials, support for external semidefinite programming solvers, and optimization over varieties.

Semidefinite relaxation for the total least squares problem with Tikhonov-like regularization

ABSTRACT We study the total least squares (TLS) with a parametric Tikhonov-like regularization. We relax it to a semidefinite programming (SDP) problem and establish a sufficient condition to guarantee the tightness of the SDP relaxation. This special-structured SDP relaxation is further reformulated as a univariate maximization and then solved by the bisection method. Numerical results demonstrate that the bisection algorithm highly outperforms the SDP solver SeDuMi. Finally, based on the newly proposed (SDP), we propose a new SDP relaxation for (TLS) with canonical Tikhonov regularization and then employ an outer approximation scheme to solve this SDP relaxation.

Observer‐based controller for positive polynomial systems with time delay

SummaryThis paper deals with some synthesis problems for a class of positive polynomial systems with time delay, for which the state vector takes nonnegative values whenever the initial conditions are nonnegative. First, the synthesis of state feedback controllers is solved using a Lyapunov‐Krasovskii approach, including the requirement of positiveness. In a second part, observers are included. In both cases, the design conditions are presented in terms of sum of squares, which can be numerically and symbolically solved via SOSTOOLS and a semidefinite program solver. Finally, four numerical examples are given to illustrate the proposed approach.

A new optimization in SDP-based learning

Many machine learning problems can be reduced to learning a low-rank positive semidefinite matrix (denoted as Z), which encounters semidefinite program (SDP). Existing SDP solvers are often expensive for large-scale learning. In order to avoid directly solving SDP, some works convert SDP into a nonconvex program by factorizing Z as XX⊤. However, this would bring higher-order nonlinearity, resulting in scarcity of structure in subsequent optimization. In this paper, we propose a novel surrogate for SDP-based learning, in which the structure of subproblem is exploited. More specifically, we surrogate unconstrained SDP by a biconvex problem, through factorizing Z as XY⊤ and using a quadratic penalty to penalize the difference of X and Y, in which the resultant subproblems are convex. Furthermore, we provide a theory bound for the associated penalty parameter under the assumption that the objective function is Lipschitz-smooth, such that the proposed surrogate will solve the original SDP when the penalty parameter is larger than this bound. Experiments on three SDP-based machine learning problems demonstrate that the proposed algorithm is as accurate as the state-of-the-art, but is faster on large-scale learning.

An oracle for the discrete-time integral quadratic constraint problem

In the theory of integral quadratic constraints (IQCs), the condition for robust performance is given in terms of a frequency-domain inequality. This inequality can be transformed into an equivalent finite-dimensional linear matrix inequality by the application of the Kalman–Yakubovich–Popov (KYP) lemma and the introduction of an auxiliary matrix variable. Thus, the IQC problem can be expressed as a semi-definite program (SDP) and solved using convex optimization tools such as general-purpose SDP solvers based on interior point methods. However, by introducing the auxiliary variable, the size of the IQC problem significantly increases, and the aforementioned solvers may fail due to the high computational cost of the SDP. To address this issue, specialized SDP solvers have been developed to efficiently solve the SDP by exploiting its structure. An alternative approach consists of using cutting plane methods to directly solve the frequency-domain inequality without resorting to the KYP lemma. Cutting plane methods require an oracle that confirms if a given candidate solution satisfies the frequency-domain inequality. If the candidate solution is not feasible, the oracle generates a hyperplane that separates the candidate solution from the set of feasible solutions. In this paper, the analytic center cutting plane method is applied to the discrete-time IQC problem. The oracle for the discrete-time IQC problem is based on the spectrum of an extended symplectic matrix pencil and does not require an invertibility assumption that holds in continuous-time but not necessarily in discrete-time. An illustrative example is provided that shows the efficacy of the proposed method and points out directions for future research.

Accelerated Inference in Markov Random Fields via Smooth Riemannian Optimization

Markov Random Fields (MRFs) are a popular model for several pattern recognition and reconstruction problems in robotics and computer vision. Inference in MRFs is intractable in general and related work resorts to approximation algorithms. Among those techniques, semidefinite programming (SDP) relaxations have been shown to provide accurate estimates while scaling poorly with the problem size and being typically slow for practical applications. Our first contribution is to design a dual ascent method to solve standard SDP relaxations that takes advantage of the geometric structure of the problem to speed up computation. This technique, named Dual Ascent Riemannian Staircase (DARS), is able to solve large problem instances in seconds. Our second contribution is to develop a second and faster approach. The backbone of this second approach is a novel SDP relaxation combined with a fast and scalable solver based on smooth Riemannian optimization. We show that this approach, named Fast Unconstrained SEmidefinite Solver (FUSES), can solve large problems in milliseconds. Contrarily to local MRF solvers, e.g., loopy belief propagation, our approaches do not require an initial guess. Moreover, we leverage recent results from optimization theory to provide per-instance sub-optimality guarantees. We demonstrate the proposed approaches in multi-class image segmentation problems. Extensive experimental evidence shows that (i) FUSES and DARS produce near-optimal solutions, attaining an objective within 0.1% of the optimum, (ii) FUSES and DARS are remarkably faster than general-purpose SDP solvers, and FUSES is more than two orders of magnitude faster than DARS while attaining similar solution quality, (iii) FUSES is faster than local search methods while being a global solver.

Efficient, certifiably optimal clustering with applications to latent variable graphical models

Motivated by the task of clustering either d variables or d points into K groups, we investigate efficient algorithms to solve the Peng–Wei (P–W) K-means semi-definite programming (SDP) relaxation. The P–W SDP has been shown in the literature to have good statistical properties in a variety of settings, but remains intractable to solve in practice. To this end we propose FORCE, a new algorithm to solve this SDP relaxation. Compared to off-the-shelf interior point solvers, our method reduces the computational complexity of solving the SDP from $${\widetilde{{\mathcal {O}}}}(d^7\log \epsilon ^{-1})$$ to $${\widetilde{{\mathcal {O}}}}(d^{6}K^{-2}\epsilon ^{-1})$$ arithmetic operations for an $$\epsilon $$ -optimal solution. Our method combines a primal first-order method with a dual optimality certificate search, which when successful, allows for early termination of the primal method. We show for certain variable clustering problems that, with high probability, FORCE is guaranteed to find the optimal solution to the SDP relaxation and provide a certificate of exact optimality. As verified by our numerical experiments, this allows FORCE to solve the P–W SDP with dimensions in the hundreds in only tens of seconds. For a variation of the P–W SDP where K is not known a priori a slight modification of FORCE reduces the computational complexity of solving this problem as well: from $${\widetilde{{\mathcal {O}}}}(d^7\log \epsilon ^{-1})$$ using a standard SDP solver to $${\widetilde{{\mathcal {O}}}}(d^{4}\epsilon ^{-1})$$ .

In SDP Relaxations, Inaccurate Solvers Do Robust Optimization

We interpret some incorrect results (due to numerical inaccuracies) already observed when solving semidefinite programming (SDP) relaxations for polynomial optimization on a double precision floati...