Exact Relaxations Research Articles

On exact and inexact RLT and SDP-RLT relaxations of quadratic programs with box constraints

Quadratic programs with box constraints involve minimizing a possibly nonconvex quadratic function subject to lower and upper bounds on each variable. This is a well-known NP-hard problem that frequently arises in various applications. We focus on two convex relaxations, namely the reformulation–linearization technique (RLT) relaxation and the SDP-RLT relaxation obtained by combining the Shor relaxation with the RLT relaxation. Both relaxations yield lower bounds on the optimal value of a quadratic program with box constraints. We show that each component of each vertex of the RLT relaxation lies in the set {0,12,1}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\{0,\\frac{1}{2},1\\}$$\\end{document}. We present complete algebraic descriptions of the set of instances that admit exact RLT relaxations as well as those that admit exact SDP-RLT relaxations. We show that our descriptions can be converted into algorithms for efficiently constructing instances with (1) exact RLT relaxations, (2) inexact RLT relaxations, (3) exact SDP-RLT relaxations, and (4) exact SDP-RLT but inexact RLT relaxations. Our preliminary computational experiments illustrate that our algorithms are capable of generating computationally challenging instances for state-of-the-art solvers.

Polyhedral properties of RLT relaxations of nonconvex quadratic programs and their implications on exact relaxations

Polyhedral properties of RLT relaxations of nonconvex quadratic programs and their implications on exact relaxations

Beyond ℓ1 sparse coding in V1.

Growing evidence indicates that only a sparse subset from a pool of sensory neurons is active for the encoding of visual stimuli at any instant in time. Traditionally, to replicate such biological sparsity, generative models have been using the ℓ1 norm as a penalty due to its convexity, which makes it amenable to fast and simple algorithmic solvers. In this work, we use biological vision as a test-bed and show that the soft thresholding operation associated to the use of the ℓ1 norm is highly suboptimal compared to other functions suited to approximating ℓp with 0 ≤ p < 1 (including recently proposed continuous exact relaxations), in terms of performance. We show that ℓ1 sparsity employs a pool with more neurons, i.e. has a higher degree of overcompleteness, in order to maintain the same reconstruction error as the other methods considered. More specifically, at the same sparsity level, the thresholding algorithm using the ℓ1 norm as a penalty requires a dictionary of ten times more units compared to the proposed approach, where a non-convex continuous relaxation of the ℓ0 pseudo-norm is used, to reconstruct the external stimulus equally well. At a fixed sparsity level, both ℓ0- and ℓ1-based regularization develop units with receptive field (RF) shapes similar to biological neurons in V1 (and a subset of neurons in V2), but ℓ0-based regularization shows approximately five times better reconstruction of the stimulus. Our results in conjunction with recent metabolic findings indicate that for V1 to operate efficiently it should follow a coding regime which uses a regularization that is closer to the ℓ0 pseudo-norm rather than the ℓ1 one, and suggests a similar mode of operation for the sensory cortex in general.

The geometry of adversarial training in binary classification

AbstractWe establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type $L^1+\text{(nonlocal)}\operatorname{TV}$, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows us to establish a series of properties of optimal solutions of the original problem, including the existence of minimal and maximal solutions (interpreted in a suitable sense) and the existence of regular solutions (also interpreted in a suitable sense). In addition, we highlight how the connection between adversarial training and perimeter minimization problems provides a novel, directly interpretable, statistical motivation for a family of regularized risk minimization problems involving perimeter/total variation. The majority of our theoretical results are independent of the distance used to define adversarial attacks.

Exact SDP relaxations for quadratic programs with bipartite graph structures

For nonconvex quadratically constrained quadratic programs (QCQPs), we first show that, under certain feasibility conditions, the standard semidefinite programming (SDP) relaxation is exact for QCQPs with bipartite graph structures. The exact optimal solutions are obtained by examining the dual SDP relaxation and the rank of the optimal solution of this dual SDP relaxation under strong duality. Our results generalize the previous results on QCQPs with sign-definite bipartite graph structures, QCQPs with forest structures, and QCQPs with nonpositive off-diagonal data elements. Second, we propose a conversion method from QCQPs with no particular structure to the ones with bipartite graph structures. As a result, we demonstrate that a wider class of QCQPs can be exactly solved by the SDP relaxation. Numerical instances are presented for illustration.

Strong duality in parametric robust semi-definite linear programming and exact relaxations

This paper addresses the issue of which strong duality holds between parametric robust semi-definite linear optimization problems and their dual programs. In the case of a spectral norm uncertainty set, it yields a corresponding strong duality result with a semi-definite programming as its dual. We also show that the dual can be reformulated as a second-order cone programming problem or a linear programming problem when the constraint uncertainty sets of parametric robust semi-definite linear programs are given in terms of affinely parameterized diagonal matrix.

Geometric Properties of Time-Optimal Controls With State Constraints Using Strong Observability

This article considers minimum time optimal control problems with linear dynamics subject to state equality constraints and control inequality constraints. In the absence of state constraints, there are well-known sufficient conditions to guarantee that optimal controls are at the boundary or extreme points of the control set. With strong observability as the key tool, analogous conditions are derived for problems subject to both extrinsic and intrinsic state constraints. Understanding these geometric properties enables exact convex relaxations. The relaxation technique is used to convert a nonconvex quadratic program to a second-order cone program and a mixed integer linear program to a linear program. The relaxations accelerate numerical solution times by factors of 18 000 and 150, respectively. As such, the theorems and relaxations are seen as important tools for real-time optimization-based control.

Finite Horizon Optimal Control With Voltage Regulation

The modern grid is undergoing unprecedented transformations with the proliferation of highly variable loads and generation systems. In order to ensure that the grid operates efficiently in the presence of these diverse entities while preserving operational limits, strategies that combine traditionally disparate transient control and steady-state coordination are necessary. However, these introduce fundamental challenges such as inscalability and non-convexities. In this paper, we formulate a finite horizon constrained optimal control problem that allows for the actuation of voltage source converters of distributed energy resources to minimize thermal losses while accounting for voltage regulation and balancing power demand with available supply. We overcome the afore-mentioned challenges by introducing strategic decompositions of the original problem over spatial and temporal domains that allow for exact convex relaxations. The efficacy of our proposal is demonstrated via comprehensive theoretical, comparative and practical studies conducted on realistic medium-voltage distribution network settings.

Exact SDP relaxations of quadratically constrained quadratic programs with forest structures

We study the exactness of the semidefinite programming (SDP) relaxation of quadratically constrained quadratic programs (QCQPs). With the aggregate sparsity matrix from the data matrices of a QCQP with n variables, the rank and positive semidefiniteness of the matrix are examined. We prove that if the rank of the aggregate sparsity matrix is not less than \(n-1\) and the matrix remains positive semidefinite after replacing some off-diagonal nonzero elements with zeros, then the standard SDP relaxation provides an exact optimal solution for the QCQP under feasibility assumptions. In particular, we demonstrate that QCQPs with forest-structured aggregate sparsity matrix, such as the tridiagonal or arrow-type matrix, satisfy the exactness condition on the rank. The exactness is attained by considering the feasibility of the dual SDP relaxation, the strong duality of SDPs, and a sequence of QCQPs with perturbed objective functions, under the assumption that the feasible region is compact. We generalize our result for a wider class of QCQPs by applying simultaneous tridiagonalization on the data matrices. Moreover, simultaneous tridiagonalization is applied to a matrix pencil so that QCQPs with two constraints can be solved exactly by the SDP relaxation.

Optimal Control on Disconnected Sets Using Extreme Point Relaxations and Normality Approximations

This article presents new mathematical results for exact and approximate relaxations of nonconvex optimal control problems defined on disconnected control sets. The exact relaxations are based upon extreme point relaxations wherein the relaxed control set is constructed so that its extreme points belong to the original set. A system property required for exactness is normality. In the absence of normality, control set approximations and system dynamic perturbations are introduced so that normality holds. The results hold for linear dynamical systems and nonlinear systems affine in the control. It is shown that the convex relaxations enable the efficient solution of problems in aerospace engineering, sparsity-promoting optimization, overactuated systems, and multiplexing systems. The computational guarantees associated with convex optimization make the relaxation techniques suitable for real-time control.

Adaptive Topology Reconfiguration in Smart Distribution Networks

The proliferation of diverse power entities (e.g. renewable generation, electric vehicles and storage) in the active Distribution Networks (DNs) offers many advantages while also introducing reliability challenges due to over-voltage and under-voltage conditions in DN buses and overcurrents in the DN lines. The focus of this work is to enable the adaptive reconfiguration of the electrical topology of the DN in order to proactively mitigate these inefficiencies. The cyber-physical nature of the modern DN facilitates the exchange of carefully crafted signals amongst actuating and monitoring power entities in the DN. These signals are designed to iteratively solve the topology reconfiguration problem that is a mixed integer non-linear program. The non-convexities inherent in this formulation are addressed by applying strategic decompositions that lead to sub-problems for which exact convex relaxations exist. Strong convergence properties and performance guarantees are established from theoretical constructs of S-procedure, Schur's complement, and additive separability. Simulation studies conducted on practical DN settings and comparisons with the state-of-the-art highlight the efficacy of the proposed algorithm.

Sparse signal reconstruction for nonlinear models via piecewise rational optimization

We propose a method to reconstruct sparse signals degraded by a nonlinear distortion and acquired at a limited sampling rate. Our method formulates the reconstruction problem as a nonconvex minimization of the sum of a data fitting term and a penalization term. In contrast with most previous works which settle for approximated local solutions, we seek for a global solution to the obtained challenging nonconvex problem. Our global approach relies on the so-called Lasserre relaxation of polynomial optimization.We here specifically include in our approach the case of piecewise rational functions, which makes it possible to address a wide class of nonconvex exact and continuous relaxations of the ℓ0 penalization function. Additionally, we study the complexity of the optimization problem. It is shown how to use the structure of the problem to lighten the computational burden efficiently. Finally, numerical simulations illustrate the benefits of our method in terms of both global optimality and signal reconstruction.

A copositive Farkas lemma and minimally exact conic relaxations for robust quadratic optimization with binary and quadratic constraints

We present a new copositive Farkas lemma for a general conic quadratic system with binary constraints under a convexifiability requirement. By employing this Farkas lemma, we establish that a minimally exact conic programming relaxation holds for a convexifiable robust quadratic optimization problem with binary and quadratic constraints under a commonly used ellipsoidal uncertainty set of robust optimization. We then derive a minimally exact copositive relaxation for a robust binary quadratic program with conic linear constraints where the convexifiability easily holds.

Exact semidefinite formulations for a class of (random and non-random) nonconvex quadratic programs

We study a class of quadratically constrained quadratic programs (QCQPs), called {\em diagonal QCQPs\/}, which contain no off-diagonal terms $x_j x_k$ for $j \ne k$, and we provide a sufficient condition on the problem data guaranteeing that the basic Shor semidefinite relaxation is exact. Our condition complements and refines those already present in the literature and can be checked in polynomial time. We then extend our analysis from diagonal QCQPs to general QCQPs, i.e., ones with no particular structure. By reformulating a general QCQP into diagonal form, we establish new, polynomial-time-checkable sufficient conditions for the semidefinite relaxations of general QCQPs to be exact. Finally, these ideas are extended to show that a class of random general QCQPs has exact semidefinite relaxations with high probability as long as the number of constraints grows no faster than a fixed polynomial in the number of variables. To the best of our knowledge, this is the first result establishing the exactness of the semidefinite relaxation for random general QCQPs.

Exact relaxations for parametric robust linear optimization problems

We first show that the closedness of the characteristic cone of the constraint system of a parametric robust linear optimization problem is a necessary and sufficient condition for each robust linear program with the finite optimal value to admit exact semidefinite linear programming relaxations. We then provide the weakest regularity condition that guarantees exact second-order cone programming relaxations for parametric robust linear programs.

Evaluating the Effects of Real Power Losses in Optimal Power Flow-Based Storage Integration

This paper proposes a dc optimal power flow (DCOPF) with losses formulation (the 1-DCOPF + S problem) and uses itto investigate the role of real power losses in OPF-based gridscale storage integration. We derive the 1-DCOPF + S problem by augmenting a standard DCOPF with storage (DCOPF + S) model to include quadratic real power loss approximations. This procedure leads to a multiperiod nonconvex quadratically constrained quadratic program, which we prove can be solved to optimality using either a semidefinite or second-order cone relaxation. Our approach has some important benefits over existing models. It is more computationally tractable than ACOPF with storage (ACOPF + S) formulations and the provably exact convex relaxations guarantee that an optimal solution can be attained for a feasible problem. Adding loss approximations to a DCOPF + S model also leads to a more accurate representation of locational marginal prices, which have been shown to be critical in determining optimal storage dispatch and siting in prior ACOPF + S-based studies. Case studies demonstrate the improved accuracy of the 1-DCOPF + S model over a DCOPF + S model and the computational advantages over an ACOPF + S formulation.

Extended trust-region problems with one or two balls: exact copositive and Lagrangian relaxations

We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush–Kuhn–Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a whole class of quadratic optimization problems that enjoys exactness of copositive relaxation while the usual Lagrangian duality gap is infinite. Finally, we also provide verifiable conditions under which both the usual Lagrangian relaxation and the copositive relaxation are exact for an extended CDT (two-ball trust-region) problem. Importantly, the sufficient conditions can be verified by solving linear optimization problems.

SOS-Convex Semialgebraic Programs and its Applications to Robust Optimization: A Tractable Class of Nonsmooth Convex Optimization

In this paper, we introduce a new class of nonsmooth convex functions called SOS-convex semialgebraic functions extending the recently proposed notion of SOS-convex polynomials. This class of nonsmooth convex functions covers many common nonsmooth functions arising in the applications such as the Euclidean norm, the maximum eigenvalue function and the least squares functions with $\ell_1$-regularization or elastic net regularization used in statistics and compressed sensing. We show that, under commonly used strict feasibility conditions, the optimal value and an optimal solution of SOS-convex semi-algebraic programs can be found by solving a single semi-definite programming problem (SDP). We achieve the results by using tools from semi-algebraic geometry, convex-concave minimax theorem and a recently established Jensen inequality type result for SOS-convex polynomials. As an application, we outline how the derived results can be applied to show that robust SOS-convex optimization problems under restricted spectrahedron data uncertainty enjoy exact SDP relaxations. This extends the existing exact SDP relaxation result for restricted ellipsoidal data uncertainty and answers the open questions left in [Optimization Letters 9, 1-18(2015)] on how to recover a robust solution from the semi-definite programming relaxation in this broader setting.

Exact Semidefinite Programming Relaxations with Truncated Moment Matrix for Binary Polynomial Optimization Problems

For binary polynomial optimization problems (POPs) of degree $d$ with $n$ variables, we prove that the $\lceil(n+d-1)/2\rceil$th semidefinite programming (SDP) relaxation in Lasserre's hierarchy of SDP relaxations provides the exact optimal value. If binary POPs involve only even-degree monomials, we show that it can be further reduced to $\lceil(n+d-2)/2\rceil$. This bound on the relaxation order coincides with the conjecture by Laurent in 2003, which was recently proved by Fawzi, Saunderson, and Parrilo, on binary quadratic optimization problems where $d=2$. We also numerically confirm that the bound is tight. More precisely, we present instances of binary POPs that require solving at least the $\lceil(n+d-1)/2\rceil$th SDP relaxation for general binary POPs and the $\lceil(n+d-2)/2\rceil$th SDP relaxation for even-degree binary POPs to obtain the exact optimal values.

Exact Conic Programming Relaxations for a Class of Convex Polynomial Cone Programs

In this paper, under a suitable regularity condition, we establish a broad class of conic convex polynomial optimization problems, called conic sum-of-squares convex polynomial programs, exhibiting exact conic programming relaxation, which can be solved by various numerical methods such as interior point methods. By considering a general convex cone program, we give unified results that apply to many classes of important cone programs, such as the second-order cone programs, semidefinite programs, and polyhedral cone programs. When the cones involved in the programs are polyhedral cones, we present a regularity-free exact semidefinite programming relaxation. We do this by establishing a sum-of-squares polynomial representation of positivity of a real sum-of-squares convex polynomial over a conic sum-of-squares convex system. In many cases, the sum-of-squares representation can be numerically checked via solving a conic programming problem. Consequently, we also show that a convex set, described by a conic sum-of-squares convex polynomial, is (lifted) conic linear representable in the sense that it can be expressed as (a projection of) the set of solutions to some conic linear systems.