Positive Semidefinite Constraint Research Articles

On the convex hull of convex quadratic optimization problems with indicators

We consider the convex quadratic optimization problem in Rn\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {R}^{n}$$\\end{document} with indicator variables and arbitrary constraints on the indicators. We show that a convex hull description of the associated mixed-integer set in an extended space with a quadratic number of additional variables consists of an (n+1)×(n+1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(n+1) \ imes (n+1)$$\\end{document} positive semidefinite constraint (explicitly stated) and linear constraints. In particular, convexification of this class of problems reduces to describing a polyhedral set in an extended formulation. While the vertex representation of this polyhedral set is exponential and an explicit linear inequality description may not be readily available in general, we derive a compact mixed-integer linear formulation whose solutions coincide with the vertices of the polyhedral set. We also give descriptions in the original space of variables: we provide a description based on an infinite number of conic-quadratic inequalities, which are “finitely generated.” In particular, it is possible to characterize whether a given inequality is necessary to describe the convex hull. The new theory presented here unifies several previously established results, and paves the way toward utilizing polyhedral methods to analyze the convex hull of mixed-integer nonlinear sets.

Calmness of partial perturbation to composite rank constraint systems and its applications

This paper is concerned with the calmness of a partial perturbation to the composite rank constraint system, an intersection of the rank constraint set and a general closed set, which is shown to be equivalent to a local Lipschitz-type error bound and also a global Lipschitz-type error bound under a certain compactness. Based on its lifted formulation, we derive two criteria for identifying those closed sets such that the associated partial perturbation possesses the calmness, and provide a collection of examples to demonstrate that the criteria are satisfied by common nonnegative and positive semidefinite rank constraint sets. Then, we use the calmness of this perturbation to obtain several global exact penalties for rank constrained optimization problems, and a family of equivalent DC surrogates for rank regularized problems.

Asymmetric Weighted Logistic Metric Learning for Hyperspectral Target Detection.

Traditional target detection methods assume that the background spectrum is subject to the Gaussian distribution, which may only perform well under certain conditions. In addition, traditional target detection methods suffer from the problem of the unbalanced number of target and background samples. To solve these problems, this study presents a novel target detection method based on asymmetric weighted logistic metric learning (AWLML). We first construct a logistic metric-learning approach as an objective function with a positive semidefinite constraint to learn the metric matrix from a set of labeled samples. Then, an asymmetric weighted strategy is provided to emphasize the unbalance between the number of target and background samples. Finally, an accelerated proximal gradient method is applied to identify the global minimum value. Extensive experiments on three challenging hyperspectral datasets demonstrate that the proposed AWLML algorithm improves the state-of-the-art target detection performance.

Partial Lasserre relaxation for sparse Max-Cut

A common approach to solve or find bounds of polynomial optimization problems like Max-Cut is to use the first level of the Lasserre hierarchy. Higher levels of the Lasserre hierarchy provide tighter bounds, but solving these relaxations is usually computationally intractable. We propose to strengthen the first level relaxation for sparse Max-Cut problems using constraints from the second order Lasserre hierarchy. We explore a variety of approaches for adding a subset of the positive semidefinite constraints of the second order sparse relaxation obtained by using the maximum cliques of the graph’s chordal extension. We apply this idea to sparse graphs of different sizes and densities, and provide evidence of its strengths and limitations when compared to the state-of-the-art Max-Cut solver BiqCrunch and the alternative sparse relaxation CS-TSSOS.

On approximations of the PSD cone by a polynomial number of smaller-sized PSD cones

We study the problem of approximating the cone of positive semidefinite (PSD) matrices with a cone that can be described by smaller-sized PSD constraints. Specifically, we ask the question: “how closely can we approximate the set of unit-trace \(n \times n\) PSD matrices, denoted by D, using at most N number of \(k \times k\) PSD constraints?” In this paper, we prove lower bounds on N to achieve a good approximation of D by considering two constructions of an approximating set. First, we consider the unit-trace \(n \times n\) symmetric matrices that are PSD when restricted to a fixed set of k-dimensional subspaces in \({\mathbb {R}}^n\). We prove that if this set is a good approximation of D, then the number of subspaces must be at least exponentially large in n for any \(k = o(n)\). Second, we show that any set S that approximates D within a constant approximation ratio must have superpolynomial \({\varvec{S}}_+^k\)-extension complexity. To be more precise, if S is a constant factor approximation of D, then S must have \({\varvec{S}}_+^k\)-extension complexity at least \(\exp ( C \cdot \min \{ \sqrt{n}, n/k \})\) where C is some absolute constant. In addition, we show that any set S such that \(D \subseteq S\) and the Gaussian width of S is at most a constant times larger than the Gaussian width of D must have \({\varvec{S}}_+^k\)-extension complexity at least \(\exp ( C \cdot \min \{ n^{1/3}, \sqrt{n/k} \})\). These results imply that the cone of \(n \times n\) PSD matrices cannot be approximated by a polynomial number of \(k \times k\) PSD constraints for any \(k = o(n / \log ^2 n)\). These results generalize the recent work of Fawzi (Math Oper Res 46(4):1479–1489, 2021) on the hardness of polyhedral approximations of \({\varvec{S}}_+^n\), which corresponds to the special case with \(k=1\).

Decomposed structured subsets for semidefinite and sum-of-squares optimization

Semidefinite programs (SDPs) are standard convex problems that are frequently found in control and optimization applications. Interior-point methods can solve SDPs in polynomial time up to arbitrary accuracy, but scale poorly as the size of matrix variables and the number of constraints increases. To improve scalability, SDPs can be approximated with lower and upper bounds through the use of structured subsets (e.g., diagonally-dominant and scaled-diagonally dominant matrices). Meanwhile, any underlying sparsity or symmetry structure may be leveraged to form an equivalent SDP with smaller positive semidefinite constraints. In this paper, we present a notion of decomposed structured subsets to approximate an SDP with structured subsets after an equivalent conversion. The lower/upper bounds found by approximation after conversion become tighter than the bounds obtained by approximating the original SDP directly. We apply decomposed structured subsets to semidefinite and sum-of-squares optimization problems with examples of H∞ norm estimation and constrained polynomial optimization. An existing basis pursuit method is adapted into this framework to iteratively refine bounds.

Hierarchical Task-Space Optimal Covariance Control With Chance Constraints

This letter presents a new control paradigm applicable to nonlinear systems such as robots subject to chance and covariance assignment constraints which we refer to as hierarchical optimal covariance control. To the best of our knowledge, this is the first study to formulate the hierarchical optimal covariance control problem involving multiple operational tasks. The framework is defined as a multi-stage optimization problem considering multiple hierarchical tasks specified in lexicographic order. Towards this goal, we first approximate the nonlinear dynamic model of a robot into multiple linear stochastic systems by linearizing the model along given trajectories. We then project these stochastic models onto the null-space of the previous task models for efficiently solving lexicographical optimization. In addition, we specify probability functions to account for chance constraints using the Whittacker M function. We formulate the chance constraints as a positive semi-definite matrix constraint and solve the hierarchical optimal covariance control problem using sequential semi-definite programming. We demonstrate that this procedure yields higher accuracy for multiple hierarchical tasks than employing deterministic operational space control models.

Fast Stochastic Ordinal Embedding With Variance Reduction and Adaptive Step Size

Learning representation from relative similarity comparisons, often called ordinal embedding, gains rising attention in recent years. Most of the existing methods are based on semi-definite programming (SDP), which is generally time-consuming and degrades the scalability, especially confronting large-scale data. To overcome this challenge, we propose a stochastic algorithm called SVRG-SBB, which has the following features: i) achieving good scalability via dropping positive semi-definite (PSD) constraints as serving a fast algorithm, i.e., stochastic variance reduced gradient (SVRG) method, and ii) adaptive learning via introducing a new, adaptive step size called the stabilized Barzilai-Borwein (SBB) step size. Theoretically, under some natural assumptions, we show the <b xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</b> (1/T) O(1T) rate of convergence to a stationary point of the proposed algorithm, where T T is the number of total iterations. Under the further Polyak-Łojasiewicz assumption, we can show the global linear convergence (i.e., exponentially fast converging to a global optimum) of the proposed algorithm. Numerous simulations and real-world data experiments are conducted to show the effectiveness of the proposed algorithm by comparing with the state-of-the-art methods, notably, much lower computational cost with good prediction performance.

Generalized solution to the problem of reconstructing the graph structure based on the transfer function matr ix

This study presents a generalized solution to the problem of restoring the structure of a graph based on the method of minimizing the transfer matrix norm, consistent with the Euclidean vector norms, with a minimum and extended set of constraints. The problem of complete reconstruction of the adjacency matrix, in the presence of pairs of vectors of exogenous and endogenous influences, expressing the intrinsic resonance properties of the network, and positive semidefinite constraints on the matrix of variables, is a generalization of the problem of reconstructing the structure of a graph from the eigenvalues of the Laplace matrix.

Numerical upper bounds to the ultimate load bearing capacity of three‐dimensional reinforced concrete structures

ABSTRACTThis contribution is addressing the ultimate limit state design of massive three‐dimensional reinforced concrete structures based on a finite‐element implementation of yield design theory. The strength properties of plain concrete are modeled either by means of a tension cutoff Mohr Coulomb or a Rankine condition, while the contribution of the reinforcing bars is taken into account by means of a homogenization method. This homogenization method can either represent regions of uniformly distributed steel rebars smeared into the concrete domain, but it can also be extended to model single rebars diluted into a larger region, thereby simplifying mesh generation and mesh size requirements in this region. The present paper is mainly focused on the implementation of the upper bound kinematic approach formulated as a convex minimization problem. The retained strength condition for the plain concrete and homogenized reinforced regions are both amenable to a formulation involving positive semidefinite constraints. The resulting semidefinite programming problems can, therefore, be solved using state‐of‐the‐art dedicated solvers. The whole computational procedure is applied to some illustrative examples, where the implementation of both static and kinematic methods produces a relatively accurate bracketing of the exact failure load for this kind of structures.

Improved noise covariance estimation in visual servoing using an autocovariance least-squares approach

Position based visual servoing is a widely adopted tool in robotics and automation. While the extended Kalman filter (EKF) has been proposed as an effective technique for this, it requires accurate noise covariance matrices to render desirable performance. Although numerous techniques for updating or estimating the covariance matrices have been developed in the literature, many of these suffer from computational limits or difficulties in imposing structural constraints such as positive semi-definiteness (PSD). In this paper, a relatively new framework, namely the autocovariance least-squares (ALS) method, is applied to estimate noise covariances using real world visual servoing data. To generate the innovations data required for the ALS method, we utilize standard position based visual servoing methods such as EKF, and also an advanced optimization-based framework, namely moving horizon estimation (MHE). A major advantage of the proposed method is that the PSD and other structural constraints on the noise covariances can be enforced conveniently in the optimization problem, which can be solved efficiently using existing software packages. Our results show that using the ALS estimated covariances in the EKF, instead of hand-tuned covariances, gives more than 20% mean error reduction in visual servoing, while utilising MHE to generate the ALS innovations provides a further 21% accuracy improvement.

Inverse semidefinite quadratic programming problem with [formula omitted] norm measure

We consider an inverse problem arising from a semidefinite quadratic programming (SDQP) problem, which is a minimization problem involving l1 vector norm with positive semidefinite cone constraint. By using convex optimization theory, the first order optimality condition of the problem can be formulated as a semismooth equation. Under two assumptions, we prove that any element of the generalized Jacobian of the equation at its solution is nonsingular. Based on this, a smoothing approximation operator is given and a smoothing Newton method is proposed for solving the solution of the semismooth equation. We need to compute the directional derivative of the smoothing operator at the corresponding point and to solve one linear system per iteration in the Newton method and its global convergence is demonstrated. Finally, we give the numerical results to show the effectiveness and stability of the smoothing Newton method for this inverse problem.

Limited memory BFGS algorithm for the matrix approximation problem in Frobenius norm

This paper proposes an L-BFGS algorithm based on the active set technique to solve the matrix approximation problem: given a symmetric matrix, find a nearest approximation matrix in the sense of Frobenius norm to make it satisfy some linear equalities, inequalities and a positive semidefinite constraint. The problem is a convex optimization problem whose dual problem is a nonlinear convex optimization problem with non-negative constraints. Under the Slater constraint qualification, it has zero duality gap with the dual problem. To handle large-scale dual problem, we make use of the active set technique to estimate the active constraints, and then the L-BFGS method is used to accelerate free variables. The global convergence of the proposed algorithm is established under certain conditions. Finally, we conduct some preliminary numerical experiments, and compare the L-BFGS method with the inexact smoothing Newton method, the projected BFGS method, the alternating direction method and the two-metric projection method based on the L-BFGS. The numerical results show that our algorithm has some advantages in terms of CPU time when a large number of linear constraints are involved.

A clique graph based merging strategy for decomposable SDPs

Chordal decomposition techniques are used to reduce large structured positive semidefinite matrix constraints in semidefinite programs (SDPs). The resulting equivalent problem contains multiple smaller constraints on the nonzero blocks (or cliques) of the original problem matrices. This usually leads to a significant reduction in the overall solve time. A further reduction is possible by remerging cliques with significant overlap. The degree of overlap for which this is effective is dependent on the particular solution algorithm and hardware to be employed. We propose a novel clique merging approach that utilizes the clique graph to identify suitable merge candidates and that is suitable for any SDP solver algorithm. We show its performance in combination with a first-order method by comparing it with two existing approaches on selected problems from a benchmark library. Our approach is implemented in the latest version of the conic ADMM-solver COSMO.

Text Co-detection in Multi-view Scene.

Multi-view scene analysis has been widely explored in computer vision, including numerous practical applications. The texts in multi-view scenes are often detected by following the existing text detection method in a single image, which however ignores the multi-view corresponding constraint. The multi-view correspondences may contain structure, location information and assist difficulties induced by factors like occlusion and perspective distortion, which are deficient in the single image scene. In this paper, we address the corresponding text detection task and propose a novel text co-detection method to identify the cooccurring texts among multi-view scene images with compositions of detection and correspondence under large environmental variations. In our text co-detection method, the visual and geometrical correspondences are designed to explore texts holding high pairwise representation similarity and guide the exploitation of texts with geometrical correspondences, simultaneously. To guarantee the pairwise consistency among multiple images, we additionally incorporate the cycle consistency constraint, which guarantees alignments of text correspondences in the image set. Finally, text correspondence is represented by a permutation matrix and solved via positive semidefinite and low-rank constraints. Moreover, we also collect a new text co-detection dataset consisting of multi-view image groups obtained from the same scene with different photographing conditions. The experiments show that our text co-detection obtains satisfactory performance and outperforms the related state-of-the-art text detection methods.

Reversing Unknown Quantum Transformations: Universal Quantum Circuit for Inverting General Unitary Operations.

Given a quantum gate implementing a d-dimensional unitary operation U_{d}, without any specific description but d, and permitted to use k times, we present a universal probabilistic heralded quantum circuit that implements the exact inverse U_{d}^{-1}, whose failure probability decays exponentially in k. The protocol employs an adaptive strategy, proven necessary for the exponential performance. It requires that k≥d-1, proven necessary for the exact implementation of U_{d}^{-1} with quantum circuits. Moreover, even when quantum circuits with indefinite causal order are allowed, k≥d-1 uses are required. We then present a finite set of linear and positive semidefinite constraints characterizing universal unitary inversion protocols and formulate a convex optimization problem whose solution is the maximum success probability for given k and d. The optimal values are computed using semidefinite programing solvers for k≤3 when d=2 and k≤2 for d=3. With this numerical approach we show for the first time that indefinite causal order circuits provide an advantage over causally ordered ones in a task involving multiple uses of the same unitary operation.

A branch-and-cut algorithm for solving mixed-integer semidefinite optimization problems

We consider a cutting-plane algorithm for solving mixed-integer semidefinite optimization (MISDO) problems. In this algorithm, the positive semidefinite (psd) constraint is relaxed, and the resultant mixed-integer linear optimization problem is solved repeatedly, imposing at each iteration a valid inequality for the psd constraint. We prove the convergence properties of the algorithm. Moreover, to speed up the computation, we devise a branch-and-cut algorithm, in which valid inequalities are dynamically added during a branch-and-bound procedure. We test the computational performance of our cutting-plane and branch-and-cut algorithms for three types of MISDO problem: random instances, computing restricted isometry constants, and robust truss topology design. Our experimental results demonstrate that, for many problem instances, our branch-and-cut algorithm delivered superior performance compared with general-purpose MISDO solvers in terms of computational efficiency and stability.

Low complexity closed‐form covariance matrix and direct constant‐envelope waveforms design for MIMO radar transmit beampattern

Multiple-input multiple-output (MIMO) radar has many advantages over the phased-array radar due to the waveform diversity, one of which is the greater flexibility to design the transmit beampattern. In this Letter, a low-complexity transmit beampattern design method, considering constant envelope (CE) waveforms, is proposed for the uniform linear array MIMO radar. First, the authors parameterise the waveform covariance matrix as the integral form of continuous spatial variable, by exploiting the beamforming weight vector in the main lobe region, to derive the closed-form solution of the waveform covariance matrix that fulfils positive semi-definiteness and uniform elemental power constraints. The transmit beampattern designed by the proposed method has the spatial characteristics of low main lobe fluctuation and peak-to-sidelobe level, and further mathematical explanation is illuminated. Next, they directly use the parameterised expression of waveform covariance matrix, approximately transformed into a discrete form of the spatial variable based on the Riemann integral definition, to efficiently synthesise correlated CE waveforms. The proposed method can provide transmit beampattern performance comparable to existing methods with a lower computational cost and ensure the CE property of the waveforms. The performance of the proposed method is verified and analysed via numerical examples.

An Adaptive Nonlocal Gaussian Prior for Hyperspectral Image Denoising

Nonlocal similar patches are effectively used in the Gaussian prior denoising model. However, it is difficult to learn an accurate Gaussian model for hyperspectral image (HSI) with noisy and limited similar patches, which will result in unstable Gaussian parameters (mean and covariance). In this letter, several techniques are proposed to overcome the noisy and small sample problems for HSI denoising. For Gaussian parameters, we propose the adaptive weighted mean of nonlocal similar patches and use a positive semidefinite constraint on the covariance parameter. In addition, an iterative manner is used to achieve more accurate parameters. The proposed method can achieve more robust Gaussian model for HSI denoising. Experiments on a HSI demonstrate the effectiveness of the proposed algorithm compared with the traditional methods for HSI denoising.

Which Factorization Machine Modeling Is Better: A Theoretical Answer with Optimal Guarantee

Factorization machine (FM) is a popular machine learning model to capture the second order feature interactions. The optimal learning guarantee of FM and its generalized version is not yet developed. For a rank k generalized FM of d dimensional input, the previous best known sampling complexity is O[k3d · polylog(kd)] under Gaussian distribution. This bound is sub-optimal comparing to the information theoretical lower bound O(kd). In this work, we aim to tighten this bound towards optimal and generalize the analysis to sub-gaussian distribution. We prove that when the input data satisfies the so-called τ-Moment Invertible Property, the sampling complexity of generalized FM can be improved to O[k2d · polylog(kd)/τ2]. When the second order self-interaction terms are excluded in the generalized FM, the bound can be improved to the optimal O[kd · polylog(kd)] up to the logarithmic factors. Our analysis also suggests that the positive semi-definite constraint in the conventional FM is redundant as it does not improve the sampling complexity while making the model difficult to optimize. We evaluate our improved FM model in real-time high precision GPS signal calibration task to validate its superiority.