The upper semicontinuity and Hölder stability of optimal solution sets for semidefinite programming

This paper characterizes the continuity property of the optimal value function in a general parametric quadratic programming problem with linear constraints. The lower semicontinuity and upper semicontinuity properties of the optimal value function are studied as well.

This paper provides stability theorems for the feasible set of optimization problems posed in locally convex topological vector spaces. The problems considered in this paper have an arbitrary number of inequality constraints and one constraint set. Different models are discussed, depending on the properties of the constraint functions (linear or not, convex or not, but at least lower semicontinuous) and one closed constraint set (but not necessarily convex). The parameter space is formed by systems of the same type as the nominal one (with the same space of variables and the same number of constraints), where the constraint set can be perturbed or not, equipped with the metric of the uniform convergence on the positive multiples of a fixed barrelled neighborhood of zero. In finite dimensions, this topology describes the uniform convergence on compact sets and, in the particular case that the constraints are linear, the uniform convergence of the vector coefficients. The paper examines, in a unified way, the lower and upper semicontinuity, and the closedness, of the feasible set mapping, the stable consistency of the constraint system with respect to arbitrary and right-hand side perturbations, Tuy and Robinson regularities, and other desirable stability properties of the feasible set.

While semidefinite programming (SDP) problems are polynomially solvable in theory, it is often difficult to solve large SDP instances in practice. One technique to address this issue is to relax the global positive-semidefiniteness (PSD) constraint and only enforce PSD-ness on smaller $$k\times k$$ principal submatrices—we call this the sparse SDP relaxation. Surprisingly, it has been observed empirically that in some cases this approach appears to produce bounds that are close to the optimal objective function value of the original SDP. In this paper, we formally attempt to compare the strength of the sparse SDP relaxation vis-a-vis the original SDP from a theoretical perspective. In order to simplify the question, we arrive at a data independent version of it, where we compare the sizes of SDP cone and the $$k$$ -PSD closure, which is the cone of matrices where PSD-ness is enforced on all $$k\times k$$ principal submatrices. In particular, we investigate the question of how far a matrix of unit Frobenius norm in the $$k$$ -PSD closure can be from the SDP cone. We provide two incomparable upper bounds on this farthest distance as a function of k and n. We also provide matching lower bounds, which show that the upper bounds are tight within a constant in different regimes of k and n. Other than linear algebra techniques, we extensively use probabilistic methods to arrive at these bounds. One of the lower bounds is obtained by observing a connection between matrices in the $$k$$ -PSD closure and matrices satisfying the restricted isometry property.

We consider the parametric programming problem (Q p ) of minimizing the quadratic function f(x,p):=x T Ax+b T x subject to the constraint Cx≤d, where x∈ℝ n , A∈ℝ n×n , b∈ℝ n , C∈ℝ m×n , d∈ℝ m , and p:=(A,b,C,d) is the parameter. Here, the matrix A is not assumed to be positive semidefinite. The set of the global minimizers and the set of the local minimizers to (Q p ) are denoted by M(p) and M loc (p), respectively. It is proved that if the point-to-set mapping M loc (·) is lower semicontinuous at p then M loc (p) is a nonempty set which consists of at most ? m,n points, where ? m,n = $\binom{m}{{\text{min}}\{[m/2],n\}}$ is the maximal cardinality of the antichains of distinct subsets of {1,2,...,m} which have at most n elements. It is proved also that the lower semicontinuity of M(·) at p implies that M(p) is a singleton. Under some regularity assumption, these necessary conditions become the sufficient ones.

The semi definite programming (SDP) problem is one of the central problems in mathematical optimization. The primal-dual interior-point method (PDIPM) is one of the most powerful algorithms for solving SDP problems, and many research groups have employed it for developing software packages. However, two well-known major bottlenecks, i.e., the generation of the Schur complement matrix (SCM) and its Cholesky factorization, exist in the algorithmic framework of the PDIPM. We have developed a new version of the semi definite programming algorithm parallel version (SDPARA), which is a parallel implementation on multiple CPUs and GPUs for solving extremely large-scale SDP problems with over a million constraints. SDPARA can automatically extract the unique characteristics from an SDP problem and identify the bottleneck. When the generation of the SCM becomes a bottleneck, SDPARA can attain high scalability using a large quantity of CPU cores and some processor affinity and memory interleaving techniques. SDPARA can also perform parallel Cholesky factorization using thousands of GPUs and techniques for overlapping computation and communication if an SDP problem has over two million constraints and Cholesky factorization constitutes a bottleneck. We demonstrate that SDPARA is a high-performance general solver for SDPs in various application fields through numerical experiments conducted on the TSUBAME 2.5 supercomputer, and we solved the largest SDP problem (which has over 2.33 million constraints), thereby creating a new world record. Our implementation also achieved 1.713 PFlops in double precision for large-scale Cholesky factorization using 2,720 CPUs and 4,080 GPUs.

In this paper, we consider a least square semidefinite programming problem under ellipsoidal data uncertainty. We show that the robustification of this uncertain problem can be reformulated as a semidefinite linear programming problem with an additional second-order cone constraint. We then provide an explicit quantitative sensitivity analysis on how the solution under the robustification depends on the size/shape of the ellipsoidal data uncertainty set. Next, we prove that, under suitable constraint qualifications, the reformulation has zero duality gap with its dual problem, even when the primal problem itself is infeasible. The dual problem is equivalent to minimizing a smooth objective function over the Cartesian product of second-order cones and the Euclidean space, which is easy to project onto. Thus, we propose a simple variant of the spectral projected gradient method (Birgin et al. in SIAM J. Optim. 10:1196---1211, 2000) to solve the dual problem. While it is well-known that any accumulation point of the sequence generated from the algorithm is a dual optimal solution, we show in addition that the dual objective value along the sequence generated converges to a finite value if and only if the primal problem is feasible, again under suitable constraint qualifications. This latter fact leads to a simple certificate for primal infeasibility in situations when the primal feasible set lies in a known compact set. As an application, we consider robust correlation stress testing where data uncertainty arises due to untimely recording of portfolio holdings. In our computational experiments on this particular application, our algorithm performs reasonably well on medium-sized problems for real data when finding the optimal solution (if exists) or identifying primal infeasibility, and usually outperforms the standard interior-point solver SDPT3 in terms of CPU time.

Semidefinite programming (SDP) covers a wide range of applications such as robust optimization, polynomial optimization, combinatorial optimization, system and control theory, financial engineering, machine learning, quantum information and quantum chemistry. In those applications, SDP problems can be large scale easily. Such large scale SDP problems often satisfy a certain sparsity characterized by a chordal graph structure. This sparsity is classified in two types. The one is the domain space sparsity (d-space sparsity) for positive semidefinite symmetric matrix variables involved in SDP problems, and the other the range space sparsity (r-space sparsity) for matrix-inequality constraints in SDP problems. In this short note, we survey how we exploit these two types of sparsities to solve large scale linear and nonlinear SDP problems. We refer to the paper [7] for more details.

The Moreau envelope function and proximal mapping in the sense of the Bregman distance

This paper presents a study of regularity of Semidefinite Programming (SDP) problems. Current methods for SDP rely on assumptions of regularity such as constraint qualifications (CQ) and well-posedness. In the absence of regularity, the characterization of optimality may fail and the convergence of algorithms is not guaranteed. Therefore, it is important to have procedures that verify the regularity of a given problem before applying any (standard) SDP solver. We suggest a simple numerical procedure to test within a desired accuracy if a given SDP problem is regular in terms of the fulfilment of the Slater CQ. Our procedure is based on the recently proposed DIIS algorithm that determines the immobile index subspace for SDP. We use this algorithm in a framework of an interactive decision support system. Numerical results using SDP problems from the literature and instances from the SDPLIB suite are presented, and a comparative analysis with other results on regularity available in the literature is made.

In this talk, we present our ongoing research project. The objective of this project is to develop advanced computing and optimization infrastructures for extremely large-scale graphs on post peta-scale supercomputers. We explain our challenge to Graph 500 and Green Graph 500 benchmarks that are designed to measure the performance of a computer system for applications that require irregular memory and network access patterns. The 1st Graph500 list was released in November 2010. The Graph500 benchmark measures the performance of any supercomputer performing a BFS (Breadth-First Search) in terms of traversed edges per second (TEPS). In 2014 and 2015, our project team was a winner of the 8th, 10th, and 11th Graph500 and the 3rd to 6th Green Graph500 benchmarks, respectively. We also present our parallel implementation for large-scale SDP (SemiDefinite Programming) problem. The semidefinite programming (SDP) problem is a predominant problem in mathematical optimization. The primal-dual interior-point method (PDIPM) is one of the most powerful algorithms for solving SDP problems, and many research groups have employed it for developing software packages. We solved the largest SDP problem (which has over 2.33 million constraints), thereby creating a new world record. Our implementation also achieved 1.774 PFlops in double precision for large-scale Cholesky factorization using 2,720 CPUs and 4,080 GPUs on the TSUBAME 2.5 supercomputer.

In this article, we study semi-definite and semi-infinite programming problems (SDSIP), which includes semi-infinite linear programs and semi-definite programs as special cases. We establish that a uniform duality between the homogeneous (SDSIP) and its Lagrangian-type dual problem is equivalent to the closedness condition of certain cone. Moreover, this closedness condition was assured by a generalized canonically closedness condition and a Slater condition. Corresponding results for the nonhomogeneous (SDSIP) problem were obtained by transforming it into an equivalent homogeneous (SDSIP) problem.

ROBUST DESIGN AND ROBUST STABILITY ANALYSIS OF ACTIVE NOISE CONTROL SYSTEMS

Since the early 1960s, polyhedral methods have played a central role in both the theory and practice of combinatorial optimization. Since the early 1990s, a new technique, semidefinite programming, has been increasingly applied to some combinatorial optimization problems. The semidefinite programming problem is the problem of optimizing a linear function of matrix variables, subject to finitely many linear inequalities and the positive semidefiniteness condition on some of the matrix variables. On certain problems, such as maximum cut, maximum satisfiability, maximum stable set and geometric representations of graphs, semidefinite programming techniques yield important new results. This monograph provides the necessary background to work with semidefinite optimization techniques, usually by drawing parallels to the development of polyhedral techniques and with a special focus on combinatorial optimization, graph theory and lift-and-project methods. It allows the reader to rigorously develop the necessary knowledge, tools and skills to work in the area that is at the intersection of combinatorial optimization and semidefinite optimization. A solid background in mathematics at the undergraduate level and some exposure to linear optimization are required. Some familiarity with computational complexity theory and the analysis of algorithms would be helpful. Readers with these prerequisites will appreciate the important open problems and exciting new directions as well as new connections to other areas in mathematical sciences that the book provides. Titles in this series are co-published with The Fields Institute for Research in Mathematical Sciences (Toronto, Ontario, Canada).|Since the early 1960s, polyhedral methods have played a central role in both the theory and practice of combinatorial optimization. Since the early 1990s, a new technique, semidefinite programming, has been increasingly applied to some combinatorial optimization problems. The semidefinite programming problem is the problem of optimizing a linear function of matrix variables, subject to finitely many linear inequalities and the positive semidefiniteness condition on some of the matrix variables. On certain problems, such as maximum cut, maximum satisfiability, maximum stable set and geometric representations of graphs, semidefinite programming techniques yield important new results. This monograph provides the necessary background to work with semidefinite optimization techniques, usually by drawing parallels to the development of polyhedral techniques and with a special focus on combinatorial optimization, graph theory and lift-and-project methods. It allows the reader to rigorously develop the necessary knowledge, tools and skills to work in the area that is at the intersection of combinatorial optimization and semidefinite optimization. A solid background in mathematics at the undergraduate level and some exposure to linear optimization are required. Some familiarity with computational complexity theory and the analysis of algorithms would be helpful. Readers with these prerequisites will appreciate the important open problems and exciting new directions as well as new connections to other areas in mathematical sciences that the book provides. Titles in this series are co-published with The Fields Institute for Research in Mathematical Sciences (Toronto, Ontario, Canada).

Theory: Optimality conditions and sensitivity analysis of cone-constrained and semi-definite programs by A. Shapiro Testing the feasibility of semidefinite programs by L. Porkolab and L. Khachiyan Polyhedra, spectrahedra, and semidefinite programming by M. V. Ramana Infinite-dimensional semidefinite programming: Regularized determinants and self-concordant barriers by L. Faybusovich Applications: A tour d'horizon on positive semidefinite and Euclidean distance matrix completion problems by M. Laurent Semidefinite programming and graph equipartition by S. E. Karisch and F. Rendl The totally nonnegative completion problem by C. R. Johnson, B. K. Kroschel, and M. Lundquist The multi-SAT algorithm by J. Gu How efficiently can we maximize threshold pseudo-Boolean functions? by M. R. Emamy-K. Faster algorithm for shortest network under given topology by G. Xue, D.-Z. Du, and F. K. Hwang Bayesian heuristic approach (BHA) and applications to discrete optimization by A. Mockus, J. Mockus, and L. Mockus Approximation clustering: A mine of semidefinite programming problems by B. Mirkin Algorithms: A long-step path following algorithm for semidefinite programming problems by K. M. Anstreicher and M. Fampa Cutting plane algorithms for semidefinite relaxations by C. Helmberg and R. Weismantel Infeasible-start semidefinite programming algorithms via self-dual embeddings by E. De Klerk, C. Roos, and T. Terlaky Solution of the trust region problem via a smooth unconstrained reformulation by S. Lucidi and L. Palagi.

Various types of upper semcontinuity properties for set-valued functions have been used in the past to obtain closure and lower closure theorems in optimal control theory as well as selection theorems and fixed-point theorems in topology. This paper unifies these various concepts by using semiclosure operators, extended topologies, and lattice theoretic operations and obtains general closure theorems. In addition, analytic criteria are given for this generalized upper semicontinuity. In particular, set-valued functions which are maximal in terms of certain properties (e.g., maximal monotone multifunctions) are shown to be necessarily upper semicontinuous.