Semidefinite programming duality and linear system theory: connections and implications for computation

Several important problems in control theory can be reformulated as semidefinite programming problems, i.e., minimization of a linear objective subject to linear matrix inequality (LMI) constraints. From convex optimization duality theory, conditions for infeasibility of the LMIs, as well as dual optimization problems, can be formulated. These can in turn be reinterpreted in control or system theoretic terms, often yielding new results or new proofs for existing results from control theory. We explore such connections for a few problems associated with linear time-invariant systems.

The semi definite programming (SDP) problem is one of the central problems in mathematical optimization. The primal-dual interior-point method (PDIPM) is one of the most powerful algorithms for solving SDP problems, and many research groups have employed it for developing software packages. However, two well-known major bottlenecks, i.e., the generation of the Schur complement matrix (SCM) and its Cholesky factorization, exist in the algorithmic framework of the PDIPM. We have developed a new version of the semi definite programming algorithm parallel version (SDPARA), which is a parallel implementation on multiple CPUs and GPUs for solving extremely large-scale SDP problems with over a million constraints. SDPARA can automatically extract the unique characteristics from an SDP problem and identify the bottleneck. When the generation of the SCM becomes a bottleneck, SDPARA can attain high scalability using a large quantity of CPU cores and some processor affinity and memory interleaving techniques. SDPARA can also perform parallel Cholesky factorization using thousands of GPUs and techniques for overlapping computation and communication if an SDP problem has over two million constraints and Cholesky factorization constitutes a bottleneck. We demonstrate that SDPARA is a high-performance general solver for SDPs in various application fields through numerical experiments conducted on the TSUBAME 2.5 supercomputer, and we solved the largest SDP problem (which has over 2.33 million constraints), thereby creating a new world record. Our implementation also achieved 1.713 PFlops in double precision for large-scale Cholesky factorization using 2,720 CPUs and 4,080 GPUs.

In this paper, we consider a least square semidefinite programming problem under ellipsoidal data uncertainty. We show that the robustification of this uncertain problem can be reformulated as a semidefinite linear programming problem with an additional second-order cone constraint. We then provide an explicit quantitative sensitivity analysis on how the solution under the robustification depends on the size/shape of the ellipsoidal data uncertainty set. Next, we prove that, under suitable constraint qualifications, the reformulation has zero duality gap with its dual problem, even when the primal problem itself is infeasible. The dual problem is equivalent to minimizing a smooth objective function over the Cartesian product of second-order cones and the Euclidean space, which is easy to project onto. Thus, we propose a simple variant of the spectral projected gradient method (Birgin et al. in SIAM J. Optim. 10:1196---1211, 2000) to solve the dual problem. While it is well-known that any accumulation point of the sequence generated from the algorithm is a dual optimal solution, we show in addition that the dual objective value along the sequence generated converges to a finite value if and only if the primal problem is feasible, again under suitable constraint qualifications. This latter fact leads to a simple certificate for primal infeasibility in situations when the primal feasible set lies in a known compact set. As an application, we consider robust correlation stress testing where data uncertainty arises due to untimely recording of portfolio holdings. In our computational experiments on this particular application, our algorithm performs reasonably well on medium-sized problems for real data when finding the optimal solution (if exists) or identifying primal infeasibility, and usually outperforms the standard interior-point solver SDPT3 in terms of CPU time.

This paper presents a study of regularity of Semidefinite Programming (SDP) problems. Current methods for SDP rely on assumptions of regularity such as constraint qualifications (CQ) and well-posedness. In the absence of regularity, the characterization of optimality may fail and the convergence of algorithms is not guaranteed. Therefore, it is important to have procedures that verify the regularity of a given problem before applying any (standard) SDP solver. We suggest a simple numerical procedure to test within a desired accuracy if a given SDP problem is regular in terms of the fulfilment of the Slater CQ. Our procedure is based on the recently proposed DIIS algorithm that determines the immobile index subspace for SDP. We use this algorithm in a framework of an interactive decision support system. Numerical results using SDP problems from the literature and instances from the SDPLIB suite are presented, and a comparative analysis with other results on regularity available in the literature is made.

Exact SDP relaxations for classes of nonlinear semidefinite programming problems

Several important problems in control theory can be reformulated as convex optimization problems. From duality theory in convex optimization, dual problems can be derived for these convex optimization problems. These dual problems can in turn be reinterpreted in control or system theoretic terms, often yielding new results or new proofs for existing results from control theory. Moreover, the most efficient algorithms for convex optimization solve the primal and dual problems simultaneously. Insight into the system-theoretic meaning of the dual problem can therefore be very helpful in developing efficient algorithms. The authors demonstrate these observations with some examples.

ROBUST DESIGN AND ROBUST STABILITY ANALYSIS OF ACTIVE NOISE CONTROL SYSTEMS

The semidefinite programming (SDP) problem is one of the central problems in mathematical optimization. The primal-dual interior-point method (PDIPM) is one of the most powerful algorithms for solving SDP problems, and many research groups have employed it for developing software packages. However, two well-known major bottlenecks, i.e., the generation of the Schur complement matrix (SCM) and its Cholesky factorization, exist in the algorithmic framework of the PDIPM. We have developed a new version of the semidefinite programming algorithm parallel version (SDPARA), which is a parallel implementation on multiple CPUs and GPUs for solving extremely large-scale SDP problems with over a million constraints. SDPARA can automatically extract the unique characteristics from an SDP problem and identify the bottleneck. When the generation of the SCM becomes a bottleneck, SDPARA can attain high scalability using a large quantity of CPU cores and some processor affinity and memory interleaving techniques. SDPARA can also perform parallel Cholesky factorization using thousands of GPUs and techniques for overlapping computation and communication if an SDP problem has over 2 million constraints and Cholesky factorization constitutes a bottleneck. We demonstrate that SDPARA is a high-performance general solver for SDPs in various application fields through numerical experiments conducted on the TSUBAME 2.5 supercomputer, and we solved the largest SDP problem (which has over 2.33 million constraints), thereby creating a new world record. Our implementation also achieved 1.713 PFlops in double precision for large-scale Cholesky factorization using 2,720 CPUs and 4,080 GPUs.

In this paper, we introduce a transformation that converts a class of linear and nonlinear semidefinite programming (SDP) problems into nonlinear optimization problems. For those problems of interest, the transformation replaces matrix-valued constraints by vector-valued ones, hence reducing the number of constraints by an order of magnitude. The class of transformable problems includes instances of SDP relaxations of combinatorial optimization problems with binary variables as well as other important SDP problems. We also derive gradient formulas for the objective function of the resulting nonlinear optimization problem and show that both function and gradient evaluations have affordable complexities that effectively exploit the sparsity of the problem data. This transformation, together with the efficient gradient formulas, enables the solution of very large-scale SDP problems by gradient-based nonlinear optimization techniques. In particular, we propose a first-order log-barrier method designed for solving a class of large-scale linear SDP problems. This algorithm operates entirely within the space of the transformed problem while still maintaining close ties with both the primal and the dual of the original SDP problem. Global convergence of the algorithm is established under mild and reasonable assumptions.

Semidefinite programming (SDP) covers a wide range of applications such as robust optimization, polynomial optimization, combinatorial optimization, system and control theory, financial engineering, machine learning, quantum information and quantum chemistry. In those applications, SDP problems can be large scale easily. Such large scale SDP problems often satisfy a certain sparsity characterized by a chordal graph structure. This sparsity is classified in two types. The one is the domain space sparsity (d-space sparsity) for positive semidefinite symmetric matrix variables involved in SDP problems, and the other the range space sparsity (r-space sparsity) for matrix-inequality constraints in SDP problems. In this short note, we survey how we exploit these two types of sparsities to solve large scale linear and nonlinear SDP problems. We refer to the paper [7] for more details.

In this talk, we present our ongoing research project. The objective of this project is to develop advanced computing and optimization infrastructures for extremely large-scale graphs on post peta-scale supercomputers. We explain our challenge to Graph 500 and Green Graph 500 benchmarks that are designed to measure the performance of a computer system for applications that require irregular memory and network access patterns. The 1st Graph500 list was released in November 2010. The Graph500 benchmark measures the performance of any supercomputer performing a BFS (Breadth-First Search) in terms of traversed edges per second (TEPS). In 2014 and 2015, our project team was a winner of the 8th, 10th, and 11th Graph500 and the 3rd to 6th Green Graph500 benchmarks, respectively. We also present our parallel implementation for large-scale SDP (SemiDefinite Programming) problem. The semidefinite programming (SDP) problem is a predominant problem in mathematical optimization. The primal-dual interior-point method (PDIPM) is one of the most powerful algorithms for solving SDP problems, and many research groups have employed it for developing software packages. We solved the largest SDP problem (which has over 2.33 million constraints), thereby creating a new world record. Our implementation also achieved 1.774 PFlops in double precision for large-scale Cholesky factorization using 2,720 CPUs and 4,080 GPUs on the TSUBAME 2.5 supercomputer.

This paper presents an iterative algorithm to compute a Robust Control Invariant (RCI) set, along with an invariance-inducing control law, for Linear Parameter-Varying (LPV) systems. As real-time measurements of the scheduling parameters are typically available, we allow the RCI set description and the invariance-inducing controller to be scheduling parameter dependent. Thus, the considered formulation leads to parameter-dependent conditions for the set invariance, which are replaced by sufficient Linear Matrix Inequalities (LMIs) via Polya’s relaxation. These LMI conditions are then combined with a novel volume maximization approach in a Semidefinite Programming (SDP) problem, which aims at computing the desirably large RCI set. Besides ensuring invariance, it is also possible to guarantee performance within the RCI set by imposing a chosen quadratic performance level as an additional constraint in the SDP problem. Using numerical examples, we show that the presented iterative algorithm can generate RCI sets for large parameter variations where commonly used robust approaches fail.

Several interesting control system design and analysis problems can be reduced to quantifier elimination (QE) problems. In this paper, we focus on semidefinite programming (SDP) problems, which are one of the generic linear matrix inequality (LMI) problems. We present a new symbolic method based on QE for the SDP problems and show some experiment by using existing QE package to demonstrate the capability of the method. Though currently this method is practically applicable to modest size problems which existing QE software can solve, it gives one exact solutions and enables one to deal with nonconvex as well as parametric cases. Moreover, in our scheme, the model or parameter uncertainties are easy to incorporate in the SDP problems.

Waveform design for target identification and classification in multiple-input multiple-output (MIMO) radar systems has been studied in several recent works. In previous works, optimal signals for an estimation algorithm are found assuming that only signal- independent noise exists. This work extends previous research by studying the case where clutter is also present. We develop a procedure to design the optimal waveform which minimizes estimation error at the output of the minimum mean squared error (MMSE) estimators in two scenarios. In the first one different transmit antennas see uncorrelated aspects of the target, and we consider the correlated target aspects in the second one. Estimation error in the first case will not zero even if the transmit power tends to infinity. This value of this error is referred to as the lower estimation error bound ε <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OUND</i> . It can be shown that since the MIMO radar receiver can null out the clutter subspace, ε <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OUND</i> is zero in the second scenario. Waveform design for MMSE estimator under the uncorrelated target aspects assumption, leads to the semi-definite programming (SDP) problem, a convex optimization problem which can be efficiently solved through numerical methods. An explicit solution is developed for this SDP problem in two cases. In the first case target and clutter covariance matrices are jointly diagonalizable, and in the second one the signal-to-noise ratio (SNR) is sufficiently high. Finding optimal transmit signals for the correlated target aspects scenario also results in an SDP problem.

Semidefinite programming has been an interesting and active area of research for several years. In semidefinite programming one optimizes a convex (often linear) objective function subject to a system of linear matrix inequality constraints. Despite its numerous applications, algorithms for solving semidefinite programming problems are restricted to problems of moderate size because the computation time grows faster than linear as the size increases. There are also storage requirements. So, it is of interest to consider how to identify redundant constraints from a semidefinite programming problem. However, it is known that the problem of determining whether or not a linear matrix inequality constraint is redundant or not is NP complete, in general. In this paper, we develop deterministic methods for identifying all redundant linear constraints in semidefinite programming. We use a characterization of the normal cone at a boundary point and semidefinite programming duality. Our methods extend certain redundancy techniques from linear programming to semidefinite programming.