A Squared Smoothing Newton Method for Semidefinite Programming

Semidefinite programming (SDP) is an extension of linear programming, with vector variables replaced by matrix variables and component wise nonnegativity replaced by positive semidefiniteness. SDP's are convex, but not polyhedral, optimization problems. SDP is well on its way to becoming an established paradigm in optimization, with many current potential applications. Consequently, efficient methods and software for solving SDP's are of great importance. During the award period, attention was primarily focused on three aspects of computational semidefinite programming: General-purpose methods for semidefinite and quadratic cone programming; Specific applications (LMI problems arising in control, minimizing a sum of Euclidean norms, a quantum mechanics application of SDP); and Optimizing matrix stability.

In this paper, we consider a semidefinite programming (SDP) relaxation of the quadratic knapsack problem. After applying low-rank factorization, we get a nonconvex problem, whose feasible region is an algebraic variety with certain good geometric properties, which we analyze. We derive a rank condition under which these two formulations are equivalent. This rank condition is much weaker than the classical rank condition if the coefficient matrix has certain special structures. We also prove that, under an appropriate rank condition, the nonconvex problem has no spurious local minima without assuming linearly independent constraint qualification. We design a feasible method that can escape from nonoptimal nonregular points. Numerical experiments are conducted to verify the high efficiency and robustness of our algorithm as compared with other solvers. In particular, our algorithm is able to solve a one-million-dimensional sparse SDP problem accurately in about 20 minutes on a modest computer. Funding: The research of K.-C. Toh is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 3 grant call [Grant MOE-2019-T3-1-010].

Support vector classification (SVC) with logistic loss has excellent theoretical properties in classification problems where the label values are not continuous. In this paper, we reformulate the hyperparameter selection for SVC with logistic loss as a bilevel optimization problem in which the upper-level problem and the lower-level problem are both based on logistic loss. The resulting bilevel optimization model is converted to a single-level nonlinear programming (NLP) based on the KKT conditions of the lower-level problem. Such NLP contains a set of nonlinear equality constraints and a simple lower-bound constraint. The second-order sufficient condition is characterized, which guarantees that the strict local optimizers are obtained. To solve such NLP, we apply the smoothing Newton method proposed in [Liang L, Sun D., Toh KC. A squared smoothing Newton method for semidefinite programming, 2023] to solve the KKT conditions, which contain one pair of complementarity constraints. We show that the smoothing Newton method has a superlinear convergence rate. Extensive numerical results verify the efficiency of the proposed approach and strict local minimizers can be achieved both numerically and theoretically. In particular, compared with other methods, our algorithm can achieve competitive results while consuming less time than other methods.

We present a successive linearization method with a trust region-type globalization for the solution of nonlinear semidefinite programs. At each iteration, the method solves a quadratic semidefinite program, which can be converted to a linear semidefinite program with a second order cone constraint. A subproblem of this kind can be solved quite efficiently by using some recent software for semidefinite and second-order cone programs. The method is shown to be globally convergent under certain assumptions. Numerical results on some nonlinear semidefinite programs including optimization problems with bilinear matrix inequalities are reported to illustrate the behaviour of the proposed method.

A semidefinite programming method with graph partitioning technique for optimal power flow problems

Since the early 1960s, polyhedral methods have played a central role in both the theory and practice of combinatorial optimization. Since the early 1990s, a new technique, semidefinite programming, has been increasingly applied to some combinatorial optimization problems. The semidefinite programming problem is the problem of optimizing a linear function of matrix variables, subject to finitely many linear inequalities and the positive semidefiniteness condition on some of the matrix variables. On certain problems, such as maximum cut, maximum satisfiability, maximum stable set and geometric representations of graphs, semidefinite programming techniques yield important new results. This monograph provides the necessary background to work with semidefinite optimization techniques, usually by drawing parallels to the development of polyhedral techniques and with a special focus on combinatorial optimization, graph theory and lift-and-project methods. It allows the reader to rigorously develop the necessary knowledge, tools and skills to work in the area that is at the intersection of combinatorial optimization and semidefinite optimization. A solid background in mathematics at the undergraduate level and some exposure to linear optimization are required. Some familiarity with computational complexity theory and the analysis of algorithms would be helpful. Readers with these prerequisites will appreciate the important open problems and exciting new directions as well as new connections to other areas in mathematical sciences that the book provides. Titles in this series are co-published with The Fields Institute for Research in Mathematical Sciences (Toronto, Ontario, Canada).|Since the early 1960s, polyhedral methods have played a central role in both the theory and practice of combinatorial optimization. Since the early 1990s, a new technique, semidefinite programming, has been increasingly applied to some combinatorial optimization problems. The semidefinite programming problem is the problem of optimizing a linear function of matrix variables, subject to finitely many linear inequalities and the positive semidefiniteness condition on some of the matrix variables. On certain problems, such as maximum cut, maximum satisfiability, maximum stable set and geometric representations of graphs, semidefinite programming techniques yield important new results. This monograph provides the necessary background to work with semidefinite optimization techniques, usually by drawing parallels to the development of polyhedral techniques and with a special focus on combinatorial optimization, graph theory and lift-and-project methods. It allows the reader to rigorously develop the necessary knowledge, tools and skills to work in the area that is at the intersection of combinatorial optimization and semidefinite optimization. A solid background in mathematics at the undergraduate level and some exposure to linear optimization are required. Some familiarity with computational complexity theory and the analysis of algorithms would be helpful. Readers with these prerequisites will appreciate the important open problems and exciting new directions as well as new connections to other areas in mathematical sciences that the book provides. Titles in this series are co-published with The Fields Institute for Research in Mathematical Sciences (Toronto, Ontario, Canada).

Elliptic localization has become a hot research topic and has been widely used in various fields. In this paper, we propose a penalized semidefinite programming (SDP) method for the elliptic localization problem when both the transmitter position and signal propagation speed are unknown. We first formulate a nonconvex constrained weighted least squares (CWLS) problem using the connections between the unknown variables in the measurement model. Since the nonconvex CWLS problem is difficult to solve, we relax it into a convex SDP problem by applying semidefinite relaxation (SDR). Although the SDP problem avoids local convergence or divergence, its solution cannot reach the Cramer-Rao lower bound (CRLB) accuracy owing to the relaxation. In order to address this issue, we propose to add a penalty term to tighten the above SDP problem. Furthermore, we propose an adaptive selection procedure for choosing a proper penalty factor. Simulation results validate the effectiveness and efficiency of the penalized SDP method and show that the proposed method is able to reach the CRLB.

A relaxed cutting plane method for semi-infinite semi-definite programming

In this paper, we present a majorized semismooth Newton-CG augmented Lagrangian method, called SDPNAL $$+$$ , for semidefinite programming (SDP) with partial or full nonnegative constraints on the matrix variable. SDPNAL $$+$$ is a much enhanced version of SDPNAL introduced by Zhao et al. (SIAM J Optim 20:1737–1765, 2010) for solving generic SDPs. SDPNAL works very efficiently for nondegenerate SDPs but may encounter numerical difficulty for degenerate ones. Here we tackle this numerical difficulty by employing a majorized semismooth Newton-CG augmented Lagrangian method coupled with a convergent 3-block alternating direction method of multipliers introduced recently by Sun et al. (SIAM J Optim, to appear). Numerical results for various large scale SDPs with or without nonnegative constraints show that the proposed method is not only fast but also robust in obtaining accurate solutions. It outperforms, by a significant margin, two other competitive publicly available first order methods based codes: (1) an alternating direction method of multipliers based solver called SDPAD by Wen et al. (Math Program Comput 2:203–230, 2010) and (2) a two-easy-block-decomposition hybrid proximal extragradient method called 2EBD-HPE by Monteiro et al. (Math Program Comput 1–48, 2014). In contrast to these two codes, we are able to solve all the 95 difficult SDP problems arising from the relaxations of quadratic assignment problems tested in SDPNAL to an accuracy of $$10^{-6}$$ efficiently, while SDPAD and 2EBD-HPE successfully solve 30 and 16 problems, respectively. In addition, SDPNAL $$+$$ appears to be the only viable method currently available to solve large scale SDPs arising from rank-1 tensor approximation problems constructed by Nie and Wang (SIAM J Matrix Anal Appl 35:1155–1179, 2014). The largest rank-1 tensor approximation problem we solved (in about 14.5 h) is nonsym(21,4), in which its resulting SDP problem has matrix dimension $$n = 9261$$ and the number of equality constraints $$m =12{,}326{,}390$$ .

This paper generalizes the modified frequency response masking (MFRM) filter structure for multirate applications. The overall filter is composed of low-delay FIR and IIR filters. A new SDP method is also proposed for designing these MFRM filters. Design results show that the structure and the design method are very useful to reduce the system delay and arithmetic complexity of traditional MFRM filters for multirate application. The application of the low-delay L-band MFRM filters to realize sharp-cutoff Discrete Fourier Transform (DFT) filter bank is also studied.

A smoothing Newton method for a type of inverse semi-definite quadratic programming problem

Duality for semidefinite programming; Entropy optimization: Interior point methods; Homogeneous selfdual methods for linear programming; Linear programming: Interior point methods; Linear programming: Karmarkar projective algorithm; Matrix completion problems; Potential reduction methods for linear programming; Semi-infinite programming, semidefinite programming and perfect duality; Semidefinite programming and determinant maximization; Semidefinite programming and structural optimization; Semidefinite programming: Optimality conditions and stability; Sequential quadratic programming:

This letter proposes a two‐degree semidefinite programming (SDP) method for achieving the global optimum of the optimal power flow (OPF) problem. The method extends the variables to four degrees, and formulates an SDP problem with a two‐degree extended matrix. It satisfies the rank‐1 condition. Experimental results show that this method is more reliable for reaching the OPF's global optimum than existing SDP methods. © 2014 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

For semidefinite programming (SDP) problems, traditional primal-dual interior-point methods based on conventional matrix operations have an upper limit on the problem size that the computer can handle due to memory constraints. But for a special kind of SDP problem, which is called the banded symmetric semidefinite programming (BSDP) problem, a memory-efficient algorithm, called a structured primal-dual interior-point method, can be applied. The method is based on the observation that both banded matrices and their inverses can be represented in sequentially semi-separable (SSS) form with numerical ranks equal to the half bandwidths of the banded matrices. Moreover, all computation can be done sequentially using the SSS form. Experiments of various problem sizes are performed to verify the feasibility of the proposed method.

For moving targets localization, incorporating frequency-difference-of-arrival (FDOA) measurements in the commonly used time-difference-of-arrival (TDOA) positioning systems will improve performance. Such an approach still has unresolved technical challenges. The commonly used maximum likelihood estimator (MLE) is nonconvex and highly nonlinear, and the parameters to be estimated are mutually coupled in the positioning process. The goal of this paper is to develop an effective iterative method that resolves these challenges for moving target localization using TDOA and FDOA. Specifically, a semidefinite programming (SDP) method is proposed to transform the MLE problem into a convex optimization problem. To improve the performance further, we develop an iterative method that uses the position and velocity estimates obtained using the SDP method as the initial values. This iterative method includes two steps: update of the velocity by using a weighted least squares method and update of the position by using SDP. The major advantage of the proposed scheme is that it significantly outperforms existing methods at moderate to high noise levels, which is validated via extensive numerical results.