Primal-dual Path-following Algorithm Research Articles

Elliptic cone optimization and primal–dual path-following algorithms

In elliptic cone optimization problems, we minimize a linear objective function over the intersection of an affine linear manifold with the Cartesian product of the so-called elliptic cones. We present some general classes of optimization problems that can be cast as elliptic cone programmes such as second-order cone programmes and circular cone programmes. We also describe some real-world applications of this class of optimization problems. We study and analyse the Jordan algebraic structure of the elliptic cones. Then, we present a glimpse of the duality theory associated with elliptic cone optimization. A primal–dual path-following interior-point algorithm is derived for elliptic cone optimization problems. We prove the polynomial convergence of the proposed algorithms by showing that the logarithmic barrier is a strongly self-concordant barrier. The numerical examples show the path-following algorithms are efficient.

Polynomial Convergence of Primal-Dual Path-Following Algorithms for Symmetric Cone Programming Based on Wide Neighborhoods and a New Class of Directions

This paper presents a class of primal-dual path-following interior-point algorithms for symmetric cone programming (SCP) based on wide neighborhoods and new directions with a parameter $$\theta $$ . When the parameter $$\theta =1$$ , the direction is exactly the classical Newton direction. When the parameter $$\theta $$ is independent of the rank of the associated Euclidean Jordan algebra, the algorithm terminates in at most $${\mathcal {O}}\left( \kappa r\log \varepsilon ^{-1}\right) $$ iterations, which coincides with the best known iteration bound for the classical wide neighborhood algorithms. When the parameter $$\theta =\sqrt{n/\beta \tau }$$ and Nesterov–Todd search direction is used, the algorithm has $${\mathcal {O}}\left( \sqrt{r}\log \varepsilon ^{-1}\right) $$ iteration complexity, the best iteration complexity obtained so far by any interior-point method for solving SCP. To our knowledge, this is the first time that a class of interior-point algorithms including the classical wide neighborhood path-following algorithm is proposed and analyzed over symmetric cone.

A New Second-Order Infeasible Primal-Dual Path-Following Algorithm for Symmetric Optimization

ABSTRACTIn this article, we propose a new second-order infeasible primal-dual path-following algorithm for symmetric cone optimization. The algorithm further improves the complexity bound of a wide infeasible primal-dual path-following algorithm. The theory of Euclidean Jordan algebras is used to carry out our analysis. The convergence is shown for a commutative class of search directions. In particular, the complexity bound is 𝒪(r5/4log ϵ−1) for the Nesterov-Todd direction, and 𝒪(r7/4log ϵ−1) for the xs and sx directions, where r is the rank of the associated Euclidean Jordan algebra and ϵ is the required precision. If the starting point is strictly feasible, then the corresponding bounds can be reduced by a factor of r3/4. Some preliminary numerical results are provided as well.

Active-set prediction for interior point methods using controlled perturbations

We propose the use of controlled perturbations to address the challenging question of optimal active-set prediction for interior point methods. Namely, in the context of linear programming, we consider perturbing the inequality constraints/bounds so as to enlarge the feasible set. We show that if the perturbations are chosen appropriately, the solution of the original problem lies on or close to the central path of the perturbed problem. We also find that a primal-dual path-following algorithm applied to the perturbed problem is able to accurately predict the optimal active set of the original problem when the duality gap for the perturbed problem is not too small; furthermore, depending on problem conditioning, this prediction can happen sooner than predicting the active set for the perturbed problem or when the original one is solved. Encouraging preliminary numerical experience is reported when comparing activity prediction for the perturbed and unperturbed problem formulations.

An iterative solver-based long-step infeasible primal-dual path-following algorithm for convex QP based on a class of preconditioners

In this paper, we present a long-step infeasible primal-dual path-following algorithm for convex quadratic programming (CQP) whose search directions are computed by means of a preconditioned iterative linear solver. In contrast to the authors’ previous paper [Z. Lu, R.D.C. Monteiro, and J.W. O'Neal. An iterative solver-based infeasible primal-dual path-following algorithm for convex quadratic programming, SIAM J. Optim. 17(1) (2006), pp. 287–310], we propose a new linear system, which we refer to as the hybrid augmented normal equation (HANE), to determine the primal-dual search directions. Since the iterative linear solver can only generate an approximate solution to the HANE, this solution does not yield a primal-dual search direction satisfying all equations of the primal-dual Newton system. We propose a recipe to compute an inexact primal-dual search direction, based on a suitable approximate solution to the HANE. The second difference between this paper and [Z. Lu, R.D.C. Monteiro, and J.W. O'Neal. An iterative solver-based infeasible primal-dual path-following algorithm for convex quadratic programming, SIAM J. Optim. 17(1)(2006), pp. 287–310] is that, instead of using the maximum weight basis (MWB) preconditioner in the aforesaid recipe for constructing the inexact search direction, this paper proposes the use of any member of a whole class of preconditioners, of which the MWB preconditioner is just a special case. The proposed recipe allows us to: (i) establish a polynomial bound on the number of iterations performed by our path-following algorithm and (ii) establish a uniform bound, depending on the quality of the preconditioner, on the number of iterations performed by the iterative solver.

Complexity of the primal–dual path-following algorithms for the weighted determinant maximization problems with linear matrix inequalities in the narrow neighbourhood

Weighted determinant maximization with linear matrix inequality constraints (maxdet-problem) is a generalization of the semidefinite programming. We give a polynomial-time complexity analysis for the path-following interior-point short-step and predictor–corrector methods for the maxdet-problem based on symmetric Newton equations for certain classes of scaling matrices.

A numerical study of an infeasible primal-dual path-following algorithm for linear programming

In this paper, we propose an infeasible primal-dual path-following interior point algorithm to solve linear programming problems. We show that the algorithm converges globally linear and finds an approximate solution in a polynomial time complexity. A numerical study is done for its numerical performance. Some numerical examples that illustrate the approach are given. Finally, an important comparison of the obtained results with those given by the feasible projective Karmarkar algorithm is done.

An Iterative Solver-Based Infeasible Primal-Dual Path-Following Algorithm for Convex Quadratic Programming

In this paper we develop a long-step primal-dual infeasible path-following algorithm for convex quadratic programming (CQP) whose search directions are computed by means of a preconditioned iterative linear solver. We propose a new linear system, which we refer to as the augmented normal equation (ANE), to determine the primal-dual search directions. Since the condition number of the ANE coefficient matrix may become large for degenerate CQP problems, we use a maximum weight basis preconditioner introduced in [A. R. L. Oliveira and D. C. Sorensen, Linear Algebra Appl., 394 (2005), pp. 1-24; M. G. C. Resende and G. Veiga, SIAM J. Optim., 3 (1993), pp. 516-537; P. Vaida, Solving Linear Equations with Symmetric Diagonally Dominant Matrices by Constructing Good Preconditioners, Tech. report, Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, 1990] to precondition this matrix. Using a result obtained in [R. D. C. Monteiro, J. W. O'Neal, and T. Tsuchiya, SIAM J. Optim., 15 (2004), pp. 96-100], we establish a uniform bound, depending only on the CQP data, forthe number of iterations needed by the iterative linear solver to obtain a sufficiently accurate solution to the ANE. Since the iterative linear solver can generate only an approximate solution to the ANE, this solution does not yield a primal-dual search direction satisfying all equations of the primal-dual Newton system. We propose a way to compute an inexact primal-dual search direction so that the equation corresponding to the primal residual is satisfied exactly, while the one corresponding to the dual residual contains a manageable error which allows us to establish a polynomial bound on the number of iterations of our method.

First- and second-order methods for semidefinite programming

In this paper, we survey the most recent methods that have been developed for the solution of semidefinite programs. We first concentrate on the methods that have been primarily motivated by the interior point (IP) algorithms for linear programming, putting special emphasis in the class of primal-dual path-following algorithms. We also survey methods that have been developed for solving large-scale SDP problems. These include first-order nonlinear programming (NLP) methods and more specialized path-following IP methods which use the (preconditioned) conjugate gradient or residual scheme to compute the Newton direction and the notion of matrix completion to exploit data sparsity.

Solving semidefinite-quadratic-linear programs using SDPT3

This paper discusses computational experiments with linear optimization problems involving semidefinite, quadratic, and linear cone constraints (SQLPs). Many test problems of this type are solved using a new release of SDPT3, a Matlab implementation of infeasible primal-dual path-following algorithms. The software developed by the authors uses Mehrotra-type predictor-corrector variants of interior-point methods and two types of search directions: the HKM and NT directions. A discussion of implementation details is provided and computational results on problems from the SDPLIB and DIMACS Challenge collections are reported.

Semidefinite Programming in the Space of Partial Positive Semidefinite Matrices

We build upon the work of Fukuda et al. [SIAM J. Optim., 11 (2001), pp. 647--674] and Nakata et al. [Math. Program., 95 (2003), pp. 303--327], in which the theory of partial positive semidefinite matrices was applied to the semidefinite programming (SDP) problem as a technique for exploiting sparsity in the data. In contrast to their work, which improved an existing algorithm based on a standard search direction, we present a primal-dual path-following algorithm that is based on a new search direction, which, roughly speaking, is defined completely within the space of partial symmetric matrices. We show that the proposed algorithm computes a primal-dual solution to the SDP problem having duality gap less than a fraction $\varepsilon > 0$ of the initial duality gap in ${\cal O}(n \log(\varepsilon^{-1}))$ iterations, where n is the size of the matrices involved. Moreover, we present computational results showing that the algorithm possesses several advantages over other existing implementations.

Polynomial convergence of primal-dual algorithms for the second-order cone program based on the MZ-family of directions

In this paper we study primal-dual path-following algorithms for the second-order cone programming (SOCP) based on a family of directions that is a natural extension of the Monteiro-Zhang (MZ) family for semidefinite programming. We show that the polynomial iteration-complexity bounds of two well-known algorithms for linear programming, namely the short-step path-following algorithm of Kojima et al. and Monteiro and Adler, and the predictor-corrector algorithm of Mizuno et al., carry over to the context of SOCP, that is they have an O( loge-1) iteration-complexity to reduce the duality gap by a factor of e, where n is the number of second-order cones. Since the MZ-type family studied in this paper includes an analogue of the Alizadeh, Haeberly and Overton pure Newton direction, we establish for the first time the polynomial convergence of primal-dual algorithms for SOCP based on this search direction.

Parallel Computing in Optimization.

Parallel Computing in Optimization.

Symmetric primal-dual path-following algorithms for semidefinite programming

We propose a framework for developing and analyzing primal-dual interior point algorithms for semidefinite programming. This framework is an extension of the v-space approach that was developed by Kojima et al. (1991) for linear complementarity problems. The extension to semidefinite programming allows us to interpret Nesterov-Todd type directions (Nesterov and Todd 1995, 1997) as Newton search directions. Our approach does not involve any barrier function. Several primal-dual path-following algorithms for semidefinite programming are analyzed. The treatment of these algorithms for semidefinite programming in our setting bears great similarity to the linear programming case.

Primal-Dual Path-Following Algorithms for Determinant Maximization Problems With Linear Matrix Inequalities

Primal-dual path-following algorithms are considered for determinant maximization problem (maxdet-problem). These algorithms apply Newton‘s method to a primal-dual central path equation similar to that in semidefinite programming (SDP) to obtain a Newton system which is then symmetrized to avoid nonsymmetric search direction. Computational aspects of the algorithms are discussed, including Mehrotra-type predictor-corrector variants. Focusing on three different symmetrizations, which leads to what are known as the AHO, H..K..M and NT directions in SDP, numerical results for various classes of maxdet-problem are given. The computational results show that the proposed algorithms are efficient, robust and accurate.

A convergence analysis of the scaling-invariant primal-dual path-following algorithms for second-order cone programming*

In this paper we study primal-dual path-following algorithms for second-order cone programming problems (SOCP). We extend the standard long-step/semilong-step/short-step primal-dual path-following alogorithms for LP and SDP to SOCP, and prove that the long-step algorithm using the NT direction and the HRVW/KSH/M direction have O(n log ε-1 ) iteration-complexity and O(n3/2log ε-1 ) iteration-complexity, respectively, to reduce the duality gap by a factor of 1/ε, where n is the number of the second-order cones. We also show that the short and semilong-step algorithms using the NT direction and the HRVW/KSH/M direction have and O(n log ε-1 ) iteration-complexities, respectively.

Extending Mehrotra and Gondzio higher order methods to mixed semidefinite-quadratic-linear programming

We discuss extensions of Mehrotra's higher order corrections scheme and Gondzio's multiple centrality corrections scheme to mixed semidefinite-quadratic-linear programming (SQLP). These extensions have been included in a solver for SQLP written in C and based on LAPACK. The code implements a primal-dual path-following algorithm for solving SQLP problems based on the XZ + ZX search direction and Mehrotra's predictor-corrector method. We present benchmarks showing that the use of the higher order schemes yields substantial reductions in both the number of iterations and the running time of the algorithm, and also improves its robustness.

A unified analysis for a class of long-step primal-dual path-following interior-point algorithms for semidefinite programming

We present a unified analysis for a class of long-step primal-dual path-following algorithms for semidefinite programming whose search directions are obtained through linearization of the symmetrized equation of the central path Hp(XS) -- [PXSP -~ + (PXSP 1)TI/2 = #I, introduced by Zhang. At an iterate (X, S), we choose a scaling matrix P from the class of nonsingular matrices P such that PXSP -~ is symmetric. This class of matrices includes the three well-known choices, namely: P = S ~/2 and P - X -~/2 proposed by Monteiro, and the matrix P corresponding to the Nesterov Todd direction. We show that within the class of algorithms studied in this paper, the one based on the Nesterov-Todd direction has the lowest possible iteration-complexity bound that can provably be derived from our analysis. More specifically, its iteration-complexity bound is of the same order as that of the corresponding long-step primal-dual path-following algorithm for linear programming introduced by Kojima, Mizuno and Yoshise. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.

A relaxed primal-dual path-following algorithm for linear programming

In this paper, we provide an easily satisfied relaxation condition for the primaldual interior path-following algorithm to solve linear programming problems. It is shown that the relaxed algorithm preserves the property of polynomial-time convergence. The computational results obtained by implementing two versions of the relaxed algorithm with slight modifications clearly demonstrate the potential in reducing computational efforts.

ALPO: Another Linear Program Optimizer

This paper describes an implementation of the one-phase primal-dual path-following algorithm for solving linear programming problems. The design is intended to be simple, portable and robust. These design goals are achieved without sacrificing state-of-the-art performance. We give a rather complete description of the algorithm and the implementation. Computational results obtained for the NETLIB suite of problems run on a Silicon Graphics workstation are also presented. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.