Methods For Convex Quadratic Programming Research Articles

A Full-Newton Step Interior-Point Method for Weighted Quadratic Programming Based on the Algebraic Equivalent Transformation

In this paper, a new full-Newton step feasible interior-point method for convex quadratic programming is presented and analyzed. The idea behind this method is to replace the complementarity condition with a non-negative weight vector and use the algebraic equivalent transformation for the obtained equation. Under the selection of appropriate parameters, the quadratic rate of convergence of the new algorithm is established. In addition, the iteration complexity of the algorithm is obtained. Finally, some numerical results are presented to demonstrate the practical performance of the proposed algorithm.

A new full-Newton step feasible interior point method for convex quadratic programming

In this paper, we propose and analyse a new full-Newton step feasible interior point method for convex quadratic programming. The basic idea of this method is to replace a complementarity condition by a non-negative variable weight vector. With a zero of weight vector, the limit of the weighted path exists and satisfies the complementarity condition, the limit yields an optimal solution of problem. In each main iteration of the new algorithm consisted of only full-Newton steps with a quadratic rate of convergence. The advantage of this method is the use of a full-Newton step, that is no calculation of the step size is required. Finally, some numerical results are reported to show the practical performance of the proposed algorithm with different parameters.

Log-domain interior-point methods for convex quadratic programming

Applying an interior-point method to the central-path conditions is a widely used approach for solving quadratic programs. Reformulating these conditions in the log-domain is a natural variation on this approach that to our knowledge is previously unstudied. In this paper, we analyze log-domain interior-point methods and prove their polynomial-time convergence. We also prove that they are approximated by classical barrier methods in a precise sense and provide simple computational experiments illustrating their superior performance.

A Unifying Complexity Certification Framework for Active-Set Methods for Convex Quadratic Programming

In model-predictive control (MPC), an optimization problem has to be solved at each time step, which in real-time applications makes it important to solve these efficiently and to have good upper bounds on worst-case solution time. Often for linear MPC problems, the optimization problem in question is a quadratic program (QP) that depends on parameters such as system states and reference signals. A popular class of methods for solving such QPs is active-set methods, where a sequence of linear systems of equations is solved. We propose an algorithm for computing which sequence of subproblems an active-set algorithm will solve, for every parameter of interest. These sequences can be used to set worst-case bounds on how many iterations, floating-point operations, and, ultimately, the maximum solution time the active-set algorithm requires to converge. The usefulness of the proposed method is illustrated on a set of QPs originating from MPC problems, by computing the exact worst-case number of iterations primal and dual active-set algorithms require to reach optimality.

Complexity analysis of interior point methods for convex quadratic programming based on a parameterized Kernel function

The kernel functions play an important role in the amelioration of the computational complexity of algorithms. In this paper, we present a primal-dual interior-point algorithm for solving convex quadratic programming based on a new parametric kernel function. The proposed kernel function is not logarithmic and not self-regular. We analysis a large and small-update versions which are based on a new kernel function. We obtain the best known iteration bound for large-update methods, which improves signicantly the so far obtained complexity results. Thisresult is the rst to reach this goal.

On a primal-dual Newton proximal method for convex quadratic programs

This paper introduces QPDO, a primal-dual method for convex quadratic programs which builds upon and weaves together the proximal point algorithm and a damped semismooth Newton method. The outer proximal regularization yields a numerically stable method, and we interpret the proximal operator as the unconstrained minimization of the primal-dual proximal augmented Lagrangian function. This allows the inner Newton scheme to exploit sparse symmetric linear solvers and multi-rank factorization updates. Moreover, the linear systems are always solvable independently from the problem data and exact linesearch can be performed. The proposed method can handle degenerate problems, provides a mechanism for infeasibility detection, and can exploit warm starting, while requiring only convexity. We present details of our open-source C implementation and report on numerical results against state-of-the-art solvers. QPDO proves to be a simple, robust, and efficient numerical method for convex quadratic programming.

Towards an efficient augmented Lagrangian method for convex quadratic programming

Interior point methods have attracted most of the attention in the recent decades for solving large scale convex quadratic programming problems. In this paper we take a different route as we present an augmented Lagrangian method for convex quadratic programming based on recent developments for nonlinear programming. In our approach, box constraints are penalized while equality constraints are kept within the subproblems. The motivation for this approach is that Newton’s method can be efficient for minimizing a piecewise quadratic function. Moreover, since augmented Lagrangian methods do not rely on proximity to the central path, some of the inherent difficulties in interior point methods can be avoided. In addition, a good starting point can be easily exploited, which can be relevant for solving subproblems arising from sequential quadratic programming, in sensitivity analysis and in branch and bound techniques. We prove well-definedness and finite convergence of the method proposed. Numerical experiments on separable strictly convex quadratic problems formulated from the Netlib collection show that our method can be competitive with interior point methods, in particular when a good initial point is available and a second-order Lagrange multiplier update is used.

Stability and Accuracy of Inexact Interior Point Methods for Convex Quadratic Programming

We consider primal–dual interior point methods where the linear system arising at each iteration is formulated in the reduced (augmented) form and solved approximately. Focusing on the iterates close to a solution, we analyze the accuracy of the so-called inexact step, i.e., the step that solves the unreduced system, when combining the effects of both different levels of accuracy in the inexact computation and different processes for retrieving the step after block elimination. Our analysis is general and includes as special cases sources of inexactness due either to roundoff and computational errors or to the iterative solution of the augmented system using typical procedures. In the roundoff case, we recover and extend some known results.

Optimality and convergence for convex ensemble learning with sparsity and diversity based on fixed point optimization

This paper discusses the classifier ensemble problem with sparsity and diversity learning, which is a central issue in machine learning. The current approach for reducing the size and increasing the accuracy of a classifier ensemble is to formulate it as a convex quadratic programming problem, which is a relaxation problem, and then solve it by using the existing methods for convex quadratic programming or by computing closed-form solutions. This paper presents a novel computational approach for solving the classifier ensemble problem with sparsity and diversity learning without any recourse to relaxation problems and their associated methods. We first show that the classifier ensemble problem can be expressed as a minimization problem for the sum of certain convex functions over the intersection of fixed point sets of quasi-nonexpansive mappings. Next, we propose fixed point optimization algorithms for solving the minimization problem and show that the algorithms converge to the solution of the minimization problem. It is shown that the proposed algorithms can directly solve the classifier ensemble problem with sparsity and diversity learning. Finally, we compare the performance of the proposed sparsity and diversity learning methods against an existing method in classification experiments using data sets from the UCI machine learning repository and the LIBSVM. The experimental results show that the proposed methods have higher classification accuracies than the existing method.

On the update of constraint preconditioners for regularized KKT systems

We address the problem of preconditioning sequences of regularized KKT systems, such as those arising in interior point methods for convex quadratic programming. In this case, constraint preconditioners (CPs) are very effective and widely used; however, when solving large-scale problems, the computational cost for their factorization may be high, and techniques for approximating them appear as a convenient alternative. Here, given a block $$LDL^T$$LDLT factorization of the CP associated with a KKT matrix of the sequence, called seed matrix, we present a technique for updating the factorization and building inexact CPs for subsequent matrices of the sequence. We have recently proposed an updating procedure that performs a low-rank correction of the Schur complement of the (1,1) block of the CP for the seed matrix. Now we focus on KKT sequences with nonzero (2,2) blocks and make a step further, by enriching the low-rank correction of the Schur complement by an additional cheap update. The latter update takes into account information not included in the former one and expressed as a diagonal modification of the low-rank correction. Theoretical results and numerical experiments show that the new strategy can be more effective than the procedure based on the low-rank modification alone.

Primal and dual active-set methods for convex quadratic programming

Computational methods are proposed for solving a convex quadratic program (QP). Active-set methods are defined for a particular primal and dual formulation of a QP with general equality constraints and simple lower bounds on the variables. In the first part of the paper, two methods are proposed, one primal and one dual. These methods generate a sequence of iterates that are feasible with respect to the equality constraints associated with the optimality conditions of the primal-dual form. The primal method maintains feasibility of the primal inequalities while driving the infeasibilities of the dual inequalities to zero. The dual method maintains feasibility of the dual inequalities while moving to satisfy the primal inequalities. In each of these methods, the search directions satisfy a KKT system of equations formed from Hessian and constraint components associated with an appropriate column basis. The composition of the basis is specified by an active-set strategy that guarantees the nonsingularity of each set of KKT equations. Each of the proposed methods is a conventional active-set method in the sense that an initial primal- or dual-feasible point is required. In the second part of the paper, it is shown how the quadratic program may be solved as a coupled pair of primal and dual quadratic programs created from the original by simultaneously shifting the simple-bound constraints and adding a penalty term to the objective function. Any conventional column basis may be made optimal for such a primal-dual pair of shifted-penalized problems. The shifts are then updated using the solution of either the primal or the dual shifted problem. An obvious application of this approach is to solve a shifted dual QP to define an initial feasible point for the primal (or vice versa). The computational performance of each of the proposed methods is evaluated on a set of convex problems.

Simulation of superconducting tapes and coils with convex quadratic programming method

Second-generation (2G) high-temperature superconducting coated conductors are playing an increasingly important role in power applications due to their large current density under high magnetic fields. In this paper, we conclude and explore the ability and possible potential of J formulation from the mathematical modeling point of view. An equivalent matrix form of J formulation has been presented and a relation between electromagnetic quantities and Karush–Kuhn–Tucker (KKT) conditions in optimization theory has been discovered. The use of the latest formulae to calculate inductance in a coil system and the primal-dual interior-point method algorithm is a trial to make the process of modeling stylized and build a bridge to commercial optimization solvers. Two different dependences of the critical current density on the magnetic field have been used in order to make a comparison with those published papers.

A polynomial arc-search interior-point algorithm for convex quadratic programming

Arc-search is developed for linear programming in [24,25]. The algorithms search for optimizers along an ellipse that is an approximation of the central path. In this paper, the arc-search method is applied to primal–dual path-following interior-point method for convex quadratic programming. A simple algorithm with iteration complexity O n log ( 1 / ϵ ) is devised. Several improvements on the simple algorithm, which improve computational efficiency, increase step length, and further reduce duality gap in every iteration, are then proposed and implemented. It is intuitively clear that the iteration with these improvements will reduce the duality gap more than the iteration of the simple algorithm without the improvements, though it is hard to show how much these improvements reduce the complexity bound. The proposed algorithm is implemented in MATLAB and tested on quadratic programming problems originating from [13]. The result is compared to the one obtained by LOQO in [22]. The proposed algorithm uses fewer iterations in all these problems and the number of total iterations is 27% fewer than the one obtained by LOQO. This preliminary result shows that the proposed algorithm is promising.

Starting-point strategies for an infeasible potential reduction method

We present two strategies for choosing a “hot” starting-point in the context of an infeasible potential reduction (PR) method for convex quadratic programming. The basic idea of both strategies is to select a preliminary point and to suitably scale it in order to obtain a starting point such that its nonnegative entries are sufficiently bounded away from zero, and the ratio between the duality gap and a suitable measure of the infeasibility is small. One of the two strategies is naturally suggested by the convergence theory of the PR method; the other has been devised to reduce the initial values of the duality gap and the infeasibility measure, with the objective of decreasing the number of PR iterations. Numerical experiments show that the second strategy generally performs better than the first, and both outperform a starting-point strategy based on the affine-scaling step.

Using an iterative linear solver in an interior-point method for generating support vector machines

This paper concerns the generation of support vector machine classifiers for solving the pattern recognition problem in machine learning. A method is proposed based on interior-point methods for convex quadratic programming. This interior-point method uses a linear preconditioned conjugate gradient method with a novel preconditioner to compute each iteration from the previous. An implementation is developed by adapting the object-oriented package OOQP to the problem structure. Numerical results are provided, and computational experience is discussed.

Numerically stable LDLT factorizations in interior point methods for convex quadratic programming

Fletcher & Powell (1974, Math. Comput., 28, 1067–1087) proposed a numerically stable method for updating the LDLT factorization of a symmetric positive-definite matrix when a symmetric low-rank term is added to it. In Goldfarb & Scheinberg (2004, Math. Program., 99, 1–34) we proposed a product-form version of the method of Fletcher and Powell for use in interior point methods for linear programming and studied its numerical stability. In this paper we extend these results to convex quadratic programming where the Hessian matrix of the objective function is, or can be approximated by, a diagonal matrix plus a matrix of low rank. Our new results are based on showing that the elements of the unit lower triangular matrix in the LDLT factorizations that arise in this context are uniformly bounded as the duality gap is driven to zero. Practicable versions of our approach are described for structured quadratic programs that arise in support vector machines and portfolio optimization.

On the iterative solution of KKT systems in potential reduction software for large-scale quadratic problems

Iterative solvers appear to be very promising in the development of efficient software, based on Interior Point methods, for large-scale nonlinear optimization problems. In this paper we focus on the use of preconditioned iterative techniques to solve the KKT system arising at each iteration of a Potential Reduction method for convex Quadratic Programming. We consider the augmented system approach and analyze the behaviour of the Constraint Preconditioner with the Conjugate Gradient algorithm. Comparisons with a direct solution of the augmented system and with MOSEK show the effectiveness of the iterative approach on large-scale sparse problems.

A new primal-dual path-following method for convex quadratic programming

In this paper, we describe a new primal-dual path-following method to solve a convex quadratic program (QP). The derived algorithm is based on new techniques for finding a new class of search directions similar to the ones developed in a recent paper by Darvay for linear programs. We prove that the short-update algorithm finds an e-solution of (QP) in a polynomial time.

On Weighted Linear Least-Squares Problems Related to Interior Methods for Convex Quadratic Programming

It is known that the norm of the solution to a weighted linear least-squares problem is uniformly bounded for the set of diagonally dominant symmetric positive definite weight matrices. This result is extended to weight matrices that are nonnegative linear combinations of symmetric positive semidefinite matrices. Further, results are given concerning the strong connection between the boundedness of weighted projection onto a subspace and the projection onto its complementary subspace using the inverse weight matrix. In particular, explicit bounds are given for the Euclidean norm of the projections. These results are applied to the Newton equations arising in a primal-dual interior method for convex quadratic programming and boundedness is shown for the corresponding projection operator.

A Long-step barrier method for convex quadratic programming

In this paper we propose a long-step logarithmic barrier function method for convex quadratic programming with linear equality constraints. After a reduction of the barrier parameter, a series of long steps along projected Newton directions are taken until the iterate is in the vicinity of the center associated with the current value of the barrier parameter. We prove that the total number of iterations isO(źnL) orO(nL), depending on how the barrier parameter is updated.