Path-following Interior-point Method Research Articles

A Primal-Dual Interior-Point Algorithm Based on a Kernel Function with a New Barrier Term

In this paper, we propose a path-following interior-point method (IPM) for solving linear optimization (LO) problems based on a new kernel function (KF). The latter differs from other KFs in having an exponential-hyperbolic barrier term that belongs to the hyperbolic type, recently developed by I. Touil and W. Chikouche \cite{filomat2021,acta2022}. The complexity analysis for large-update primal-dual IPMs based on this KF yields an $\mathcal{O}\left( \sqrt{n}\log^2n\log \frac{n}{\epsilon }\right)$ iteration bound which improves the classical iteration bound. For small-update methods, the proposed algorithm enjoys the favorable iteration bound, namely, $\mathcal{O}\left( \sqrt{n}\log \frac{n}{\epsilon }\right)$. We back up these results with some preliminary numerical tests which show that our algorithm outperformed other algorithms with better theoretical convergence complexity. To our knowledge, this is the first feasible primal-dual interior-point algorithm based on an exponential-hyperbolic KF.

A semidefinite programming approach for the projection onto the cone of negative semidefinite symmetric tensors with applications to solid mechanics

We propose an algorithm for computing the projection of a symmetric second-order tensor onto the cone of negative semidefinite symmetric tensors with respect to the inner product defined by an assigned positive definite symmetric fourth-order tensor {mathcal {C}}. The projection problem is written as a semidefinite programming problem and an algorithm based on a primal-dual path-following interior point method coupled with a Mehrotra’s predictor-corrector approach is proposed. Implementations based on well-known symmetrization schemes and on direct methods are theoretically and numerically investigated taking into account tensors {mathcal {C}} arising in the modelling of masonry-like materials. For these special cases, indications on the preferable symmetrization scheme that take into account the conditioning of the arising linear systems are given.

A dual spectral projected gradient method for log-determinant semidefinite problems

We extend the result on the spectral projected gradient method by Birgin et al. in 2000 to a log-determinant semidefinite problem with linear constraints and propose a spectral projected gradient method for the dual problem. Our method is based on alternate projections on the intersection of two convex sets, which first projects onto the box constraints and then onto a set defined by a linear matrix inequality. By exploiting structures of the two projections, we show that the same convergence properties can be obtained for the proposed method as Birgin’s method where the exact orthogonal projection onto the intersection of two convex sets is performed. Using the convergence properties, we prove that the proposed algorithm attains the optimal value or terminates in a finite number of iterations. The efficiency of the proposed method is illustrated with the numerical results on randomly generated synthetic/deterministic data and gene expression data, in comparison with other methods including the inexact primal–dual path-following interior-point method, the Newton-CG primal proximal-point algorithm, the adaptive spectral projected gradient method, and the adaptive Nesterov’s smooth method. For the gene expression data, our results are compared with the quadratic approximation for sparse inverse covariance estimation method. We show that our method outperforms the other methods in obtaining a better objective value fast.

An Interior-Point Path-Following Method to Compute Stationary Equilibria in Stochastic Games

Subgame perfect equilibrium in stationary strategies (SSPE) is the most important solution concept used in applications of stochastic games, which makes it imperative to develop efficient numerical methods to compute an SSPE. For this purpose, this paper develops an interior-point path-following method (IPM), which remedies a number of issues with the existing method called stochastic linear tracing procedure (SLTP). The homotopy system of IPM is derived from the optimality conditions of an artificial barrier game, whose objective function is a combination of the original payoff function and a logarithmic term. Unlike SLTP, the starting stationary strategy profile can be arbitrarily chosen and IPM does not need switching between different systems of equations. The use of a perturbation term makes IPM applicable to all stochastic games, whereas SLTP only works for a generic stochastic game. A transformation of variables reduces the number of equations and variables of by roughly one half. Numerical results show that our method is more than three times as efficient as SLTP.

A wide neighborhood interior-point algorithm for linear optimization based on a specific kernel function

This paper presents an interior point algorithm for solving linear optimization problems in a wide neighborhood of the central path introduced by Ai and Zhang (SIAM J Optim 16:400–417, 2005). In each iteration, the algorithm computes the new search directions by using a specific kernel function. The convergence of the algorithm is shown and it is proved that the algorithm has the same iteration bound as the best short-step algorithms. We demonstrate the computational efficiency of the proposed algorithm by testing some Netlib problems in standard form. To best our knowledge, this is the first wide neighborhood path-following interior-point method with the same complexity as the best small neighborhood path-following interior-point methods that uses the kernel function.

Path-following interior point method: Theory and applications for the Stokes flow with a stick-slip boundary condition

A path-following interior point method is proposed for minimization of quadratic functions subject to box and equality constraints. The problems with the singular Hessian that is symmetric, positive definite on the null space of the equality constraint matrix are considered. The inner linear systems are solved by the projected conjugate gradient method preconditioned by oblique projectors. Numerical experiments include large-scale problems arising from the TFETI domain decomposition method applied for solving the Stokes flow with the stick-slip boundary condition.

New complexity analysis of a full Nesterov–Todd step interior-point method for semidefinite optimization

In this paper, we propose a new primal-dual path-following interior-point method for semidefinite optimization based on a new reformulation of the nonlinear equation of the system which defines the central path. The proposed algorithm takes only full Nesterov and Todd steps and therefore no line-searches are needed for generating the new iterations. The convergence of the algorithm is established and the complexity result coincides with the best-known iteration bound for semidefinite optimization problems.

Randomized Block Proximal Damped Newton Method for Composite Self-Concordant Minimization

In this paper we consider the composite self-concordant (CSC) minimization problem, which minimizes the sum of a self-concordant function $f$ and a (possibly nonsmooth) proper closed convex function $g$. The CSC minimization is the cornerstone of the path-following interior point methods for solving a broad class of convex optimization problems. It has also found numerous applications in machine learning. The proximal damped Newton (PDN) methods have been well studied in the literature for solving this problem that enjoy a nice iteration complexity. Given that at each iteration these methods typically require evaluating or accessing the Hessian of $f$ and also need to solve a proximal Newton subproblem, the cost per iteration can be prohibitively high when applied to large-scale problems. Inspired by the recent success of block coordinate descent methods, we propose a randomized block proximal damped Newton (RBPDN) method for solving the CSC minimization. Compared to the PDN methods, the computational cos...

A corrector–predictor path-following method for second-order cone optimization

In this paper, we present a corrector–predictor path-following interior-point method for second-order cone optimization (SOCO) based on a new proximity measure. The algorithm produces a sequence of iterates in a neighbourhood of the central path based on a new proximity measure. We show that the algorithm is well-defined and derive the complexity bound for the algorithm. We obtain the best-known result for SOCO. The numerical results show that the proposed algorithm is effective.

A Class of Path-Following Interior-Point Methods for $$P_*(\kappa )$$ P ∗ ( κ ) -Horizontal Linear Complementarity Problems

In this paper, a class of polynomial interior-point algorithms for \(P_*\left(\kappa \right)\)-horizontal linear complementarity problems based on a new parametric kernel function is presented. The new parametric kernel function is used both for determining the search directions and for measuring the distance between the given iterate and the \(\mu \)-center of the problem. We derive the complexity analysis for the algorithm, both with large and small updates.

An Interior-Point Path-Following Method for Computing a Perfect Stationary Point of a Polynomial Mapping on a Polytope

As a refinement of the concept of stationary point, the notion of perfect stationary point was formulated in the literature. Although simplicial methods could be applied to approximate such a point, these methods do not make use of the possible differentiability of the problem and can be very time-consuming even for small-scale problems. To fully exploit the differentiability of the problem, this paper develops an interior-point path-following method for computing a perfect stationary point of a polynomial mapping on a polytope. By incorporating a logarithmic barrier term into the linear objective function with an appropriate convex combination, the method closely approximates some stationary points of the mapping on a perturbed polytope, especially when the perturbation is sufficiently small. It is proved that there exists a smooth path which starts from a point in the interior of a polytope and ends at a perfect stationary point. A predictor-corrector method is adopted for numerically following the path. Numerical results further confirm the effectiveness of the method.

An interior point method in function space for the efficient solution of state constrained optimal control problems

We propose and analyze an interior point path-following method in function space for state constrained optimal control. Our emphasis is on proving convergence in function space and on constructing a practical path-following algorithm. In particular, the introduction of a pointwise damping step leads to a very efficient method, as verified by numerical experiments.

Comparison of Levenberg-Marquardt Method and Path Following Interior Point Method for the Solution of Optimal Power Flow Problem

Abstract The Levenberg-Marquardt (L-M) method and the path following interior point (PFIP) method are two most popular methods to solve the optimal power flow (OPF) problem. The performance comparison between the L-M method and the PFIP method applied to solve the OPF problem has not been made so far. The main objective of this paper provides a more comprehensive reference for researcher by analysing, comparing and evaluating the performances of the two methods. The L-M method is employed to solve the OPF problem formulated by the nonlinear complementarity problem (NCP). After the Karush-Kuhn-Tucker (KKT) conditions of OPF model are represented in term of a set of semismooth equations by introducing the NCP function, the semismooth equations are solved by the L-M method. The PFIP method, combined logarithmic barrier function with Newton-Raphson method, has been then applied to solve the OPF problem. Finally, the two methods are tested on the 5-bus, the IEEE 30-bus and the IEEE 118-bus system, respectively. Simulation results show that the two methods get the same optimal value of the objective function under the convergent criterion, while the L-M method has higher computation precision and faster convergence speed than the PFIP method.

A New Iteration Large-Update Primal-Dual Interior-Point Method for Second-Order Cone Programming

In this article, we extend the Ai-Zhang direction for solving LCP to the class of second-order cone programming. Each iterate always follows the usual wide neighborhood , not necessarily staying within it, but must stay within the wider neighborhood 𝒩(β, τ). In addition, we decompose the classical Newton direction into two separate parts according to the positive and negative parts. We show that the algorithm has iteration complexity bound, where n is the dimension of the problem and with ϵ the required precision and (X 0, S 0) the initial interior solution. To the best of our knowledge, this is the first large-neighborhood path-following interior point method (IPMs) with the same complexity as small neighborhood path-following IPMs for second-order cone programming. It is the best result in regard to the iteration complexity bound in the context of the large-update path-following method for second-order cone programming.

An $\ell _{1}$-Laplace Robust Kalman Smoother

Robustness is a major problem in Kalman filtering and smoothing that can be solved using heavy tailed distributions; e.g., l1-Laplace. This paper describes an algorithm for finding the maximum a posteriori (MAP) estimate of the Kalman smoother for a nonlinear model with Gaussian process noise and l1 -Laplace observation noise. The algorithm uses the convex composite extension of the Gauss-Newton method. This yields convex programming subproblems to which an interior point path-following method is applied. The number of arithmetic operations required by the algorithm grows linearly with the number of time points because the algorithm preserves the underlying block tridiagonal structure of the Kalman smoother problem. Excellent fits are obtained with and without outliers, even though the outliers are simulated from distributions that are not l1 -Laplace. It is also tested on actual data with a nonlinear measurement model for an underwater tracking experiment. The l1-Laplace smoother is able to construct a smoothed fit, without data removal, from data with very large outliers.

A polynomial arc-search interior-point algorithm for convex quadratic programming

Arc-search is developed for linear programming in [24,25]. The algorithms search for optimizers along an ellipse that is an approximation of the central path. In this paper, the arc-search method is applied to primal–dual path-following interior-point method for convex quadratic programming. A simple algorithm with iteration complexity O n log ( 1 / ϵ ) is devised. Several improvements on the simple algorithm, which improve computational efficiency, increase step length, and further reduce duality gap in every iteration, are then proposed and implemented. It is intuitively clear that the iteration with these improvements will reduce the duality gap more than the iteration of the simple algorithm without the improvements, though it is hard to show how much these improvements reduce the complexity bound. The proposed algorithm is implemented in MATLAB and tested on quadratic programming problems originating from [13]. The result is compared to the one obtained by LOQO in [22]. The proposed algorithm uses fewer iterations in all these problems and the number of total iterations is 27% fewer than the one obtained by LOQO. This preliminary result shows that the proposed algorithm is promising.

A New Class of Large Neighborhood Path-Following Interior Point Algorithms for Semidefinite Optimization with $O(\sqrt{n}\log\frac{\mathrm{Tr}(X^0S^0)}{\epsilon})$ Iteration Complexity

In this paper, we extend the Ai-Zhang direction to the class of semidefinite optimization problems. We define a new wide neighborhood $\mathcal{N}(\tau_1,\tau_2,\eta)$, and, as usual but with a small change, we make use of the scaled Newton equations for symmetric search directions. After defining the “positive part” and the “negative part” of a symmetric matrix, we recommend solving the Newton equation with its right-hand side replaced first by its positive part and then by its negative part, respectively. In such a way, we obtain a decomposition of the classical Newton direction and use different step lengths for each of them. Starting with a feasible point $(X^0,y^0,S^0)$ in $\mathcal{N}(\tau_1,\tau_2,\eta)$, the algorithm terminates in at most $O(\eta\sqrt{\kappa_{\infty}n}\log({Tr}(X^0S^0)/\epsilon))$ iterations, where $\kappa_{\infty}$ is a parameter associated with the scaling matrix $P$ and $\epsilon$ is the required precision. To our best knowledge, when the parameter $\eta$ is a constant, this is the first large neighborhood path-following interior point method (IPM) with the same complexity as small neighborhood path-following IPMs for semidefinite optimization that use the Nesterov-Todd direction. In the case where $\eta$ is chosen to be in the order of $\sqrt{n}$, our result coincides with the results for classical large neighborhood IPMs. Some preliminary numerical results also confirm the efficiency of the algorithm.

Some disadvantages of a Mehrotra-type primal-dual corrector interior point algorithm for linear programming

Employing a new primal-dual corrector algorithm, we investigate the impact that corrector directions may have on the convergence behaviour of predictor–corrector methods. The Primal-Dual Corrector ( pdc) algorithm that we propose computes on each iteration a corrector direction in addition to the direction of the standard primal-dual path-following interior point method [M. Kojima, S. Mizuno, A. Yoshise, A primal-dual interior point algorithm for linear programming, in: Progress in Mathematical Programming, Pacific Grove, CA, 1987, Springer, New York, 1989, pp. 29–47] for Linear Programming ( lp), in an attempt to improve performance. The new iterate is chosen by moving along the sum of these directions, from the current iterate. This technique is similar to the construction of Mehrotra's highly popular predictor–corrector algorithm [S. Mehrotra, On finding a vertex solution using interior point methods, Linear Algebra Appl. 152 (1991) 233–253]. We present examples, however, that show that the pdc algorithm may fail to converge to a solution of the lp problem, in both exact and finite arithmetic, regardless of the choice of stepsize that is employed. The cause of this bad behaviour is that the correctors exert too much influence on the direction in which the iterates move.

Mesh independence and fast local convergence of a primal-dual active-set method for mixed control-state constrained elliptic control problems

A class of mixed control-state constrained optimal control problems for elliptic partial differential equations arising, for example, in Lavrentiev-type regularized state constrained optimal control is considered. Its numerical solution is obtained via a primal-dual activeset method, which is equivalent to a class of semi-smooth Newton methods. The locally superlinear convergence of the active-set method in function space is established, and its mesh independence is proved. The paper contains a report on numerical test runs including a comparison with a short-step path-following interior-point method and a coarse-to-fine mesh sweep, that is, a nested iteration technique, for accelerating the overall solution process. Finally, convergence and regularity properties of the regularized problems with respect to a vanishing Lavrentiev parameter are considered. 2000 Mathematics subject classification: primary 65K05; secondary 90C33.

How good are interior point methods? Klee–Minty cubes tighten iteration-complexity bounds

By refining a variant of the Klee–Minty example that forces the central path to visit all the vertices of the Klee–Minty n-cube, we exhibit a nearly worst-case example for path-following interior point methods. Namely, while the theoretical iteration-complexity upper bound is $$O(2^{n}n^{\frac{5}{2}})$$, we prove that solving this n-dimensional linear optimization problem requires at least 2 n −1 iterations.