Large-scale Trust-region Subproblem Research Articles

On solving L-SR1 trust-region subproblems

In this article, we consider solvers for large-scale trust-region subproblems when the quadratic model is defined by a limited-memory symmetric rank-one (L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact representation of L-SR1 matrices. Our approach makes use of both an orthonormal basis for the eigenspace of the L-SR1 matrix and the Sherman---Morrison---Woodbury formula to compute global solutions to trust-region subproblems. To compute the optimal Lagrange multiplier for the trust-region constraint, we use Newton's method with a judicious initial guess that does not require safeguarding. A crucial property of this solver is that it is able to compute high-accuracy solutions even in the so-called hard case. Additionally, the optimal solution is determined directly by formula, not iteratively. Numerical experiments demonstrate the effectiveness of this solver.

INCOMPLETE CHOLESKY FACTORIZATION IN FIXED MEMORY WITH FLEXIBLE DROP-TOLERANCE STRATEGY

We propose an incomplete Cholesky factorization for the solution of large positive definite systems of equations and for the solution of large-scale trust region sub-problems. The factorization is based on the two- parameter (m, p) drop-tolerance strategy for insignificant elements in the incomplete factor matrix. The factorization proposed essentially reduces the negative processes of irregular distribution and accumulation of errors in factor matrix and provides the optimal rate of memory filling with essential nonzero elements. On the contrary to the known p - retain and t - drop-tolerance strategies, the (m, p) strategy allows to form the factor matrix in fixed memory.

Computational and sensitivity aspects of eigenvalue-based methods for the large-scale trust-region subproblem

The trust-region subproblem (TRS) of minimizing a quadratic function subject to a norm constraint arises in the context of trust-region methods in optimization and in the regularization of discrete forms of ill-posed problems, including non-negative regularization by means of interior-point methods. A class of efficient methods and software for solving large-scale trust-region subproblems (TRSs) is based on a parametric-eigenvalue formulation of the subproblem. The solution of a sequence of large symmetric eigenvalue problems is the main computation in these methods. In this work, we study the robustness and performance of eigenvalue-based methods for the large-scale TRS. We describe the eigenvalue problems and their features, and discuss the computational challenges they pose as well as some approaches to handle them. We present results from a numerical study of the sensitivity of solutions to the TRS to eigenproblem solutions.

Accelerating the LSTRS Algorithm

The LSTRS software for the efficient solution of the large-scale trust-region subproblem was proposed in [M. Rojas, S. A. Santos, and D. C. Sorensen, ACM Trans. Math. Software, 34 (2008), article 11]. The LSTRS method is based on recasting the problem in terms of a parameter-dependent eigenvalue problem and adjusting the parameter iteratively. The essential work at each iteration is the solution of an eigenvalue problem for the smallest eigenvalue of a bordered Hessian matrix (or two smallest eigenvalues in the potential hard case) and associated eigenvector(s). Using the nonlinear Arnoldi method to solve the eigenvalue problems makes it possible to recycle most of the information from previous iterations which can substantially accelerate LSTRS.

A practical method for solving large-scale TRS

We present a nearly-exact method for the large scale trust region subproblem (TRS) based on the properties of the minimal-memory BFGS method. Our study is concentrated in the case where the initial BFGS matrix can be any scaled identity matrix. The proposed method is a variant of the More–Sorensen method that exploits the eigenstructure of the approximate Hessian B, and incorporates both the standard and the hard case. The eigenvalues of B are expressed analytically, and consequently a direction of negative curvature can be computed immediately by performing a sequence of inner products and vector summations. Thus, the hard case is handled easily while the Cholesky factorization is completely avoided. An extensive numerical study is presented, for covering all the possible cases arising in the TRS with respect to the eigenstructure of B. Our numerical experiments confirm that the method is suitable for very large scale problems.

Solving the quadratic trust-region subproblem in a low-memory BFGS framework

We present a new matrix-free method for the large-scale trust-region subproblem, assuming that the approximate Hessian is updated by the L-BFGS formula with m=1 or 2. We determine via simple formulas the eigenvalues of these matrices and, at each iteration, we construct a positive definite matrix whose inverse can be expressed analytically, without using factorization. Consequently, a direction of negative curvature can be computed immediately by applying the inverse power method. The computation of the trial step is obtained by performing a sequence of inner products and vector summations. Furthermore, it immediately follows that the strong convergence properties of trust region methods are preserved. Numerical results are also presented.

Matrix-free algorithm for the large-scale constrained trust-region subproblem†

A new ‘matrix-free’ algorithm for the solution of linear inequality constrained, large-scale trust-region subproblems is presented. The matrix-free nature of the algorithm eliminates the need for any matrix factorizations and only requires matrix–vector products. Numerical results that demonstrate the viability of the approach are included. †Contribution of the National Institute of Standards and Technology and not subject to copyright in the UnitedStates.

Gauss Quadrature Applied to Trust Region Computations

This paper discusses the application of the Lanczos process to the solution of large-scale trust-region subproblems. Techniques pioneered by Golub based on Gauss quadrature are applied to derive inexpensively computable upper and lower bounds for quantities of interest. These bounds help determine how many steps of the Lanczos process should be carried out.

An interior-point trust-region-based method for large-scale non-negativeregularization

We present a new method for large-scale non-negative regularization based on aquadratically and non-negatively constrained quadratic problem. Such problemsarise for example in the regularization of ill posed problems in image restorationwhere the matrices involved are very ill conditioned. The method is aninterior-point iteration that requires the solution of a large-scale and possibly illconditioned parametrized trust-region subproblem at each step. The method usesrecently developed techniques for the large-scale trust-region subproblem. Wedescribe the method and present preliminary numerical results on test problemsand image restoration problems.

A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem

We present a new method for the large-scale trust-region subproblem. The method is matrix-free in the sense that only matrix-vector products are required. We recast the trust-region subproblem as a parameterized eigenvalue problem and compute an optimal value for the parameter. We then find the solution of the trust-region subproblem from the eigenvectors associated with two of the smallest eigenvalues of the parameterized eigenvalue problem corresponding to the optimal parameter. The new algorithm uses a different interpolating scheme than existing methods and introduces a unified iteration that naturally includes the so-called hard case. We show that the new iteration is well defined and convergent at a superlinear rate. We present computational results to illustrate convergence properties and robustness of the method.

Incomplete Cholesky Factorizations with Limited Memory

We propose an incomplete Cholesky factorization for the solution of large-scale trust region subproblems and positive definite systems of linear equations. This factorization depends on a parameter p that specifies the amount of additional memory (in multiples of n, the dimension of the problem) that is available; there is no need to specify a drop tolerance. Our numerical results show that the number of conjugate gradient iterations and the computing time are reduced dramatically for small values of p. We also show that in contrast with drop tolerance strategies, the new approach is more stable in terms of number of iterations and memory requirements.

Minimization of a Large-Scale Quadratic FunctionSubject to a Spherical Constraint

An important problem in linear algebra and optimization is the trust-region subproblem: minimize a quadratic function subject to an ellipsoidal or spherical constraint. This basic problem has several important large-scale applications including seismic inversion and forcing convergence in optimization methods. Existing methods to solve the trust-region subproblem require matrix factorizations, which are not feasible in the large-scale setting. This paper presents an algorithm for solving the large-scale trust-region subproblem that requires a fixed-size limited storage proportional to the order of the quadratic and that relies only on matrix-vector products. The algorithm recasts the trust-region subproblem in terms of a parameterized eigenvalue problem and adjusts the parameter with a superlinearly convergent iteration to find the optimal solution from the eigenvector of the parameterized problem. Only the smallest eigenvalue and corresponding eigenvector of the parameterized problem needs to be computed. The implicitly restarted Lanczos method is well suited to this subproblem.