Newton-SOR Iteration for Solving Large-Scale Unconstrained Optimization Problems with an Arrowhead Hessian Matrices

Abstract

Solving unconstrained optimization problems using Newton method requires calculating Newton’s direction, which involves inverse Hessian matrices. Once the order of Hessian matrices is large, it may be impossible to store the inverse of the Hessian matrices using the direct method. To overcome this problem, we used a point iterative method as an inner iteration in finding Newton direction. Therefore in this paper, we proposed a combination between Newton method and successive overrelaxation (SOR) point iterative method for solving large scale unconstrained optimization problems in which the Hessian of the Newton direction is arrowhead matrices. To calculate and validate the performance of the proposed method, we used a combination of Newton method with Gauss-Seidel point iteration and Jacobi point iteration scheme as a reference method. The proposed method provides results that are more efficient compared to the reference methods in terms of execution time and a number of iteration.