An Alternative Approach for Finding Newton's Direction in Solving Large-Scale Unconstrained Optimization for Problems with an Arrowhead Hessian Matrix

Khadizah Ghazali,Darmesah Gabda,Jumat Sulaiman,Yosza Dasril

doi:10.13189/ms.2020.081307

Khadizah Ghazali, Darmesah Gabda + Show 2 more

Open Access

https://doi.org/10.13189/ms.2020.081307

Copy DOI

Abstract

In this paper, we proposed an alternative way to find the Newton direction in solving large-scale unconstrained optimization problems where the Hessian of the Newton direction is an arrowhead matrix. The alternative approach is a two-point Explicit Group Gauss-Seidel (2EGGS) block iterative method. To check the validity of our proposed Newton’s direction, we combined the Newton method with 2EGGS iteration for solving unconstrained optimization problems and compared it with a combination of the Newton method with Gauss-Seidel (GS) point iteration and the Newton method with Jacobi point iteration. The numerical experiments are carried out using three different artificial test problems with its Hessian in the form of an arrowhead matrix. In conclusion, the numerical results showed that our proposed method is more superior than the reference method in term of the number of inner iterations and the execution time.

Highlights

We consider large-scale unconstrained optimization problem: min f (x) (1)x n where f : n → is twice continuously differentiable and n 1000
A new generalized nonmonotone line search for solving large-scale unconstrained optimization is proposed by Shuai et al [3] to integrate advantages of the existing line searches
We focus our attention on seeking the Newton direction through equation (7) for the large-scale problem with n by n real arrowhead matrix

Summary

Introduction

X n where f : n → is twice continuously differentiable and n 1000. Theoretically, classical Newton’s method is one of the most powerful gradient descent techniques that possess the fast quadratic rate of convergence for solving problem (1). In large-scale problems, many researchers avoid this method by using other gradient descent methods such as in [2,3,4,5] or by merely modifying this classical Newton’s method [6,7,8]. In this paper, inspired by the advantages of the Newton method, we proposed 2-point explicit group Gauss-Seidel (2EGGS) block iterative method (as inner iteration) to find the Newton direction and solve problem (1) using Newton’s method (as outer iteration). We named this method as the Newton-2EGGS method.

Approaches to Newton Scheme with an Arrowhead Hessian Matrix

The Proposed Approach for Finding the Newton Direction

Formation of 2EGGS Iteration

Numerical Experiments and Computational Results Conclusion

Conclusions