An Alternative Scaling Factor In Broyden's Class Methods for Unconstrained Optimization

Embong Embong

doi:10.3844/jmssp.2010.163.167

Abstract

Problem statement: In order to calculate step size, a suitable line search method can be employed. As the step size usually not exact, the error is unavoidable, thus radically affect quasi-Newton method by as little as 0.1 percent of the step size error. Approach: A suitable scaling factor has to be introduced to overcome this inferiority. Self-scaling Variable Metric algorithms (SSVM's) are commonly used method, where a parameter is introduced, altering Broyden's single parameter class of approximations to the inverse Hesssian to a double parameter class. This study proposes an alternative scaling factor for the algorithms. Results: The alternative scaling factor had been tried on several commonly test functions and the numerical results shows that the new scaled algorithm shows significant improvement over the standard Broyden's class methods. Conclusion: The new algorithm performance is comparable to the algorithm with initial scaling on inverse Hessian approximation by step size. An improvement over unscaled BFGS is achieved, as for most of the cases, the number of iterations are reduced.

Highlights

The quasi-Newton methods are very popular and efficient methods for solving unconstrained optimization problem: min f (x) (1) x ∈ RnHk = The inverse Hessian approximation
Scaling the Broyden’s class method: Self-scaling Variable Metric (SSVM) method: Many modifications have been applied on quasi-Newton methods in attempt to improve its efficiency
An alternative scale factor: In this article, the smallest eigenvalue of inverse Hessian approximation was proposed as an alternative scaling factor of initial scaling on H as in (15)

Summary

INTRODUCTION

The quasi-Newton methods are very popular and efficient methods for solving unconstrained optimization problem: min f (x). The step size αk is a positive step length chosen by a line search so that at each iteration either:. There are a large number of quasi-Newton methods but the Broyden’s class of update is more popular. Byrd and Nocedal (1989) They cover a large class of line search strategies under suitable conditions. If the gradient of f is Lipschitz continuous, for several well known line search satisfy the Wolfe conditions:. Byrd and Nocedal (1989) prove that if the ratio between successive trial values of α is bounded away from zero, the new iteration produced by a backtracking line search satisfies (4) and (5). If φ1∈[0,1-σ] for σ∈[0,1] (1.8) is called the restricted Broyden’s class method (Byrd et al, 1987)

MATERIALS AND METHODS

Findings

DISCUSSION

CONCLUSION