A randomized adaptive trust region line search method

Saman Babaie-Kafaki,Saeed Rezaee

doi:10.11121/ijocta.01.2020.00900

Abstract

Hybridizing the trust region, line search and simulated annealing methods, we develop a heuristic algorithm for solving unconstrained optimization problems. We make some numerical experiments on a set of CUTEr test problems to investigate efficiency of the suggested algorithm. The results show that the algorithm is practically promising.

Highlights

As the most basic nonlinear optimization problem with continuous variables, unconstrained optimization naturally arises in many disciplines such as regression, image and signal processing, physical systems, optimal control and so on
In each iteration of a TR method, a neighborhood is defined around the available approximation of the solution, called the trust region, and an approximation of the objective function is minimized within the region to achieve the new estimation
In each iteration of an line search (LS) method a search direction is defined at the available approximation of the solution and the objective function is minimized along the given direction to achieve the new estimation

Summary

Introduction

As the most basic nonlinear optimization problem with continuous variables, unconstrained optimization naturally arises in many disciplines such as regression, image and signal processing, physical systems, optimal control and so on. In each iteration of a TR method, a neighborhood is defined around the available approximation of the solution, called the trust region, and an approximation of the objective function is minimized within the region to achieve the new estimation. In each iteration of an LS method a search direction is defined at the available approximation of the solution and the objective function is minimized along the given direction to achieve the new estimation. When the TR ratio is small, the approximate model is found to be a poor predictor of the objective function behavior. In such situation, the model should be resolved in a smaller region.

A randomized trust region line search algorithm

Numerical experiments

Findings

Conclusions