Optimization by Direct Search in Matrix Computations

Abstract

A direct search method attempts to maximize a function $f :{\bf R}^n \to {\bf R}$ using function values only. Many questions about the stability and accuracy of algorithms in matrix computations can be expressed in terms of the maximum value of some easily computable function f. For a variety of algorithms it is shown that direct search is capable of revealing instability or poor performance, even when such failure is difficult to discover using theoretical analysis or numerical tests with random or nonrandom data. Informative numerical examples generated by direct search provide the impetus for further analysis and improvement of an algorithm. The direct search methods used are the method of alternating directions and the multi-directional search method of Dennis and Torczon. The problems examined include the reliability of matrix condition number estimators and the stability of Strassen’s fast matrix inversion method.