Bayesian heuristic approach to global optimization and examples

Abstract

The traditional numerical analysis considers optimization algorithms which guarantee some accuracy for all functions to be optimized. This includes the exact algorithms. Limiting the maximal error requires a computational effort that in many cases increases exponentially with the size of the problem (Horst and Pardalos, 1995, Handbook of Global Optimization, Kluwer). That limits practical applications of the worst case analysis. An alternative is the average case analysis where the average error is made as small as possible (Calvin and Glynn, 1997, J. Appl. Prob., 32: 157). The average is taken over a set of functions to be optimized. The average case analysis is called the Bayesian Approach (BA) (Diaconis, 1988, Statistical Decision Theory and Related Topics, Springer; Mockus and Mockus, 1987, Theory of Optimal Decisions, Nauk, Lithuania). Application of BA to optimization of heuristics is called the Bayesian Heuristic Approach (BHA) (Mockus, 2000, A Set of Examples of Global and Discrete Optimization, Kluwer). In this paper a short presentation of the basic ideas of BHA (described in detail in Mockus (1989), Bayesian Approach to Global Optimization, Kluwer and Mockus (2000), A Set of Examples of Global and Discrete Optimization, Kluwer) is given using the knapsack problem as an example. The application potential is illustrated by the school scheduling example. In addition the new heuristic algorithm for solving a bimatrix game problem is investigated. The results ae applied while solving real life optimization problems and also as examples for distance graduate level studies of the theory of games and markets in the Internet environment.