2 - Reinforcement Schemes for Average Loss Function Minimization

Abstract

This chapter discusses reinforcement schemes for average loss function minimization. The standard problem of average loss function minimization is formulated as a linear programming problem. This formulation considers the problem of adaptive control as a minimization of a linear function on a simplex. The recurrent control algorithms are classified into two categories: nonprojectional and projectional algorithms. For every reinforcement scheme, the pseudogradient condition must be fulfilled to guarantee the property of learning. The chapter presents all known reinforcement schemes and their classification from the point of view of fulfilling of pseudogradient condition. Following the definitions of different types of automata behavior, it is shown that the majority of learning automata possess symptotically an optimal behavior only in a special class of environments. The analysis of the behavior of learning automata is carried out using the martingale theory. The reinforcement scheme is the heart of the learning automaton. It is the mechanism used to adapt the probability distribution. The reinforcement schemes can be classified on the basis of the properties that they induce in the learning automaton or on the basis of their own characteristics.