Stochastic processes with optimization — A model of learning in higher systems

Abstract

A special class of stochastic processes with optimization (SPO) is considered and their long-run behaviour is investigated. At each step of the process {X h} h ≧0 (where X h is a discrete random variable) a loss function expressing the distance with respect to the moments in the previous step is minimized. The transformation leading from a certain probability distribution F k (step k) to the next probability distribution F k+1 (step k+1) is accomplished by means of an optimization operator, or simply optimator, that is, a nonlinear operator performing an optimization. The constraints involved by the optimator are typically regarded as a message conveying the information needed for the stepwise evolution of the system. In other words, the behaviour of the system is expressed by an ordered set of events (actions) to be realized with given probabilities and minimum losses, while the message is viewed as a set of constraints providing an inferior bound for that probabilities, hence, ensuring that the required actions are performed. Besides, in order to account for some relaxation phenomena taking place in higher systems each active step (active optimator) is followed by a relaxation step (recovery optimator). Under these conditions it is shown that repeated presentation of a stimulus pattern leads to a convergent process, so that the action is finally performed with minimum minimorum losses. This reveals some fundamental relations between optimization and learning in higher systems, highlighting also the key role of the relaxation processes. Further the behaviour of the system is described in terms of a special class of Non-Markovian processes termed stochastic processes with optimization and relaxation (SPORs). It is shown that two basic subclasses of SPORs exist, namely the monoergodic and the biergodic SPORs. Sufficient conditions for both monoergodicity and biergodicity are given. Finally, a particular feature of the optimators, the so-called nonredundancy is shown to be relevant with respect to the influence of the past on the current evolution of the system.