Self-organizing Strategies in Game of Agent Movement

Abstract

In this paper, a stochastic game model of self-organization of strategies of stochastic game of mobile agents in the form of cyclic behavioral patterns, which consist of coordinated strategies for moving agents in a limited discrete space, is developed. The behavioral pattern of a multi-agent system is a visualized form of orderly movement of agents that arises from their initial chaotic movement during the learning of a stochastic game. The mobility of multi-step stochastic game agents is ensured by the fact that in discrete moments of time they randomly, simultaneously and independently choose their own pure strategy of moving in one of the possible directions. Current player payments are defined as loss functions that depend on the strategies of neighboring players. These functions are formed from the penalty for irregular spacing of agents in a limited discrete space and the penalty for collisions when moving agents. Random selection of players’ strategies aims to minimize their average loss functions. The generation of sequences of pure strategies is performed by a discrete distribution based on the vectors of mixed strategies. The elements of the vectors of mixed strategies are the conditional probabilities of choosing the appropriate pure displacement strategies. Mixed strategies change over time, adaptively taking into account the value of current losses. This provides an increase in the probability of choosing those strategies that lead to a decrease in the functions of average losses. The dynamics of the vectors of mixed strategies is determined by the Markov recurrent method, for the construction of which a stochastic approximation of the modified condition of complementary non- rigidity, which is valid at Nash equilibrium points, is performed, and a projection operator for expandable unit epsilon simplex is applied. The convergence of the recurrent game method is ensured by compliance with the fundamental conditions and limitations of stochastic approximation. The stochastic game begins with untrained mixed strategies that set a chaotic picture of moving agents. During the learning of the stochastic game, the vectors of mixed strategies are purposefully changed so as to ensure an orderly, conflict-free movement of agents. As a result of computer simulation of stochastic game, cyclic patterns of self-organization of mobile agents on the surface of a discrete torus and within a rectangular region on a plane are obtained. The reliability of the experimental studies was confirmed by the similarity of the obtained patterns of moving agents for different sequences of random variables. The results of the study are proposed to be used in practice for the construction of distributed systems with elements of self-organization, solving various flow and transport problems and collective decision-making in conditions of uncertainty.