On asynchronous implementations of fictitious play for distributed learning

Abstract

The classical Fictitious Play (FP) algorithm is defined within a framework of synchronous repeated play. In practice, the global synchronization assumed in classical FP can be difficult to achieve in large-scale multi-agent settings. The paper considers FP with asynchronous updates—a variant of FP in which players are permitted to be either active or idle in each stage of the repeated play process. The FP process with asynchronous updates is shown to be a generalization of classical FP. Analytical convergence results are given for the asynchronous variant of FP. Furthermore, the paper studies an asynchronous continuous-time embedding of FP. The continuous- time embedded FP process may be implemented in a real-world setting where no global clock is available. Sufficient conditions for convergence of the continuous-time embedded process are provided as a consequence of the convergence analysis for FP with asynchronous updates. Example implementations that attain the sufficient condition are presented.