Bayesian inference and simulation approaches improve the assessment of Elo‐ratings in the analysis of social behaviour

Abstract

Abstract The construction of rank hierarchies based on agonistic interactions between two individuals (“dyads”) is an important component in the characterization of the social structure of groups. To this end, winner‐loser matrices are typically created, which collapse the outcome of dyadic interactions over time, resulting in the loss of all information contained in the temporal domain. Methods that track changes in the outcome of dyadic interactions (such as “Elo‐ratings”) are receiving increasing interest. Critically, individual ratings are not just based on the succession of wins and losses, but depend on the values of start ratings and a shift coefficient. Recent studies improved existing methods by introducing a point estimation of these auxiliary parameters on the basis of a maximum likelihood (ML) approach. For a sound assessment of the rank hierarchies generated this way, we argue that measures of uncertainty of the estimates, as well as a quantification of the robustness of the methods, are also needed. We introduce a Bayesian inference (BI) approach using “partial pooling”, which rests on the assumption that all start ratings are samples from the same distribution. We compare the outcome of the ML approach to that of the BI approach using real‐world data. In addition, we simulate different scenarios to explore in which way the Elo‐rating responds to social events (such as rank changes), and low numbers of observations. Estimates of the start ratings based on “partial pooling” are more robust than those based on ML, also in scenarios where some individuals have only few observations. Our simulations show that assumed rank differences may fall well within the “uncertain” range, and that low sampling density, unbalanced designs, and coalitionary leaps involving several individuals within the hierarchy may yield unreliable results. Our results support the view that Elo‐rating can be a powerful tool in the analysis of social behaviour, when the data meet certain criteria. Assessing the uncertainty greatly aids in the interpretation of results. We advocate running simulation approaches to test how well Elo‐ratings reflect the (simulated) true structure and how sensitive the rating is to true changes in the hierarchy.