Social interaction is a prime driver for the evolution of animal behaviour. Dyadic interaction, in particular, has been the focus of intensive research on the evolution of mutualistic, altruistic, selfish or spiteful behaviours. Meanwhile, triadic interaction has been the minimal framework for the study of animal coalition as observed in some species of primates, as well as in carnivores and cetaceans, where two or more individuals act jointly against a third party in a competitive context. Previous mathematical models of animal coalition have either failed to explain the observed diversity in the configuration of coalition or presumed fine-tuned decision-making that may be unrealistic for non-human animals. To approach these issues, the present study develops a new model that is fairly simple, but still able to account for the observed diversity in animal coalitions. Analysis of the model specifies key parameters affecting the predicted types of coalition: the nature of the benefit being contested, the cost-to-benefit ratio associated with fighting and the synergistic effect in coalition formation. Additionally, the model is used to evaluate the social selection hypothesis, which claims that coalition formation induced social selection favouring reduced aggression and lower fighting abilities during human evolution.