Bradley-Terry Model Research Articles

The Rayleigh Paired Comparison Model with Bayesian Analysis

A paired comparison (PC) method is more reliable to rank or compare more than two items/ objects at the same time. It is a welldeveloped method of ordering attributes or characteristics of a given set of items.The PC model is developed using Rayleigh random variables on the basis of Stern's criteria [17]. The Rayleigh PC model is analyzed in Bayesian framework using non-informative (Jeffreys and Uniform) priors. The Bayesian inference of the developed model is compared with existing the Bradley-Terry model. The preference and predictive probabilities for current and future comparisons are calculated. The posterior probabilities of hypotheses for comparing two parameters are evaluated. The Bayesian 95% credible interval are calculated.Appropriateness of the model is also examined. Graphs of marginal posterior distributions of the parameters are drawn. The Bayesian analysis is performed using real life data sets.

Clustering and Inference From Pairwise Comparisons

Given a set of pairwise comparisons, the classical ranking problem computes a single ranking that best represents the preferences of all users. In this paper, we study the problem of inferring individual preferences, arising in the context of making personalized recommendations. In particular, we assume users form clusters; users of the same cluster provide similar pairwise comparisons for the items according to the Bradley-Terry model. We propose an efficient algorithm to estimate the preference for each user: first, compute the net-win vector for each user using the comparisons; second, cluster the users based on the net-win vectors; third, estimate a single preference for each cluster separately. We show that the net-win vectors are much less noisy than the high dimensional vectors of pairwise comparisons, therefore our algorithm can cluster the users reliably. Moreover, we show that, when a cluster is only approximately correct, the maximum likelihood estimation for the Bradley-Terry model is still close to the true preference.

Evaluating Human Pairwise Preference Judgments

Human evaluation plays an important role in NLP, often in the form of preference judgments. Although there has been some use of classical non-parametric and bespoke approaches to evaluating these sorts of judgments, there is an entire body of work on this in the context of sensory discrimination testing and the human judgments that are central to it, backed by rigorous statistical theory and freely available software, that NLP can draw on. We investigate one approach, Log-Linear Bradley-Terry models, and apply it to sample NLP data.

Forest management and regional tree composition drive the host preference of saproxylic beetle communities

Summary Among saproxylic beetles, many early colonizers prefer particular host species. Ranking of preferred hosts of local saproxylic beetle communities is critical for effective dead‐wood management in forests, but is rarely done because experiments with numerous tree species are labour and cost intensive. We analysed the host preference of local saproxylic beetle communities on logs of 13 tree species in relation to management (unmanaged and managed beech stands, conifer plantations on natural beech sites) in three regions of Germany during the most critical period for host specificity, that is the first two years after harvesting. Hosts were ranked quantitatively based on the ordinal ranking of hosts by single beetle species, which in turn was based on the variation in beetle abundance. First, we employed a Bradley–Terry model in which ranking was derived from paired comparisons of host trees. Then, a model‐based recursive partitioning of the Bradley–Terry model tested whether host preference of beetle communities is affected by stand management, region and decay progress of dead wood. Our results indicated that beetle communities overall avoided logs of Fraxinus, Pseudotsuga, Larix and Tilia, and Carpinus ranked highest in preference. Carpinus also ranked highest for communities of broadleaf specialists; Picea abies ranked highest for communities of conifer specialists. Model‐based recursive partitioning revealed that ranking of local hosts in conifer stands differed from that of broadleaf stands, and that ranking of hosts in broadleaf stands differed between regions, but only in the first year for both. Synthesis and applications. Early‐colonizing saproxylic beetle communities vary locally in their choice of host trees. Therefore, forest managers should focus on the enrichment of dead wood of regional tree species and tree species of the local stand to successfully promote early‐colonizing beetle.

Sound quality preference modeling usiung a nested artificial neural network architecture

Sound quality analysis is a process that is used to study subjective perception of the functionality or preference of a product based on the sound that it makes. Paired comparison jury testing is often used to understand preference on a pairwise basis and map the results onto an interval scale using a Bradley-Terry model. This provides a merit score for each of the sounds used in the study. A statistical model, such as multiple linear regression (MLR), is then used to model the merit score using objective metrics. This statistical model, or preference equation, then allows the results of the jury testing to be generalized for use with other sounds. One drawback of this approach is the necessity to assume the form of the preference prior to applying the statistical model and the requirement of the Bradley-Terry model for a full and balanced jury study. The present paper proposes a method of using a nested artificial neural network (nested ANN) to learn the paired preferences directly. The nested ANN architecture can uncover non-linear preferences and does not require a full and balanced jury study. The nested ANN model is applied to real golf club jury study results to identify the presence of a non-linear preference for the pitch of the ball impact sound. The paper concludes by demonstrating how the nested ANN model can be simplified to ease interpretation of the preference model and provide a means of optimizing the sound quality in the context of active noise control for sound quality.

Subject-specific Bradley–Terry–Luce models with implicit variable selection

The Bradley–Terry–Luce (BTL) model for paired comparison data is able to obtain a ranking of the objects that are compared pairwise by subjects. The task of each subject is to make preference decisions in favour of one of the objects. This decision is binary when subjects prefer either the first object or the second object, but can also be ordinal when subjects make their decisions on more than two preference categories. Since subject-specific covariates, which reflect characteristics of the subject, may affect the preference decision, it is essential to incorporate subject-specific covariates into the model. However, the inclusion of subject-specific covariates yields a model that contains many parameters and thus estimation becomes challenging. To overcome this problem, we propose a procedure that is able to select and estimate only relevant variables.

Systemic Testing on Bradley-Terry Model against Nonlinear Ranking Hierarchy

We take a system point of view toward constructing any power or ranking hierarchy onto a society of human or animal players. The most common hierarchy is the linear ranking, which is habitually used in nearly all real-world problems. A stronger version of linear ranking via increasing and unvarying winning potentials, known as Bradley-Terry model, is particularly popular. Only recently non-linear ranking hierarchy is discussed and developed through recognition of dominance information contents beyond direct dyadic win-and-loss. We take this development further by rigorously arguing for the necessity of accommodating system's global pattern information contents, and then introducing a systemic testing on Bradley-Terry model. Our test statistic with an ensemble based empirical distribution favorably compares with the Deviance test equipped with a Chi-squared asymptotic approximation. Several simulated and real data sets are analyzed throughout our development.

Modelling Assumed Metric Paired Comparison Data - Application to Learning Related Emotions

In this article we suggest a beta regression model that accounts for the degree of preference in paired comparisons measured on a bounded metric paired comparison scale. The beta distribution for bounded continuous random variables assumes values in the open unit interval (0,1). However, in practice we will observe paired comparison responses that lie within a fixed or arbitrary fixed interval [-a,a] with known value of a. We therefore transform the observed responses into the interval (0,1) and assume that these transformed responses are each a realization of a random variable which follows a beta distribution. We propose a simple paired comparison regression model for beta distributed variables which allows us to model the mean of the transformed response using a linear predictor and a logit link function -- where the linear predictor is defined by the parameters of the logit-linear Bradley-Terry model. For illustration we applied the presented model to a data set obtained from a student survey of learning related emotions in mathematics.

Markov models of dependence in longitudinal paired comparisons: an application to course design

This article suggests a new approach for modelling longitudinal paired comparison data. As individual preferences may change from one time point to another, we propose extending the basic log-linear Bradley–Terry model by incorporating a Markovian structure with temporal within-comparison dependence parameters and parameters indicating the amount of change of the unknown preference parameters of the objects. We illustrate this approach by analysing a student survey relating to statistics course design with three time points.

T2FS-Based Adaptive Linguistic Assessment System for Semantic Analysis and Human Performance Evaluation on Game of Go

The game of Go is a board game with a long history that is much more complex than chess. The uncertainties of this game will be higher when the board size gets bigger. For evaluating the human performance on Go games, one human could be advanced to a higher rank based on the number of winning games via a formal human against human competition. However, a human Go player's performance could be influenced by factors such as the on-the-spot environment, as well as physical and mental situations of the day, which causes difficulty and uncertainty in certificating the human's rank. Thanks to a sample of one player's games, evaluating his/her strength by classical models such as the Bradley-Terry model is possible. However, due to inhomogeneous game conditions and limited access to archives of games, such estimates can be imprecise. In addition, classical rankings (1 Dan, 2 Dan, ...) are integers, which lead to a rather imprecise estimate of the opponent's strengths. Therefore, we propose to use a sample of games played against a computer to estimate the human's strength. In order to increase the precision, the strength of the computer is adapted from one move to the next by increasing or decreasing the computational power based on the current situation and the result of games. The human can decide some specific conditions, such as komi and board size. In this paper, we use type-2 fuzzy sets (T2FSs) with parameters optimized by a genetic algorithm for estimating the rank in a stable manner, independently of board size. More precisely, an adaptive Monte Carlo tree search (MCTS) estimates the number of simulations, corresponding to the strength of its opponents. Next, the T2FS-based adaptive linguistic assessment system infers the human performance and presents the results using the linguistic description. The experimental results show that the proposed approach is feasible for application to the adaptive linguistic assessment on a human Go player's performance.

Preference Learning for Move Prediction and Evaluation Function Approximation in Othello

This paper investigates the use of preference learning as an approach to move prediction and evaluation function ap- proximation, using the game of Othello as a test domain. Using the same sets of features, we compare our approach with least squares temporal difference learning, direct classification, and with the Bradley-Terry model, fitted using minorization-maximization (MM). The results show that the exact way in which preference learning is applied is critical to achieving high performance. Best results were obtained using a combination of board inversion and pair-wise preference learning. This combination significantly out- performed the others under test, both in terms of move prediction accuracy, and in the level of play achieved when using the learned evaluation function as a move selector during game play.

Rank Consistent Bradley-Terry Models for Repeated Tournaments

The primary objective of this paper is to model the win-loss records of matches in a repeated tournament using the ranks of the teams. The work proposes modifications of Bradley-Terry (BT) model to make the estimation consistent with the ranks of the participating teams. The BT model with restricted maximum likelihood estimation involves too many parameters and the estimates typically lack strict monotonicity. A proposed class of rank-percentile BT models based on different parametric distribution resolves both the issues. Parameter estimation, goodness-of-fit using suitably framed test statistic and its null distribution, change point analysis in a nested model framework, as well as other estimation aspects are discussed in this article. Adaptive variations of the model that allow estimates to alter are also discussed. For demonstration, National Collegiate Athletic Association (NCAA) men and women basketball tournament data are considered. The discussed models provide excellent fit to the historical data using only a few parameters. The fit validates the ranking procedure implemented by the NCAA. The models can be extended in more general tournament structures, as shown through an analysis of results from the Indian Premiere League. The work done has potential for application in the wider domain of paired comparisons.

A chess rating system for evolutionary algorithms: A new method for the comparison and ranking of evolutionary algorithms

The Null Hypothesis Significance Testing (NHST) is of utmost importance for comparing evolutionary algorithms as the performance of one algorithm over another can be scientifically proven. However, NHST is often misused, improperly applied and misinterpreted. In order to avoid the pitfalls of NHST usage this paper proposes a new method, a Chess Rating System for Evolutionary Algorithms (CRS4EAs) for the comparison and ranking of evolutionary algorithms. A computational experiment in CRS4EAs is conducted in the form of a tournament where the evolutionary algorithms are treated as chess players and a comparison between the solutions of two algorithms on the objective function is treated as one game outcome. The rating system used in CRS4EAs was inspired by the Glicko-2 rating system, based on the Bradley–Terry model for dynamic pairwise comparisons, where each algorithm is represented by rating, rating deviation, a rating/confidence interval, and rating volatility. The CRS4EAs was empirically compared to NHST within a computational experiment conducted on 16 evolutionary algorithms and a benchmark suite of 20 numerical minimisation problems. The analysis of the results shows that the CRS4EAs is comparable with NHST but may also have many additional benefits. The computations in CRS4EAs are less complicated and sensitive than those in statistical significance tests, the method is less sensitive to outliers, reliable ratings can be obtained over a small number of runs, and the conservativity/liberality of CRS4EAs is easier to control.

Why do we Attend Refresher Courses? - A Case Study of Preference Data Analysis

Abtsrcat Participants of a refresher course sponsored by the University Grants Commission (UGC) of India were asked to rank six possible reasons for attending the course. They were later also asked to rate the reasons on a five point preference scale. In this paper, our objective is to characterise and quantify the preference for these six reasons. Standard methods for the analysis of contingency tables are not appropriate for ranked preference data so we develop a variant of the y measure of association for ordinal data to examine excess preference in pairwise comparisons. The bootstrap is used to derive p-values and condence intervals for the proposed statistic. Finally, we fit a Bradley-Terry regression model to the rankings and estimate the preference for each reason in terms of ‘worth’ parameters. We repeat this analysis on the ratings data and estimate the worth parameters as functions of covariates. The model is further extended to include dependencies between the ratings. Salient results indicate that career advancement has the highest preference across all covariate classes while the preference for service to society is by far the lowest. However, the magnitude of the preferences is influenced by the number of years for which a participant has taught and whether or not he holds a doctoral degree. There are some differences in conclusions from the analysis of the ratings and rankings datasets.

A dynamic paired comparisons model: Who is the greatest tennis player?

We present a methodology for fitting time-varying paired comparisons models in which the parameters are allowed to vary deterministically, as opposed to stochastically, with time. Our dynamic paired comparisons model is based on a new closed-form for Stern’s continuum of paired comparisons models which include the Bradley–Terry model and the Thurstone–Mosteller model. The dynamic element of our model is facilitated by utilising barycentric rational interpolants BRIs. An incidental result of our work is to show that BRIs often provide a better fit to data than the obvious alternative of spline interpolation. We use our model to shed light on the debate of who is the greatest tennis player of the Open Era of men’s professional tennis since 1968. Constructing a single rankings list from our model is not trivial as there are many alternative metrics that could be used to identify which player was the best ever. We present three alternative rankings lists derived from our model. In general our rankings lists largely agree with the rankings list based on number of Grand Slam titles won, which, to some extent, validates our choice of metrics. So who is the greatest tennis player of the Open Era? Roger Federer seems like the most likely candidate, with Bjorn Borg and Jimmy Connors close behind.

A Study of Volleyball Match’s Line-up Analysis Theory Using by Bradley-Terry Model

A Study of Volleyball Match’s Line-up Analysis Theory Using by Bradley-Terry Model

Revisiting the Bradley-Terry model and its application to information retrieval

The Bradley-Terry model is widely used for analysis of pairwise preference data. We explain that the popularity of Bradley-Terry model is gained due to not only easy computation but also some nice asymptotic properties when the model is misspecified. For information retrieval required to analyze big ranking data, we propose to use a pseudo likelihood based on the Bradley-Terry model even when the true model is different from the Bradley-Terry model. We justify using the Bradley-Terry model by proving that the estimated ranking based on the proposed pseudo likelihood is consistent when the true model belongs to the class of Thurstone models, which is much bigger than the Bradley-Terry model.

Measurement of Diet Preference and its Variability in Captive Northern Fur Seals (Callorhinus ursinus)

Diet preferences of three captive northern fur seals for seven prey items (Okhotsk atka mackerel, rainbow trout, common mackerel, walleye pollock, Pacific sand lance, horse mackerel, and Japanese common squid) were measured through repeated two-choice tests. Preference indices estimated from the paired consumption data using the normal Bradley-Terry model revealed the existence of diet preferences and their individual variation. The three seals generally preferred Okhotsk atka mackerel, rainbow trout, and common mackerel, but showed individual differences for less preferred food items. The preferred items had moderate food size and high energy content per one item. Once diet preferences were determined, animals were given rainbow trout laced with lithium chloride (LiCl) to induce conditioned taste aversion (CTA). Four to five LiCl treatments were required for CTA, but could only induce temporary aversion of rainbow trout in favor of horse mackerel. Choice tests after CTA demonstrated a decrease in preference indices for rainbow trout in the three animals, but the ranking and preference indices of rainbow trout did not fall below those for horse mackerel. These results suggest that preference and aversion are relative phenomena, and the success of CTA is dependent on the reduction of preference relative to other food items.

(Psycho-)analysis of benchmark experiments: A formal framework for investigating the relationship between data sets and learning algorithms

It is common knowledge that the performance of different learning algorithms depends on certain characteristics of the data—such as dimensionality, linear separability or sample size. However, formally investigating this relationship in an objective and reproducible way is not trivial. A new formal framework for describing the relationship between data set characteristics and the performance of different learning algorithms is proposed. The framework combines the advantages of benchmark experiments with the formal description of data set characteristics by means of statistical and information-theoretic measures and with the recursive partitioning of Bradley–Terry models for comparing the algorithms’ performances. The formal aspects of each component are introduced and illustrated by means of an artificial example. Its real-world usage is demonstrated with an application example consisting of thirteen widely-used data sets and six common learning algorithms. The Appendix provides information on the implementation and the usage of the framework within the R language.

A Model Based Ranking System for Soccer Teams

In competitive sports, the design of an effective ranking system attracts increasing interests. For sports with 2-way results, the Bradley-Terry model has proven to be an effective approach. However, a well-known limitation of standard Bradley-Terry models is that they cannot be applied to sports which allow to end with a tie after regular playing time. During the past decades various methods have been explored to model soccer games in the literature. In this paper, a new extended Bradley-Terry model with various covariates is devised to construct a more advanced ranking system for international soccer teams based on their games played within the past 4 years. First a standard likelihood (ML) estimation method was employed. A weighted likelihood (WML) estimation procedure was run for further improvement. Finally an evolutional model was developed to capture the dynamic trend in performance during big events. This paper examines the extent to which our ranking differs from the existing FIFA ranking. In addition, the 2012 UEFA European Football Championships was analyzed.