Win Ratio Research Articles

Rejoinder to "on the alternative hypotheses for the win ratio".

Rejoinder to "on the alternative hypotheses for the win ratio".

Use of composite outcomes to assess risk–benefit in clinical trials

Before a novel treatment can be deemed a clinical success, an assessment of its risk-benefit profile must be made. One of the inherent challenges for this assessment comes from the multiplicity that arises from comparing treatment groups across multiple outcomes. Composite outcomes that summarize a patient's clinical status, or severity, across a prioritized list of safety and efficacy outcomes have become increasing popular. In this article, we review these approaches and illustrate through examples some of the challenges and complexities of a composite derived from prioritized outcomes, such as the win ratio. These challenges include the difficult tension between the analytical validity that comes from choosing a pre-specified outcome and an evaluation that is responsive to unexpected safety events that arise during the course of a trial. Other challenges include a sensitivity of the resulting test statistic to the underlying censoring distribution and other nuisance parameters. Approaches that resolve some of the difficulties of the analytical challenges associated with prioritized outcomes are then discussed. Ultimately, a composite outcome of net clinical benefit is another decision tool, but one to be used alongside more traditional analyses of efficacy and safety, and with the broader perspective that investigators, the data safety monitoring board, and regulators bring to an evaluation of risk-benefit.

Degree of vision impairment influence the fight outcomes in the Paralympic judo: a 10-year retrospective analysis.

In the International Blind Sports Federation (IBSA) judo Sports Classes B1, B2, and B3 compete against each other within weight- and gender-specific categories. B1 athletes are totally blind, whereas B2 and B3 are partially sighted. The aim of this study was to examine the impact of the degree of vision impairment on fight outcomes. We analyzed 1936 fights from official competitions and compare win ratio in fights between discordant Sports Classes. B1 won B2 ratio was 34.8%, B1 won B3 ratio was 36.5%, B2 won B3 ratio = 50.6%. The B1 percent of IBSA judo athletes decreased from 25.9% in 2007 to 19.4% in 2016, while the total number of participants increased over this period. These findings confirm that blind athletes have fewer chances to win a fight in IBSA Judo and become a member of National Team. Loss of vision functions affects movement coordination, balance, and emotional state, which are important for martial arts and may explain why B1 athletes are more likely to lose competitions against either B2 or B3. Creation of the evidence-based and sport-specific classification system and/or improvement of the technical rules are necessary for fair IBSA Judo competitions.

The win ratio approach did not alter study conclusions and may mitigate concerns regarding unequal composite end points in kidney transplant trials

ObjectivesThe aim of the study was to determine the impact of using the win ratio approach and investigate whether this approach alters the interpretations or conclusions of a randomized trial in kidney transplantation. Study Design and SettingWe present an application of the win ratio approach in a kidney transplant trial that assessed the clinical effectiveness of ramipril treatment vs. placebo. The primary composite outcome included the time to death, kidney transplant failure, or doubling of serum creatinine. We compare the win ratio to a conventional hazard ratio (HR) from the original trial. A win ratio with a lower 95% confidence limit greater than 1 indicates a positive treatment effect with statistical significance. ResultsFor the primary composite end point, ramipril treatment resulted in a win ratio of 1.21 (95% confidence interval [CI], 0.55-2.59) vs. a HR of 0.76 (95% CI, 0.38-1.51). With extended follow-up (mean 48 months), ramipril was associated with a win ratio of 1.02 (95% CI, 0.54-1.83) vs. a HR of 0.96 (95% CI, 0.55-1.65). ConclusionThe win ratio approach produced results similar to the original time-to-event analysis. Using this approach would not alter the conclusion of the original trial and lessens concerns associated with composite components of unequal clinical importance.

Performance Evaluation Gaps in a Real-Time Strategy Game Between Human and Artificial Intelligence Players

Since 2010, annual StarCraft artificial intelligence (AI) competitions have promoted the development of successful AI players for complex real-time strategy games. In these competitions, AI players are ranked based on their win ratio over thousands of head-to-head matches. Although simple and easily implemented, this evaluation scheme may less adequately help develop more human-competitive AI players. In this paper, we recruited 45 human StarCraft players at different expertise levels (expert/medium/novice) and asked them to play against the 18 top AI players selected from the five years of competitions (2011-2015). The results show that the human evaluations of AI players differ substantially from the current standard evaluation and ranking method. In fact, from a human standpoint, there has been little progress in the quality of StarCraft AI players over the years. It is even possible that AI-only tournaments can lead to AIs being created that are unacceptable competitors for humans. This paper is the first to systematically explore the human evaluation of AI players, the evolution of AI players, and the differences between human perception and tournament-based evaluations. The discoveries from this paper can support AI developers in game companies and AI tournament organizers to better incorporate the perspective of human users into their AI systems.

The stratified win ratio

ABSTRACTThe win ratio was first proposed in 2012 by Pocock and his colleagues to analyze a composite endpoint while considering the clinical importance order and the relative timing of its components. It has attracted considerable attention since then, in applications as well as methodology. It is not uncommon that some clinical trials require a stratified analysis. In this article, we propose a stratified win ratio statistic in a similar way as the Mantel-Haenszel stratified odds ratio, derive a general form of its variance estimator with a plug-in of existing or potentially new variance/covariance estimators of the number of wins for the two treatment groups, and assess its statistical performance using simulation studies. Our simulations show that our proposed Mantel-Haenszel-type stratified win ratio performs similarly to the Mantel-Haenszel stratified odds ratio for the simplified situation when the win ratio reduces to the odds ratio, and our proposed stratified win ratio is preferred compared to the inverse-variance weighted win ratio and unweighted win ratio particularly when the data are sparse. We also formulate a homogeneity test following Cochran’s approach that assesses whether the stratum-specific win ratios are homogeneous across strata, as this method is used frequently in meta-analyses and a better test for the win ratio homogeneity is not available yet.

Adaptation Method of the Exploration Ratio Based on the Orientation of Equilibrium in Multi-Agent Reinforcement Learning Under Non-Stationary Environments

In this paper, we propose a method to adapt the exploration ratio in multi-agent reinforcement learning. The adaptation of exploration ratio is important in multi-agent learning, as this is one of key parameters that affect the learning performance. In our observation, the adaptation method can adjust the exploration ratio suitably (but not optimally) according to the characteristics of environments. We investigated the evolutionarily adaptation of the exploration ratio in multi-agent learning. We conducted several experiments to adapt the exploration ratio in a simple evolutionary way, namely, mimicking advantageous exploration ratio (MAER), and confirmed that MAER always acquires relatively lower exploration ratio than the optimal value for the change ratio of the environments. In this paper, we propose a second evolutionary adaptation method, namely, win or update exploration ratio (WoUE). The results of the experiments showed that WoUE can acquire a more suitable exploration ratio than MAER, and the obtained ratio was near-optimal.

Biased Arbitrators and Tribunal Decisions Against Developing Countries: Stylized Facts on Investor-State Dispute Settlement

Based on UNCTAD's database on investor-state dispute settlements (ISDS), we find that many cases raised against developing countries are handled by fairly unbiased tribunals. As for ISDS outcomes, the overall ratio of investor wins over state wins is below one. Nevertheless, it is harder for developing countries, compared to high-income countries, to achieve state wins in international arbitration of investment disputes. Copyright © 2017 John Wiley & Sons, Ltd.

Win Ratio -An Intuitive and Easy-To-Interpret Composite Outcome in Medical Studies.

SummaryIn medical studies with multiple outcomes, researchers always need to make choices as to whether to use a composite outcome (after combining multiple outcomes) as their primary outcome. In this paper we review a new measurement of the treatment effect – win ratio, which can be easily used in studies with prioritized multiple outcomes. We also propose some research topics to be done in this area.

Central contracts in Test cricket: A model of best practice?

Across the last two decades, management of international cricket players has changed substantially, with the main Test-playing nations using central contracts to guide their team selection. Increased management control over player workload has been a key focus of this. This paper aims to analyse selection in relation to performance for eight Test-playing nations in 1135 matches over 30 years (1985–2015), particularly in relation to the introduction of central contracts. The results demonstrated a relationship between selection stability (i.e. changes made) and performance (overall results and win ratio). The improvement was more pronounced immediately following the introduction of a contract system, as the competitive advantage appears to be at its highest in the two years following their introduction. The data presented argue that the implementation of central contracts as a best-practice model has been a beneficial addition to nations’ performance in Test matches. Despite this, team managers, coaches and selectors should focus their work on developing an organisational culture where the elite environment has long-term stability as its focus. This is particularly pertinent, as selection uncertainty can be a de-stabilising factor, as suggested in this paper and in previous research.

A generalized analytic solution to the win ratio to analyze a composite endpoint considering the clinical importance order among components.

A composite endpoint consists of multiple endpoints combined in one outcome. It is frequently used as the primary endpoint in randomized clinical trials. There are two main disadvantages associated with the use of composite endpoints: a) in conventional analyses, all components are treated equally important; and b) in time-to-event analyses, the first event considered may not be the most important component. Recently Pocock et al. (2012) introduced the win ratio method to address these disadvantages. This method has two alternative approaches: the matched pair approach and the unmatched pair approach. In the unmatched pair approach, the confidence interval is constructed based on bootstrap resampling, and the hypothesis testing is based on the non-parametric method by Finkelstein and Schoenfeld (1999). Luo et al. (2015) developed a close-form variance estimator of the win ratio for the unmatched pair approach, based on a composite endpoint with two components and a specific algorithm determining winners, losers and ties. We extend the unmatched pair approach to provide a generalized analytical solution to both hypothesis testing and confidence interval construction for the win ratio, based on its logarithmic asymptotic distribution. This asymptotic distribution is derived via U-statistics following Wei and Johnson (1985). We perform simulations assessing the confidence intervals constructed based on our approach versus those per the bootstrap resampling and per Luo et al. We have also applied our approach to a liver transplant Phase III study. This application and the simulation studies show that the win ratio can be a better statistical measure than the odds ratio when the importance order among components matters; and the method per our approach and that by Luo et al., although derived based on large sample theory, are not limited to a large sample, but are also good for relatively small sample sizes. Different from Pocock et al. and Luo et al., our approach is a generalized analytical method, which is valid for any algorithm determining winners, losers and ties. Copyright © 2016 John Wiley & Sons, Ltd.

On the win-ratio statistic in clinical trials with multiple types of event

Pocock et al. (2012), following Finkelstein & Schoenfeld (1999), has popularized the win ratio for analysis of controlled clinical trials with multiple types of outcome event. The approach uses pairwise comparisons between patients in the treatment and control groups using a primary outcome, say the time to death, with ties broken using a secondary outcome, say the time to hospitalization. In general the observed pairwise preferences and the weight they attach to the component rankings will depend on the distribution of potential follow-up time. We present expressions for the win and loss probabilities for general bivariate survival models when follow-up of all patients is limited to a specified time horizon. In the special case of a bivariate Lehmann model we show that the win ratio does not depend on this horizon. We show how the win ratio may be estimated nonparametrically or from a parametric model. Extensions to events of three or more types are described. Application of the method of marginal estimation due to Wei et al. (1989) to this problem is described.

The Comparison Of Competitive Balance Between Super Rugby (Sanzar) And English Premiership Rugby: A Case Study From 1996-2014 Season Or Not Attractive – No People – No Money

Most sports are attractive because they are almost unpredictable. The more the competitiveness of league teams, the harder to predict the games and as a result, that league will be more attractive. Message is: more attractive leagues= bigger audience= more attractive for sponsorship= more money in sport. Competitive balance (CB) refers to the balance in sport capabilities of teams. The aim of this paper was to compare the competitive balance between Super Rugby league named SANZAR, which consist of three nations (New Zealand, Australia and South Africa) and English Premiership Rugby League in 1996-2014 seasons and compare them. The data were secondary and collected from the final tables. It was used five models in this study: the three-club and five-club concentration ratio (C3 and C5) and C3/C5 index of competitive balance (C3ICB/C5ICB) were used to analyse the data. The less the index C3ICB/C5ICB and C3/C5 are, the more competitive balance is, and conversely. Standard deviation of game results, the ratio of actual and ideal standard deviation wins, numbers of winners and a placement in the k-th place. The results showed that the Salary cap in Premiership League does not work very effectively and the competitive balance is in last few seasons still worst and worst. Super Rugby has mirror position to Premiership. Last few years are the competitive balance on the right track. Comparison result is for Super Rugby strategy with involving more teams to the league.

Research note: An evaluation of the Elo algorithm for pairwise visual assessment surveys

A growing number of online visual preference surveys are conducted which ask respondents to make pairwise comparisons between visual stimuli, raising questions about how to best analyze the resulting data. This research note directly compares three potential methods for analyzing such data, two which have been previously described (Win Ratio and Q Score) and one newly applied to this context (Elo Score). Drawing on a dataset of 103,200 votes among 362 randomly selected streets in Philadelphia, the note finds that although the three metrics are highly correlated overall, the Elo Score results in more dispersed standardized scores and a somewhat different rank order than the Win Ratio (Kendall’s tau-b 0.864). Although the Win Ratio may be adequate for some research designs, the Elo Score provides a more discriminating measure for images that elicit strong responses. The note discusses how studies using pairwise comparisons might incorporate respondent demographics, and recommends further investigation of respondent demographics for online preference surveys.

The Comparison of Competitive Balance between Super Rugby (Sanzar) and English Premiership Rugby: A Case Study from 1996-2014 Season or Not Attractive – No People – No Money

Abstract Most sports are attractive because they are almost unpredictable. The more the competitiveness of league teams, the harder to predict the games and as a result, that league will be more attractive. Message is: more attractive leagues= bigger audience= more attractive for sponsorship= more money in sport. Competitive balance (CB) refers to the balance in sport capabilities of teams. The aim of this paper was to compare the competitive balance between Super Rugby league named SANZAR, which consist of three nations (New Zealand, Australia and South Africa) and English Premiership Rugby League in 1996-2014 seasons and compare them. The data were secondary and collected from the final tables. It was used five models in this study: the three-club and five-club concentration ratio (C3 and C5) and C3/C5 index of competitive balance (C3ICB/C5ICB) were used to analyse the data. The less the index C3ICB/C5ICB and C3/C5 are, the more competitive balance is, and conversely. Standard deviation of game results, the ratio of actual and ideal standard deviation wins, numbers of winners and a placement in the k-th place. The results showed that the Salary cap in Premiership League does not work very effectively and the competitive balance is in last few seasons still worst and worst. Super Rugby has mirror position to Premiership. Last few years are the competitive balance on the right track. Comparison result is for Super Rugby strategy with involving more teams to the league.

The win ratio approach to analyzing composite outcomes: An application to the EVOLVE trial

BackgroundUnlike conventional time-to-event analysis of composite endpoints in clinical trials, the “win ratio” method allows for flexibility in prioritizing their components. Here, we compare the EVOLVE trial findings using the win ratio with those from time-to-event analysis. MethodsExposure: Randomization to cinacalcet or placebo.Outcome: The primary composite endpoint combining all-cause mortality and non-fatal myocardial infarction, hospitalization for unstable angina, heart failure, and peripheral vascular events.Analysis: In an unadjusted analysis, we paired each participant from the cinacalcet arm with every participant from the placebo arm within randomization strata. Pairs were classified as “winners” or “losers,” according to which participant died first during the shared follow-up time, or experienced the next ranked event first. We ranked non-fatal events in two ways: 1) all ranked evenly; and 2) prioritized by their effect on health-related quality of life. The win ratio equaled the total winners divided by total losers. Further analyses were conducted where the win ratio was stratified by, or adjusted for, age. ResultsThe unadjusted win ratio for the primary composite endpoint was 1.09 (95% CI 0.97 to 1.21), a statistically non-significant result which supports the primary trial result — unadjusted hazard ratio 0.93 (95% CI 0.85 to 1.02). Age-stratified analyses showed a nominally significant benefit of cinacalcet (win ratio 1.14, 95% CI 1.04 to 1.26). Ranking of non-fatal outcomes by their relative effects on quality of life did not materially alter the results. ConclusionThe win ratio method corroborated the findings of EVOLVE based on conventional time-to-event analysis.EVOLVE ClinicalTrials.gov number: NCT00345839.

A win ratio approach to comparing continuous non-normal outcomes in clinical trials.

Clinical trials are often designed to compare continuous non-normal outcomes. The conventional statistical method for such a comparison is a non-parametric Mann-Whitney test, which provides a P-value for testing the hypothesis that the distributions of both treatment groups are identical, but does not provide a simple and straightforward estimate of treatment effect. For that, Hodges and Lehmann proposed estimating the shift parameter between two populations and its confidence interval (CI). However, such a shift parameter does not have a straightforward interpretation, and its CI contains zero in some cases when Mann-Whitney test produces a significant result. To overcome the aforementioned problems, we introduce the use of the win ratio for analysing such data. Patients in the new and control treatment are formed into all possible pairs. For each pair, the new treatment patient is labelled a 'winner' or a 'loser' if it is known who had the more favourable outcome. The win ratio is the total number of winners divided by the total numbers of losers. A 95% CI for the win ratio can be obtained using the bootstrap method. Statistical properties of the win ratio statistic are investigated using two real trial data sets and six simulation studies. Results show that the win ratio method has about the same power as the Mann-Whitney method. We recommend the use of the win ratio method for estimating the treatment effect (and CI) and the Mann-Whitney method for calculating the P-value for comparing continuous non-Normal outcomes when the amount of tied pairs is small. Copyright © 2016 John Wiley & Sons, Ltd.

Debunking the curse of the rainbow jersey: historical cohort study

Objective To understand the underlying mechanism of the “curse of the rainbow jersey,” the lack of wins that purportedly affects the current cycling world champion. Design Historical cohort study. Setting...

Large sample inference for a win ratio analysis of a composite outcome based on prioritized components

Composite outcomes are common in clinical trials, especially for multiple time-to-event outcomes (endpoints). The standard approach that uses the time to the first outcome event has important limitations. Several alternative approaches have been proposed to compare treatment versus control, including the proportion in favor of treatment and the win ratio. Herein, we construct tests of significance and confidence intervals in the context of composite outcomes based on prioritized components using the large sample distribution of certain multivariate multi-sample U-statistics. This non-parametric approach provides a general inference for both the proportion in favor of treatment and the win ratio, and can be extended to stratified analyses and the comparison of more than two groups. The proposed methods are illustrated with time-to-event outcomes data from a clinical trial.

Pushing through evolution? Incidence and fight records of left-oriented fighters in professional boxing history

The fighting hypothesis proposes that left-oriented athletes enjoy a negative frequency-dependent advantage in combat sports such as boxing. Supporting evidence, however, is restricted to cross-sectional frequency data from small samples. Here, we examined the incidence and fight records of 2,403 left- and right-oriented fighters who were listed in the annual ratings of professional boxing from 1924 to 2012. Unexpectedly, left-oriented boxers were overrepresented in no more than 7 of the 89 years considered, their percentages varied up to 30% and increased over the entire period, and frequencies varied substantially between weight divisions. In support of the fighting hypothesis, lose–win ratios indicated larger fighting strength in left- compared to right-oriented boxers, which, however, was not reflected in different proportions of wins and losses by knockout. Our findings are partly consistent with an assumed left-oriented fighters' advantage in combat sports. Such advantage could be explained by negative frequency-dependent selection mechanisms; however, our study also revealed potential limits of the fighting hypothesis such that alternative explanations cannot be fully excluded. We propose that interference by factors not related to performance could also limit the suitability of data from elite sporting competition for testing evolutionary models of human handedness.