Ranking Datasets Research Articles

Combinatorics and Statistical Issues Related to the Kruskal–Wallis Statistic

We explore criteria that data must meet in order for the Kruskal–Wallis test to reject the null hypothesis by computing the number of unique ranked datasets in the balanced case where each of the m alternatives has n observations. We show that the Kruskal–Wallis test tends to be conservative in rejecting the null hypothesis, and we offer a correction that improves its performance. We then compute the number of possible datasets producing unique rank-sums. The most commonly occurring data lead to an uncommonly small set of possible rank-sums. We extend prior findings about row- and column-ordered data structures.

An R package for analyzing and modeling ranking data

BackgroundIn medical informatics, psychology, market research and many other fields, researchers often need to analyze and model ranking data. However, there is no statistical software that provides tools for the comprehensive analysis of ranking data. Here, we present pmr, an R package for analyzing and modeling ranking data with a bundle of tools. The pmr package enables descriptive statistics (mean rank, pairwise frequencies, and marginal matrix), Analytic Hierarchy Process models (with Saaty’s and Koczkodaj’s inconsistencies), probability models (Luce model, distance-based model, and rank-ordered logit model), and the visualization of ranking data with multidimensional preference analysis.ResultsExamples of the use of package pmr are given using a real ranking dataset from medical informatics, in which 566 Hong Kong physicians ranked the top five incentives (1: competitive pressures; 2: increased savings; 3: government regulation; 4: improved efficiency; 5: improved quality care; 6: patient demand; 7: financial incentives) to the computerization of clinical practice. The mean rank showed that item 4 is the most preferred item and item 3 is the least preferred item, and significance difference was found between physicians’ preferences with respect to their monthly income. A multidimensional preference analysis identified two dimensions that explain 42% of the total variance. The first can be interpreted as the overall preference of the seven items (labeled as “internal/external”), and the second dimension can be interpreted as their overall variance of (labeled as “push/pull factors”). Various statistical models were fitted, and the best were found to be weighted distance-based models with Spearman’s footrule distance.ConclusionsIn this paper, we presented the R package pmr, the first package for analyzing and modeling ranking data. The package provides insight to users through descriptive statistics of ranking data. Users can also visualize ranking data by applying a thought multidimensional preference analysis. Various probability models for ranking data are also included, allowing users to choose that which is most suitable to their specific situations.

Construction Path Analysis Model from DNA Microarray Data

A fundamental problem in human health is to predict the effect of genes that cause disease; this is an important step to diagnosis and treatment. Also prediction gene functions are still a challenge for biologists in the post-genomic era. DNA microarray simultaneously monitors expression levels thousands of genes; the massive gene expression data provide us with unique opportunities to analyze the functional and regulatory relationships among genes. In this paper, new approach to estimate the relationship among genes and the effects of each gene on diseases is proposed. This approach consists of four main steps: First, extract the subset of high informative genes. Second, constructing genes network; thus, we propose to predict a gene’s functions according to its context graph, which is defined as the gene interaction network composed of the genes interacting directly and indirectly based on focal gene. Third, path analysis model is used to estimate the effect of genes on the disease and among each other. Fourth, obtains the relation among genes and the target (disease) by applying logistic regression analysis for predict new samples. This approach evaluated by lung cancer Microarray dataset. The proposed path diagram fit the subset of top ranked genes expression dataset, the Goodness of Fit Index (GFI) was > 0.832. The value of this approach is that it not only tackles the measurement problem by path analysis but also provides a visualization of the relationship among genes. The proposed approach also is useful for feature reduction, due to evaluate all genes simultaneously vs. lung state.

Coclustering—a useful tool for chemometrics

Nowadays, chemometric applications in biology can readily deal with tens of thousands of variables, for instance, in omics and environmental analysis. Other areas of chemometrics also deal with distilling relevant information in highly information‐rich data sets. Traditional tools such as the principal component analysis or hierarchical clustering are often not optimal for providing succinct and accurate information from high rank data sets. A relatively little known approach that has shown significant potential in other areas of research is coclustering, where a data matrix is simultaneously clustered in its rows and columns (objects and variables usually).Coclustering is the tool of choice when only a subset of variables is related to a specific grouping among objects. Hence, coclustering allows a select number of objects to share a particular behavior on a select number of variables.In this paper, we describe the basics of coclustering and use three different example data sets to show the advantages and shortcomings of coclustering. Copyright © 2012 John Wiley & Sons, Ltd.

Mixtures of weighted distance-based models for ranking data with applications in political studies

Analysis of ranking data is often required in various fields of study, for example politics, market research and psychology. Over the years, many statistical models for ranking data have been developed. Among them, distance-based ranking models postulate that the probability of observing a ranking of items depends on the distance between the observed ranking and a modal ranking. The closer to the modal ranking, the higher the ranking probability is. However, such a model assumes a homogeneous population, and the single dispersion parameter in the model may not be able to describe the data well. To overcome these limitations, we formulate more flexible models by considering the recently developed weighted distance-based models which can allow different weights for different ranks. The assumption of a homogeneous population can be relaxed by an extension to mixtures of weighted distance-based models. The properties of weighted distance-based models are also discussed. We carry out simulations to test the performance of our parameter estimation and model selection procedures. Finally, we apply the proposed methodology to analyze synthetic ranking datasets and a real world ranking dataset about political goals priority.

Uncovering the riffled independence structure of ranked data

Representing distributions over permutations can be a daunting task due to the fact that the number of permutations of n objects scales factorially in n. One recent way that has been used to reduce storage complexity has been to exploit probabilistic independence, but as we argue, full independence assumptions impose strong sparsity constraints on distributions and are unsuitable for modeling rankings. We identify a novel class of independence structures, called riffled independence, encompassing a more expressive family of distributions while retaining many of the properties necessary for performing efficient inference and reducing sample complexity. In riffled independence, one draws two permutations independently, then performs the riffle shuffle, common in card games, to combine the two permutations to form a single permutation. Within the context of ranking, riffled independence corresponds to ranking disjoint sets of objects independently, then interleaving those rankings. In this paper, we provide a formal introduction to riffled independence and propose an automated method for discovering sets of items which are riffle independent from a training set of rankings. We show that our clustering-like algorithms can be used to discover meaningful latent coalitions from real preference ranking datasets and to learn the structure of hierarchically decomposable models based on riffled independence.

A Signal Processing Approach to Fourier Analysis of Ranking Data: The Importance of Phase

Ranking data is a type of data obtained in some elections, in customer surveys, as well as from web search results. Such data may be considered as a type of signal defined on the group of permutations of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> objects, denoted <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sn</i> . There exists a Fourier transform for <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sn</i> obtained from group representation theory, which is well known in the mathematics literature. However, previous work has not approached the transform from a signal processing perspective: in particular, there is no discussion of what constitutes “magnitude” and “phase,” nor any analysis of what phase information might tell us beyond a well-known connection to group translation. This paper explores the properties of the phase spectrum of ranking data; in particular, a novel contribution is the formulation of the bispectrum for ranking data, which may be used for studying phase linearity. Analysis of two well-known ranking data sets shows that they are surprisingly well fit by linear phase approximations.

Boosted multi-task learning

In this paper we propose a novel algorithm for multi-task learning with boosted decision trees. We learn several different learning tasks with a joint model, explicitly addressing their commonalities through shared parameters and their differences with task-specific ones. This enables implicit data sharing and regularization. Our algorithm is derived using the relationship between l 1-regularization and boosting. We evaluate our learning method on web-search ranking data sets from several countries. Here, multi-task learning is particularly helpful as data sets from different countries vary largely in size because of the cost of editorial judgments. Further, the proposed method obtains state-of-the-art results on a publicly available multi-task dataset. Our experiments validate that learning various tasks jointly can lead to significant improvements in performance with surprising reliability.

Distance-based tree models for ranking data

Ranking data has applications in different fields of studies, like marketing, psychology and politics. Over the years, many models for ranking data have been developed. Among them, distance-based ranking models, which originate from the classical rank correlations, postulate that the probability of observing a ranking of items depends on the distance between the observed ranking and a modal ranking. The closer to the modal ranking, the higher the ranking probability is. However, such a model basically assumes a homogeneous population and does not incorporate the presence of covariates. To overcome these limitations, we combine the strength of a tree model and the existing distance-based models to build a model that can handle more complexity and improve prediction accuracy. We will introduce a recursive partitioning algorithm for building a tree model with a distance-based ranking model fitted at each leaf. We will also consider new weighted distance measures which allow different weights for different ranks in formulating more flexible distance-based tree models. Finally, we will apply the proposed methodology to analyze a ranking dataset of Inglehart's items collected in the 1999 European Values Studies.

Aggregation and decision making using ranked data

Nonparametric procedures are frequently used to rank order alternatives. Often, information from several data sets must be aggregated to derive an overall ranking. When using nonparametric procedures, Simpson-like paradoxes can occur in which the conclusion drawn from the aggregate ranked data set seems contradictory to the conclusions drawn from the individual data sets. Extending previous results found in the literature for the Kruskal–Wallis test, this paper presents a strict condition that ranked data must satisfy in order to avoid this type of inconsistency when using nonparametric pairwise procedures or Bhapkar’s V procedure to extract an overall ranking. Aggregating ranked data poses further difficulties because there exist numerous ways to combine ranked data sets. This paper illustrates these difficulties and derives an upper bound for the number of possible ways that two ranked data sets can be combined.

Self‐ and Peer‐Assessment Disparities in University Ranking Schemes

University rankings often include surveys that involve self‐assessments, but little is known about the effects of these self‐assessments on results. This paper tests for the existence of leniency effects and differential leniency effects, terms borrowed from the performance assessment literature, with a data set of ratings and rankings of Australian university disciplines by Australian academics. It finds support for the leniency effect – these academics rate and rank their own universities higher than others do. It finds less support, however, for the differential leniency effect – academics at the lower‐rated universities do not seem to overrate their universities more than do academics from the higher‐rated universities.

Symmetry of Nonparametric Statistical Tests on Three Samples

Problem statement: Many different nonparametric statistical procedures can be used to analyze ranked data. Inconsistencies among the outcomes of such procedures can occur when analyzing the same ranked data set. Understanding why these peculiarities can occur is imperative to providing an accurate analysis of the ranking data. In this context, this study addressed why inconsistent outcomes can occur and which types of data structures cause the different procedures to yield different outcomes. Approach: Appropriate properties were identified and developed to explain why different methods can define different rankings of three samples with the same data. The approach identifies certain symmetry structures that are implicitly contained within the data and analyzes how the procedures utilize these structures to produce an outcome. Results: We proved that all possible differences among the nonparametric rules are caused because different rules place different levels of emphasis on the specified symmetry configurations of data. Our findings explain and characterize why different procedures can output different results using the same data set. Conclusion: This study therefore served as crucial step in deciding which nonparametric procedure to use when analyzing ranked data. In addition, it serves as the building block to defining new techniques to analyze rankings. Because different procedures use different aspects of the data in different ways, then one may determine the choice of analysis procedure based on what parts of the data one deems important.

Machine learning for regulatory analysis and transcription factor target prediction in yeast.

High throughput technologies, including array-based chromatin immunoprecipitation, have rapidly increased our knowledge of transcriptional maps—the identity and location of regulatory binding sites within genomes. Still, the full identification of sites, even in lower eukaryotes, remains largely incomplete. In this paper we develop a supervised learning approach to site identification using support vector machines (SVMs) to combine 26 different data types. A comparison with the standard approach to site identification using position specific scoring matrices (PSSMs) for a set of 104 Saccharomyces cerevisiae regulators indicates that our SVM-based target classification is more sensitive (73 vs. 20%) when specificity and positive predictive value are the same. We have applied our SVM classifier for each transcriptional regulator to all promoters in the yeast genome to obtain thousands of new targets, which are currently being analyzed and refined to limit the risk of classifier over-fitting. For the purpose of illustration we discuss several results, including biochemical pathway predictions for Gcn4 and Rap1. For both transcription factors SVM predictions match well with the known biology of control mechanisms, and possible new roles for these factors are suggested, such as a function for Rap1 in regulating fermentative growth. We also examine the promoter melting temperature curves for the targets of YJR060W, and show that targets of this TF have potentially unique physical properties which distinguish them from other genes. The SVM output automatically provides the means to rank dataset features to identify important biological elements. We use this property to rank classifying k-mers, thereby reconstructing known binding sites for several TFs, and to rank expression experiments, determining the conditions under which Fhl1, the factor responsible for expression of ribosomal protein genes, is active. We can see that targets of Fhl1 are differentially expressed in the chosen conditions as compared to the expression of average and negative set genes. SVM-based classifiers provide a robust framework for analysis of regulatory networks. Processing of classifier outputs can provide high quality predictions and biological insight into functions of particular transcription factors. Future work on this method will focus on increasing the accuracy and quality of predictions using feature reduction and clustering strategies. Since predictions have been made on only 104 TFs in yeast, new classifiers will be built for the remaining 100 factors which have available binding data.Electronic Supplementary MaterialSupplementary material is available in the online version of this article at http://dx.doi.org/10.1007/s11693-006-9003-3 and is accessible for authorized users.

Ranking Data Sets ‐ an easy method

Ranking Data Sets ‐ an easy method