Traditional Ranking Research Articles

A Quantitative Experimental Paradigm to Optimize Construction of Rank Order Lists in the National Resident Matching Program

As part of the National Resident Matching Program, programs must submit a rank order list of desired applicants. Despite the importance of this process and the numerous manifest limitations with traditional approaches, minimal research has been conducted to examine the accuracy of different ranking strategies. The authors developed the Moore Optimized Ordinal Rank Estimator (MOORE), a novel algorithm for ranking applicants that is based on college sports ranking systems. Because it is not possible to study the Match in vivo, the authors then designed the Recruitment Outcomes Simulation System (ROSS). This program was used to simulate a series of interview seasons and to compare MOORE and traditional approaches under different conditions. The accuracy of traditional ranking and the MOORE approach are equally and adversely affected with higher levels of intrarater variability. However, compared with traditional ranking methods, MOORE produces a more accurate rank order list as interrater variability increases. The present data demonstrate three key findings. First, they provide proof of concept that it is possible to scientifically test the accuracy of different rank methods used in the Match. Second, they show that small amounts of variability can have a significant adverse impact on the accuracy of rank order lists. Finally, they demonstrate that an ordinal approach may lead to a more accurate rank order list in the presence of interviewer bias. The ROSS-MOORE approach offers programs a novel way to optimize the recruitment process and, potentially, to construct a more accurate rank order list.

New page ranking algorithm based on website force

The traditional sorting algorithm merely works on interlinks among Web pages while putting aside the mutually reinforcing relationship between sites and pages and user evaluation of the Web pages.A new page ranking algorithm was developed based on update time,Web-based authority and feedback from users.The algorithm reckoned on a website node for each site authoritative value.While distributing the authority value,the site location of the webpage and the feedback from users were taken into account.Meanwhile,the mutual-influencing relationship between website and webpages was used to get mutul-feeback.The experimental results show that,compared with the traditional PageRank,HITS and other ranking algorithms,the algorithm has obvious improvement in retrieval performance.

A Unified Approach to Factorial Designs with Randomization Restrictions

Abtsrcat Factorial designs are commonly used to assess the impact of factors and factor combinations in industrial and agricultural experiments. Though preferred, complete randomization of trials is often infeasible, and randomization restrictions are imposed. In this paper, we discuss a finite projective geometric (PG) approach to unify the existence, construction and analysis of multistage factorial designs with randomization restrictions using randomization defining contrast subspaces (or flats of a PG). Our main focus will be on the construction of such designs, and developing a word length pattern scheme that can be used for generalizing the traditional design rank- ing criteria for factorial designs. We also present a novel isomorphism check algorithm for these designs.

Servicer Performance Scoring: Using Cash Flowto Measure Servicer Success

Investors in residential mortgage-backed securities have a vested interest in the operational performance of mortgage servicers, who collect monthly mortgage payments, forward principal and interest payments, complete loss-mitigation actions, foreclose on defaulted loans, and manage the sale of real estate owned (REO) properties with real estate agents. Traditional rankings of how servicers manage their operational risk, based on analysis of their management and organization, loan administration, and default management, have not fully taken into account performance forces outside the control of the servicer. Over the course of the mortgage meltdown and the financial crisis, it became apparent that additional quantitative measures were required to evaluate servicer success based on the collection of cash and the minimization of severity upon liquidation. The authors describe a statistically based analytical method that considers all forces affecting mortgage cash flow and provide a single measure based on cash collection of how a servicer has performed against expectations and relative to its peers. <b>TOPICS:</b>Performance measurement, MBS and residential mortgage loans, CMBS and commercial mortgage loans

Data envelopment analysis (DEA) based performance evaluation system for investment companies: Case study of Tehran Stock Exchange

An investment company is a company that invests money on behalf of its shareholders who are participant in profits and losses. Therefore, appraisal of performance of such companies is most important for related investors. As we know, data envelopment analysis (DEA) is a nonparametric method that measures the relationship between produced outputs and consumed inputs and determines an efficiency score and has been widely applied in performance evaluation domain. This paper provides a DEA based system for evaluation of the performance of Iranian investment companies using financial statement analysis. Four different versions of DEA are applied for efficiency analysis and AP-DEA is applied for ranking of twelve considered companies. Ten different financial ratios are considered as inputs and outputs of each decision making unit. The results show that the BCC form of DEA is not able to distinguish between efficient and inefficient units and AP-DEA is an appropriate method to obtain full ranking of companies. Using this performance evaluation system, a new ranking of investment companies is obtained compared to the traditional ranking presented by Tehran stock exchange (TSE). Key words: Performance evaluation, investment companies, data envelopment analysis (DEA).

Is there a gap between students’ preference and university presidents’ concern over college ranking indicators?: a case study of “College Navigator in Taiwan”

In order to help students make well-informed choices, reliable college ranking systems with comparable information about higher education institutions worldwide have been welcomed by many students. Because traditional college rankings had many methodological problems, a new type of user-based ranking, called “personalized college ranking” started to develop in many nations in the late 1990s. In 2008, Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT), launched a ranking project called “College Navigator in Taiwan” which developed the first Asian student-based college search engine to provide local and international students with transparent information on Taiwan’s higher education institutions. The main objective of this paper, therefore, is to compare the rational, strategies and pathways for establishing personalized college rankings. In order to analyze the gap between students’ preferences and university presidents’concerns over ranking indicators, HEEACT’s “College Navigator in Taiwan” is adopted as a case study at the end of paper.

Proximity measures for rank join

We introduce the proximity rank join problem, where we are given a set of relations whose tuples are equipped with a score and a real-valued feature vector. Given a target feature vector, the goal is to return the K combinations of tuples with high scores that are as close as possible to the target and to each other, according to some notion of distance or dissimilarity. The setting closely resembles that of traditional rank join, but the geometry of the vector space plays a distinctive role in the computation of the overall score of a combination. Also, the input relations typically return their results either by distance from the target or by score. Because of these aspects, it turns out that traditional rank join algorithms, such as the well-known HRJN , have shortcomings in solving the proximity rank join problem, as they may read more input than needed. To overcome this weakness, we define a tight bound (used as a stopping criterion) that guarantees instance optimality, that is, an I/O cost is achieved that is always within a constant factor of optimal. The tight bound can also be used to drive an adaptive pulling strategy, deciding at each step which relation to access next. For practically relevant classes of problems, we show how to compute the tight bound efficiently. An extensive experimental study validates our results and demonstrates significant gains over existing solutions.

On the Iteratively Reweighted Rank Regression Estimator

The finite sample performance of the rank estimator of regression coefficients obtained using the iteratively reweighted least squares (IRLS) of Sievers and Abebe (2004) is evaluated. Efficiency comparisons show that the IRLS method does quite well in comparison to least squares or the traditional rank estimates in cases of moderate-tailed error distributions; however, the IRLS method does not appear to be suitable for heavy-tailed data. Moreover, our results show that the IRLS estimator will have an unbounded influence function even if we use an initial estimator with a bounded influence function.

Encouraging creativity in mathematics and science through photography

Based on the results of a survey of the science and mathematics students at our university, we observed that students do not consider mathematics and science to be creative endeavors, though the traditional artistic disciplines rank high in this regard. To address this problem in perception, the authors used photography as a means to encourage students to find the deep-rooted connections between science and mathematics and the arts. The photography project was used in a formal classroom setting as well as an outside activity, i.e. in a more informal setting. In this article, we describe the project and its outcomes.

Ranking Production Potential From Key Geological Drivers - Bakken Case Study

This article, written by Senior Technology Editor Dennis Denney, contains highlights of paper IPTC 14733, ’Ranking Production Potential Based on Key Geological Drivers - Bakken Case Study,’ by Robin S. Pilcher, Jessica McDonough Ciosek, Kelsey McArthur, John Hohman, and Peter J. Schmitz, SPE, Hess Corporation, prepared for the 2011 International Petroleum Technology Conference, Bangkok, Thailand, 15-17 November 2011. The paper has not been peer reviewed. Conference rescheduled to 7-9 February 2012. The Bakken play is a complex, low-porosity/-permeability system with variable source maturity, reservoir pressure and temperature, and fluid type. Target reservoirs are thin and variably affected by subtle structure, faults, or natural fractures. Many horizontal wells with multistage completions will be required to develop the Bakken play. A method is presented to assess the underlying production potential of the reservoirs on the basis of key geological drivers. With an extensive well-log database, petrophysical uncertainty can be addressed and stock-tank oil originally in place (STOOIP) can be calculated and mapped. Bakken Play The Williston basin oil province was discovered in 1951, and the Bakken formation was formally described in 1953. First oil was produced from the play in the same year. However, most of the early vertical wells, with the exception of those in the heavily fractured Antelope field, proved uneconomical to produce. The Bakken play is a hybrid of conventional (although tight) reservoirs and continuous resource plays that require development by use of closely spaced horizontal wells and multistage-hydraulic-fracture completions. Production Potential and Ranking Goals Regional ranking of future drilling locations is inherently challenging because of the many geological factors involved, and is complicated further by the influence of drilling, completion, and production operations. Traditional ranking schemes try to describe the relative potential of acreage and provide empirical support for the ranking by correlating the resulting rank to some measure of production performance. The production measure may be either a cumulative volume over a certain time frame (e.g., 90-day cumulative production) or a modeled estimated ultimate recovery (EUR) for the life of the well. Horizontal wells and multistage completions are a relatively new technology in the basin, with short production histories. As a result, use of actual production or actual production normalized per hydraulic-fracture stage is preferred because EUR introduces more assumptions and unknowns. A minimum period of 90 days of production was used in this study. Calibrating the regional-ranking predictions to actual production data will, inevitably, prove impossible to match past performance because the producing wells are subject to rapidly evolving technology and to variations in drilling, completion, production history, and operatorship.

Sentence-based relevance flow analysis for high accuracy retrieval

Traditional ranking models for information retrieval lack the ability to make a clear distinction between relevant and nonrelevant documents at top ranks if both have similar bag-of-words representations with regard to a user query. We aim to go beyond the bag-of-words approach to document ranking in a new perspective, by representing each document as a sequence of sentences. We begin with an assumption that relevant documents are distinguishable from nonrelevant ones by sequential patterns of relevance degrees of sentences to a query. We introduce the notion of relevance flow, which refers to a stream of sentence-query relevance within a document. We then present a framework to learn a function for ranking documents effectively based on various features extracted from their relevance flows and leverage the output to enhance existing retrieval models. We validate the effectiveness of our approach by performing a number of retrieval experiments on three standard test collections, each comprising a different type of document: news articles, medical references, and blog posts. Experimental results demonstrate that the proposed approach can improve the retrieval performance at the top ranks significantly as compared with the state-of-the-art retrieval models regardless of document type. © 2011 Wiley Periodicals, Inc.

Probabilistic Ranking Techniques in Relational Databases

Ranking queries are widely used in data exploration, data analysis and decision making scenarios. While most of the currently proposed ranking techniques focus on deterministic data, several emerging applications involve data that are imprecise or uncertain. Ranking uncertain data raises new challenges in query semantics and processing, making conventional methods inapplicable. Furthermore, the interplay between ranking and uncertainty models introduces new dimensions for ordering query results that do not exist in the traditional settings. This lecture describes new formulations and processing techniques for ranking queries on uncertain data. The formulations are based on marriage of traditional ranking semantics with possible worlds semantics under widely-adopted uncertainty models. In particular, we focus on discussing the impact of tuple-level and attribute-level uncertainty on the semantics and processing techniques of ranking queries. Under the tuple-level uncertainty model, we describe new processing techniques leveraging the capabilities of relational database systems to recognize and handle data uncertainty in score-based ranking. Under the attribute-level uncertainty model, we describe new probabilistic ranking models and a set of query evaluation algorithms, including sampling-based techniques. We also discuss supporting rank join queries on uncertain data, and we show how to extend current rank join methods to handle uncertainty in scoring attributes. Table of Contents: Introduction / Uncertainty Models / Query Semantics / Methodologies / Uncertain Rank Join / Conclusion

Removing Size as a Determinant of Quality: A Per Capita Approach to Ranking Doctoral Programs in Finance

Rankings of finance doctoral programs generally fall into two categories: a qualitative opinion survey or a quantitative analysis of research productivity. The consistency of these rankings suggests either the best programs have the most productive faculty, or that the university affiliations most often seen in publications are correlated with institutional quality, which biases the rankings towards larger programs. The authors introduce a per capita measure of research output to evaluate finance programs in a context that removes absolute size as a variable. The results indicate that smaller programs in the field are frequently overlooked in traditional rankings.

Information Collection on a Graph

We derive a knowledge gradient policy for an optimal learning problem on a graph, in which we use sequential measurements to refine Bayesian estimates of individual edge values in order to learn about the best path. This problem differs from traditional ranking and selection in that the implementation decision (the path we choose) is distinct from the measurement decision (the edge we measure). Our decision rule is easy to compute and performs competitively against other learning policies, including a Monte Carlo adaptation of the knowledge gradient policy for ranking and selection.

How Much is too Much? Using Magnitude Estimation for User Experience Research

This paper discusses the use of magnitude estimation, a method with roots in psychophysics, as a tool for evaluating user experience. Using two healthcare-related case studies for illustration, we describe the advantages and disadvantages of applying this method, relative to more traditional ranking and rating scales. We also discuss lessons learned and provide recommendations for researchers considering the use of magnitude estimation to evaluate user experience.

Proximity rank join

We introduce the proximity rank join problem, where we are given a set of relations whose tuples are equipped with a score and a real-valued feature vector. Given a target feature vector, the goal is to return the K combinations of tuples with high scores that are as close as possible to the target and to each other, according to some notion of distance. The setting closely resembles that of traditional rank join, but the geometry of the vector space plays a distinctive role in the computation of the overall score of a combination. Also, the input relations typically return their results either by distance from the target or by score. Because of these aspects, it turns out that traditional rank join algorithms, such as the well-known HRJN , have shortcomings in solving the proximity rank join problem, as they may read more input than needed. To overcome this weakness, we define a tight bound (used as a stopping criterion) that guarantees instance optimality, i.e., an I/O cost is achieved that is always within a constant factor of optimal. The tight bound can also be used to drive an adaptive pulling strategy, deciding at each step which relation to access next. For practically relevant classes of problems, we show how to compute the tight bound efficiently. An extensive experimental study validates our results and demonstrates significant gains over existing solutions.

Academic rankings: an approach to rank portuguese universities

The academic rankings are a controversial subject in higher education. However, despite all the criticism, academic rankings are here to stay and more and more different stakeholders use rankings to obtain information about the institutions' performance. The two most well-known rankings, The Times and the Shanghai Jiao Tong University rankings have different methodologies. The Times ranking is based on peer review, whereas the Shanghai ranking has only quantitative indicators and is mainly based on research outputs. In Germany, the CHE ranking uses a different methodology from the traditional rankings, allowing the users to choose criteria and weights. The Portuguese higher education institutions are performing below their European peers, and the Government believes that an academic ranking could improve both performance and competitiveness between institutions. The purpose of this paper is to analyse the advantages and problems of academic rankings and provide guidance to a new Portuguese ranking.

Pastoralists’ Conceptions of Poverty: An Analysis of Traditional and Conventional Indicators from Borana, Ethiopia

Summary We examine the traditional wealth ranking system of Borana pastoralists in Ethiopia. We investigate the proximity of conventional poverty indicators to the traditional rankings, revealing the relative lack of proximity of income and expenditure indicators. An examination of the poverty rates generated by different indicators and poverty lines reveals that conventional measures produce poverty rates that are significantly higher than traditional rankings and measures based on herd size. Poverty diagnosis among pastoralists such as the Borana, in order to generate appropriate prescriptions, should to a greater extent rely on asset-based indicators; these should ideally be complemented by qualitative approaches.

Designing a composite indicator for road safety

This study set out to develop a composite road safety indicator for benchmarking countries’ road safety performance, which would combine the main layers of the road safety pyramid which describes the complex nature of road safety activities, performance and outcomes. Four groups of basic safety indicators were considered, which refer to: policy performance (road safety programmes), final road safety outcomes (fatality rates, scope of traffic injury), intermediate outcomes (wearing rates of seat belts, crashworthiness and composition of vehicle fleet, alcohol-impaired driving), and background characteristics of countries (motorization level, population density). The analysis used the data collected for 27 European countries. Weights based on statistical models were used to combine the basic indicators into a composite one. Principal Component Analysis and Common Factor Analysis weighting were examined. The composite indicators, estimated by several methods, enabled us to rank and group the countries according to their safety performance. The analysis revealed that the countries’ ranking based on the composite indicators is not necessarily similar to the traditional ranking of countries based on fatality rates only. Furthermore, it was observed that the indicators belonging to the final outcomes and intermediate outcomes are not uniform in their behaviour. Indicators which were found to be more consistent and influential and termed a ‘core set of basic indicators’ are recommended for future uses. The general conclusion is that the design of a composite road safety indicator in which relevant information from the different components of the road safety pyramid has been captured and weighted is realistic and meaningful. Such an indicator gives a more enriched picture of road safety than a ranking based only on fatality rates, which is the common practice at present. Grouping countries in this process is promising and seems to be preferable to simply ranking countries.

Uncontrolled therapeutic observations in complementary medicine—What is the benefit?

Objective A considerable part of articles about complementary therapies focus on uncontrolled interventions administered in the context of clinical practice. According to the therapy-related hierarchy of scientific evidence 〈 http://www.cebm.net/index.aspx?o=1025 〉, however, such publications are associated with a very low evidence level. Since a perceived positive therapeutic result gained after an uncontrolled intervention may be due to at least seven factors, six of which are unspecific [2] , no conclusion can be drawn regarding a specific therapeutic effect (i.e., post hoc ergo propter hoc ). Does that mean that such reports do not have any benefit? Method A systematic search was carried out in electronic databases as well as in current textbooks on evidence-based medicine and complementary medicine. The reference lists of all relevant articles were perused. Results The identified contributions suggest that the (anecdotal) information gained in uncontrolled therapeutic interventions may indeed be valuable, particularly in (but no limited to) the following cases: • detection of possible side effect, potential new therapeutic indications, and/or problems regarding compliance [1] ; • generation of new hypotheses [3] . Discussion Vandenbroucke [3] suggested differentiating between two hierarchies of study design: one for intended effects of therapy ( cf ., 〈 http://www.cebm.net/index.aspx?o=1025 〉) and another one for discovery and studying new explanations. In the latter hierarchy, the traditional ranking of the levels of evidence is reversed: uncontrolled interventions administered in the context of clinical practice (as well as findings in patient or laboratory data and in the literature) are given the greatest importance because they may serve as catalysts for follow-up research (using the traditional hierarchy), which in turn might lead to new knowledge regarding diagnosis, therapy, and/or prevention. This double strategy may open another door towards the concept of integrative medicine.