Learning To Rank Research Articles

Ranking Learning Objects through Integration of Different Quality Indicators

The solutions used to-date for recommending learning objects have proved unsatisfactory. In an attempt to improve the situation, this document highlights the insufficiencies of the existing approaches, and identifies quality indicators that might be used to provide information on which materials to recommend to users. Next, a synthesized quality indicator that can facilitate the ranking of learning objects, according to their overall quality, is proposed. In this way, explicit evaluations carried out by users or experts will be used, along with the usage data; thus, completing the information on which the recommendation is based. Taking a set of learning objects from the Merlot repository, we analyzed the relationships that exist between the different quality indicators to form an overall quality indicator that can be calculated automatically, guaranteeing that all resources will be rated.

A Computer Assisted Method for Nuclear Cataract Grading From Slit-Lamp Images Using Ranking

In clinical diagnosis, a grade indicating the severity of nuclear cataract is often manually assigned by a trained ophthalmologist to a patient after comparing the lens' opacity severity in his/her slit-lamp images with a set of standard photos. This grading scheme is often subjective and time-consuming. In this paper, a novel computer-aided diagnosis method via ranking is proposed to facilitate nuclear cataract grading following conventional clinical decision-making process. The grade of nuclear cataract in a slit-lamp image is predicted using its neighboring labeled images in a ranked image list, which is achieved using a learned ranking function. This ranking function is learned via direct optimization on a newly proposed approximation to a ranking evaluation measure. Our proposed method has been evaluated by a large dataset composed of 1000 different cases, which are collected from an ongoing clinical population-based study. Both experimental results and comparison with several existing methods demonstrate the benefit of grading via ranking by our proposed method.

Ranking Metrics and Search Guidance for Learning Object Repository

In line with the popularity of the Internet and the development of search engine, users request information through web-based services. Although general-purpose searching such as one provided by Google is powerful, searching mechanism for specific purposes could rely on metadata. In distance learning (or e-learning), SCORM provides an efficient metadata definition for learning objects to be searched and shared. To facilitate searching in a federated repository, CORDRA provides a common architecture for discovering and sharing Learning Objects. We followed SCORM and CORDRA specifications to develop a registry system, called the MINE Registry, for storing and sharing 20,738 Learning Objects created in the past five years. As a contribution, we propose the concept of “Reusability Tree” to represent the relationships among relevant Learning Objects and enhance CORDRA. We further collect relevant information, while users are utilizing Learning Objects, such as citations and time period persisted. The feedbacks from the user community are also considered as critical elements for evaluating significance degree of Learning Objects. Through theses factors, we propose a mechanism to weight and rank Learning Objects in the MINE Registry, in addition to other external learning objects repositories. As a practical contribution, we provide a tool called “Search Guider” to assist users in finding relevant information in Learning Objects based on individual requirements.

Cost-Sensitive Rank Learning From Positive and Unlabeled Data for Visual Saliency Estimation

This paper presents a cost-sensitive rank learning approach for visual saliency estimation. This approach avoids the explicit selection of positive and negative samples, which is often used by existing learning-based visual saliency estimation approaches. Instead, both the positive and unlabeled data are directly integrated into a rank learning framework in a cost-sensitive manner. Compared with existing approaches, the rank learning framework can take the influences of both the local visual attributes and the pair-wise contexts into account simultaneously. Experimental results show that our algorithm outperforms several state-of-the-art approaches remarkably in visual saliency estimation.

Enabling multi-level relevance feedback on PubMed by integrating rank learning into DBMS

BackgroundFinding relevant articles from PubMed is challenging because it is hard to express the user's specific intention in the given query interface, and a keyword query typically retrieves a large number of results. Researchers have applied machine learning techniques to find relevant articles by ranking the articles according to the learned relevance function. However, the process of learning and ranking is usually done offline without integrated with the keyword queries, and the users have to provide a large amount of training documents to get a reasonable learning accuracy. This paper proposes a novel multi-level relevance feedback system for PubMed, called RefMed, which supports both ad-hoc keyword queries and a multi-level relevance feedback in real time on PubMed.ResultsRefMed supports a multi-level relevance feedback by using the RankSVM as the learning method, and thus it achieves higher accuracy with less feedback. RefMed "tightly" integrates the RankSVM into RDBMS to support both keyword queries and the multi-level relevance feedback in real time; the tight coupling of the RankSVM and DBMS substantially improves the processing time. An efficient parameter selection method for the RankSVM is also proposed, which tunes the RankSVM parameter without performing validation. Thereby, RefMed achieves a high learning accuracy in real time without performing a validation process. RefMed is accessible at http://dm.postech.ac.kr/refmed.ConclusionsRefMed is the first multi-level relevance feedback system for PubMed, which achieves a high accuracy with less feedback. It effectively learns an accurate relevance function from the user’s feedback and efficiently processes the function to return relevant articles in real time.

A ranking-based scoring function for peptide-spectrum matches.

The analysis of the large volume of tandem mass spectrometry (MS/MS) proteomics data that is generated these days relies on automated algorithms that identify peptides from their mass spectra. An essential component of these algorithms is the scoring function used to evaluate the quality of peptide-spectrum matches (PSMs). In this paper, we present new approach to scoring of PSMs. We argue that since this problem is at its core a ranking task (especially in the case of de novo sequencing), it can be solved effectively using machine learning ranking algorithms. We developed a new discriminative boosting-based approach to scoring. Our scoring models draw upon a large set of diverse feature functions that measure different qualities of PSMs. Our method improves the performance of our de novo sequencing algorithm beyond the current state-of-the-art, and also greatly enhances the performance of database search programs. Furthermore, by increasing the efficiency of tag filtration and improving the sensitivity of PSM scoring, we make it practical to perform large-scale MS/MS analysis, such as proteogenomic search of a six-frame translation of the human genome (in which we achieve a reduction of the running time by a factor of 15 and a 60% increase in the number of identified peptides, compared to the InsPecT database search tool). Our scoring function is incorporated into PepNovo+ which is available for download or can be run online at http://bix.ucsd.edu.

Advancing the relationship between business school ranking and student learning

This commentary advances a positive relationship between a business school's ranking in the popular press and student learning by advocating market‐oriented measures of student learning. A framework for student learning is based on the Assurance of Learning mandated by the Association to Advance Collegiate Schools of Business International, requiring business schools to measure and report demonstrations of student learning as a prerequisite for successful graduates. Examples of student demonstrations of performance and their associated market‐oriented learning goals are offered to help define and position business schools for advantage in published rankings.

A Fast Algorithm for Learning a Ranking Function from Large-Scale Data Sets

We consider the problem of learning the ranking function that maximizes a generalization of the Wilcoxon-Mann-Whitney statistic on the training data. Relying on an $\epsilon$-accurate approximation for the error-function, we reduce the computational complexity of each iteration of a conjugate gradient algorithm for learning ranking functions from O(m2) to O(m2), where m is the number of training samples. Experiments on public benchmarks for ordinal regression and collaborative filtering indicate that the proposed algorithm is as accurate as the best available methods in terms of ranking accuracy, when the algorithms are trained on the same data. However, since it is several orders of magnitude faster than the current state-of-the-art approaches, it is able to leverage much larger training datasets.

Improving Semantic Concept Detection Through Optimizing Ranking Function

In this paper, a kernel-based learning algorithm, kernel rank, is presented for improving the performance of semantic concept detection. By designing a classifier optimizing the receiver operating characteristic (ROC) curve using kernel rank, we provide a generic framework to optimize any differentiable ranking function using effective smoothing functions. kernel rank directly maximizes a 1-D quality measure of ROC, i.e., AUC (area under the ROC). It exploits the kernel density estimation to model the ranking score distributions and approximate the correct ranking count. The ranking metric is then derived and the learnable parameters are naturally embedded. To address the issues of computation and memory in learning, an efficient implementation is developed based on the gradient descent algorithm. We apply kernel rank with two types of kernel density functions to train the linear discriminant function and the Gaussian mixture model classifiers. From our experiments carried out on the development set for TREC Video Retrieval 2005, we conclude that (1) kernel rank is capable of training any differentiable classifier with various kernels; and (2) the learned ranking function performs better than traditional maximization likelihood or classification error minimization based algorithms in terms of AUC and average precision (AP).

How can I rank processes, learning and growth?

A thoughtfully designed, repurposed, creatively used employee survey can solve this and many other measurement problems. Who is in a better position to comment knowledgeably about matters such as internal business processes and learning and development than the front‐line employees who do the day‐to‐day work?

Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results

We present a meta-learning method to support selection of candidate learning algorithms. It uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand. The distance between datasets is assessed using a relatively small set of data characteristics, which was selected to represent properties that affect algorithm performance. The performance of the candidate algorithms on those datasets is used to generate a recommendation to the user in the form of a ranking. The performance is assessed using a multicriteria evaluation measure that takes not only accuracy, but also time into account. As it is not common in Machine Learning to work with rankings, we had to identify and adapt existing statistical techniques to devise an appropriate evaluation methodology. Using that methodology, we show that the meta-learning method presented leads to significantly better rankings than the baseline ranking method. The evaluation methodology is general and can be adapted to other ranking problems. Although here we have concentrated on ranking classification algorithms, the meta-learning framework presented can provide assistance in the selection of combinations of methods or more complex problem solving strategies.

The Creative Elite in America by Nathaniel Weyl (review)

BOOK REVIEWS The Creative Elite in America. By Nathaniel Weyl. Washington: Public Affairs Press, 1966. Pp. 236. $6.00. The brains of our country, and incidentally ofmost other lands, are re-explained and measured by rather new and powerful statistical techniques instead of by the subjective appraisals or citing of instances which have hitherto been our mainstay, most unreliable from their snobbish ethnocentric tendency. With his customary energy, assistance, and computer, Weyl has selected seventy-five different "rosters ofeminence," such as Who's Who, members ofvarious professions, and attainers of various indicia of success. Next, he found about 140 surnames distinctive ofnational origin, including 8 per cent ofall social security card holders, and corrected this for the frequency ofNegroes bearing each name. Then, by comparing the frequency ofa national group ofnames, in the various occupations or indicia ofsuccess, against the frequency ofthose names in the general population, he obtains very significant measures ofthe fields ofwork, degrees ofsuccess, and essential merit of each national contingent in our population. Further abundant data lie here for each reader to make his own inquiries. For instance, some ofthe seventy-five rosters of eminence represent bygone decades, most represent the present, and some the future, in the persons ofcollege students and new Phi Beta Kappans, so one can perceive the marked changes which each group has been undergoing and now faces for its future. These well-based findings are often surprising and valuable for our population policy as to immigration and the encouragement of births or birth control in various groups. In science, generally, theJews are outstanding, as they are in most other categories except the Social Register and indexes ofwealth. And in superior degree, the Germans, Dutch, Puritans, Scotch, and Scandinavians lead in science. Four rosters of medical professions show the Jews with about four times the American average, followed far after by the Chinese and Scotch. The scientific eminence of the Scandinavians, despite their fairly recent and humble immigration, may be explained by Scandinavia's having known only five centuries ofenforced clerical celibacy, which exterminated so many people choosing an intellectual life; whereas the rest of Europe which became Protestant had kept the priests and monastics celibate for about eleven centuries and as they still are in southern Europe. Many explanations are sought, in past selections ofpeople favored to get born, survive, or immigrate, explaining the present differences, sought with great learning and acumen, and not claimed as proved, but good ideas. Special attention is given to the sexual and social history of the Jews, explaining their intellectual preeminence, especially through their tradition that a learned man or his daughter is supremely desirable to marry. But the Jews are not given favored treatment: Their prudish and unsavory 319 traditions are not spared, nor is their present fivefold preeminence in lists ofsubversives. Weyl's study ofsurnames is carried a novel step further, proving within a nationality wide differences ofrank among its names—especially among theJews whose surnames are ofrecent origin. Levine, Epstein, Stern, Shapiro, and Kaplan outrank by nine to four in descending order such names as Cohen in twenty rosters of eminence. The explanation is to be found in the learned rank of the original bearers. So the English names Clark, Palmer, Draper, and Miner outrank other English names for similar reasons, and the Clarkes are a bit better than the Clarks, while the Puritan names from southern New England far outrank the common English names, as Huntington had shown. A fine book, crammed with novel and well-proved data on all the races and kiths in our country. S. Colum Gilfillan, Ph.D. I2i¡ Ocean Avenue Santa Monica, California 90401 The Living Races ofMan. By Carleton S. Coon, with Edward E. Hunt,Jr. New York: Knopf, 1965. Pp. xxxii+344+xx. $10.00. Carleton Coon is a great anthropologist because he has recognized fundamental problems ofhis subject and has given fresh and reasoned insights into them. These problems are: (1) How did the modem races ofman evolve in the Pleistocene? (2) What adaptive significance is there in the differences among human races? The first ofthese was scarcely even noted before his recent book, The Origin ofRaces fi], and the publication in 1950...

The Living Races of Man by Carleton S. Coon, with Edward E. Hunt, Jr. (review)

traditions are not spared, nor is their present fivefold preeminence in lists ofsubversives. Weyl's study ofsurnames is carried a novel step further, proving within a nationality wide differences ofrank among its names—especially among theJews whose surnames are ofrecent origin. Levine, Epstein, Stern, Shapiro, and Kaplan outrank by nine to four in descending order such names as Cohen in twenty rosters of eminence. The explanation is to be found in the learned rank of the original bearers. So the English names Clark, Palmer, Draper, and Miner outrank other English names for similar reasons, and the Clarkes are a bit better than the Clarks, while the Puritan names from southern New England far outrank the common English names, as Huntington had shown. A fine book, crammed with novel and well-proved data on all the races and kiths in our country. S. Colum Gilfillan, Ph.D. I2i¡ Ocean Avenue Santa Monica, California 90401 The Living Races ofMan. By Carleton S. Coon, with Edward E. Hunt,Jr. New York: Knopf, 1965. Pp. xxxii+344+xx. $10.00. Carleton Coon is a great anthropologist because he has recognized fundamental problems ofhis subject and has given fresh and reasoned insights into them. These problems are: (1) How did the modem races ofman evolve in the Pleistocene? (2) What adaptive significance is there in the differences among human races? The first ofthese was scarcely even noted before his recent book, The Origin ofRaces fi], and the publication in 1950 of Races by Coon, Garn, and Birdsell [2] was a milestone in the study of the second. The Living Races ofMan is devoted to a description of the major physical differences among the peoples ofthe world and to an analysis of the possible causes ofthese differences . Somewhat over one-half the book concerns the traits, environments, languages, implements, and history (up to the present) ofthe various human groups, all integrated in a geographical framework. Numerous maps and about two hundred photographs of individuals of different groups add clarity. I am a geneticist and paleontologist, not an anthropologist, and can only say that I found this part ofthe book (and the rest) well and interestingly written. The literature on human diversity is cluttered with middens of observations that are seldom made part ofan intelligible picture. A major simplification advocated by Coon is the importance of what he calls "Movius' Line," for its discoverer, stretching west of Sinkiang and south of Tibet to the Bay of Bengal. The Mongoloids and Australoids evolved east of this line for hundreds ofthousands ofyears, according to Coon, and the other races west ofit. He postulates less interchange across the line than among the groups on either side. In particular, he hypothesizes that the Negroes have a relatively large amount ofCaucasoid ancestry and that their origin as a group may be due to this admixture (original or secondary) with the forest pygmies. This and other original hypotheses will undoubtedly stir the air and perhaps a few minds for some time. It is obvious that different human populations differ in the frequencies ofmany traits. 320 Book Reviews Perspectives in Biology and Medicine · Winter 1967 But why do they differ? How important relative to each other are the antipodal forces ofadaptation andhistorical accident? Coon sets himselfsquarely in the camp ofadaptation and (with Hunt involved in this part) presents an impressive amount ofdirect and indirect evidence for an adaptive significance ofa wide variety oftrends. The aspects ofthe environment invoked are mainly epidemic diseases and climate, and the geography of different aspects ofclimate itselfoccupies fifteen pages with maps. Some ofthe characters discussed are relatively recondite, and for some no adaptive significance is suggested. It is peculiar, although not physiologically inexplicable, that both increased heat flow through the lower arms (some Mongoloids) and decreased flow (cold-adapted populations of other races) are regarded by workers in the area as specific adaptations to cold. Coon does not, I think, mention chance as even a real alternative to selection (including pleiotropic adaptation). However, human populations have often been small, and realistic models now indicate that random elements may have been moderately important in human evolution for monogenic or even oligogenic traits. The general importance of these random elements is...