Contextual Ranking Research Articles

Data Augmentation for Sample Efficient and Robust Document Ranking

Contextual ranking models have delivered impressive performance improvements over classical models in the document ranking task. However, these highly over-parameterized models tend to be data-hungry and require large amounts of data even for fine-tuning. In this article, we propose data-augmentation methods for effective and robust ranking performance. One of the key benefits of using data augmentation is in achieving sample efficiency or learning effectively when we have only a small amount of training data. We propose supervised and unsupervised data augmentation schemes by creating training data using parts of the relevant documents in the query-document pairs. We then adapt a family of contrastive losses for the document ranking task that can exploit the augmented data to learn an effective ranking model. Our extensive experiments on subsets of the MS MARCO and TREC-DL test sets show that data augmentation, along with the ranking-adapted contrastive losses, results in performance improvements under most dataset sizes. Apart from sample efficiency, we conclusively show that data augmentation results in robust models when transferred to out-of-domain benchmarks. Our performance improvements in in-domain and more prominently in out-of-domain benchmarks show that augmentation regularizes the ranking model and improves its robustness and generalization capability.

A Contextual Ranking and Selection Method for Personalized Medicine

Problem definition: Personalized medicine (PM) seeks the best treatment for each patient among a set of available treatment methods. Because a specific treatment does not work well on all patients, traditionally, the best treatment was selected based on the doctor’s personal experience and expertise, which is subject to human errors. In the meantime, stochastic models have been well developed in the literature for a lot of major diseases. This gives rise to a simulation-based solution for PM, which uses the simulation tool to evaluate the performance for pairs of treatment and patient biometric characteristics and, based on that, selects the best treatment for each patient characteristic. Methodology/results: In this research, we extend the ranking and selection (R&S) model in simulation-based decision making to solving PM. The biometric characteristics of a patient are treated as a context for R&S, and we call it contextual ranking and selection (CR&S). We consider two formulations of CR&S with small and large context spaces, respectively, and develop new techniques for solving them and identifying the rate-optimal budget allocation rules. Based on them, two selection algorithms are proposed, which can be shown to be numerically superior via a set of tests on abstract and real-world examples. Managerial implications: This research provides a systematic way of conducting simulation-based decision-making for PM. To improve the overall decision quality for the possible contexts, more simulation efforts should be devoted to contexts in which it is difficult to distinguish between the best treatment and non-best treatments, and our results quantify the optimal trade-off of the simulation efforts between the pairs of contexts and treatments. Funding: J. Du is partially supported by the National Natural Science Foundation of China [Grant 72091211]. C.-H. Chen is partially supported by the National Science Foundation under Awards FAIN212368. Supplemental Material: The e-companion is available at https://doi.org/10.1287/msom.2022.0232 .

Contextual Ranking and Selection with Gaussian Processes and OCBA

In many real world problems, we are faced with the problem of selecting the best among a finite number of alternatives, where the best alternative is determined based on context specific information. In this work, we study the contextual Ranking and Selection problem under a finite-alternative-finite-context setting, where we aim to find the best alternative for each context. We use a separate Gaussian process to model the reward for each alternative, and derive the large deviations rate function for both the expected and worst-case contextual probability of correct selection. We propose the GP-C-OCBA sampling policy, which uses the Gaussian process posterior to iteratively allocate observations to maximize the rate function. We prove its consistency and show that it achieves the optimal convergence rate under the assumption of a non-informative prior. Numerical experiments show that our algorithm is highly competitive in terms of sampling efficiency, while having significantly smaller computational overhead.

Correlating Resting-State Functional Magnetic Resonance Imaging Connectivity by Independent Component Analysis-Based Epileptogenic Zones with Intracranial Electroencephalogram Localized Seizure Onset Zones and Surgical Outcomes in Prospective Pediatric Intractable Epilepsy Study

The purpose of this study was to prospectively investigate the agreement between the epileptogenic zone(s) (EZ) localization by resting-state functional magnetic resonance imaging (rs-fMRI) and the seizure onset zone(s) (SOZ) identified by intracranial electroencephalogram (ic-EEG) using novel differentiating and ranking criteria of rs-fMRI abnormal independent components (ICs) in a large consecutive heterogeneous pediatric intractable epilepsy population without an a priori alternate modality informing EZ localization or prior declaration of total SOZ number. The EZ determination criteria were developed by using independent component analysis (ICA) on rs-fMRI in an initial cohort of 350 pediatric patients evaluated for epilepsy surgery over a 3-year period. Subsequently, these rs-fMRI EZ criteria were applied prospectively to an evaluation cohort of 40 patients who underwent ic-EEG for SOZ identification. Thirty-seven of these patients had surgical resection/disconnection of the area believed to be the primary source of seizures. One-year seizure frequency rate was collected postoperatively. Among the total 40 patients evaluated, agreement between rs-fMRI EZ and ic-EEG SOZ was 90% (36/40; 95% confidence interval [CI], 0.76–0.97). Of the 37 patients who had surgical destruction of the area believed to be the primary source of seizures, 27 (73%) rs-fMRI EZ could be classified as true positives, 7 (18%) false positives, and 2 (5%) false negatives. Sensitivity of rs-fMRI EZ was 93% (95% CI 78–98%) with a positive predictive value of 79% (95% CI, 63–89%). In those with cryptogenic localization-related epilepsy, agreement between rs-fMRI EZ and ic-EEG SOZ was 89% (8/9; 95% CI, 0.52–99), with no statistically significant difference between the agreement in the cryptogenic and symptomatic localization-related epilepsy subgroups. Two children with negative ic-EEG had removal of the rs-fMRI EZ and were seizure free 1 year postoperatively. Of the 33 patients where at least 1 rs-fMRI EZ agreed with the ic-EEG SOZ, 24% had at least 1 additional rs-fMRI EZ outside the resection area. Of these patients with un-resected rs-fMRI EZ, 75% continued to have seizures 1 year later. Conversely, among 75% of patients in whom rs-fMRI agreed with ic-EEG SOZ and had no anatomically separate rs-fMRI EZ, only 24% continued to have seizures 1 year later. This relationship between extraneous rs-fMRI EZ and seizure outcome was statistically significant (p = 0.01). rs-fMRI EZ surgical destruction showed significant association with postoperative seizure outcome. The pediatric population with intractable epilepsy studied prospectively provides evidence for use of resting-state ICA ranking criteria, to identify rs-fMRI EZ, as developed by the lead author (V.L.B.). This is a high yield test in this population, because no seizure nor particular interictal epilepiform activity needs to occur during the study. Thus, rs-fMRI EZ detected by this technique are potentially informative for epilepsy surgery evaluation and planning in this population. Independent of other brain function testing modalities, such as simultaneous EEG-fMRI or electrical source imaging, contextual ranking of abnormal ICs of rs-fMRI localized EZs correlated with the gold standard of SOZ localization, ic-EEG, across the broad range of pediatric epilepsy surgery candidates, including those with cryptogenic epilepsy.

Contextual Ranking by Passive Safety of Generational Classes of Light Vehicles

Summary Each year, the Bulletin d’Analyse des Accidents Corporels (BAAC) data set gathers descriptions of traffic accidents on French public roads involving one or several light vehicles and injuring at least one of the passengers. Each light vehicle can be associated with its ‘generational class’ (GC), a raw description of the vehicle including its date of design, date of entry into service and size class. In two given contexts of accident, two light vehicles with two different GCs do not necessarily offer the same level of safety to their passengers. The objective of this study is to assess to what extent more recent generations of light vehicles are safer than older vehicles on the basis of the BAAC data set. We rely on ‘scoring’: we look for a score function that associates any context of accident and any GC with a real number in such a way that, the smaller is this number, the safer is the GC in the given context. A better score function is learned from the BAAC data set by cross-validation, under the form of an optimal convex combination of score functions produced by a library of ranking algorithms by scoring. An oracle inequality illustrates the performances of the resulting meta-algorithm. We implement it, apply it and show some results.

Community-Based Question Answering via Contextual Ranking Metric Network Learning

The exponential growth of information on Community-based Question Answering (CQA) sites has raised the challenges for the accurate matching of high-quality answers to the given questions. Many existing approaches learn the matching model mainly based on the semantic similarity between questions and answers, which can not effectively handle the ambiguity problem of questions and the sparsity problem of CQA data. In this paper, we propose to solve these two problems by exploiting users' social contexts. Specifically, we propose a novel framework for CQA task by exploiting both the question-answer content in CQA site and users' social contexts. The experiment on real-world dataset shows the effectiveness of our method.

A Non-Stochastic Learning Approach to Energy Efficient Mobility Management

Energy efficient mobility management is an important problem in modern wireless networks with heterogeneous cell sizes and increased nodes densities. We show that optimization-based mobility protocols cannot achieve long-term optimal energy consumption, particularly for ultra-dense networks (UDN). To address the complex dynamics of UDN, we propose a non-stochastic online-learning approach which does not make any assumption on the statistical behavior of the small base station (SBS) activities. In addition, we introduce handover cost to the overall energy consumption, which forces the resulting solution to explicitly minimize frequent handovers. The proposed Batched Randomization with Exponential Weighting (BREW) algorithm relies on batching to explore in bulk, and hence reduces unnecessary handovers. We prove that the regret of BREW is sublinear in time, thus guaranteeing its convergence to the optimal SBS selection. We further study the robustness of the BREW algorithm to delayed or missing feedback. Moreover, we study the setting where SBSs can be dynamically turned on and off. We prove that sublinear regret is impossible with respect to arbitrary SBS on/off, and then develop a novel learning strategy, called ranking expert (RE), that simultaneously takes into account the handover cost and the availability of SBS. To address the high complexity of RE, we propose a contextual ranking expert (CRE) algorithm that only assigns experts in a given context. Rigorous regret bounds are proved for both RE and CRE with respect to the best expert. Simulations show that not only do the proposed mobility algorithms greatly reduce the system energy consumption, but they are also robust to various dynamics which are common in practical ultra-dense wireless networks.

A Free Access, Automated Law Citator with International Scope: The LawCite Project

Twelve free access and non-profit providers of legal information from across the common law world have collaborated to produce an automated international citator for case law and legal scholarship, access to end-users free of any user charges. This context imposes constraints on project development: only modest grant funding for development; open source plus purpose-developed software; no significant editorial input to develop application data, only automation; very limited maintenance funds; and no user charges or advertising revenue. The LawCite citator is the principal output to date of the LawCite Project, developed by the Australasian Legal Information Institute (AustLII) for the consortium of participating legal information institutes (LIIs). LawCite currently contains index records of the citation histories of almost five million cases, law journal articles, law reform documents and treaties. The citator is international, containing citation records in significant numbers from court decisions in 75 countries (primarily but not exclusively from common law countries).The purpose of this paper is to explain how the components of the LawCite project work, in some technical detail, and to outline the applications that have already resulted (both the citator and others), and future applications that are planned and possible. Our goal is for the LawCite project to play a key role in future global development of free access to legal information, provided collaboratively by free access providers across the globe.The LawCite project has developed and uses 3 main databases in an iterative fashion to produce its applications (the citator and other applications). They are: (i) The Citations database (basic information related to a citation); (ii) The Series database (information about each series of law reports, law journals, treaties, or law reform reports recognised by LawCite); and (iii) The Document database (an XML record for each recorded case or journal article). Oracle Berkeley DB (database software) is used for data mining and markup, and for construction of the Citations and Series databases. The Series and Document databases are used to generate the Citations database. The Series database is manually updated periodically. The Document and Citations databases are recreated automatically during each iteration of the data mining and ‘unmining’ processes. Raw citation lists are gathered from participating LII and non-LII data sources by the Citation Miner. Once collected, these are analysed, combined and normalised by the “Unminer” and from the combined list, the LawCite databases are generated. These databases are used both by the LawCite Citator as well as for the markup of text. The article explains these processes.The article concludes with a brief account of the citator from the user perspective, other outputs from the project (the publicly accessible LawCite Automated Markup Tool, and the markup tools used by LIIs), and potential future applications (informing policy debates through citation information; data visualisation of citation flows; and contextual ranking).

Lightweight Contextual Ranking of City Pictures: Urban Sociology to the Rescue

To increase mobile user engagement, photo sharing sites are trying to identify interesting and memorable pictures. Past proposals for identifying such pictures have relied on either metadata (e.g., likes) or visual features. In practice, techniques based on those two inputs do not always work: metadata is sparse (only few pictures have considerable number of likes), and extracting visual features is computationally expensive. In mobile solutions, geo-referenced content becomes increasingly important. The premise behind this work is that pictures of a neighborhood is linked to the way the neighborhood is perceived by people: whether it is, for instance, distinctive and beautiful or not. Since 1970s, urban theories proposed by Lynch, Milgram and Peterson aimed at systematically capturing the way people perceive neighborhoods. Here we tested whether those theories could be put to use for automatically identifying appealing city pictures. We did so by gathering geo-referenced Flickr pictures in the city of London; selecting six urban qualities associated with those urban theories; computing proxies for those qualities from online social media data; and ranking Flickr pictures based on those proxies. We find that our proposal enjoys three main desirable properties: it is effective, scalable, and aware of contextual changes such as time of day and weather condition. All this suggests new promising research directions for multi-modal learning approaches that automatically identify appealing city pictures.

On contextual ranking queries in databases

In this paper, we identify a novel and interesting type of queries, contextual ranking queries, which return the ranks of query tuples among some context tuples given in the queries. Contextual ranking queries are useful for olap and decision support applications in non-traditional data exploration. They provide a mechanism to quickly identify where tuples stand within the context. In this paper, we extend the sql language to express contextual ranking queries and propose a general partition-based framework for processing them. In this framework, we use a novel method that utilizes bitmap indices built on ranking functions. This method can efficiently identify a small number of candidate tuples, thus achieves lower cost than alternative methods. We analytically investigate the advantages and drawbacks of these methods, according to a preliminary cost model. Experimental results suggest that the algorithm using bitmap indices on ranking functions can be substantially more efficient than other methods.

A model based on multi-features to enhance healthcare and medical document retrieval

Objective. A major problem in biomedical informatics is the contextual retrieval and ranking of medical and healthcare information. In this article, we present a model for extracting semantic relations among medical and clinical documents. The purpose is to maximise contextual retrieval and ranking performance with minimum input from users.Design. We developed and evaluated a medical search engine that relies on a multi-features similarity model. The indexed documents are represented as a network that reflects the semantic relations among documents to assess topical rankings.Measurement. The evaluation measurements include the following: recall, precision and R-precision. We used OHSUMED collection to evaluate our work with runs submitted to TREC-9. We provide a comparison of the top five runs that achieved the highest average precision scores. In addition, we used questionnaire-based evaluation to measure the effectiveness of the ranking task.Results. The results indicated that the proposed model achieved a higher average precision in comparison with top-scored runs submitted to TREC-9; the improvement of our model over other methods is statistically significant (p-value <0.0001). Furthermore, a questionnaire-based experiment showed that the proposed model performed quite well in ranking retrieved documents according to their topics.

Search structures and algorithms for personalized ranking

As data of an unprecedented scale are becoming accessible on the Web, personalization, of narrowing down the retrieval to meet the user-specific information needs, is becoming more and more critical. For instance, while web search engines traditionally retrieve the same results for all users, they began to offer beta services to personalize the results to adapt to user-specific contexts such as prior search history or other application contexts. In a clear contrast to search engines dealing with unstructured text data, this paper studies how to enable such personalization in the context of structured data retrieval. In particular, we adopt contextual ranking model to formalize personalization as a cost-based optimization over collected contextual rankings. With this formalism, personalization can be abstracted as a cost-optimal retrieval of contextual ranking, closely matching user-specific retrieval context. With the retrieved matching context, we adopt a machine learning approach, to effectively and efficiently identify the ideal personalized ranked results for this specific user. Our empirical evaluations over synthetic and real-life data validate both the efficiency and effectiveness of our framework.