Effective Retrieval Model Research Articles

TriSampler: A Better Negative Sampling Principle for Dense Retrieval

Negative sampling stands as a pivotal technique in dense retrieval, essential for training effective retrieval models and significantly impacting retrieval performance. While existing negative sampling methods have made commendable progress by leveraging hard negatives, a comprehensive guiding principle for constructing negative candidates and designing negative sampling distributions is still lacking. To bridge this gap, we embark on a theoretical analysis of negative sampling in dense retrieval. This exploration culminates in the unveiling of the quasi-triangular principle, a novel framework that elucidates the triangular-like interplay between query, positive document, and negative document. Fueled by this guiding principle, we introduce TriSampler, a straightforward yet highly effective negative sampling method. The keypoint of TriSampler lies in its ability to selectively sample more informative negatives within a prescribed constrained region. Experimental evaluation show that TriSampler consistently attains superior retrieval performance across a diverse of representative retrieval models.

Dense Text Retrieval Based on Pretrained Language Models: A Survey

Text retrieval is a long-standing research topic on information seeking, where a system is required to return relevant information resources to user’s queries in natural language. From heuristic-based retrieval methods to learning-based ranking functions, the underlying retrieval models have been continually evolved with the ever-lasting technical innovation. To design effective retrieval models, a key point lies in how to learn text representations and model the relevance matching. The recent success of pretrained language models (PLM) sheds light on developing more capable text-retrieval approaches by leveraging the excellent modeling capacity of PLMs. With powerful PLMs, we can effectively learn the semantic representations of queries and texts in the latent representation space, and further construct the semantic matching function between the dense vectors for relevance modeling. Such a retrieval approach is called dense retrieval , since it employs dense vectors to represent the texts. Considering the rapid progress on dense retrieval, this survey systematically reviews the recent progress on PLM-based dense retrieval. Different from previous surveys on dense retrieval, we take a new perspective to organize the related studies by four major aspects, including architecture, training, indexing and integration, and thoroughly summarize the mainstream techniques for each aspect. We extensively collect the recent advances on this topic, and include 300+ reference papers. To support our survey, we create a website for providing useful resources, and release a code repository for dense retrieval. This survey aims to provide a comprehensive, practical reference focused on the major progress for dense text retrieval.

A novel benchmark dataset of color steel sheds for remote sensing image retrieval

Benchmark datasets play an important role in evaluating remote sensing image retrieval (RSIR) methods. The current datasets cover many scene categories, but omit an important scene of color steel sheds, which are widely distributed with a large number on the earth’s surface. Therefore, we propose a new benchmark dataset of color steel sheds (CSS) from Google map imagery for RSIR and share it open access in our V-RSIR system. The new dataset has rich intra-class and inter-class diversity, and is composed of blue, red and white color steel sheds with the total number of 2407 remote sensing images. We conduct evaluation experiments on the new dataset by using ten low/mid feature-based and ten deep learning feature-based methods. Experimental results indicate that the dataset is effective for evaluating RSIR methods and using the dataset can construct an effective retrieval model for color steel sheds. Besides, we have experimentally demonstrated that color constancy does affect retrieval performance on our CSS dataset. Furthermore, some experiments of merging the CSS dataset with the PatternNet, VGoogle and NWPU45 datasets are also conducted. Experimental results demonstrate that our dataset can be used as a complement to other retrieval datasets. Furthermore, these experimental results can be used as baseline for future applications on RSIR.

Modeling user information needs on mobile devices

Recent advances in the development of mobile devices equipped with multiple sensors, together with the availability of millions of applications, have made these devices more pervasive in our lives than ever. The availability of the diverse set of sensors, as well as high computational power, enable information retrieval (IR) systems to sense a user's context, and personalize their results accordingly. Relevant studies show that people use their mobile devices to access information in a wide range of topics in various contextual situations, highlighting the fact that modeling user information need on mobile devices involves studying several means of information access. In this thesis, we study three significant aspects of information access on mobile devices. First, we focus on proactive approaches to modeling users for venue suggestion. We investigate three methods of user modeling, namely, content-based, collaborative, and hybrid, focusing on personalization and context-awareness. Our content-based model is based on multiple relevance scores derived from multi-modal information that we collected from multiple sources of information [1]. We further propose a two-phase collaborative ranking algorithm for leveraging users' implicit feedback while incorporating temporal and geographical information into the model [3]. We then extend our collaborative model to include multiple cross-venue similarity scores and combine it with our content-based approach to produce a hybrid recommendation [2]. Second, we introduce and investigate a new task on mobile search, that is, unified mobile search. We take the first step in defining, studying, and modeling this task by collecting two datasets [5, 4]. Our analyses show that a limited number of popular apps attract most of the search queries. We further observe notable differences between queries submitted to different apps. We show that query length and content differ among apps. We also show that 39% of search queries were done in Google Search, and it was the top choice of users in 35% of the tasks. Given that more than 71% of the defined tasks could be done with the current features of Google Search, this indicates that users prefer to search using a more specific app. Also, we conduct experiments on one of the main components of unified mobile search frameworks, which is target apps selection. To this end, we propose two neural approaches that learn high-dimensional app representations based on their respective queries. Finally, we address the conversational aspect of mobile search , where we propose an offline evaluation protocol and build a dataset for asking clarifying questions for conversational search [6]. Also, we propose a retrieval framework consisting of three main components: question retrieval, question selection, and document retrieval. The experiments and analyses indicate that asking clarifying questions should be an essential part of a conversational system, resulting in a high performance gain. In particular, experiments on the oracle model demonstrated that asking only one good clarifying question leads to over 150% relative improvement in terms of P@1 and nDCG@1. Moreover, we observed that asking clarifying questions improves the model's performance for a substantial percentage of the facets, even though a more effective retrieval model than the one we used could potentially improve the performance.

Explainable Product Search with a Dynamic Relation Embedding Model

Product search is one of the most popular methods for customers to discover products online. Most existing studies on product search focus on developing effective retrieval models that rank items by their likelihood to be purchased. However, they ignore the problem that there is a gap between how systems and customers perceive the relevance of items. Without explanations, users may not understand why product search engines retrieve certain items for them, which consequentially leads to imperfect user experience and suboptimal system performance in practice. In this work, we tackle this problem by constructing explainable retrieval models for product search. Specifically, we propose to model the “search and purchase” behavior as a dynamic relation between users and items, and create a dynamic knowledge graph based on both the multi-relational product data and the context of the search session. Ranking is conducted based on the relationship between users and items in the latent space, and explanations are generated with logic inferences and entity soft matching on the knowledge graph. Empirical experiments show that our model, which we refer to as the Dynamic Relation Embedding Model (DREM), significantly outperforms the state-of-the-art baselines and has the ability to produce reasonable explanations for search results.

Binary Independence Language Model in a Relevance Feedback Environment

Model construction is a kind of knowledge engineering, and building retrieval models is critical to the success of search engines. This article proposes a new (retrieval) language model, called binary independence language model (BILM). It integrates two document-context based language models together into one by the log-odds ratio where these two are language models applied to describe document-contexts of query terms. One model is based on relevance information while the other is based on the non-relevance information. Each model incorporates link dependencies and multiple query term dependencies. The probabilities are interpolated between the relative frequency and the background probabilities. In a simulated relevance feedback environment of top 20 judged documents, our BILM performed statistically significantly better than the other highly effective retrieval models at 95% confidence level across four TREC collections using fixed parameter values for the mean average precision. For the less stable performance measure (i.e. precision at the top 10), no statistical significance is shown between the different models for the individual test collections although numerically our BILM is better than two other models with a confidence level of 95% based on a paired sign test across the test collections of both relevance feedback and retrospective experiments.

Multi-scale CNNs for 3D model retrieval

Recent advancements in low-cost 3D sensors and mobile devices of virtual 3D models have additionally facilitated the accessibility of 3D data. 3D model retrieval is becoming an indispensable function for modern search engines. An effective retrieval model is at the core of computer vision. With the continuous improvement of 3D data, there are large number of methods to solve this problem. Existing works proposed numerous works to deal with feature extraction and object matching. Most of them are unable to fully exploit the information of 3D representations. To address this problem, we propose a novel multi-layer deep network in this paper. First, multiple rendered images are extracted from a 3D object, and combined into one representative view, which is the actual input of the network. Then, the novel multi-layer network structure is trained and tested on these representative views, generating the feature leaning model, which owns the local and global information of a 3D object. Finally, simple Euclidean metric is used to compute the similarity between two different 3D models to complete the retrieval problem. Extensive experiments and corresponding experimental results have demonstrated the superiority of our approach.

Similarity Measurements of Vector Space Model on Arabic Text

This study presented an effective retrieval model through appling a successful comparison between sets of measurment within Vector Space Model (VSM) and to proof that, we use two mechanism in inverted file, the first is word-oriented mechanism for indexing a text collection and the second is block-oriented mechanism, after removing the stop words. This study use 242 collection of arabic abstract and 60 building collection of arabic queries. During building an inverted file as index file, time and space factors are computed. And after running the system, recall and precision calculated to compare the retrieval efficiency of using inverted file. The VSM have many measurment: Cosine measure, Dice measure, Jaccard measure and Inner product similarity. The study achived an effective retrieval system through appled VSM with jaccard measure comparison with the other measurements, jaccard mesure obtain a good result particularly when using the arabic collection documents. The study also obtained a good result from block-oriented mechanism rather than word-oriented mechanism. As a conclusion, the best information retrival model for arabic documents is VSM with jaccard measure using block-oriented technique.

Opinion-Based Entity Ranking using learning to rank

As social media and e-commerce on the Internet continue to grow, opinions have become one of the most important sources of information for users to base their future decisions on. Unfortunately, the large quantities of opinions make it difficult for an individual to comprehend and evaluate them all in a reasonable amount of time. The users have to read a large number of opinions of different entities before making any decision. Recently a new retrieval task in information retrieval known as Opinion-Based Entity Ranking (OpER) has emerged. OpER directly ranks relevant entities based on how well opinions on them are matched with a user's preferences that are given in the form of queries. With such a capability, users do not need to read a large number of opinions available for the entities. Previous research on OpER does not take into account the importance and subjectivity of query keywords in individual opinions of an entity. Entity relevance scores are computed primarily on the basis of occurrences of query keywords match, by assuming all opinions of an entity as a single field of text. Intuitively, entities that have positive judgments and strong relevance with query keywords should be ranked higher than those entities that have poor relevance and negative judgments. This paper outlines several ranking features and develops an intuitive framework for OpER in which entities are ranked according to how well individual opinions of entities are matched with the user's query keywords. As a useful ranking model may be constructed from many ranking features, we apply learning to rank approach based on genetic programming (GP) to combine features in order to develop an effective retrieval model for OpER task. The proposed approach is evaluated on two collections and is found to be significantly more effective than the standard OpER approach.

Extended Vector Space Model with Semantic Relatedness on Java Archive Search Engine

Byte code as information source is a novel approach which enable Java archive search engine to be built without relying on another resources except the Java archive itself [1]. Unfortunately, its effectiveness is not considerably high since some relevant documents may not be retrieved because of vocabulary mismatch. In this research, a vector space model (VSM) is extended with semantic relatedness to overcome vocabulary mismatch issue in Java archive search engine. Aiming the most effective retrieval model, some sort of equations in retrieval models are also proposed and evaluated such as sum up all related term, substituting non-existing term with most related term, logaritmic normalization, context-specific relatedness, and low-rank query-related retrieved documents. In general, semantic relatedness improves recall as a tradeoff of its precision reduction. We also proposed a scheme to take the advantage of relatedness without affected by its disadvantage (VSM + considering non-retrieved documents as low-rank retrieved documents using semantic relatedness). This scheme assures that relatedness score should be ranked lower than standard exact-match score. This scheme yields 1.754% higher effectiveness than our standard VSM.

Semantic concept-enriched dependence model for medical information retrieval

ObjectiveIn medical information retrieval research, semantic resources have been mostly used by expanding the original query terms or estimating the concept importance weight. However, implicit term-dependency information contained in semantic concept terms has been overlooked or at least underused in most previous studies. In this study, we incorporate a semantic concept-based term-dependence feature into a formal retrieval model to improve its ranking performance. DesignStandardized medical concept terms used by medical professionals were assumed to have implicit dependency within the same concept. We hypothesized that, by elaborately revising the ranking algorithms to favor documents that preserve those implicit dependencies, the ranking performance could be improved. The implicit dependence features are harvested from the original query using MetaMap. These semantic concept-based dependence features were incorporated into a semantic concept-enriched dependence model (SCDM). We designed four different variants of the model, with each variant having distinct characteristics in the feature formulation method. MeasurementsWe performed leave-one-out cross validations on both a clinical document corpus (TREC Medical records track) and a medical literature corpus (OHSUMED), which are representative test collections in medical information retrieval research. ResultsOur semantic concept-enriched dependence model consistently outperformed other state-of-the-art retrieval methods. Analysis shows that the performance gain has occurred independently of the concept’s explicit importance in the query. ConclusionBy capturing implicit knowledge with regard to the query term relationships and incorporating them into a ranking model, we could build a more robust and effective retrieval model, independent of the concept importance.

Diagnostic Evaluation of Information Retrieval Models

Developing effective retrieval models is a long-standing central challenge in information retrieval research. In order to develop more effective models, it is necessary to understand the deficiencies of the current retrieval models and the relative strengths of each of them. In this article, we propose a general methodology to analytically and experimentally diagnose the weaknesses of a retrieval function, which provides guidance on how to further improve its performance. Our methodology is motivated by the empirical observation that good retrieval performance is closely related to the use of various retrieval heuristics. We connect the weaknesses and strengths of a retrieval function with its implementations of these retrieval heuristics, and propose two strategies to check how well a retrieval function implements the desired retrieval heuristics. The first strategy is to formalize heuristics as constraints, and use constraint analysis to analytically check the implementation of retrieval heuristics. The second strategy is to define a set of relevance-preserving perturbations and perform diagnostic tests to empirically evaluate how well a retrieval function implements retrieval heuristics. Experiments show that both strategies are effective to identify the potential problems in implementations of the retrieval heuristics. The performance of retrieval functions can be improved after we fix these problems.

Quantitative estimation of chlorophyll-a concentration in the northern part of Lake Taihu using spectral reflectance

利用地物光谱仪研究了太湖水体的反射光谱特征与叶绿素a浓度之间的定量关系,结果表明太湖水体的叶绿素a浓度可以用720 nm附近的反射率估算,同时也可以用806 nm和571 nm两个波段的反射率比值来估算,前者建立的估算模型具有较好的通用性,而后者只能较好的估算<10μg/L的叶绿素a浓度;通过对光谱微分的分析,发现叶绿素a浓度与690 nm附近的一阶微分和702 nm附近的二阶微分相关性最好,但基于反射光谱一阶微分的叶绿素a浓度估算模型,并没有显著的提高太湖叶绿素a浓度的估测精度,二阶微分后的估测精度好于一阶微分,但其估测精度仍没有利用720 nm反射光谱的反演模型高．太湖水体的叶绿素a浓度可以利用720 nm附近的反射光谱有效地估算．;The quantitative relationship between Spectral Reflectance (SR) characteristics of water and chlorophyll-a( Chi. a) concentration in Lake Taihu was analysed based on the in-situ data collected by ASD FieldSpec Hand-Held Spectroradiometer. The results reveal that the Chi. a concentration can be quantitative retrieved using SR at wavelength of 720 nm，or using the ratio of SR at wavelength of 806 nm and 507 nm. The model using SR at wavelength of 720 nm is an effective retrieval model whether Chi. a concentration > 10ug/L or Chi. a concentration < 10 ug/L, while the model using the ratio only retrieve Chi. a concentration effectively for Chi. a concentration < 10ug/L. It was found that there is high correlation between Chi. a and the first derivative at 690 nm, also high correlation between Chi. a and the second derivative at 702 nm through spectral derivative method，but the model retrieve accuracy using the first derivative or the second derivative is not significantly improved compared to the model directly using SR. But the retrieval accuracy using the second derivative is a little higher than the model using the first derivative. So it was concluded that the Chi. a Concentration in Lake Taihu can be retrieved using SR at wavelength 720.