Information Retrieval Model Research Articles

An end-to-end pseudo relevance feedback framework for neural document retrieval

Pseudo relevance feedback (PRF) is commonly used to boost the performance of traditional information retrieval (IR) models by using top-ranked documents to identify and weight new query terms, thereby reducing the effect of query-document vocabulary mismatches. While neural retrieval models have recently demonstrated promising results for ad-hoc document retrieval, combining them with PRF is not straightforward due to incompatibilities between existing PRF approaches and neural architectures. To bridge this gap, we propose an end-to-end neural PRF framework, coined NPRF, that enriches the representation of user information need from a single query to multiple PRF documents. NPRF can be used with existing neural IR models by embedding different neural models as building blocks. Three state-of-the-art neural retrieval models, including the unigram DRMM and KNRM models, and the position-aware PACRR model, are utilized to instantiate the NPRF framework. Extensive experiments on two standard test collections, TREC1-3 and Robust04, confirm the effectiveness of the proposed NPRF framework in improving the performance of three state-of-the-art neural IR models. In addition, analysis shows that integrating the existing neural IR models within the NPRF framework results in reduced training and validation losses, and consequently, improved effectiveness of the learned ranking functions.

Neural models for information retrieval without labeled data

Recent developments of machine learning models, and in particular deep neural networks, have yielded significant improvements on several computer vision, natural language processing, and speech recognition tasks. Progress with information retrieval (IR) tasks has been slower, however, due to the lack of large-scale training data as well as neural network models specifically designed for effective information retrieval [9]. In this dissertation, we address these two issues by introducing task-specific neural network architectures for a set of IR tasks and proposing novel unsupervised or weakly supervised solutions for training the models. The proposed learning solutions do not require labeled training data. Instead, in our weak supervision approach, neural models are trained on a large set of noisy and biased training data obtained from external resources, existing models, or heuristics. We first introduce relevance-based embedding models [3] that learn distributed representations for words and queries. We show that the learned representations can be effectively employed for a set of IR tasks, including query expansion, pseudo-relevance feedback, and query classification [1, 2]. We further propose a standalone learning to rank model based on deep neural networks [5, 8]. Our model learns a sparse representation for queries and documents. This enables us to perform efficient retrieval by constructing an inverted index in the learned semantic space. Our model outperforms state-of-the-art retrieval models, while performing as efficiently as term matching retrieval models. We additionally propose a neural network framework for predicting the performance of a retrieval model for a given query [7]. Inspired by existing query performance prediction models, our framework integrates several information sources, such as retrieval score distribution and term distribution in the top retrieved documents. This leads to state-of-the-art results for the performance prediction task on various standard collections. We finally bridge the gap between retrieval and recommendation models, as the two key components in most information systems. Search and recommendation often share the same goal: helping people get the information they need at the right time. Therefore, joint modeling and optimization of search engines and recommender systems could potentially benefit both systems [4]. In more detail, we introduce a retrieval model that is trained using user-item interaction (e.g., recommendation data), with no need to query-document relevance information for training [6]. Our solutions and findings in this dissertation smooth the path towards learning efficient and effective models for various information retrieval and related tasks, especially when large-scale training data is not available.

Neural generative models and representation learning for information retrieval

Information Retrieval (IR) concerns about the structure, analysis, organization, storage, and retrieval of information. Among different retrieval models proposed in the past decades, generative retrieval models, especially those under the statistical probabilistic framework, are one of the most popular techniques that have been widely applied to Information Retrieval problems. While they are famous for their well-grounded theory and good empirical performance in text retrieval, their applications in IR are often limited by their complexity and low extendability in the modeling of high-dimensional information. Recently, advances in deep learning techniques provide new opportunities for representation learning and generative models for information retrieval. In contrast to statistical models, neural models have much more flexibility because they model information and data correlation in latent spaces without explicitly relying on any prior knowledge. Previous studies on pattern recognition and natural language processing have shown that semantically meaningful representations of text, images, and many types of information can be acquired with neural models through supervised or unsupervised training. Nonetheless, the effectiveness of neural models for information retrieval is mostly unexplored. In this thesis, we study how to develop new generative models and representation learning frameworks with neural models for information retrieval. Specifically, our contributions include three main components: (1) Theoretical Analysis : We present the first theoretical analysis and adaptation of existing neural embedding models for ad-hoc retrieval tasks; (2) Design Practice : Based on our experience and knowledge, we show how to design an embedding-based neural generative model for practical information retrieval tasks such as personalized product search; And (3) Generic Framework : We further generalize our proposed neural generative framework for complicated heterogeneous information retrieval scenarios that concern text, images, knowledge entities, and their relationships. Empirical results show that the proposed neural generative framework can effectively learn information representations and construct retrieval models that outperform the state-of-the-art systems in a variety of IR tasks.

A Question-Answering System for Applicant Support Using Modern Messaging Apps

There is an increasing user interest in instant messaging applications. These applications allow us to interact with other users and include a functionality that can help us to implement bots that automate various business processes or provide information services. This paper considers a specialized question answering system that employs today’s messaging services infrastructure to support university applicants. Over two years, we gathered a corpus of typical questions of applicants and developed an information retrieval model for finding similar questions in this corpus. Applicants can type their questions using natural language without any formal requirements to phrase construction or using special templates. If the system is unable to find a relevant answer, the user can directly address the question to representatives of the university. The system was implemented using modern cloud services provided by Amazon. We used serverless computations and NoSQL databases, so we had to develop an appropriate architecture of the service. Since the system operates sensitive personal data and provides personalized service, we analyzed methods of ensuring system security and approaches to user authorization. We are currently testing our system and estimating its information retrieval quality.

A topic‐based term frequency normalization framework to enhance probabilistic information retrieval

AbstractMany well‐known probabilistic information retrieval models have shown promise for use in document ranking, especially BM25. Nevertheless, it is observed that the control parameters in BM25 usually need to be adjusted to achieve improved performance on different data sets; additionally, the assumption in BM25 on the bag‐of‐words model prevents its direct utilization of rich information that lies at the sentence or document level. Inspired by the above challenges with respect to BM25, we first propose a new normalization method on the term frequency in BM25 (called BM25QL in this paper); in addition, the method is incorporated into CRTER2, a recent BM25‐based model, to construct CRTER2QL. Then, we incorporate topic modeling and word embedding into BM25 to relax the assumption of the bag‐of‐words model. In this direction, we propose a topic‐based retrieval model, TopTF, for BM25, which is then further incorporated into the language model (LM) and the multiple aspect term frequency (MATF) model. Furthermore, an enhanced topic‐based term frequency normalization framework, ETopTF, based on embedding is presented. Experimental studies demonstrate the great effectiveness and performance of these methods. Specifically, on all tested data sets and in terms of the mean average precision (MAP), our proposed models, BM25QL and CRTER2QL, are comparable to BM25 and CRTER2 with the best b parameter value; the TopTF models significantly outperform the baselines, and the ETopTF models could further improve the TopTF in terms of the MAP.

Adaptive and Optimization of Personalized Information Retrieval Model in Semantic Web

The recognition of user’s visited set of web pages for the prediction of web page is a key drawback. Thus the work is employed with the web access log files which is stored in the server. To understand the user interest patterns, the web access log files are extracted that depicts the user behavior. Various applications can be employed to predicting user’s behavior while serving the web. During this work, the proposed framework analyze the user usage, reinforced the content and the content retrieved with the semantic manner. The semantic information retrieval supported the user access pages are preprocessed and the web log data of the particular user is analyzed to identify the user profile. Then the retrieved information is graded with clustering the semantic content based results. The ranked content is then analyzed with the user profile to produce an optimized search results for the users based on the user classification.

Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering

Adversarial evaluation stress-tests a model’s understanding of natural language. Because past approaches expose superficial patterns, the resulting adversarial examples are limited in complexity and diversity. We propose human- in-the-loop adversarial generation, where human authors are guided to break models. We aid the authors with interpretations of model predictions through an interactive user interface. We apply this generation framework to a question answering task called Quizbowl, where trivia enthusiasts craft adversarial questions. The resulting questions are validated via live human–computer matches: Although the questions appear ordinary to humans, they systematically stump neural and information retrieval models. The adversarial questions cover diverse phenomena from multi-hop reasoning to entity type distractors, exposing open challenges in robust question answering.

A question-entailment approach to question answering

BackgroundOne of the challenges in large-scale information retrieval (IR) is developing fine-grained and domain-specific methods to answer natural language questions. Despite the availability of numerous sources and datasets for answer retrieval, Question Answering (QA) remains a challenging problem due to the difficulty of the question understanding and answer extraction tasks. One of the promising tracks investigated in QA is mapping new questions to formerly answered questions that are “similar”.ResultsWe propose a novel QA approach based on Recognizing Question Entailment (RQE) and we describe the QA system and resources that we built and evaluated on real medical questions. First, we compare logistic regression and deep learning methods for RQE using different kinds of datasets including textual inference, question similarity, and entailment in both the open and clinical domains. Second, we combine IR models with the best RQE method to select entailed questions and rank the retrieved answers. To study the end-to-end QA approach, we built the MedQuAD collection of 47,457 question-answer pairs from trusted medical sources which we introduce and share in the scope of this paper. Following the evaluation process used in TREC 2017 LiveQA, we find that our approach exceeds the best results of the medical task with a 29.8% increase over the best official score.ConclusionsThe evaluation results support the relevance of question entailment for QA and highlight the effectiveness of combining IR and RQE for future QA efforts. Our findings also show that relying on a restricted set of reliable answer sources can bring a substantial improvement in medical QA.

Smart System for the Retrieval of Digital Educational Content

The education sector is a major generator, consumer, and depositary of educational content. Thanks to technological advances, today’s educators and learners have ubiquitous and on-demand access to information. Technology has made it possible for us to communicate and share information effortlessly from anywhere in the world. However, the availability of vast amounts of heterogeneous educational content will not be useful unless we search, retrieve and integrate it, creating interoperable educational environments. The current challenges to integrating educational content arise from its distribution over several repositories. This research proposes AIREH (architecture for intelligent retrieval of educational content from heterogeneous environments), for the retrieval of digital content through agent-based virtual organizations. This flexible architecture facilitates the search for and integration of heterogeneous content through an information retrieval model that involves both case-based reasoning and federated search. Moreover, AIREH is based on an adaptive organization model for distributed planning, thanks to which, it manages open systems flexibly, dynamically, and effectively. The conducted case study gives very promising results and demonstrates the advantages of using agent-based virtual organizations in the retrieval of labeled digital content. The proposed model is flexible, customizable, comprehensive and efficient.

Multilingual Information Access (MLIA) Tools on Google and WorldCat: Bi/Multilingual University Students’ Experience and Perceptions

This article reports on the results of an exploratory user-centered study that examined how technological advancements in natural language processing (NLP) such as the availability of multilingual information access (MLIA) tools impact the information searching behavior of bi/multilingual academic users. Thirty-one bi/multilingual students participated in a controlled lab-based user experiment in which they carried out two assigned tasks each on Google and WorldCat for a total of four tasks, and then completed a post experiment questionnaire. The captures from the experiment showed 86.7% of the participants using multilingual information access tools. Further analyses of the captures also showed that participants were more likely to use MLIA tools when the instructions for the task were stated in their native language. An independent samples t-test revealed that participants spent less time on their searches when they used MLIA tools. The study revealed considerable diversity in the information searching behavior of the participants, even within the same pair of languages, and even for the same user. Diversity was noted for instance, on which tasks MLIA tools were used and in how these tools were used. User-centered designed, personalized multilingual information retrieval (PMLIR) models could hold promise for best representing the information searching behavior of bi/multilingual users.

Focal elements of neural information retrieval models. An outlook through a reproducibility study

This paper analyzes two state-of-the-art Neural Information Retrieval (NeuIR) models: the Deep Relevance Matching Model (DRMM) and the Neural Vector Space Model (NVSM). Our contributions include: (i) a reproducibility study of two state-of-the-art supervised and unsupervised NeuIR models, where we present the issues we encountered during their reproducibility; (ii) a performance comparison with other lexical, semantic and state-of-the-art models, showing that traditional lexical models are still highly competitive with DRMM and NVSM; (iii) an application of DRMM and NVSM on collections from heterogeneous search domains and in different languages, which helped us to analyze the cases where DRMM and NVSM can be recommended; (iv) an evaluation of the impact of varying word embedding models on DRMM, showing how relevance-based representations generally outperform semantic-based ones; (v) a topic-by-topic evaluation of the selected NeuIR approaches, comparing their performance to the well-known BM25 lexical model, where we perform an in-depth analysis of the different cases where DRMM and NVSM outperform the BM25 model or fail to do so. We run an extensive experimental evaluation to check if the improvements of NeuIR models, if any, over the selected baselines are statistically significant.

An Efficient Information System for Providing Location Based Services in Network Environments

In information retrieval systems, the location information can be collected by using Global Positioning Systems for providing Location Based Services (LBS) to the information system users. The LBS can be either person-oriented or device-oriented. In the person-oriented approach, the LBS focuses on the location of the user and the user can control the services provided by the service provider. In the device-oriented approach, the LBS focuses on the object and it is not controlled by the mobile user. The LBS also manages Points of Interest (POI) of mobile users in order to perform effective route finding for providing the services fast. Moreover, the users of mobile devices tend to query based on Points of Interest such as for restaurants and markets where the users can query based on its attributes like price, type of food, user ratings which are close to their current location. The POI is calculated primarily on the interest of mobile users and the ratings given by existing user’s with their feedback. The users can select the features to obtain specific objects. In this paper, a new information retrieval model is proposed for improving the performance of information systems by providing new data structures and algorithms for storage and retrieval of data. The primary idea behind this work is to minimize the number of user requests and to achieve high accuracy for the user’s query result by using an information repository called as the Path Log. The Path Log is used to store the information about the search time and distance of the service provider locations by providing an optimal route to serve the users information request. For this purpose, a new spatial indexing is used in this work to obtain an accurate and optimal path to reach the location of the services. The tree has been constructed by giving weights for the leaf nodes. Thus the solution proposed in this work for providing LBS is achieved by processing the K-Nearest Neighbor and Range queries such that the user requests for information retrieval is carried out more optimally.

Bi lingual Information Retrieval System for English and Italian Tweets using Python

Bi-lingual text analysis is competent in present scenario as the information gathered in various languages is flattering. The bi-lingual text classification is yet an obscure area whereas the text classification in a single language is well known. The concept of bi-lingual text has been left in a shell, apart from the lame stream of both theory as well as practical. The use of social media is increasing day by day and thus the amount of data too in increasing with a rapid rate. So, it is an alarming stage to analyze the big data and extract the useful information. In this paper, we are developing a dynamic information retrieval model and extricating the sentiments of people on global warming of English and Italian tweets and corresponding to it its heat map and affinity map are generated as it produces the output after harmonizing different objects which diverge in the rung of relevancy to the question.

Design and Evaluation of a Contextual Model for Information Retrieval From Web-Scale Discovery Services to Improve Evidence-Based Practice by Health Care Practitioners: Mixed Methods Study.

BackgroundPracticing evidence-based health care is challenging because of overwhelming results presented to practitioners by Google-like Web-scale discovery (WSD) services that index millions of resources while retrieving information based on relevancy algorithms with limited consideration for user information need.ObjectiveOn the basis of the user-oriented theory of information need and following design science principles, this study aimed to develop and evaluate an innovative contextual model for information retrieval from WSD services to improve evidence-based practice (EBP) by health care practitioners.MethodsWe identified problems from literature to support real-world requirements for this study. We used design science research methodology to guide artefact design. We iteratively improved prototype of the context model using artificial formative evaluation. We performed naturalistic summative evaluation using convergent interviewing of health care practitioners and content analysis from a confirmatory focus group consisting of health researchers to evaluate the model’s validity and utility.ResultsThe study iteratively designed and applied the context model to a WSD service to meet 5 identified requirements. All 5 health care practitioners interviewed found the artefact satisfied the 5 requirements to successfully evaluate the model as having validity and utility. Content analysis results from the confirmatory focus group mapped top 5 descriptors per requirement to support a true hypothesis that there is significant discussion among participants to justify concluding that the artefact had validity and utility.ConclusionsThe context model for WSD satisfied all requirements and was evaluated successfully for information retrieval to improve EBP. Outcomes from this study justify further research into the model.

Automatic recall of software lessons learned for software project managers

Lessons learned (LL) records constitute the software organization memory of successes and failures. LL are recorded within the organization repository for future reference to optimize planning, gain experience, and elevate market competitiveness. However, manually searching this repository is a daunting task, so it is often disregarded. This can lead to the repetition of previous mistakes or even missing potential opportunities. This, in turn, can negatively affect the profitability and competitiveness of organizations. We aim to present a novel solution that provides an automatic process to recall relevant LL and to push those LL to project managers. This will dramatically save the time and effort of manually searching the unstructured LL repositories and thus encourage the LL exploitation. We exploit existing project artifacts to build the LL search queries on-the-fly in order to bypass the tedious manual searching. An empirical case study is conducted to build the automatic LL recall solution and evaluate its effectiveness. The study employs three of the most popular information retrieval models to construct the solution. Furthermore, a real-world dataset of 212 LL records from 30 different software projects is used for validation. Top-k and MAP well-known accuracy metrics are used as well. Our case study results confirm the effectiveness of the automatic LL recall solution. Also, the results prove the success of using existing project artifacts to dynamically build the search query string. This is supported by a discerning accuracy of about 70% achieved in the case of top-k. The automatic LL recall solution is valid with high accuracy. It will eliminate the effort needed to manually search the LL repository. Therefore, this will positively encourage project managers to reuse the available LL knowledge, which will avoid old pitfalls and unleash hidden business opportunities.

Passage-Based Text Summarization for Legal Information Retrieval

Automatic text summarization is a process of condensing the content of a text document to pursue the most important information. It plays a significant role in various tasks like text categorization, question answering and information retrieval (IR). As legal information retrieval (LIR) is a subfield of IR, the produced summaries are combined into IR system, with the objective of decreasing the length of the document. In this way, we can improve the access time for searching the information, and relevant documents are retrieved. In this article, we present the creation of passage-level summaries (generic and legal) with different compression ratios and evaluate their performance. The generic summaries present the overall description of the essential information of a document and legal summaries, produced by taking into account the domain-specific features that are present in the document. Next, we propose Boosting Okapi BM25 which is the modified model of Okapi BM25 to increase the efficiency of the LIR. We have evaluated proposed LIR approach in terms of MAP and R-precision and summarization approach using ROUGE tool on FIRE2013 and FIRE2014 datasets. To show the efficacy of the proposed system, we compare the experimental results with different IR models like PL2, \(In\_expB2\), \(In\_expC2\), InL2, \(DFR\_BM25\), Okapi BM25 in terms of MAP. The experimental results of the proposed system show better performance than the existing various IR models in terms of various performance metrics. The empirical results also exhibit that the integration of text summarization and IR techniques helps in retrieving relevant information with less access time.

Data Augmentation Based on Adversarial Autoencoder Handling Imbalance for Learning to Rank

Data imbalance is a key limiting factor for Learning to Rank (LTR) models in information retrieval. Resampling methods and ensemble methods cannot handle the imbalance problem well since none of them incorporate more informative data into the training procedure of LTR models. We propose a data generation model based on Adversarial Autoencoder (AAE) for tackling the data imbalance in LTR via informative data augmentation. This model can be utilized for handling two types of data imbalance, namely, imbalance regarding relevance levels for a particular query and imbalance regarding the amount of relevance judgements in different queries. In the proposed model, relevance information is disentangled from the latent representations in this AAE-based model in order to reconstruct data with specific relevance levels. The semantic information of queries, derived from word embeddings, is incorporated in the adversarial training stage for regularizing the distribution of the latent representation. Two informative data augmentation strategies suitable for LTR are designed utilizing the proposed data generation model. Experiments on benchmark LTR datasets demonstrate that our proposed framework can significantly improve the performance of LTR models.

DeepTileBars: Visualizing Term Distribution for Neural Information Retrieval

Most neural Information Retrieval (Neu-IR) models derive query-to-document ranking scores based on term-level matching. Inspired by TileBars, a classical term distribution visualization method, in this paper, we propose a novel Neu-IR model that handles query-to-document matching at the subtopic and higher levels. Our system first splits the documents into topical segments, “visualizes” the matchings between the query and the segments, and then feeds an interaction matrix into a Neu-IR model, DeepTileBars, to obtain the final ranking scores. DeepTileBars models the relevance signals occurring at different granularities in a document’s topic hierarchy. It better captures the discourse structure of a document and thus the matching patterns. Although its design and implementation are light-weight, DeepTileBars outperforms other state-of-the-art Neu-IR models on benchmark datasets including the Text REtrieval Conference (TREC) 2010-2012 Web Tracks and LETOR 4.0.

Convergence of Learning Dynamics in Information Retrieval Games

We consider a game-theoretic model of information retrieval with strategic authors. We examine two different utility schemes: authors who aim at maximizing exposure and authors who want to maximize active selection of their content (i.e., the number of clicks). We introduce the study of author learning dynamics in such contexts. We prove that under the probability ranking principle (PRP), which forms the basis of the current state-of-the-art ranking methods, any betterresponse learning dynamics converges to a pure Nash equilibrium. We also show that other ranking methods induce a strategic environment under which such a convergence may not occur.

A Deep Look into neural ranking models for information retrieval

Ranking models lie at the heart of research on information retrieval (IR). During the past decades, different techniques have been proposed for constructing ranking models, from traditional heuristic methods, probabilistic methods, to modern machine learning methods. Recently, with the advance of deep learning technology, we have witnessed a growing body of work in applying shallow or deep neural networks to the ranking problem in IR, referred to as neural ranking models in this paper. The power of neural ranking models lies in the ability to learn from the raw text inputs for the ranking problem to avoid many limitations of hand-crafted features. Neural networks have sufficient capacity to model complicated tasks, which is needed to handle the complexity of relevance estimation in ranking. Since there have been a large variety of neural ranking models proposed, we believe it is the right time to summarize the current status, learn from existing methodologies, and gain some insights for future development. In contrast to existing reviews, in this survey, we will take a deep look into the neural ranking models from different dimensions to analyze their underlying assumptions, major design principles, and learning strategies. We compare these models through benchmark tasks to obtain a comprehensive empirical understanding of the existing techniques. We will also discuss what is missing in the current literature and what are the promising and desired future directions.