Information Retrieval Approach Research Articles

Enhancing Suggestion Detection in Online User Reviews through Integrated Information Retrieval and Deep Learning Approaches

In the aftermath of the COVID-19 pandemic, using web platforms as a communication medium and decision-making tool in online commerce has become widely acknowledged. User-generated comments, reflecting positive and negative sentiments towards specific items, serve as invaluable indicators, offering recommendations for product and organizational improvements. Consequently, the extraction of suggestions from mined opinions can enhance the efficacy of companies and organizations in this domain. Prevailing research in suggestion mining predominantly employs rule-based methodologies and statistical classifiers, relying on manually identified features. However, a recent trend has emerged wherein researchers explore solutions grounded in deep learning tools and techniques. This study aims to employ information retrieval techniques for the automated identification of suggestions. To this end, various methodologies, including distance measurement approaches, multilayer perceptron neural networks, support vector machines, regression logistics, convolutional neural networks utilizing TF-IDF, Bag of Words (BOW), and Word2Vec vectors, along with keyword extraction, have been integrated. The proposed approach is assessed using the SemEval2019 dataset to extract suggestions from the textual content of online user reviews. The obtained results demonstrate a notable enhancement in the F1 score, reaching 0.76 compared to prior research. The experiments further suggest that information retrieval-based approaches exhibit promising potential for this specific task.

Deep Is Better? An Empirical Comparison of Information Retrieval and Deep Learning Approaches to Code Summarization

Code summarization aims to generate short functional descriptions for source code to facilitate code comprehension. While Information Retrieval (IR) approaches that leverage similar code snippets and corresponding summaries have led the early research, Deep Learning (DL) approaches that use neural models to capture statistical properties between code and summaries are now mainstream. Although some preliminary studies suggest that IR approaches are more effective in some cases, it is currently unclear how effective the existing approaches can be in general, where and why IR/DL approaches perform better, and whether the integration of IR and DL can achieve better performance. Consequently, there is an urgent need for a comprehensive study of the IR and DL code summarization approaches to provide guidance for future development in this area. This article presents the first large-scale empirical study of 18 IR, DL, and hybrid code summarization approaches on five benchmark datasets. We extensively compare different types of approaches using automatic metrics, we conduct quantitative and qualitative analyses of where and why IR and DL approaches perform better, respectively, and we also study hybrid approaches for assessing the effectiveness of integrating IR and DL. The study shows that the performance of IR approaches should not be underestimated, that while DL models perform better in predicting tokens from method signatures and capturing structural similarities in code, simple IR approaches tend to perform better in the presence of code with high similarity or long reference summaries, and that existing hybrid approaches do not perform as well as individual approaches in their respective areas of strength. Based on our findings, we discuss future research directions for better code summarization.

Joint Representation Learning for Retrieval and Annotation of Genomic Interval Sets.

As available genomic interval data increase in scale, we require fast systems to search them. A common approach is simple string matching to compare a search term to metadata, but this is limited by incomplete or inaccurate annotations. An alternative is to compare data directly through genomic region overlap analysis, but this approach leads to challenges like sparsity, high dimensionality, and computational expense. We require novel methods to quickly and flexibly query large, messy genomic interval databases. Here, we develop a genomic interval search system using representation learning. We train numerical embeddings for a collection of region sets simultaneously with their metadata labels, capturing similarity between region sets and their metadata in a low-dimensional space. Using these learned co-embeddings, we develop a system that solves three related information retrieval tasks using embedding distance computations: retrieving region sets related to a user query string, suggesting new labels for database region sets, and retrieving database region sets similar to a query region set. We evaluate these use cases and show that jointly learned representations of region sets and metadata are a promising approach for fast, flexible, and accurate genomic region information retrieval.

PhraseMap: Attention-Based Keyphrases Recommendation for Information Seeking.

Many Information Retrieval (IR) approaches have been proposed to extract relevant information from a large corpus. Among these methods, phrase-based retrieval methods have been proven to capture more concrete and concise information than word-based and paragraph-based methods. However, due to the complex relationship among phrases and a lack of proper visual guidance, achieving user-driven interactive information-seeking and retrieval remains challenging. In this study, we present a visual analytic approach for users to seek information from an extensive collection of documents efficiently. The main component of our approach is a PhraseMap, where nodes and edges represent the extracted keyphrases and their relationships, respectively, from a large corpus. To build the PhraseMap, we extract keyphrases from each document and link the phrases according to word attention determined using modern language models, i.e., BERT. As can be imagined, the graph is complex due to the extensive volume of information and the massive amount of relationships. Therefore, we develop a navigation algorithm to facilitate information seeking. It includes (1) a question-answering (QA) model to identify phrases related to users' queries and (2) updating relevant phrases based on users' feedback. To better present the PhraseMap, we introduce a resource-controlled self-organizing map (RC-SOM) to evenly and regularly display phrases on grid cells while expecting phrases with similar semantics to stay close in the visualization. To evaluate our approach, we conducted case studies with three domain experts in diverse literature. The results and feedback demonstrate its effectiveness, usability, and intelligence.

DocFlow: A Visual Analytics System for Question-Based Document Retrieval and Categorization.

A systematic review (SR) is essential with up-to-date research evidence to support clinical decisions and practices. However, the growing literature volume makes it challenging for SR reviewers and clinicians to discover useful information efficiently. Many human-in-the-loop information retrieval approaches (HIR) have been proposed to rank documents semantically similar to users' queries and provide interactive visualizations to facilitate document retrieval. Given that the queries are mainly composed of keywords and keyphrases retrieving documents that are semantically similar to a query does not necessarily respond to the clinician's need. Clinicians still have to review many documents to find the solution. The problem motivates us to develop a visual analytics system, DocFlow, to facilitate information-seeking. One of the features of our DocFlow is accepting natural language questions. The detailed description enables retrieving documents that can answer users' questions. Additionally, clinicians often categorize documents based on their backgrounds and with different purposes (e.g., populations, treatments). Since the criteria are unknown and cannot be pre-defined in advance, existing methods can only achieve categorization by considering the entire information in documents. In contrast, by locating answers in each document, our DocFlow can intelligently categorize documents based on users' questions. The second feature of our DocFlow is a flexible interface where users can arrange a sequence of questions to customize their rules for document retrieval and categorization. The two features of this visual analytics system support a flexible information-seeking process. The case studies and the feedback from domain experts demonstrate the usefulness and effectiveness of our DocFlow.

Two-Level Information-Retrieval-Based Model for Bug Localization Based on Bug Reports

Software bugs are a noteworthy concern for developers and maintainers. When a failure is detected late, it costs more to be fixed. To repair the bug that caused the software failure, the location of the bug must first be known. The process of finding the defective source code elements that led to the failure of the software is called bug localization. Effective approaches for automatically locating bugs using bug reports are highly desirable, as they would reduce bug-fixing time, consequently lowering software maintenance costs. With the increasing size and complexity of software projects, manual bug localization methods have become complex, challenging, and time-consuming tasks, which motivates research on automated bug localization techniques. This paper introduces a novel bug localization model, which works on two levels. The first level localizes the buggy classes using an information retrieval approach and it has two additional sub-phases, namely the class-level feature scoring phase and the class-level final score and ranking phase, which ranks the top buggy classes. The second level localizes the buggy methods inside these classes using an information retrieval approach and it has two sub-phases, which are the method-level feature scoring phase and the method-level final score and ranking phase, which ranks the top buggy methods inside the localized classes. A model is evaluated using an AspectJ dataset, and it can correctly localize and rank more than 350 classes and more than 136 methods. The evaluation results show that the proposed model outperforms several state-of-the-art approaches in terms of the mean reciprocal rank (MRR) metrics and the mean average precision (MAP) in class-level bug localization.

Empirical Study And Analysis Of Software Bug Localization Approaches Using Deep Learning

Bugs comes in high volume and documenting them properly in a specific format in the bug reports is difficult. Locating these bugs in correctly identified buggy files is a challenging task which needs to be automated. Numerous tools and techniques are proposed by researchers to support developers and testers to detect buggy files and automate the process of bug localization with greater accuracy. Recent research deal with the automation of Bug Localization process by using different techniques and tools. In this paper, we presented a comprehensive review of few papers in the domain of bug localization. This review helps us to know the benchmark datasets that are used in this process of bug localization, the major techniques that are worked upon, the findings and evaluation criteria and the architecture of various models and frameworks developed by researchers to automate the task of bug localization. This paper works on IR and DNN approaches and attempted to improve the previous results of DNNLOC successfully. The optimized version of the model improved the accuracy from 0.815 to 0.969 for enhanced rvsm and 0.83 to 0.971 for enhanced dnn. It is apparent that information retrieval approach and deep learning approach complement together in the domain of bug localization.

Annotate and retrieve in vivo images using hybrid self-organizing map

AbstractMultimodal retrieval has gained much attention lately due to its effectiveness over uni-modal retrieval. For instance, visual features often under-constrain the description of an image in content-based retrieval; however, another modality, such as collateral text, can be introduced to abridge the semantic gap and make the retrieval process more efficient. This article proposes the application of cross-modal fusion and retrieval on real in vivo gastrointestinal images and linguistic cues, as the visual features alone are insufficient for image description and to assist gastroenterologists. So, a cross-modal information retrieval approach has been proposed to retrieve related images given text and vice versa while handling the heterogeneity gap issue among the modalities. The technique comprises two stages: (1) individual modality feature learning; and (2) fusion of two trained networks. In the first stage, two self-organizing maps (SOMs) are trained separately using images and texts, which are clustered in the respective SOMs based on their similarity. In the second (fusion) stage, the trained SOMs are integrated using an associative network to enable cross-modal retrieval. The underlying learning techniques of the associative network include Hebbian learning and Oja learning (Improved Hebbian learning). The introduced framework can annotate images with keywords and illustrate keywords with images, and it can also be extended to incorporate more diverse modalities. Extensive experimentation has been performed on real gastrointestinal images obtained from a known gastroenterologist that have collateral keywords with each image. The obtained results proved the efficacy of the algorithm and its significance in aiding gastroenterologists in quick and pertinent decision making.

A Purely Entity-Based Semantic Search Approach for Document Retrieval

Over the past decade, knowledge bases (KB) have been increasingly utilized to complete and enrich the representation of queries and documents in order to improve the document retrieval task. Although many approaches have used KB for such purposes, the problem of how to effectively leverage entity-based representation still needs to be resolved. This paper proposes a Purely Entity-based Semantic Search Approach for Information Retrieval (PESS4IR) as a novel solution. The approach includes (i) its own entity linking method and (ii) an inverted indexing method, and for document retrieval and ranking, (iii) an appropriate ranking method is designed to take advantage of all the strengths of the approach. We report the findings on the performance of our approach, which is tested by queries annotated by two known entity linking tools, REL and DBpedia-Spotlight. The experiments are performed on the standard TREC 2004 Robust and MSMARCO collections. By using the REL method on the Robust collection, for the queries whose terms are all annotated and whose average annotation scores are greater than or equal to 0.75, our approach achieves the maximum nDCG@5 score (1.00). Also, it is shown that using PESS4IR alongside another document retrieval method would improve performance, unless that method alone achieves the maximum nDCG@5 score for those highly annotated queries.

Information retrieval algorithms and neural ranking models to detect previously fact-checked information

Although in the last decade several fact-checking organizations have emerged to verify misinformation, fake news has continued to proliferate, especially through social media platforms. Even though adopting improved detection strategies is of utmost importance, the fact-checking process could be optimized by verifying whether a claim has been previously fact-checked. Despite some ad-hoc information retrieval approaches having been recently proposed, the utility of modern (neural) retrieval systems have not been investigated yet. In this paper, we consider the standard two-phases retriever-reranker architecture and benchmark different state-of-the-art techniques from the information retrieval and Q&A literature. We design several experiments on a real-world Twitter dataset to analyze the efficiency and the effectiveness of the benchmark approaches. Our results show that combining standard and neural approaches is the most promising research direction to improve retrievers performance and that complex (neural) rerankers might still be efficient in practice since there is no need to process a high number of documents to improve ranking performance.

A structured approach to information retrieval improved identification of funding and researchers’ conflicts of interest in trials included in Cochrane reviews

ObjectivesTo compare the contemporary Cochrane review approach for retrieving information on trial funding and researchers’ conflicts of interest with a structured approach for information retrieval. Study Design and SettingMethodological study of 100 Cochrane reviews from August to December 2020 and one randomly selected trial from each review. Reporting of trial funding and researchers’ conflicts of interest in reviews was compared with information identified using a structured retrieval process, and time to retrieve information was noted. We also formulated a guide to systematic reviewers for efficient information retrieval. ResultsSixty-eight of 100 Cochrane reviews reported trial funding and 24 reported trial researchers’ conflicts of interest. A simple structured approach, searching only trial publications (including conflicts of interest disclosure forms), identified funding for 16 additional trials and conflicts of interest information for 39 additional trials. A comprehensive structured approach, searching multiple information sources, identified funding for two additional trials and conflicts of interest for 14 additional trials. The median time to retrieve information was 10 minutes per trial (interquartile range: 7–15) for the simple approach and 20 minutes (11–43) for the comprehensive approach. ConclusionA structured information retrieval approach improves identification of funding and researchers’ conflicts of interest in trials included in Cochrane reviews.

SPRF: A semantic Pseudo-relevance Feedback enhancement for information retrieval via ConceptNet

Pseudo-relevance feedback is a widely acclaimed technique for information retrieval. However, traditional information retrieval approaches typically process the original query into individual terms, often overlooking the consideration of the semantic information in the query term itself, by instead focusing on features such as term frequency and inverse document frequency. In this paper, we propose a new semantic-based Pseudo-relevance Feedback model (SPRF) based on the PRF framework. Our SPRF model leverages ConceptNet to provide comprehensive semantic information between terms. It not only considers the query’s importance in the collection but also integrates its semantic information into the PRF framework to enhance the selection of query expansion terms, leading to more precise feedback documents for users. A series of experimental results show that our proposed SPRF model is feasible. Our model achieves good performance in terms of the MAP, P@10, NDCG and MRR metrics and demonstrates advantages over the baseline models, the state-of-the-art models and several neural network-based methods. By comparing and analyzing the methods in a sample case, the extended terms resulting from the proposed model are shown to be in better semantic agreement with the given query.

Unsupervised Numerical Information Extraction via Exploiting Syntactic Structures

Numerical information plays an important role in various fields such as scientific, financial, social, statistics, and news. Most prior studies adopt unsupervised methods by designing complex handcrafted pattern-matching rules to extract numerical information, which can be difficult to scale to the open domain. Other supervised methods require extra time, cost, and knowledge to design, understand, and annotate the training data. To address these limitations, we propose QuantityIE, a novel approach to extracting numerical information as structured representations by exploiting syntactic features of both constituency parsing (CP) and dependency parsing (DP). The extraction results may also serve as distant supervision for zero-shot model training. Our approach outperforms existing methods from two perspectives: (1) the rules are simple yet effective, and (2) the results are more self-contained. We further propose a numerical information retrieval approach based on QuantityIE to answer analytical queries. Experimental results on information extraction and retrieval demonstrate the effectiveness of QuantityIE in extracting numerical information with high fidelity.

Pronouns in the Workplace: Developing Sociotechnical Systems for Digitally Mediated Gender Expression

Sharing personal pronouns (such as they/them, she/her, or he/him) in the workplace helps colleagues respectfully address each other. Many companies currently seek to design and implement new software tools to facilitate pronoun-sharing among employees. This paper analyzes the social processes of communication about pronouns in the workplace and identifies best practices for creating and using workplace pronoun-sharing software. We conducted 78 semi-structured qualitative interviews with various stakeholders involved in the launch of pronoun-sharing tools in workplace collaboration software, including transgender and queer people, HR and IT professionals, and LGBTQ advocacy organizations. We used an anthropological approach to qualitatively analyze interview materials and notes from interactions with research participants. We find that sharing personal pronouns is an ongoing communication process rather than a single act of information provision. Pronoun sharing tools encapsulate the tension between dynamic social processes of self-expression and technical systems of classification and information retrieval. People communicate their pronouns differently as they navigate identity expression across social contexts. Sharing pronouns is therefore both an individual expression of self-presentation and a complex act of social communication. Developers must create new methods for building pronoun-sharing tools that equip people to control ongoing social processes of self-expression instead of using an information retrieval approach that treats pronouns merely as stable, unchanging data.

CASBERT: BERT-based retrieval for compositely annotated biosimulation model entities.

Maximising FAIRness of biosimulation models requires a comprehensive description of model entities such as reactions, variables, and components. The COmputational Modeling in BIology NEtwork (COMBINE) community encourages the use of Resource Description Framework with composite annotations that semantically involve ontologies to ensure completeness and accuracy. These annotations facilitate scientists to find models or detailed information to inform further reuse, such as model composition, reproduction, and curation. SPARQL has been recommended as a key standard to access semantic annotation with RDF, which helps get entities precisely. However, SPARQL is unsuitable for most repository users who explore biosimulation models freely without adequate knowledge of ontologies, RDF structure, and SPARQL syntax. We propose here a text-based information retrieval approach, CASBERT, that is easy to use and can present candidates of relevant entities from models across a repository's contents. CASBERT adapts Bidirectional Encoder Representations from Transformers (BERT), where each composite annotation about an entity is converted into an entity embedding for subsequent storage in a list of entity embeddings. For entity lookup, a query is transformed to a query embedding and compared to the entity embeddings, and then the entities are displayed in order based on their similarity. The list structure makes it possible to implement CASBERT as an efficient search engine product, with inexpensive addition, modification, and insertion of entity embedding. To demonstrate and test CASBERT, we created a dataset for testing from the Physiome Model Repository and a static export of the BioModels database consisting of query-entities pairs. Measured using Mean Average Precision and Mean Reciprocal Rank, we found that our approach can perform better than the traditional bag-of-words method.

An efficient ontology model with query execution for accurate document content extraction

<span lang="EN-US">The technique of extracting important documents from massive data collections is known as information retrieval (IR). The dataset provider coupled with the increasing demand for high-quality retrieval results, has resulted in traditional information retrieval approaches being increasingly insufficient to meet the challenge of providing high-quality search results. Research has concentrated on information retrieval and interactive query formation through ontologies in order to overcome these challenges, with a specific emphasis on enhancing the functionality between information and search queries in order to bring the outcome sets closer to the research requirements of users. In the context of document retrieval technologies, it is a process that assists researchers in extracting documents from data collections. It is discussed in this research how to use ontology-based information retrieval approaches and techniques, taking into account the issues of ontology modelling, processing, and the transformation of ontological knowledge into database search queries. In this research work, an efficient optimized ontology model with query execution for content extraction from documents (OOM-QE-CE) is proposed. The existing ontology-to-database transformation and mapping methodologies are also thoroughly examined in terms of data and semantic loss, structural mapping and domain knowledge applicability.</span>

Information retrieval approaches

The area of information retrieval (IR) has taken on increasing importance in recent years. This field is now of interest to large communities in several application domains (security, medicine, aeronautics, etc.). IR studies find relevant information from the semi-unstructured type of data. As the information resources generated after the search can be extensive in quantity and different in quality, it is essential to rank these results according to the degree of relevance. This paper focuses on text information retrieval (TIR) and emphasizes the importance of each IR approach. This study presents insightful aspects of TIR and provides a comparative study between some proposed approaches and models. Each model offers IR advantages and suffers from several limitations.

Detecting non-natural language artifacts for de-noising bug reports

Textual documents produced in the software engineering process are a popular target for natural language processing (NLP) and information retrieval (IR) approaches. However, issue tickets often contain artifacts such as code snippets, log outputs and stack traces. These artifacts not only inflate the issue ticket sizes, but also can this noise constitute a real problem for some NLP approaches, and therefore has to be removed in the pre-processing of some approaches. In this paper, we present a machine learning based approach to classify textual content into natural language and non-natural language artifacts at line level. We show how data from GitHub issue trackers can be used for automated training set generation, and present a custom preprocessing approach for the task of artifact removal. The training sets are automatically created from Markdown annotated issue tickets and project documentation files. We use these generated training sets to train a Markdown agnostic model that is able to classify un-annotated content. We evaluate our approach on issue tickets from projects written in C++, Java, JavaScript, PHP, and Python. Our approach achieves ROC-AUC scores between 0.92 and 0.96 for language-specific models. A multi-language model trained on the issue tickets of all languages achieves ROC-AUC scores between 0.92 and 0.95. The provided models are intended to be used as noise reduction pre-processing steps for NLP and IR approaches working on issue tickets.

Improving Biomedical Information Retrieval with Neural Retrievers

Information retrieval (IR) is essential in search engines and dialogue systems as well as natural language processing tasks such as open-domain question answering. IR serve an important function in the biomedical domain, where content and sources of scientific knowledge may evolve rapidly. Although neural retrievers have surpassed traditional IR approaches such as TF-IDF and BM25 in standard open-domain question answering tasks, they are still found lacking in the biomedical domain. In this paper, we seek to improve information retrieval (IR) using neural retrievers (NR) in the biomedical domain, and achieve this goal using a three-pronged approach. First, to tackle the relative lack of data in the biomedical domain, we propose a template-based question generation method that can be leveraged to train neural retriever models. Second, we develop two novel pre-training tasks that are closely aligned to the downstream task of information retrieval. Third, we introduce the ``Poly-DPR'' model which encodes each context into multiple context vectors. Extensive experiments and analysis on the BioASQ challenge suggest that our proposed method leads to large gains over existing neural approaches and beats BM25 in the small-corpus setting. We show that BM25 and our method can complement each other, and a simple hybrid model leads to further gains in the large corpus setting.

The Problem on Determining "How to Incorporate Knowledge into Information Mining Algorithms"

There is a massive amount of data on the Internet right now, but essentially little classification information. The main issue is figuring out how to incorporate knowledge into information mining algorithms. The authors examine information retrieval approaches and discuss their advantages and disadvantages. Although most Web documents are text oriented, there are plenty of them that contain multimedia elements, which are not easily accessible through common search methods. Web information is dynamic, semi-structured, and interwound with hyperlinks. Several advanced methods for Web information mining are analysed: 1) syntax analysis, 2) metadata-based searching using RDF, 3) knowledge annotation by use of conceptual graphs (CGs), 4) KPS: Keyword, Pattern, Sample search techniques, and 5) techniques of obtaining descriptions by fuzzification and back-propagation. The problem of choosing proper keywords is