Information Retrieval Methods Research Articles

Development of an Evaluation Instrument on Artificial Intelligence Search Tools for Evidence Synthesis

What Was the Question? There are conflicting calls for evidence synthesis producers to adopt or avoid recent Artificial Intelligence (AI) technologies. Our question was: How do we evaluate rapidly evolving AI tools to enhance the production of evidence syntheses and maintain quality standards? What Did We Do? To advance information retrieval science for producing evidence syntheses at Canada’s Drug Agency (CDA-AMC), the Research Information Services team developed a process to evaluate promising AI search tools. We inventoried 51 tools in the fall of 2023, established selection criteria, assessed specific attributes, and built a standalone instrument to support continuous monitoring and evaluation of new tools. What Did We Find? Rapid development of AI search tools requires a flexible evaluation instrument to inform adoption decisions and enable comparison between tools. We identified mandatory and desirable characteristics for suitable AI tools to assist with information retrieval tasks conducted by CDA-AMC. This work enabled the development of a flexible instrument to evaluate novel AI search tools for evidence synthesis. What Does This Mean? CDA-AMC operationalized a replicable process to monitor and evaluate AI search tools. Our approach to evaluating AI technologies will advance information retrieval methods by CDA-AMC, and our evaluation instrument will assist any evidence synthesis producer interested in adopting AI search tools.

Advancing continual lifelong learning in neural information retrieval: Definition, dataset, framework, and empirical evaluation

Continual learning refers to the capability of a machine learning model to learn and adapt to new information, without compromising its performance on previously learned tasks. Although several studies have investigated continual learning methods for neural information retrieval (NIR) tasks, a well-defined task definition is still lacking, and it is unclear how typical learning strategies perform in this context. To address this challenge, a systematic task definition of continual NIR is presented, along with a multiple-topic dataset that simulates continuous information retrieval. A comprehensive continual neural information retrieval framework consisting of typical retrieval models and continual learning strategies is then proposed. Empirical evaluations illustrate that the proposed framework can successfully prevent catastrophic forgetting in neural information retrieval and enhance performance on previously learned tasks. The results also indicate that embedding-based retrieval models experience a decline in their continual learning performance as the topic shift distance and dataset volume of new tasks increase. In contrast, pretraining-based models do not show any such correlation. Adopting suitable learning strategies can mitigate the effects of topic shift and data augmentation in continual neural information retrieval.

Empirical Testing of Alternative Search Methods to Retrieve Utility Values for Health Economic Modelling.

The objective of this study is to compare different information retrieval methods that can be used to identify utility inputs for health economic models. Theusual practice of using systematic review methods was compared with two alternatives (iterative searching and rapid review), using a health technology assessment (HTA) case study in ulcerative colitis (UC). We analysed whether there were differences in the utility values identified when using the alternative search methods. Success was evaluated in terms of time, burden and relevance of identified information. The identified utility values were tested in an executable health economic model developed for UC, and the model results were compared. The usual practice of using systematic review search approaches identified the most publications but was also the least precise method and took longest to complete.The inclusion of data from the different search methods in the model did not lead to different conclusions across search methods. In this case study, usual practice was less efficient and resulted in the same health economic model conclusions as the alternative search methods. Further case studies are required to examine whether this conclusion might be generalisable.

A support system for the detection of abusive clauses in B2C contracts

AbstractMany countries employ systemic methods of protecting consumers from unfair business practices. One such practice is the use of abusive clauses in business-to-consumer (B2C) contracts, which unfairly impose additional obligations on the consumer or deprive them of their due rights. This article presents an information system that utilizes artificial intelligence methods to automate contract analysis and to detect abusive clauses. The goal of the system is to support the entire administrative process, from contract acquisition, through text extraction and the recommendation of potentially abusive clauses, to the generation of official administrative documents that can be sent to court or to the owners of firms. This article focuses on on the components that use machine learning methods. The first is an intelligent crawler that is responsible for automatically detecting contract templates on websites and retrieving them into the system. The second is a document analysis module that implements a clause recommendation algorithm. The algorithm employs transformer-based language models and information retrieval methods to identify abusive passages in text. Our solution achieved first place in a competition on the automatic analysis of B2C contracts organized by the Polish Office of Competition and Consumer Protection (UOKiK), and has since been implemented as an official tool to support the contract analysis process in Poland.

A Cross-Language Information Retrieval Method Based on Multi-Task Learning

This study introduces a novel Cross-Language Information Retrieval (CLIR) method employing multi-task learning and soft parameter sharing to enhance neural retrieval models' feature extraction across languages. The approach integrates an interaction-based neural retrieval model with a semantic-based text classification model, exchanging hidden vectors for richer feature representation. Experimental results across four language pairs—English-Chinese, English-Arabic, English-French, and English-German—demonstrate significant performance improvements. The proposed method achieved the highest Mean Average Precision (MAP) scores: 0.419 for EN-ZH, 0.403 for EN-AR, 0.427 for EN-FR, and 0.441 for EN-DE, surpassing other models like BM25, BPNRM, KNRM, KNRM-Trans, and KNRM-Embed. This research underscores the potential of multi-task learning for CLIR, showcasing improved retrieval performance through semantic information and knowledge transfer.

Capital as an economic category. Elements of capital as accounting objects

Subject. This article examines the category of Capital in its various manifestations, the classification of its types, and the characteristics of capital components as accounting objects. Objectives. The article aims to assess the meaning and role of the category of Capital in economic, financial and accounting aspects, substantiate the importance of understanding the content of capital elements as accounting objects in order to improve the quality of accounting and information support of the organization's management system. Methods. For the study, we used the methods of information retrieval, simulation of situations, analysis and synthesis. Results. Based on the study and generalization of historical and modern approaches to substantiating the category of Capital, the article describes the multifaceted nature of this category, evaluates the content of individual elements of capital, and offers recommendations for the classification of capital in its various manifestations in order to form information in the accounting system and accounting (financial) statements. Conclusions and Relevance. The application of the research results in the educational process and practical activities will contribute to the most thorough and deep understanding of the category of Capital, its complexity and versatility. This can help reflect information on capital in accounting (financial) statements in a more correct way, assess the amount of capital and calculate indicators related to it when analyzing the economic activities of economic entities. All this together will affect the quality of management decisions of the organization's management system. The research materials can be used for the study and teaching of accounting disciplines, as well as in the practical activities of specialists in the field of accounting and finance.

Application Research of Intelligent Search System Based on Clustering Algorithm

In this paper, a multi-agent oriented personalized information retrieval method is proposed. The system is composed of 5 agents and 2 other components. The design highlights the personalization and real-time of the information obtained by users, and has the characteristics of intelligent retrieval and active retrieval. Then this paper studies it from three aspects: user model, system learning model and individual information model. Then a grouping gravity search algorithm is proposed. Compared with conventional gravity search, group gravity search is carried out for a specific decoding strategy. This algorithm is a new gravity search algorithm suitable for packet coding, which makes the packet search process approximate to the iterative optimization of classical gravity algorithm. Finally, the effectiveness of the proposed method is verified on various typical experimental cases. Experiments show that the proposed algorithm has higher classification performance than other intelligent group algorithms.

Towards a Regulatory Framework for Electronic Medical Record Data Visualization.

Physicians struggle to retrieve data from electronic medical records. We evaluated a digital tool that enhances physician efficiency in retrieving and analyzing patient information for treatment decision-making. Our use case is the care of diabetic patients. Evaluation results showed that healthcare providers who used the i4C (Insights for Care) dashboard experienced greater time efficiency than those who used traditional EMR information retrieval methods. A comprehensive evaluation of the i4C Dashboard confirms its effectiveness in facilitating diabetic care data management, as well as its potential application to a wide range of healthcare scenarios. In order to further maximize its effectiveness on clinical efficiency and patient care, future research should focus on improving its usability and scalability.

Web Page Ranking Based on Text Content and Link Information Using Data Mining Techniques

Thanks to the rapid expansion of the Internet, anyone can now access a vast array of information online. However, as the volume of web content continues to grow exponentially, search engines face challenges in delivering relevant results. Early search engines primarily relied on the words or phrases found within web pages to index and rank them. While this approach had its merits, it often resulted in irrelevant or inaccurate results. To address this issue, more advanced search engines began incorporating the hyperlink structures of web pages to help determine their relevance. While this method improved retrieval accuracy to some extent, it still had limitations, as it did not consider the actual content of web pages. The objective of the work is to enhance Web Information Retrieval methods by leveraging three key components: text content analysis, link analysis, and log file analysis. By integrating insights from these multiple data sources, the goal is to achieve a more accurate and effective ranking of relevant web pages in the retrieved document set, ultimately enhancing the user experience and delivering more precise search results the proposed system was tested with both multi-word and single-word queries, and the results were evaluated using metrics such as relative recall, precision, and F-measure. When compared to Google’s PageRank algorithm, the proposed system demonstrated superior performance, achieving an 81% mean average precision, 56% average relative recall, and a 66% F-measure.

The significance and role of contra-asset accounts in accounting theory and practice

Subject. This article deals with the issues of classification of accounts by purpose and structure, as well as the application of contra-asset accounts. Objectives. The article aims to assess the significance and role of contra-asset accounts in the accounting system in its theoretical and practical aspects, as well as clarify the name of this classification group of accounts. Methods. For the study, we used the methods of information retrieval, generalization, modeling, analysis and synthesis. Results. Based on the study and generalization of theoretical provisions that reveal the issues and problems of the use of accounting accounts, the article presents proposals for clarifying the classification of contra-asset accounts and determines their theoretical and practical significance in the formation of information in the accounting system and financial reporting. Conclusions and Relevance. The use of the results of the study can help deeper understand the significance and role of contra-asset accounts and the multifunctional nature of information provided by their use in the formation of accounting (financial) statements and the solution of various management tasks for the management of processes related to the acquisition, use, storage, assessment of the state of objects recorded in the main accounts. This will create prerequisites for the competent use of this group of accounts in the practical activities of accountants. The materials of the study can be used to study and teach the theory of accounting, financial accounting, accounting (financial) reporting, as well as in the practical activities of specialists in the field of accounting.

Information Retrieval and Machine Learning Methods for Academic Expert Finding

In the context of academic expert finding, this paper investigates and compares the performance of information retrieval (IR) and machine learning (ML) methods, including deep learning, to approach the problem of identifying academic figures who are experts in different domains when a potential user requests their expertise. IR-based methods construct multifaceted textual profiles for each expert by clustering information from their scientific publications. Several methods fully tailored for this problem are presented in this paper. In contrast, ML-based methods treat expert finding as a classification task, training automatic text classifiers using publications authored by experts. By comparing these approaches, we contribute to a deeper understanding of academic-expert-finding techniques and their applicability in knowledge discovery. These methods are tested with two large datasets from the biomedical field: PMSC-UGR and CORD-19. The results show how IR techniques were, in general, more robust with both datasets and more suitable than the ML-based ones, with some exceptions showing good performance.

TenseMusic: An automatic prediction model for musical tension.

The perception of tension and release dynamics constitutes one of the essential aspects of music listening. However, modeling musical tension to predict perception of listeners has been a challenge to researchers. Seminal work demonstrated that tension is reported consistently by listeners and can be accurately predicted from a discrete set of musical features, combining them into a weighted sum of slopes reflecting their combined dynamics over time. However, previous modeling approaches lack an automatic pipeline for feature extraction that would make them widely accessible to researchers in the field. Here, we present TenseMusic: an open-source automatic predictive tension model that operates with a musical audio as the only input. Using state-of-the-art music information retrieval (MIR) methods, it automatically extracts a set of six features (i.e., loudness, pitch height, tonal tension, roughness, tempo, and onset frequency) to use as predictors for musical tension. The algorithm was optimized using Lasso regression to best predict behavioral tension ratings collected on 38 Western classical musical pieces. Its performance was then tested by assessing the correlation between the predicted tension and unseen continuous behavioral tension ratings yielding large mean correlations between ratings and predictions approximating r = .60 across all pieces. We hope that providing the research community with this well-validated open-source tool for predicting musical tension will motivate further work in music cognition and contribute to elucidate the neural and cognitive correlates of tension dynamics for various musical genres and cultures.

Age- and Sex-Related Differences in Music Requests on a Radio Show for Children

In research on the development of musical preferences, children are often asked either to evaluate musical pieces that have been previously selected by adult experimenters, or to give self-reports on preferences for different musical styles. This study involved analyzing a sample of 1,412 freely and publicly expressed music requests from children aged between 4 and 11 years taken from a German radio program, with regard to age- and sex-specific differences. The music was categorized into genres, as listed on Spotify, and examined using methods of music information retrieval provided by the Spotify Developer application programming interface. Results showed that, at younger ages, the requests were generally more evenly distributed across different genres. Regarding single genres, we observed small positive relationships between age and the likelihood of requesting the genres pop or electro and small negative relationships between age and the likelihood of requesting A capella, German songwriter, indie, theme songs, or children's music. Furthermore, we found that boys requested significantly more rock or hip-hop, whereas girls had a higher tendency to ask for pop. Finally, age- (but no sex-) related differences regarding the Spotify features of valence and liveness of the requested music were found, which were related to the preference for children's music at younger ages. The results thus suggest that previous findings regarding differences between boys and girls and an increasing formation of distinct genre preferences during infancy also apply to single song requests made on a radio show.

Category Decomposition-based Within Pixel Information Retrieval Method and its Application to Partial Cloud Extraction from Satellite Imagery Pixels

Category Decomposition-based Within Pixel Information Retrieval Method and its Application to Partial Cloud Extraction from Satellite Imagery Pixels

Developing a Base Domain Ontology from Geoscience Report Collection to Aid in Information Retrieval towards Spatiotemporal and Topic Association

The efficient and precise retrieval of desired information from extensive geological databases is a prominent and pivotal focus within the realm of geological information services. Conventional information retrieval methods primarily rely on keyword matching approaches, which often overlook the contextual and semantic aspects of the keywords, consequently impeding the retrieval system’s ability to accurately comprehend user query requirements. To tackle this challenge, this study proposes an ontology-driven information-retrieval framework for geological data that integrates spatiotemporal and topic associations. The framework encompasses the development of a geological domain ontology, extraction of key information, establishment of a multi-feature association and retrieval framework, and validation through a comprehensive case study. By employing the proposed framework, users are empowered to actively and automatically retrieve pertinent information, simplifying the information access process, mitigating the burden of comprehending information organization and software application models, and ultimately enhancing retrieval efficiency.

FRAILTY ASSESSMENT FROM PAPER TO ONLINE: THE E-FI-CGA WEB APP

Abstract Background Early frailty assessment is crucial for improving care for older adults through enhancing frailty-informed care planning and execution. To increase the access and use of the established electronic Frailty Index based on the Comprehensive Geriatric Assessment tool, we developed an online software application: the web-based eFI-CGA. Methods End-user requirements primarily adapted the standalone version of the eFI-CGA and covered assessment competency, interface familiarity, accessibility and convenient use by front-line health professionals. Web app-bounded features included user-account authentication and authorization, user-space management, record-search and retrieval, and functions for data management, analysis and display. The web app development used the Microsoft Azure web server with the ASP.NET framework, combined Webforms and Model-View-Controller architecture, and C# programming language with standard client-side libraries incorporating scripting and mark-up languages. Results The web-based eFI-CGA was technically released and accessible online (efi-cga). Tests on the web pages for all the web pages (e.g., Home, Signup Login, Assessment, Search, and Analysis) using large sizes of systematically designed test data and mocked patient cohorts showed that the web software functions meet the requirements with 100% accuracy. Conclusion The web-based eFI-CGA provides a valuable remote frailty assessment and information retrieval method for healthcare professionals, promoting effective multidisciplinary integrated care of older adults. Further work is needed to allow the safe use of the web app. We are also planning a patient-oriented frailty assessment tool for community-dwelling older adults.

Investigating the current state of the Russian market for drugs treating infectious and inflammatory conditions of oral cavity and oropharynx to demonstrate the need for new drug development

Objective: to assess the relevance and commercial prospects of bringing to market a drug for the local treatment of infectious and inflammatory diseases of oral cavity and oropharynx.Material and methods. The objects of the study were the drug nomenclatures in the State Register of Medicines, the Register of Medicines of Russia, as well as data from the Federal Service for Surveillance in Healthcare. The analysis of scientific publications in the field of marketing research of drugs, found in eLibrary.ru and PubMed/MEDLINE databases was carried out. For analyzing the structure and marketing research of drugs in the study group, specific methods of information retrieval, content analysis, description, data aggregation, ranking, data grouping method, system approach, as well as such methods of marketing analysis as determining the breadth and depth of the assortment, the renewal index, the vital importance index, SWOT analysis (strengths, weaknesses, opportunities, threats) were used.Results. The assortment of drugs for the local treatment of oral cavity and oropharyngeal infectious and inflammatory diseases was structured by the nature of the active substance, type of dosage form, and country of origin. The main trends in the development of the market of drugs of this group were identified. An analysis of the range of drugs using marketing research methods made it possible to establish the breadth and completeness of the range. The calculated renewal and vitality indices confirmed the relevance of developing new drugs with the considered pharmacotherapeutic action.Conclusion. The study revealed the main trends in the development of the market of drugs of the action in question. The SWOT analysis showed a possible strategy for developing and launching a new drug on the market.

Rethinking E-Commerce Search

E-commerce search and recommendation usually operate on structured data such as product catalogs and taxonomies. However, creating better search and recommendation systems often requires a large variety of unstructured data including customer reviews and articles on the web. Traditionally, the solution has always been converting unstructured data into structured data through information extraction, and conducting search over the structured data. However, this is a costly approach that often has low quality. In this paper, we envision a solution that does entirely the opposite. Instead of converting unstructured data (web pages, customer reviews, etc) to structured data, we instead convert structured data (product inventory, catalogs, taxonomies, etc) into textual data, which can be easily integrated into the text corpus that trains LLMs. Then, search and recommendation can be performed through a Q/A mechanism through an LLM instead of using traditional information retrieval methods over structured data.

A Knowledge Graph Embedding Model Based on Cyclic Consistency—Cyclic_CKGE

Most of the existing medical knowledge maps are incomplete and need to be completed/predicted to obtain a complete knowledge map. To solve this problem, we propose a knowledge graph embedding model (Cyclic_CKGE) based on cyclic consistency. The model first uses the “graph” constructed with the head entity and relationship to predict the tail entity, and then uses the “inverse graph” constructed with the tail entity and relationship to predict the head entity. Finally, the semantic space distance between the head entity and the original head entity should be very close, which solves the reversibility problem of the network. The Cyclic_CKGE model with a parameter of 0.46 M has the best results on FB15k-237, reaching 0.425 Hits@10. Compared with the best model R-GCN, its parameter exceeds 8 M and reaches 0.417 Hits@10. Overall, Cyclic_CKGE’s parametric efficiency is more than 17 times that of R-GCNs and more than 8 times that of DistMult. In order to better show the practical application of the model, we construct a visual medical information platform based on a medical knowledge map. The platform has three kinds of disease information retrieval methods: conditional query, path query and multi-symptom disease inference. This provides a theoretical method and a practical example for realizing knowledge graph visualization.

Span prompt dense passage retrieval for Chinese open domain question answering

Dense passage retrieval is a popular method in information retrieval recently, especially in open domain question answering. It aims to retrieve related articles from massive passages to answer the question. Retriever can increase retrieval speed with less loss of accuracy compared to other methods. However, the pretrained language models used in recent research are often ineffective in semantic embedding, which will reduce accuracy. In addition, we find that contrastive learning will diverge the representation space, and Siamese models with independent parameters on both sides will decrease generalization performance. Therefore, we propose span prompt dense passage retrieval (SPDPR) based on span mask prompt tuning and parameter sharing in Chinese open-domain dense retrieval. This model can generate more efficient representation embeddings and effectively counteract the separation tendency between positive samples. We evaluate the effectiveness of SPDPR in DYKzh, as well as two Chinese datasets. SPDPR surpasses all SOTAs implemented in DYKzh and achieves a competitive result in other datasets.