Information Retrieval Research Articles

Microservices Patterns Recommendation based on Information Retrieval

Software developers encounter recurring problems during software development, which can be solved using proven solutions known as design patterns. Microservices architecture, a Service-Oriented Architecture (SOA) variant, presents common communication, deployment, and service definition challenges. However, selecting the appropriate design pattern from a vast pool of patterns to solve a problem is difficult for novice and experienced developers. This paper proposes a recommendation tool based on Information Retrieval (IR) to assist developers in choosing the suitable microservices pattern to solve a given problem. The tool leverages textual descriptions given by developers to provide relevant indications of microservices patterns. It has been evaluated using both toy and industrial problems, demonstrating promising results. The results showed that the tool was able to solve 60% of toy design problems, indicating that it can provide valuable and accurate recommendations. Furthermore, tests with industrial problems revealed that over 70% of the recommended patterns helped to address the problems at hand. Interviews with developers who work in the software industry corroborated the recommendation tool’s effectiveness and practicality.

Editorial

Dear Readers, It gives me great pleasure to announce the tenth regular issue of 2024. I would like to thank all the authors for their sound research papers and the editorial board and our guest reviewers for their extremely valuable reviews and suggestions for improvement. These contributions and the generous support of the consortium members enable us to run our journal and maintain its quality. I would also like to thank our broader community for reading and incorporating sound J.UCS papers into their research. Still, I would like to expand our editorial board: If you are a tenured associate professor or above with a good publication record, please apply to join our editorial board. We are also interested in receiving high-quality proposals for special issues on new topics and emerging trends. In this regular issue, I am very pleased to introduce six accepted papers involving 17 authors from six different countries (Argentina, Brazil, Ecuador, Hong Kong, India, Saudi Arabia). Álex dos Santos Moura, Fábio Gomes Rocha, and Michel S. Soares from Brazil propose in their research a recommendation tool based on information retrieval to assist developers in choosing the suitable microservices pattern to solve a given problem. In a collaborative research effort between Ecuador and Argentina, Gonzalo P. Espinel-Mena, José L. Carrillo-Medina, Eddie E. Galarza, and Mario Matias Urbieta discuss a systematic mapping of configuration management activities in software product line. Edward Kai Fung Dang, Robert Wing Pong Luk, and Qing Li from Hong Kong contribute to the forum for negative results with their study on word bigrams for pseudo-relevance feedback in information retrieval. Prateek Thakral and Yugal Kumar from India present in their work an improved water flow optimizer (IWFO) algorithm for cluster analysis that can address the issues of traditional and heuristic algorithms. Samit Bhanja, Banani Ghose, and Abhishek Das from India highlight their findings on multi-step-ahead time series forecasting using a deep learning and fuzzy time series-based error correction method. And last but not least, Abdullilah A. Alotaibi and Salman A. AlQahtani from Saudi Arabia present in their research an intelligent distributed channel selection framework with hybrid mode selection for interference mitigation in D2D based 5G networks.  Enjoy Reading! Best regards,  Christian Gütl, Managing Editor Graz University of Technology, Graz, Austria

Optimizing the Utilization of Generative Artificial Intelligence (AI) in the AEC Industry: ChatGPT Prompt Engineering and Design

Generative Artificial Intelligence (AI) holds significant potential for revolutionizing the Architecture, Engineering, and Construction (AEC) industry by automating complex tasks such as construction scheduling, hazard recognition, resource leveling, information retrieval from BIM, etc. However, realizing this potential requires a strategic approach to ensure effective utilization and maximum benefit. This paper presents guidelines for prompt design and engineering to elicit desired responses from ChatGPT, a Generative AI tool, in AEC applications. Key steps include understanding user intent, leveraging model capabilities, and optimizing prompt structures. By following these guidelines, stakeholders in the AEC industry can harness the power of Generative AI to improve construction scheduling processes, increase project efficiency, and ultimately drive innovation and growth in the industry. Several illustrative examples on construction scheduling and hazard recognition are provided to demonstrate the methodology proposed in this research. It is concluded that Generative AI, when effectively utilized, significantly enhances project scheduling and hazard recognition capability in the AEC industry with minimal error.

A Personal Health Record for patients with multiple sclerosis: A 2-year evaluation study

Abstract Background Multiple sclerosis (MS) is a chronic disease impacting the central nervous system, affecting 2.8 million people worldwide. Effective self-management is crucial for treating chronic diseases, including MS. Personal health records (PHRs) show promise in supporting self-management, potentially empowering patients and enhancing their engagement in treatment and health. However, implementation challenges persist. Objectives This study aimed to) assess the feasibility and usability of the PHR according to patients and healthcare professionals (HCPs), and 2) evaluate implementation determinants, and exploring preliminary effects on a range of outcomes related to the quality, efficiency, and costs of care, as well as health-related outcomes. Methods A mixed-methods study was conducted in a Dutch hospital. Quantitative data from 80 patients and 12 HCPs were collected via questionnaires at baseline (T0), after one (T1) and 2 years (T2). Focus group interview were conducted at T2 with 7 patients and 4 HCPs. Results Most patients never logged in during the first year and logged in a couple of times during the second year, with 15 minutes per log-in session. The HCPs mainly logged in a couple of times per year, averaging six minutes per session. Patients’ and HCPs’ usability and satisfaction scores were below average and moderate respectively. Various facilitators and barriers were identified, including knowledge gaps, staff capacity issues, and ICT obstacles. No significant differences were found in preliminary effects. Qualitative data highlighted benefits in terms of gaining insight into health data but revealed challenges with log-in issues and information retrieval. Conclusions Implementation challenges hindered PHR adoption. The evolving nature of PHRs requires ongoing evaluation and adaptation to optimize their potential benefits. Utilizing a participatory design approach and a dedicated implementation team could enhance adoption and maximize benefits. Key messages • A personal health record (PHR) for patients with MS was evaluated within the care setting of a Dutch hospital, demonstrating limited usability and moderate satisfaction. • Diverse obstacles impeded the adoption and uptake of the PHR, but potential solutions lie in continuous development with a participatory design approach and dedicated implementation team.

Enhancing integration testing efficiency through AI-driven combined structural and textual class coupling metric

Integration testing, a critical and resource-intensive phase in the software development lifecycle, can account for up to a high percentage of the total testing cost. Identifying classes with high coupling is crucial for efficient integration testing, as these classes are more susceptible to the impact of maintenance-related changes. This research introduces a novel metric called <i>combined structural and textual class coupling</i> (CSTCC), which harnesses the power of artificial intelligence (AI) techniques to predict and rank the most critical classes in an object-oriented software system. CSTCC integrates structural coupling metrics with <i>latent semantic indexing</i> (LSI)-based textual coupling, providing a comprehensive measure of class coupling. LSI, an information retrieval technique, analyses the semantic relationships between classes based on their textual content, enabling CSTCC to capture both structural and conceptual dependencies, resulting in a more accurate identification of high-risk classes. The effectiveness of the proposed approach is rigorously evaluated using mutation testing on four Java open-source projects, and the results demonstrate that test cases developed based on CSTCC achieve high mutation scores, indicating their ability to detect a significant percentage of integration faults. By focusing testing efforts on high-coupling classes identified by CSTCC, developers can potentially save time and cost during integration testing. The results demonstrate that test cases developed based on CSTCC achieve high mutation scores, ranging from 98% to 100%, indicating their ability to detect a significant percentage of integration faults. Additionally, the approach results in substantial efficiency gains, with a notable reduction in the number of test cases needed, saving up to 33.3% of the testing effort in some cases. By focusing testing efforts on high-coupling classes identified by CSTCC, developers can potentially save time and cost during integration testing. The CSTCC metric provides a novel and effective approach to prioritize testing resources and improve the efficiency of integration testing in object-oriented software systems.

Study on the Method of Vineyard Information Extraction Based on Spectral and Texture Features of GF-6 Satellite Imagery

Accurately identifying the distribution of vineyard cultivation is of great significance for the development of the grape industry and the optimization of planting structures. Traditional remote sensing techniques for vineyard identification primarily depend on machine learning algorithms based on spectral features. However, the spectral reflectance similarities between grapevines and other orchard vegetation lead to persistent misclassification and omission errors across various machine learning algorithms. As a perennial vine plant, grapes are cultivated using trellis systems, displaying regular row spacing and distinctive strip-like texture patterns in high-resolution satellite imagery. This study selected the main oasis area of Turpan City in Xinjiang, China, as the research area. First, this study extracted both spectral and texture features based on GF-6 satellite imagery, subsequently employing the Boruta algorithm to discern the relative significance of these remote sensing features. Then, this study constructed vineyard information extraction models by integrating spectral and texture features, using machine learning algorithms including Naive Bayes (NB), Support Vector Machines (SVMs), and Random Forests (RFs). The efficacy of various machine learning algorithms and remote sensing features in extracting vineyard information was subsequently evaluated and compared. The results indicate that three spectral features and five texture features under a 7 × 7 window have significant sensitivity to vineyard recognition. These spectral features include the Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index (EVI), and Normalized Difference Water Index (NDWI), while texture features include contrast statistics in the near-infrared band (B4_CO) and the variance statistic, contrast statistic, heterogeneity statistic, and correlation statistic derived from NDVI images (NDVI_VA, NDVI_CO, NDVI_DI, and NDVI_COR). The RF algorithm significantly outperforms both the NB and SVM models in extracting vineyard information, boasting an impressive accuracy of 93.89% and a Kappa coefficient of 0.89. This marks a 12.25% increase in accuracy and a 0.11 increment in the Kappa coefficient over the NB model, as well as an 8.02% enhancement in accuracy and a 0.06 rise in the Kappa coefficient compared to the SVM model. Moreover, the RF model, which amalgamates spectral and texture features, exhibits a notable 13.59% increase in accuracy versus the spectral-only model and a 14.92% improvement over the texture-only model. This underscores the efficacy of the RF model in harnessing the spectral and textural attributes of GF-6 imagery for the precise extraction of vineyard data, offering valuable theoretical and methodological insights for future vineyard identification and information retrieval efforts.

A Study of Word Bigrams for Pseudo-relevance Feedback in Information Retrieval

Traditional information retrieval models mostly adopt a term independence assumption and are based on single terms or unigrams. Past efforts have attempted to go beyond this assumption, such as by using contiguous terms (i.e. word n-grams) or terms appearing in proximity. One such approach employs pseudo-relevance feedback (PRF) in an extended BM25 model, with an expanded query containing bigrams and proximity word pairs besides unigrams. However, the benefit of this approach over the traditional unigram PRF remains inconclusive. We speculate the uncertain effectiveness of bigram PRF in this past work is due to: (1) The new bigrams obtained from the expanded query may be formed by pairing unigrams drawn from different documents. These are potentially noise instead of relevant concepts; (2) The collection statistics of n-grams needed to calculate the document ranking functions, such as their document frequency, is not available in retrieval. Only estimates of these quantities are used instead. We suggest that these issues may be overcome by extracting word n-grams as single units in query expansion, and employing a document index that contains both unigrams and word n-grams. We demonstrate the approach for the case of bigram PRF in an extended BM25. Retrieval experiments are conducted on a range of standard test collections. For the majority of tested collections, the difference between values of the evaluation metrics (Mean Average Precision and the precision-oriented NDCG@20) obtained by our bigram PRF and the unigram PRF baseline is not statistically significant. Thus, our bigram PRF fails to improve over unigram PRF robustly across collections. An analysis of our results reveals ‘query drift’ due to bigram query expansion terms that represent too-broad topics as a cause for the failure of our approach.

The impact of widely used algorithms of large language models on information retrieval and the associated potential risks

Thesis. The aim of this work is to perform an analysis concerning the subject of arti- ficial intelligence integration and its application in modern knowledge acquisition. The article will be an overview of scientific research and papers to raise a discussion of the ad- vantages and potential hazards associated with the widespread implementation of large language models. Concept. The main part of the article will focus on describing the AI enhanced tools available on the market and common ways of utilising them. This knowledge will be jux- taposed with psychological research on human consciousness in the context of decision- making, creativity, and held beliefs to list potential threads related. Results. The analysis of tools that have recently been enhanced with algorithms un- equivocally indicates the dominance of the sector providing informational and creative support. These tools possess a measurably high level of reliability among the public, re- sulting in a reduced number of verification actions. A review of existing literature and research on human psychology shows a very strong correlation between the influence of previous social authorities on decision-making behaviour and an uncritical approach to information obtained through AI. Studies have shown that access to these tools, without proper controlling actions of results, exposes users to bias, manipulation, or susceptibility to marketing activities. Alternatively, with proper and rational use of these implemented algorithms, humans are able to obtain extraordinarily precise and knowledge-based sup- port in decision-making. Similar polarisations of results have been observed in the con- text of creativity, innovation, and inquisitiveness. Originality. The article addresses topics that relate to the psychological nature of hu- mans in the context of new and expanded AI augmented tools. The results of the explora- tory study clearly indicate that programmes supported by algorithms show potential in terms of both scientific and social development. Unfortunately, they also present many potential dangers for both, which, discussed in the foundation, provide the groundwork for more in-depth research.

Perception of antibiotics in Polish Internet during the infection season 2023/2024

Aim. Public health researchers have used the Internet for disease surveillance, the assessment of risk perception and behaviour changes during epidemics in the past. Cur- rently, investigations of antibiotics (ABx) mentions and the closely related concept of antimicrobial resistance (AMR) on the Internet are mostly limited to information retrieval as well as simple analysis of Google Trends and media narratives. Methods. To address these shortcomings, we perform media listening and analysis for monitoring of ABx perception in the Polish segment of the Internet during November and December 2023 with Brand24 – a comprehensive Internet analysis tool – based on 2668 mentions. Results. Although the scientific community is aware of the importance of AMR, it was not reflected by the general public perception. Real life examples of public engagement related to practical solutions – such as natural and alternative medicine, and ABx acqui- sition with special focus on children among others – dominated the Internet discourse. Conclusion. Our findings demonstrate that while the interest in health topics in the general Polish population is high, it is often not matched by a sound biomedical literacy. A long-term patient engagement could be difficult, as they tend to think from an indi- vidual perspective (how to help myself/my family/my companion animal) rather than in the context of public health and one health.

CPEQA: A Large Language Model Based Knowledge Base Retrieval System for Chinese Confidentiality Knowledge Question Answering

Large language models (LLMs) have exhibited remarkable performance on various natural language processing (NLP) tasks, particularly in the construction of the intelligent question-answering system. These systems, especially in specialized fields, usually rely on NLP through the retrieval of corpus and answering databases to efficiently provide accurate and concise answers. This paper focuses on the national confidentiality publicity and education field, aiming to address the dilemma of inaccurate knowledge retrieval in this field. Therefore, we design an intelligent confidentiality question-answering system CPEQA by comprehensively utilizing the LLMs platform and information retrieval technique. CPEQA is capable of providing professional answers to questions about Chinese confidentiality publicity and education raised by users. Additionally, we also integrate the conventional database retrieval technique and LLMs into the database query construction, enabling CPEQA to perform real-time queries and data analysis for both single-table and multi-table querying tasks. Through extensive experiments with generated query sentences, we show both methodological comparisons and empirical evaluations of CPEQA’s performance. Experimental results indicate that CPEQA has achieved competitive results on answering precision, recall rate and other metrics. Finally, we explore the challenges of the CPEQA system associated with these techniques and outline potential avenues for future research in this emerging field.

Studying technical sciences without borders: We use the navigator for open educational resources. Part 1. The Russian experience

Introduction. The introduction of information and communication technologies into learning processes expands the possibilities of using information resources and becomes one of the priorities of the transforming higher education system. These changes are reflected in the trends in the development and application of open educational resources. However, the scale and fragmentation of data on the Internet, as well as the lack of open educational resources standards, makes the information retrieval process a difficult task and requires professional skills.Purpose setting. To determine the relevance and prospects for the development of sectoral open educational resources, it is necessary to analyze their role in the modern educational system. It is important to identify the possibilities of using open access research in technical sciences as one of the tools to eliminate social inequality in higher education. To identify and present arguments for the purpose of justifying the need to develop an industry-specific «Navigator on OER of Technical Sciences» (hereinafter referred to as the «Navigator»).Methodology and methods of the study. The research methodology covers the study and analysis of scientific papers devoted to the development of tools and initiatives in the field of open educational resources in technical sciences both in Russia and abroad. This study is also based on the method of oral questioning of students, postgraduates and teachers of technical disciplines at universities in Novosibirsk. Results. An array of scientific literature and web resources has been studied, providing free access to scientific and educational information in the field of research under consideration. Taking into account empirical data, the necessity of developing a «Navigator» on open educational resources in the system of higher technical education is justified (http://lib-os.ru/issledovatelyam/resursy/obrazovatelnye- resursy/navigator-oor-techn-nauk/).Conclusion. The study of technical sciences using open educational resources contributes to the effective improvement of professional level and allows you to be aware of current solutions to innovative activities in the field of knowledge under consideration.

Applications of Big Data and Blockchain Technology in Food Testing and Their Exploration on Educational Reform

This study reviews the applications of big data (BD) and blockchain technology in modern food testing and explores their impact on educational reform. The first part highlights the critical role of BD in ensuring food safety across the supply chain, discussing various data collection methods, such as national and international food safety databases, while addressing the challenges related to data storage and real-time information retrieval. Additionally, blockchain technology has been explored for its ability to enhance transparency, traceability, and security in the food-testing process by creating immutable records of testing data, ensuring data integrity, and reducing the risk of tampering or fraud. The second part focuses on the influence of BD and blockchain on educational reform, particularly within food science curricula. BD enables data-driven curriculum design, supporting personalized learning and more effective educational outcomes, while blockchain ensures transparency in course management and credentials. This study advocates integrating these technologies into curriculum reform to enhance both the efficiency and quality of education.

The current trends in the development of methods and regulatory framework for indexing library information resources

The author examines the development of the basic concepts of indexing information resources using classifications and dictionaries of information retrieval languages. The impact of digital communications and the Internet on the indexing methods in the information environment, including the libraries, is emphasized. The modern library collections comprise various types of resources, both traditional paper publications and digital resources of various types, widely available on the Internet. In these conditions, harmonization of indexing methods with the principles of Internet information retrieval grows in importance. The author discusses the regulatory framework for indexing in the library sector and its relationship with Internet standards within the concept of Semantic Web. The author reviews the main stages of the development of the regulatory framework both on the international level and in this country. The author emphasizes the need to study the experience of other organizations in the field of standardization for further development of indexing technology.

Identification of ocular artifact in EEG signals using VMD and Hurst exponent.

Electroencephalographic (EEG) readings are usually infected with unavoidable artifacts, especially physiological ones. One such physiological artifact is the ocular artifacts (OAs) that are generally related to eyes and are characterized by high magnitude and a specific spike pattern in the prefrontal region of the brain. During the long-duration EEG acquisition, the retrieval of important information becomes quite complicated in prefrontal regions as ocular artifacts dominate the EEG recorded, making it difficult to discern underlying brain activity. With the progress and development in signal processing techniques, artifact handling has become a progressive field of investigation. This paper presents a framework for the detection and correction of ocular artifacts. This study emphasizes improving the quality and reducing the time complexity by using higher-order statistics (HOS) for artifact identification and variational mode decomposition (VMD) for OA correction. An overall SNR of 14 dB, MAE of 0.09, and PSNR of 33.59 dB has been attained by the proposed framework. It was observed that the proposed HOS-VMD surpassed the state-of-the-art mode decomposition techniques.

Paperless libraries: The trending way to go in the fifth industrial revolution

The idea that spurred the author’s interest in carrying out this study results from understanding a cashless society, where users of banking systems only move around with their ATM cards. With their ATM cards, they can access their money anywhere globally. This approach could resonate in libraries, where we now have a paperless society, having cost-benefit analysis in the era of the fifth Industrial Revolution (5IR). The 5IR has to do with humans interacting between technology or machines either to solve individual and organizational societal problems from a general or specific perspective. The interaction between humans and technology would assist in recognizing the synergistic in addressing the support for task accomplishment. The methodology applied is in two stages, first, a systematic literature review and second, interpretive content analysis of literature harvested from the database of Scopus. Findings revealed that technological advancements have enabled paperless libraries such that on-screen reading is increasingly prevalent, rapid development support through mobile computing, software, and internet/connections/Wi-Fi, multifarious electronic information retrieval among others. Furthermore, other findings were anchored on the relevance of paperless libraries in the 5IR, the functionality of paperless library systems, enhanced accessibility, and inclusivity, collaboration and connectivity, and environmental impact. Meanwhile, certain challenges and mitigation strategies were proffered. The study concludes that as we navigate this transformative era, the emergence of paperless libraries stands as a testament to the harmonious integration of technological prowess and human aspirations. Therefore, envision human-machine interactions be encouraged to bring about paperless libraries, considering the significance of AI and human-centric innovation.

Transforming Libraries: The Impact of Artificial Intelligence

Artificial intelligence (AI) is revolutionising library services and enhancing user experience, creating a new era of efficiency, accessibility, and innovation. This editorial explores how AI reshapes the library landscape, ushering in a new generation of efficiency, accessibility, and innovation. The AI’s most notable contribution to libraries is the automation of routine tasks, cataloguing and organising; librarians can now rely on AI algorithms to streamline. It saves librarians valuable time engaging in intellectually stimulating activities and ensures a more accurate and organised library system. AI-driven systems can efficiently analyse vast amounts of data, leading to improved search functionalities and more seamless information retrieval for library patrons. Moreover, AI is instrumental in the personalisation of library services. Through machine learning algorithms, libraries can analyse user behaviour, preferences, and historical data to provide customised recommendations. This enhances the user experience by offering tailored suggestions for reading materials, resources, and services, making libraries more relevant and engaging for diverse user groups. Another essential role of AI in libraries is the development of virtual assistants. These digital assistants, powered by natural language processing and machine learning, can help library patrons navigate the catalogue system, answer queries, and even provide Realtime language translation service. Virtual assistants help libraries become more inclusive and accessible, reaching more audiences with diverse language backgrounds and information needs.

GENERALIZED PASCAL’S PYRAMIDS AND COMBINATORIAL INFORMATION RETRIEVAL PROBLEMS

The work studies combinatorial objects of pyramidal structure. It proposes fundamental relations for generalizations of Pascal’s triangle and pyramid. It considers problems of finding new ways of constructing information relevance indices and offers a way of constructing such an index with the help of Pascal’s generalized pyramids. Using structures of a type of Pascal’s generalized pyramids with weight factors allow construction of an index that reflects a degree of relevance of keywords, as well as makes it possible to allocate subsets of keywords with specific properties. The offered methods of the combinatorial analysis of hierarchical structures in information retrieval problems can be used in creating and analyzing knowledge databases.

Pattern separation during encoding and Subsequent Memory Effect

Memory retrieval has been extensively studied in relation to the encoding processes that precede access to stored information. Event related potentials (ERP) research has compared brain potentials elicited during the study phase of successful and unsuccessful retrieval, finding greater activation for the subsequent retrieval information. In this work we were interested in exploring the neural markers associated to subsequent recognition when similar memories are subsequently encoded. We used a Subsequent Memory paradigm in which we manipulated the number of similar items within a category (2 or 6) that participants encoded. Manipulating the number of similar encoded items within a category allowed us to test whether encoding markers of subsequent recognition depend solely on memory trace strength or, on the contrary, successful recognition is influenced by subsequently presented similar memories, and consequently may not be reflected in higher activation in such cases. After a 20-minute period, participants performed a recognition task providing one of a three-option judgement: “old”, “similar” and “new”, which allowed us to test if the amplitude of ERP waveforms varied based on the similarity judgement of the unrecognized encoded item. We did not observe a significant parietal subsequent memory effect, however, old hits and similar false alarms were both significantly different from similar correct rejections and old false alarms in ERP retrieval. These findings suggest that differences in brain responses between conditions are specifically related to the retrieval process and not the encoding process, indicating potential differential effects on memory during retrieval. Moreover, it is also possible that differences in brain responses develop or change over the rest time between phases, influencing how these conditions manifest across different stages of information processing.

On Elastic Language Models

Large-scale pretrained language models have achieved compelling performance in a wide range of language understanding and information retrieval tasks. While their large scales ensure capacity, they also hinder deployment. Knowledge distillation offers an opportunity to compress a large language model to a small one, in order to reach a reasonable latency-performance tradeoff. However, for scenarios where the number of requests (e.g., queries submitted to a search engine) is highly variant, the static tradeoff attained by the compressed language model might not always fit. Once a model is assigned with a static tradeoff, it could be inadequate in that the latency is too high when the number of requests is large, or the performance is too low when the number of requests is small. To this end, we propose an elastic language model ( ElasticLM ) that elastically adjusts the tradeoff according to the request stream. The basic idea is to introduce a compute elasticity to the compressed language model, so that the tradeoff could vary on-the-fly along a scalable and controllable compute. Specifically, we impose an elastic structure to equip ElasticLM with compute elasticity and design an elastic optimization method to learn ElasticLM under compute elasticity. To serve ElasticLM , we apply an elastic schedule. Considering the specificity of information retrieval, we adapt ElasticLM to dense retrieval and reranking, and present an ElasticDenser and an ElasticRanker, respectively. Offline evaluation is conducted on a language understanding benchmark GLUE, and several information retrieval tasks including Natural Question, Trivia QA and MS MARCO. The results show that ElasticLM along with ElasticDenser and ElasticRanker can perform correctly and competitively compared with an array of static baselines. Furthermore, an online simulation with concurrency is also carried out. The results demonstrate that ElasticLM can provide elastic tradeoffs with respect to varying request stream.

Development of an Evaluation Instrument on Artificial Intelligence Search Tools for Evidence Synthesis

What Was the Question? There are conflicting calls for evidence synthesis producers to adopt or avoid recent Artificial Intelligence (AI) technologies. Our question was: How do we evaluate rapidly evolving AI tools to enhance the production of evidence syntheses and maintain quality standards? What Did We Do? To advance information retrieval science for producing evidence syntheses at Canada’s Drug Agency (CDA-AMC), the Research Information Services team developed a process to evaluate promising AI search tools. We inventoried 51 tools in the fall of 2023, established selection criteria, assessed specific attributes, and built a standalone instrument to support continuous monitoring and evaluation of new tools. What Did We Find? Rapid development of AI search tools requires a flexible evaluation instrument to inform adoption decisions and enable comparison between tools. We identified mandatory and desirable characteristics for suitable AI tools to assist with information retrieval tasks conducted by CDA-AMC. This work enabled the development of a flexible instrument to evaluate novel AI search tools for evidence synthesis. What Does This Mean? CDA-AMC operationalized a replicable process to monitor and evaluate AI search tools. Our approach to evaluating AI technologies will advance information retrieval methods by CDA-AMC, and our evaluation instrument will assist any evidence synthesis producer interested in adopting AI search tools.