Information Retrieval Algorithms Research Articles

LESS LiDAR: A Full-Waveform and Discrete-Return Multispectral LiDAR Simulator Based on Ray Tracing Algorithm

Light detection and ranging (LiDAR) is a widely used technology for the acquisition of three-dimensional (3D) information about a wide variety of physical objects and environments. However, before conducting a campaign, a test is typically conducted to assess the potential of the utilized algorithm for information retrieval. It might not be a real campaign but rather a simulation to save time and costs. Here, a multi-platform LiDAR simulation model considering the location, direction, and wavelength of each emitted laser pulse was developed based on the large-scale remote sensing (RS) data and image simulation framework (LESS) model, which is a 3D radiative transfer model for simulating passive optical remote sensing signals using the ray tracing algorithm. The LESS LiDAR simulator took footprint size, returned energy, multiple scattering, and multispectrum LiDAR into account. The waveform and point similarity were assessed with the LiDAR module of the discrete anisotropic radiative transfer (DART) model. Abstract and realistic scenes were designed to assess the simulated LiDAR waveforms and point clouds. A waveform comparison in the abstract scene with the DART LiDAR module showed that the relative error was lower than 1%. In the realistic scene, airborne and terrestrial laser scanning were simulated by LESS and DART LiDAR modules. Their coefficients of determination ranged from 0.9108 to 0.9984. Their mean was 0.9698. The number of discrete returns fitted well and the coefficient of determination was 0.9986. A terrestrial point cloud comparison in the realistic scene showed that the coefficient of determination between the two sets of data could reach 0.9849. The performance of the LESS LiDAR simulator was also compared with the DART LiDAR module and HELIOS++. The results showed that the LESS LiDAR simulator is over three times faster than the DART LiDAR module and HELIOS++ when simulating terrestrial point clouds in a realistic scene. The proposed LiDAR simulator offers two modes for simulating point clouds: single-ray and multi-ray modes. The findings demonstrate that utilizing a single-ray simulation approach can significantly reduce the simulation time, by over 28 times, without substantially affecting the overall point number or ground pointswhen compared to employing multiple rays for simulations. This new LESS model integrating a LiDAR simulator has great potential in terms of simultaneously simulating LiDAR data and optical images based on the same 3D scene and parameters. As a proof of concept, the normalized difference vegetation index (NDVI) results from multispectral images and the vertical profiles from multispectral LiDAR waveforms were simulated and analyzed. The results showed that the proposed LESS LiDAR simulator can fulfill its design goals.

Information retrieval algorithms and neural ranking models to detect previously fact-checked information

Although in the last decade several fact-checking organizations have emerged to verify misinformation, fake news has continued to proliferate, especially through social media platforms. Even though adopting improved detection strategies is of utmost importance, the fact-checking process could be optimized by verifying whether a claim has been previously fact-checked. Despite some ad-hoc information retrieval approaches having been recently proposed, the utility of modern (neural) retrieval systems have not been investigated yet. In this paper, we consider the standard two-phases retriever-reranker architecture and benchmark different state-of-the-art techniques from the information retrieval and Q&A literature. We design several experiments on a real-world Twitter dataset to analyze the efficiency and the effectiveness of the benchmark approaches. Our results show that combining standard and neural approaches is the most promising research direction to improve retrievers performance and that complex (neural) rerankers might still be efficient in practice since there is no need to process a high number of documents to improve ranking performance.

Bibliometric‐enhanced legal information retrieval: Combining usage and citations as flavors of impact relevance

AbstractBibliometric‐enhanced information retrieval uses bibliometrics (e.g., citations) to improve ranking algorithms. Using a data‐driven approach, this article describes the development of a bibliometric‐enhanced ranking algorithm for legal information retrieval, and the evaluation thereof. We statistically analyze the correlation between usage of documents and citations over time, using data from a commercial legal search engine. We then propose a bibliometric boost function that combines usage of documents with citation counts. The core of this function is an impact variable based on usage and citations that increases in influence as citations and usage counts become more reliable over time. We evaluate our ranking function by comparing search sessions before and after the introduction of the new ranking in the search engine. Using a cost model applied to 129,571 sessions before and 143,864 sessions after the intervention, we show that our bibliometric‐enhanced ranking algorithm reduces the time of a search session of legal professionals by 2 to 3% on average for use cases other than known‐item retrieval or updating behavior. Given the high hourly tariff of legal professionals and the limited time they can spend on research, this is expected to lead to increased efficiency, especially for users with extremely long search sessions.

Analysis on the Multimedia Information Retrieval Algorithm Based on Enterprise Correlation Financial Analysis under the Background of Big Data

At present, based on the background of the big data era, the operation of enterprises has a great impact, and relevance is a principle set to meet the needs of enterprise operation and management. In today’s digital and network era, multimedia has become the main part of the Internet information transmission, and multidisciplinary art and science communion has promoted the development of the multimedia information retrieval. As a result, this paper examines the correlation of big data against the background of financial and multimedia information retrieval algorithms.

Clustering and Mining Algorithm of Factor Structure Data of Physical Education Teachers' Teaching Behavior Based on Internet Information Retrieval Algorithm

Integrating visual word list and Rocchio algorithm, constructs an implicit relevance feedback retrieval model in semantic space, and judges information retrieval preference. User demand mining is introduced, Jensen-Shannon divergence is used to calculate the relative entropy distance between the probability distributions of document sets, and similarity matching is calculated to complete interactive information retrieval. Teaching ability is gradually formed by teachers in long-term teaching practice, and is stably and comprehensively reflected in teaching viewpoints, teaching methods, teaching skills, teaching style and so on. The weight analysis of each index in the evaluation index system was carried out by using the analytic hierarchy process, and the selection of the evaluation subject was completed through the third round of expert questionnaires, and the evaluation criteria were formulated in combination with the opinions of tutors and some expert teachers.

Similarity matching method of power distribution system operating data based on neural information retrieval

Operation control of power systems has become challenging with an increase in the scale and complexity of power distribution systems and extensive access to renewable energy. Therefore, improvement of the ability of data-driven operation management, intelligent analysis, and mining is urgently required. To investigate and explore similar regularities of the historical operating section of the power distribution system and assist the power grid in obtaining high-value historical operation, maintenance experience, and knowledge by rule and line, a neural information retrieval model with an attention mechanism is proposed based on graph data computing technology. Based on the processing flow of the operating data of the power distribution system, a technical framework of neural information retrieval is established. Combined with the natural graph characteristics of the power distribution system, a unified graph data structure and a data fusion method of data access, data complement, and multi-source data are constructed. Further, a graph node feature-embedding representation learning algorithm and a neural information retrieval algorithm model are constructed. The neural information retrieval algorithm model is trained and tested using the generated graph node feature representation vector set. The model is verified on the operating section of the power distribution system of a provincial grid area. The results show that the proposed method demonstrates high accuracy in the similarity matching of historical operation characteristics and effectively supports intelligent fault diagnosis and elimination in power distribution systems.

Study on Library Management System Based on Data Mining and Clustering Algorithm

In order to improve the information retrieval and resource sharing abilities of the library and establish an intelligent library information management system, alibrary management system based on data mining and clustering algorithm is proposed. The system development is divided into two modules: library mining algorithm design and information management system software development. Library data mining design and clustering algorithm of library management system development. System development is divided into two modules: library mining algorithm design and information management system software development. Big data fusion and feature clustering methods is used to design intelligent information retrieval algorithms in library information management to establish library management database. Realizing the development and design of library management system under embedded environment, and finally experimental test analysis is carried out. Results show that the use of the proposed method for library information management can improve the accuracy of book recommendation, which is 16.7% higher than traditional methods on average. It has good data mining, and has good performance indicators in data recall and precision. Therefore, this algorithm is not only a means of automated management of library systems, but also an effective means to realize library information modernization.

Benchmarking for Recommender System (MFRISE)

The advent of the internet age offers overwhelming choices of movies and shows to viewers which create need of comprehensive Recommendation Systems (RS). Recommendation System will suggest best content to viewers based on their choice using the methods of Information Retrieval, Data Mining and Machine Learning algorithms. The novel Multifaceted Recommendation System Engine (MFRISE) algorithm proposed in this paper will help the users to get personalized movie recommendations based on multi-clustering approach using user cluster and Movie cluster along with their interaction effect. This will add value to our existing parameters like user ratings and reviews. In real-world scenarios, recommenders have many non-functional requirements of technical nature. Evaluation of Multifaceted Recommendation System Engine must take these issues into account in order to produce good recommendations. The paper will show various technical evaluation parameters like RMSE, MAE and timings, which can be used to measure accuracy and speed of Recommender system. The benchmarking results also helpful for new recommendation algorithms. The paper has used MovieLens dataset for purpose of experimentation. The studied evaluation methods consider both quantitative and qualitative aspects of algorithm with many evaluation parameters like mean squared error (MSE), root mean squared error (RMSE), Test Time and Fit Time are calculated for each popular recommender algorithm (NMF, SVD, SVD++, SlopeOne, Co- Clustering) implementation. The study identifies the gaps and challenges faced by each above recommender algorithm. This study will also help researchers to propose new recommendation algorithms by overcoming identified research gaps and challenges of existing algorithms.

Retraction Note: Research on Chinese and English language information retrieval algorithm based on bilingual theme model

Retraction Note: Research on Chinese and English language information retrieval algorithm based on bilingual theme model

Natural Language Semantic Answering: Application to Medicinal Plants and Coronavirus

This paper describes PryRe (Preguntas y Respuestas) {Questions and answers}, a question answering system that receives as input a question in Spanish and returns the answer. An information retrieval algorithm identifies the meaning of the question using its semantic properties. To do this, it is necessary to implement natural language processing modules. PryRe identifies the question using a triplet R (C, V), where R is the relation or link, C is the concept or main idea, and V is the value of the concept. For example, in the question: ¿Cuál es la hierba que mejora la digestion? {What is the herb that improves digestion?}, the answer is Manzanilla {Chamomile}. To obtain this response, a process of query analysis and design is executed. The analysis consists of identifying the answer, we call this process the semantic identification of the question. The PryRe algorithms are explained in detail. The tests have been carried out with some sources on herbalism [15] and on 2019-nCoV [14] in an Ontology; they show an acceptable accuracy of 83%. Resources used in this work are (1) the OM notation, which is used to describe ontologies, and (2) the deductive capability of PryRe. PryRe consults this ontology to answer the questions using cases that are described in its algorithm.

LibACA, pyACA, and ACA-Code: Audio content analysis in 3 languages

The three packages libACA, pyACA, and ACA-Code provide reference implementations for basic approaches and algorithms for the analysis of musical audio signals in three different languages: C++, Python, and Matlab. All three packages cover the same algorithms, such as extraction of low level audio features, fundamental frequency estimation, as well as simple approaches to chord recognition, musical key detection, and onset detection. In addition, it implementations of more generic algorithms useful in audio content analysis such as dynamic time warping and the Viterbi algorithm are provided. The three packages thus provide a practical cross-language and cross-platform reference to students and engineers implementing audio analysis algorithms and enable implementation-focused learning of algorithms for audio content analysis and music information retrieval.

Machine Reading at Scale: A Search Engine for Scientific and Academic Research

The Internet, much like our universe, is ever-expanding. Information, in the most varied formats, is continuously added to the point of information overload. Consequently, the ability to navigate this ocean of data is crucial in our day-to-day lives, with familiar tools such as search engines carving a path through this unknown. In the research world, articles on a myriad of topics with distinct complexity levels are published daily, requiring specialized tools to facilitate the access and assessment of the information within. Recent endeavors in artificial intelligence, and in natural language processing in particular, can be seen as potential solutions for breaking information overload and provide enhanced search mechanisms by means of advanced algorithms. As the advent of transformer-based language models contributed to a more comprehensive analysis of both text-encoded intents and true document semantic meaning, there is simultaneously a need for additional computational resources. Information retrieval methods can act as low-complexity, yet reliable, filters to feed heavier algorithms, thus reducing computational requirements substantially. In this work, a new search engine is proposed, addressing machine reading at scale in the context of scientific and academic research. It combines state-of-the-art algorithms for information retrieval and reading comprehension tasks to extract meaningful answers from a corpus of scientific documents. The solution is then tested on two current and relevant topics, cybersecurity and energy, proving that the system is able to perform under distinct knowledge domains while achieving competent performance.

Automatic construction of generic stop words list for hausa text

<span lang="EN-US">Stop-words are words having the highest frequencies in a document without any significant information. They are characterized by having common relations within a cluster. They are the noise of the text that are evenly distributed over a document. Removal of stop words improve the performance and accuracy of information retrieval algorithms and machine learning at large. It saves the storage space by reducing the vector space dimension, and helps in effective documents indexing. This research generated a list of Hausa stop words automatically using aggregated method by combining frequency and statistics methods. The experiments are conducted using a primarily collected Hausa corpus consisting of 841 Hausa news articles of size 646862 words and finally a list of distinct 81 Hausa stop words is generated.</span>

Transferrable Framework Based on Knowledge Graphs for Generating Explainable Results in Domain-Specific, Intelligent Information Retrieval

In modern industrial systems, collected textual data accumulates over time, offering an important source of information for enhancing present and future industrial practices. Although many AI-based solutions have been developed in the literature for a domain-specific information retrieval (IR) from this data, the explainability of these systems was rarely investigated in such domain-specific environments. In addition to considering the domain requirements within an explainable intelligent IR, transferring the explainable IR algorithm to other domains remains an open-ended challenge. This is due to the high costs, which are associated with intensive customization and required knowledge modelling, when developing new explainable solutions for each industrial domain. In this article, we present a transferable framework for generating domain-specific explanations for intelligent IR systems. The aim of our work is to provide a comprehensive approach for constructing explainable IR and recommendation algorithms, which are capable of adopting to domain requirements and are usable in multiple domains at the same time. Our method utilizes knowledge graphs (KG) for modeling the domain knowledge. The KG provides a solid foundation for developing intelligent IR solutions. Utilizing the same KG, we develop graph-based components for generating textual and visual explanations of the retrieved information, taking into account the domain requirements and supporting the transferability to other domain-specific environments, through the structured approach. The use of the KG resulted in minimum-to-zero adjustments when creating explanations for multiple intelligent IR algorithms in multiple domains. We test our method within two different use cases, a semiconductor manufacturing centered use case and a job-to-applicant matching one. Our quantitative results show a high capability of our approach to generate high-level explanations for the end users. In addition, the developed explanation components were highly adaptable to both industrial domains without sacrificing the overall accuracy of the intelligent IR algorithm. Furthermore, a qualitative user-study was conducted. We recorded a high level of acceptance from the users, who reported an enhanced overall experience with the explainable IR system.

Method to integrate speakers identification, speech recognition, and information retrieval algorithms for speaker-based information retrieval.

Method to integrate speakers identification, speech recognition, and information retrieval algorithms for speaker-based information retrieval.

Object Pre-processing using Motion Stabilization and Key Frame Extraction with Machine Learning Techniques

Video information processing is one of the most important application areas in research and to solve in various pre-processing issues. The pre-processing issues such as unstable video frame rates or capture angle, noisy data and large size of the video data prevent the researchers to apply information retrieval or categorization algorithms. The video data itself plays a vital role in various areas. This work aims to solve the motion stabilization, noise reduction and key frame extraction, without losing the information and in reduced time. The work results into 66% reduction in key frame extraction and nearly 6 ns time for complete video data processing.

From Information Search to the Loss of Personality: The Phenomenon of Dataism

Purpose. The research is devoted to the analysis of the urgent problem of the information society: the overload of a person with information and, as a result, the impossibility of adequate formation and development of the personality; as well as the problem of "digitization" of human existence and the formation of a new reality of dataism. Theoretical basis. A lot of modern scientific works are devoted to the analysis of the information society, its problems and features. The information society is a logical continuation of the scientific and technological revolution, which led to the rapid growth of scientific knowledge and the technology development. In the 20th century, technologies have touched the sphere of knowledge and information, as a result of which the formation and development of the information society, or "knowledge society", takes place. Information becomes the main resource in it, and, one way or another, a person’s life is inextricably connected with the information space. With the information society formation, the problem of information search and processing becomes one of the most urgent. It turns out that despite the abundance and availability of information, it is very difficult to determine its relevance. A lot of effort is spent on developing information retrieval algorithms. Another problem is related to the person’s inability to process large amounts of data. This situation begins to influence not only the education process and professional activity, but also the formation of a person’s personality. A person is "lost" in the information space and gradually loses his/her "I". Algorithms for data analysis come to the rescue, but gradually, instead of giving a person material for thought, they begin to make decisions on their own, and therefore, live life instead of a person. With the advent of Big Data processing algorithms, a new ideological paradigm appears – dataism, which predicts the merger of a person with the general data flow. Originality. The authors make assumptions that the dataistic future is "natural". A lot of works, including in the framework of philosophy, are focused on the problem of "dissolving" a person in the information space and finding ways to overcome it. But, in our opinion, this process is a completely logical continuation of human evolution. Conclusions. Existence of a person as a data flow is not a problem and a threat, but a new dimension of his/her being, and, therefore, requires a careful study and formulation of the main principles of this form of existence.

An information retrieval benchmarking model of satisficing and impatient users’ behavior in online search environments

This study analyzes the effects that the position of the alternatives ranked by a search engine and the relative impatience of users have on their information retrieval behavior. We design a stochastic information retrieval algorithm calibrated to mimic the click-through rates (CTRs) of users observed in real-life environments. We introduce two versions of the mimicking algorithm designed to demonstrate the importance of impatience as a determinant of CTRs conditioned by the alternatives’ ranking position. The first version assumes that users proceed sequentially through the ranking until they find an alternative satisficing their expectations. Once they find a satisficing alternative, they continue retrieving information until they observe an alternative that violates their expectations. The second version increases users’ impatience, who stop retrieving information as soon as an alternative does not satisfy their expectations – even if it is the top-ranked one. All three algorithmic structures are sufficiently malleable to incorporate any potential modification to users’ beliefs and preferences. We simulate sets of 1,000,000 queries to illustrate how the CTRs of the top three ranked alternatives remain stable as users grow impatient, with differences widening as growingly impatient users proceed halfway through the ranking.

Fluence adaptation for contrast-based dose optimization in x-ray phase-contrast imaging.

X-ray phase-contrast imaging (XPCI) can provide multiple contrasts with great potentials for clinical and industrial applications, including conventional attenuation, phase contrast, and dark field. Grating-based imaging (GBI) and edge-illumination (EI) are two promising types of XPCI as the conventional x-ray sources can be directly utilized. For the GBI and EI systems, the phase-stepping acquisition with multiple exposures at a constant fluence is usually adopted in the literature.This work, however, attempts to challenge such a constant fluence concept during the phase-stepping process and proposes a fluence adaptation mechanism for dosereduction. Given the importance of patient radiation dose for clinical applications, numerous studies have tried to reduce patient dose in XPCI by altering imaging system designs, data acquisition, and information retrieval. Recently, analytic multiorder moment analysis has been proposed to improve the computing efficiency. In these algorithms, multiple contrasts can be calculated by summing together the weighted phase-stepping curves (PSCs) with some kernel functions, which suggests us that the raw data at different steps have different contributions for the noise in retrieved contrasts. Therefore, it is possible to improve the noise performance by adjusting the fluence distribution during the phase-stepping process directly. Based on analytic retrieval formulas and the Gaussian noise model for detected signals, we derived an optimal adaptive fluence distribution, which is proportional to the absolute weighting kernel functions and the root of original sample PSCs acquired under the constant fluence. Considering that the original sample PSC might be unavailable, we proposed two practical forms for the GBI and EI systems, which are also able to reduce the contrast noise when comparing with the constant fluence distribution. Since the kernel functions are target contrast-dependent, our proposed fluence adaptation mechanism provides a way of realizing a contrast-based dose optimization while keeping the same noiselevel. To validate our analyses, simulations and experiments are conducted for the GBI and EI systems. Simulated results demonstrate that the dose reduction ratio between our proposed fluence distributions and the typical constant one can be about 20% for the phase contrast, which is consistent with our theoretical predictions. Although the experimental noise reduction ratios are a little smaller than the theoretical ones, low-dose experiments observe better noise performance by our proposed method. Our simulated results also give out the effective ranges of the parameters of the PSCs, such as the visibility in the GBI, the standard deviation, and the mean value in the EI, providing a guidance for the use of our proposed approach inpractice. In this paper, we propose a fluence adaptation mechanism for contrast-based dose optimization in XPCI, which can be applied to the GBI and EI systems. Our proposed method explores a new direction for dose reduction, and may also be further extended to other types of XPCI systems and information retrievalalgorithms.

Disperse and preserve the perverse: computing how hip-hop censorship changed popular music genres in China

How do political interventions reshape genre boundaries? Previous studies on genres only tangentially touch on this question as they mostly focus on the artistic, economic, or critical consequences of genre spanning. This paper fills in this gap by exploring the impact of music censorship on the censored genre and other related genres. Using an original dataset of 53,364 songs released on a Chinese online music platform, I study how Hip-Hop censorship in China in 2018 impacted Hip-Hop as well as Pop, Rock, and Folk songs in terms of how they sound and what topics are engaged in the lyrics. I propose a novel, computational approach to measure sound similarities between songs by using Music Information Retrieval (MIR) algorithms which synthesize audio signal processing and neural networks. I also measure the change of topic prevalence in song lyrics by using topic models supplemented with a dictionary approach. I found that post-censorship Hip-Hop songs sound significantly different from pre-censorship ones, with a bigger impact on the high-profile songs. Moreover, Rock, as a close genre to Hip-Hop, became more “Hip-Hoppy”, while Pop, a mixed category that reflects trending genres, became less “Hip-Hoppy”; the impact on these two genres is more salient among their low-profile songs. Folk, a genre distanced from Hip-Hop, remained generally untouched. The censorship also made Hip-Hop musicians engage less with topics related to violence and deviant behaviors but more with sexual terms, albeit in a covered form and not necessarily related to sexual conduct per se. The findings suggest a dispersion model in explaining the outcome of political interventions in genres, where stylistic conventions of the censored genre are dispersed from salient works of that genre to less influential ones as well as to adjacent genres.