Effectiveness Of Information Retrieval Research Articles

Efficient Document Retrieval System using Locality Sensitive Hashing Nearest Neighbor Algorithm and Weighted Jaccard Distance for Retrieving Closest Personalities

The process of retrieving significant documents based on the search key from a corpus has been a vital research problem in the information retrieval field. This paper proposes an efficient way to retrieve documents related to different personalities extracted from Wikipedia. The proposed method utilizes the Locality Sensitive Hashing Nearest Neighbor algorithm combined with Weighted Jaccard Distance to measure document similarity with enhanced precision. This document retrieval system demonstrates competitive performance compared to existing methods in the Personality Identification domain. The introduction of a document centroid normalization technique significantly improves the effectiveness of information retrieval by enabling better discrimination between documents. The personality document search results were compared for different distance measures using performance metrics like Normalized Discounted Cumulative Gain and Mean Average Precision. The results presented in this paper show that the TF-IDF scheme with Locality Sensitive Hashing Nearest Neighbor Algorithm using the Weighted Jaccard Distance can yield superior retrieval efficiency when contrasted with alternative approaches found in the existing literature.

A model of the relationship between the variations of effectiveness and fairness in information retrieval

The requirement that, for fair document retrieval, the documents should be ranked in the order to equally expose authors and organizations has been studied for some years. The fair exposure of a ranking, however, undermines the optimality of the Probability Ranking Principle and as a consequence retrieval effectiveness. It is shown how the variations of fairness and effectiveness can be related by a model. To this end, the paper introduces a fairness measure inspired in Gini’s index of mutability for non-ordinal variables and relates it to a general enough measure of effectiveness, thus modeling the connection between these two dimensions of Information Retrieval. The paper also introduces the measurement of the statistical significance of the fairness measure. An empirical study completes the paper.

Retracted: Research on Information Retrieval Effectiveness of University Scientific Researchers Based on Mental Model

Retracted: Research on Information Retrieval Effectiveness of University Scientific Researchers Based on Mental Model

Query expansion based on modified Concept2vec model using resource description framework knowledge graphs

The enormous size of the web and the vagueness of the terms used to formulate queries still pose a huge problem in achieving user satisfaction. To solve this problem, queries need to be disambiguated based on their context. One well-known technique for enhancing the effectiveness of information retrieval (IR) is query expansion (QE). It reformulates the initial query by adding similar terms that help in retrieving more relevant results. In this paper, we propose a new QE semantic approach based on the modified Concept2vec model using linked data. The novelty of our work is the use of query-dependent linked data from DBpedia as training data for the Concept2vec skip-gram model. We considered only the top feedback documents, and we did not use them directly to generate embeddings; we used their interlinked data instead. Also, we used the linked data attributes that have a long value, e.g., “dbo: abstract”, as training data for neural network models, and, we extracted from them the valuable concepts for QE. Our experiments on the Associated Press collection dataset showed that retrieval effectiveness can be much improved when a skip-gram model is used along with a DBpedia feature. Also, we demonstrated significant improvements compared to other approaches.

Usability of KOHA OPAC by Undergraduate Users for Information Retrieval with Regard to Usability Testing in Technical University of Kenya Library

The intention of this paper was to determine usability of KOHA OPAC in information retrieval with regard to usability testing by undergraduate users at Technical University of Kenya Library (TU-KL) in Nairobi County. Usability testing can be realized by way of assigning tasks to a representative of the user population in order to determine whether users can be able to achieve the desired goal of retrieving information. To a great extent, success will depend on how users perform the tasks assigned to them. This study conducted usability test on actual users of KOHA OPAC at Technical University of Kenya Library. Qualitative data was collected using the think aloud method where study participants were required to verbalize their thought process as they performed search and retrieval tasks using KOHA OPAC. The number of errors made, time taken to complete search and retrieval activities were coded, recorded and analysed during usability testing. This helped the researcher to determine whether KOHA OPAC is easy to use and whether it saves time for the user. Undergraduate students for the study were selected using a stratified random sampling technique. The population of the study (n = 382) was determined using a simplified formula developed by Yamane. The study initiated a usability test which presented search scenarios where participants were invited to participate in searching and retrieving information resources using KOHA OPAC. The results of the usability test of KOHA OPAC at TU-KL showed that users were slow at accomplishing search tasks, they took long duration to access TU-KL website that hosts KOHA OPAC and there was inefficiency in accessing library materials. This study concluded that usability challenges may have an impact on users' overall satisfaction with KOHA OPAC and information retrieval effectiveness. In order to enhance user experience and boost the effectiveness of information retrieval, the study recommends improvement of KOHA OPAC usability by revamping its interface and enhancing its features.

Evaluation of Average Term Occurrences Weighting Technique for Arabic Textual Information Retrieval

Information retrieval of documents is an important process in the current time, and the vector space retrieval model uses a term weighting scheme as a basic method for matching queries with documents. Term frequency-Inverse document frequency is a widely used and famous term weighting scheme, and many studies proved its effectiveness in information retrieval. However, this term weighting scheme has some drawbacks like retrieving irrelevant documents, which sometimes reduces effectiveness. From this point, a new term weighting scheme called Term Frequency with Average Term Occurrence was proposed and experienced in the English language to minimize retrieving unnecessary documents. In this paper, an information retrieval system is built for the Arabic language, and Open-Source Arabic Corpora was used to complete experiments. Calculations were made using two schemes which are traditional Term frequency-inverse Document Frequency and proposed Term Frequency with Average Term Occurrence. After that, comparisons of results were made using evaluation measures. With all obtained queries, four case studies with two approaches (stop word removal and stemming) are implemented. In English experiments, stop word removal was applied with another discriminative approach, which calculates the centroid of documents. After the analysis of the results, it was found that the proposed scheme is applicable on Arabic text and applied approaches enhance IR effectiveness if they are both implemented. Furthermore, it was found that stop word removal has a favorable effect on both schemes which was also proved in English experiments.

On the role of human and machine metadata in relevance judgment tasks

In order to evaluate the effectiveness of Information Retrieval (IR) systems it is key to collect relevance judgments from human assessors. Crowdsourcing has successfully been used as a method to scale-up the collection of manual relevance judgments, and previous research has investigated the impact of different judgment task design elements (e.g., highlighting query keywords in the document) on judgment quality and efficiency. In this work we investigate the positive and negative impacts of presenting crowd human assessors with more than just the topic and the document to be judged. We deploy different variants of crowdsourced relevance judgment tasks following a between-subjects design in which we present different types of metadata to the human assessor. Specifically, we investigate the effect of human metadata (e.g., what other human assessors think of the current document, as in which relevance level has already been selected by the majority crowd workers), machine metadata (e.g., how IR systems scored this document such as its average position in ranked lists, statistics about the document such as term frequencies). We look at the impact of metadata on judgment quality (i.e., the level of agreement with trained assessors) and cost (i.e., the time it takes for workers to complete the judgments) as well as at how metadata quality positively or negatively impact the collected judgments.

Application of Convolution Neural Networks in Web Search Log Mining for Effective Web Document Clustering

The volume of web search data stored in search engine log is increasing and has become big search log data. The web search log has been the source of data for mining based on web document clustering techniques to improve the efficiency and effectiveness of information retrieval. In this paper Deep Learning Model Convolution Neural Network(CNN) is used in big web search log data mining to learn the semantic representation of a document. These semantic documents vectors are clustered using K-means to group relevant documents for effective web document clustering. Experiment was done on the data set of web search query and associated clicked URLs to measure the quality of clusters based on document semantic representation using Deep learning model CNN. The clusters analysis was performed based on WCSS(the sum of squared distances of documents samples to their closest cluster center) and decrease in the WCSS in comparison to TF.IDF keyword based clusters confirm the effectiveness of CNN in web search log mining for effective web document clustering.

Research on Semantic Information Retrieval Based on Improved Fish Swarm Algorithm

In order to improve the effectiveness of semantic information retrieval, the improved fish swarm algorithm is proposed to carry out semantic information retrieval. Firstly, the system of semantic information retrieval is designed, and theory model of search engine is established. Secondly, the information retrieval model based on semantic similarity is constructed, and the mathematical model is deduced. Thirdly, the improved fish algorithm is established, and the analysis procedure of it is designed. Finally, the simulation analysis of semantic information retrieval is carried out, results show that the proposed model can obtain higher classification accuracy, precision rare and recall rate, therefore it has higher performance on semantic information retrieval.

Vietnamese Information Retrieval Test Collection Development

Linguistic features processing can remarkably affect information retrieval effectiveness. Evaluation is an important phase in the development process of any information retrieval solutions. In the current state, due to the lack of test collections, it’s difficult to carry out research and share achieved results, since the development of Vietnamese test collections are required. This report will provide general information about information retrieval evaluation problems, the test collection development and evaluation processes. The achieved results show that information retrieval test collection development can take a lot of time and effort, but can be divided into many phases and can be accomplished incrementally.

Influence of GUJarati STEmmeR in Supervised Learning of Web Page Categorization

With the large quantity of information offered on-line, it's equally essential to retrieve correct information for a user query. A large amount of data is available in digital form in multiple languages. The various approaches want to increase the effectiveness of on-line information retrieval but the standard approach tries to retrieve information for a user query is to go looking at the documents within the corpus as a word by word for the given query. This approach is incredibly time intensive and it's going to miss several connected documents that are equally important. So, to avoid these issues, stemming has been extensively utilized in numerous Information Retrieval Systems (IRS) to extend the retrieval accuracy of all languages. These papers go through the problem of stemming with Web Page Categorization on Gujarati language which basically derived the stem words using GUJSTER algorithms [1]. The GUJSTER algorithm is based on morphological rules which is used to derived root or stem word from inflected words of the same class. In particular, we consider the influence of extracted a stem or root word, to check the integrity of the web page classification using supervised machine learning algorithms. This research work is intended to focus on the analysis of Web Page Categorization (WPC) of Gujarati language and concentrate on a research problem to do verify the influence of a stemming algorithm in a WPC application for the Gujarati language with improved accuracy between from 63% to 98% through Machine Learning supervised models with standard ratio 80% as training and 20% as testing.

Parallel sentence extraction to improve cross-language information retrieval from Wikipedia

Translation language resources, such as bilingual word lists and parallel corpora, are important factors affecting the effectiveness of cross-language information retrieval (CLIR) systems. In particular, when large domain-appropriate parallel corpora are not available, developing an effective CLIR system is particularly difficult. Furthermore, creating a large parallel corpus is costly and requires considerable effort. Therefore, we here demonstrate the construction of parallel corpora from Wikipedia as well as improved query translation, wherein the queries are used for a CLIR system. To do so, we first constructed a bilingual dictionary, termed WikiDic. Then, we evaluated individual language resources and combinations of them in terms of their ability to extract parallel sentences; the combinations of our proposed WikiDic with the translation probability from the Web’s bilingual example sentence pairs and WikiDic was found to be best suited to parallel sentence extraction. Finally, to evaluate the parallel corpus generated from this best combination of language resources, we compared its performance in query translation for CLIR to that of a manually created English–Korean parallel corpus. As a result, the corpus generated by our proposed method achieved a better performance than did the manually created corpus, thus demonstrating the effectiveness of the proposed method for automatic parallel corpus extraction. Not only can the method demonstrated herein be used to inform the construction of other parallel corpora from language resources that are readily available, but also, the parallel sentence extraction method will naturally improve as Wikipedia continues to be used and its content develops.

A quantum-like semantic model for text retrieval in Arabic

The subject of study. The paper focuses on the extraction of semantics from texts in Arabic. In particular, the applicability of the Bell test to word pairs is investigated as a measure of the semantic words relatedness in a context. The study applies the quantum formalism to the task of information retrieval in Arabic texts and presents the results of this work. The authors also examine the influence of the context width on the effectiveness of information retrieval. Method. The research is based on the vector representation of the context. It uses the well-known approach based on the HAL (Hyperspace Analogue to Language) matrix and Bell test. The HAL matrix allows taking into account both the frequency of the words occurrence in the context and the distance to the target word. Quantum theory operates with probability density matrices. Quantum theory allows describing probabilities in the vector space in a more natural way i.e., words can be represented as vectors. Main results. The results demonstrate that using the Bell’s test for texts in Arabic provides a better ranking of search results compared to the results of search services. Practical significance. The research results can be used in the development of the information retrieval systems, as well as for the further development of methods based on the distributive hypothesis.

Cheap IR evaluation

To evaluate Information Retrieval (IR) effectiveness, a possible approach is to use test collections, which are composed of a collection of documents, a set of description of information needs (called topics), and a set of relevant documents to each topic. Test collections are modelled in a competition scenario: for example, in the well known TREC initiative, participants run their own retrieval systems over a set of topics and they provide a ranked list of retrieved documents; some of the retrieved documents (usually the first ranked) constitute the so called pool, and their relevance is evaluated by human assessors; the document list is then used to compute effectiveness metrics and rank the participant systems. Private Web Search companies also run their in-house evaluation exercises; although the details are mostly unknown, and the aims are somehow different, the overall approach shares several issues with the test collection approach. The aim of this work is to: (i) develop and improve some state-of-the-art work on the evaluation of IR effectiveness while saving resources, and (ii) propose a novel, more principled and engineered, overall approach to test collection based effectiveness evaluation. In this thesis we focus on three main directions: the first part details the usage of few topics (i.e., information needs) in retrieval evaluation and shows an extensive study detailing the effect of using fewer topics for retrieval evaluation in terms of number of topics, topics subsets, and statistical power. The second part of this thesis discusses the evaluation without relevance judgements, reproducing, extending, and generalizing state-of-the-art methods and investigating their combinations by means of data fusion techniques and machine learning. Finally, the third part uses crowdsourcing to gather relevance labels, and in particular shows the effect of using fine grained judgement scales; furthermore, explores methods to transform judgements between different relevance scales. Awarded by: University of Udine, Udine, Italy on 19 March 2020. Supervised by: Professor Stefano Mizzaro. Available at: https://kevinroitero.com/resources/kr-phd-thesis.pdf.

Root Based Stemmer for Telugu Script

In this paper, a new stemmer has been proposed named as “Root based stemmer”. This stemmer is strictly based on Dravidian script. Stemming can be used to pick up the effectiveness of information retrieval. In proposed Root based stemming technique, each and every token is compared against with all the words of a valid root words dictionary until a match is found. Then extract the matched string or substring from a token and identified as valid root. The present work is aimed to build dictionary based stemmer to extract valid root words for Indian languages especially for Telugu and compare the results with existing stemmers.

Fewer topics? A million topics? Both?! On topics subsets in test collections

When evaluating IR run effectiveness using a test collection, a key question is: What search topics should be used? We explore what happens to measurement accuracy when the number of topics in a test collection is reduced, using the Million Query 2007, TeraByte 2006, and Robust 2004 TREC collections, which all feature more than 50 topics, something that has not been examined in past work. Our analysis finds that a subset of topics can be found that is as accurate as the full topic set at ranking runs. Further, we show that the size of the subset, relative to the full topic set, can be substantially smaller than was shown in past work. We also study the topic subsets in the context of the power of statistical significance tests. We find that there is a trade off with using such sets in that significant results may be missed, but the loss of statistical significance is much smaller than when selecting random subsets. We also find topic subsets that can result in a low accuracy test collection, even when the number of queries in the subset is quite large. These negatively correlated subsets suggest we still lack good methodologies which provide stability guarantees on topic selection in new collections. Finally, we examine whether clustering of topics is an appropriate strategy to find and characterize good topic subsets. Our results contribute to the understanding of information retrieval effectiveness evaluation, and offer insights for the construction of test collections.

A new approach for query expansion using Wikipedia and WordNet

Query expansion (QE) is a well-known technique used to enhance the effectiveness of information retrieval. QE reformulates the initial query by adding similar terms that help in retrieving more relevant results. Several approaches have been proposed in literature producing quite favorable results, but they are not evenly favorable for all types of queries (individual and phrase queries). One of the main reasons for this is the use of the same kind of data sources and weighting scheme while expanding both the individual and the phrase query terms. As a result, the holistic relationship among the query terms is not well captured or scored. To address this issue, we have presented a new approach for QE using Wikipedia and WordNet as data sources. Specifically, Wikipedia gives rich expansion terms for phrase terms, while WordNet does the same for individual terms. We have also proposed novel weighting schemes for expansion terms: in-link score (for terms extracted from Wikipedia) and a tf-idf based scheme (for terms extracted from WordNet). In the proposed Wikipedia-WordNet-based QE technique (WWQE), we weigh the expansion terms twice: first, they are scored by the weighting scheme individually, and then, the weighting scheme scores the selected expansion terms concerning the entire query using correlation score. The proposed approach gains improvements of 24% on the MAP score and 48% on the GMAP score over unexpanded queries on the FIRE dataset. Experimental results achieve a significant improvement over individual expansion and other related state-of-the-art approaches. We also analyzed the effect on retrieval effectiveness of the proposed technique by varying the number of expansion terms.

In quest of new document relations: evaluating co-opinion relations between co-citations and its impact on Information retrieval effectiveness

Document relational network has been effective in retrieving and evaluating papers. Despite their effectiveness, relational measures, including co-citation, are far from ideal and need improvements. The assumption underlying the co-citation relation is the content relevance and opinion relatedness of cited and citing papers. This may imply existence of some kind of co-opinionatedness between co-cited papers which may be effective in improving the measure. Therefore, the present study tries to test the existence of this phenomenon and its role in improving information retrieval. To do so, based on CITREC, a medical test collection was developed consisting of 30 queries (seed documents) and 4823 of their co-cited papers. Using NLP techniques, the co-citances of the queries and their co-cited papers were analyzed and their similarities were computed by 4 g similarity measure. Opinion scores were extracted from co-citances using SentiWordnet. Also, nDCG values were calculated and then compared in terms of the citation proximity index (CPI) and co-citedness measures before and after being normalized by the co-opinionatedness measure. The reliability of the test collection was measured by generalizability theory. The findings suggested that a majority of the co-citations exhibited a high level of co-opinionatedness in that they were mostly similar either in their opinion strengths or in their polarities. Although anti-polar co-citations were not trivial in their number, a significantly higher number of the co-citations were co-polar, with a majority being positive. The evaluation of the normalization of the CPI and co-citedness by the co-opinionatedness indicated a generally significant improvement in retrieval effectiveness. While anti-polar similarity reduced the effectiveness of the measure, the co-polar similarity proved to be effective in improving the co-citedness. Consequently, the co-opinionatedness can be presented as a new document relation and used as a normalization factor to improve retrieval performance and research evaluation.

A PSO Strategy of Finding Relevant Web Documents using a New Similarity Measure

In the world of the Internet and World Wide Web, which offers a tremendous amount of information, an increasing emphasis is being given to searching services and functionality. Currently, a majority of web portals offer their searching utilities, be it better or worse. These can search for the content within the sites, mainly text the textual content of documents. In this paper a novel similarity measure called SMDR (Similarity Measure for Documents Retrieval) is proposed to help retrieve more similar documents from the repository thus contributing considerably to the effectiveness of Web Information Retrieval (WIR) process. Bio-inspired PSO methodology is used with the intent to reduce the response time of the system and optimizes WIR process, hence contributes to the efficiency of the system. This paper also demonstrates a comparative study of the proposed system with the existing method in terms of accuracy, sensitivity, F-measure and specificity. Finally, extensive experiments are conducted on CACM collections. Better precision-recall rates are achieved than the existing system. Experimental results demonstrate the effectiveness and efficiency of the proposed system.

Document-based and term-based linear methods for pseudo-relevance feedback

Query expansion is a successful approach for improving Information Retrieval effectiveness. This work focuses on pseudo-relevance feedback (PRF) which provides an automatic method for expanding queries without explicit user feedback. These techniques perform an initial retrieval with the original query and select expansion terms from the top retrieved documents. We propose two linear methods for pseudo-relevance feedback, one document-based and another term-based, that models the PRF task as a matrix decomposition problem. These factorizations involve the computation of an inter-document or inter-term similarity matrix which is used for expanding the original query. These decompositions can be computed by solving a least squares regression problem with regularization and a non-negativity constraint. We evaluate our proposals on five collections against state-of-the-art baselines. We found that the term-based formulation provides high figures of MAP, nDCG and robustness index whereas the document-based formulation provides very cheap computation at the cost of a slight decrease in effectiveness.