Vector Space Retrieval Model Research Articles

Evaluation of Average Term Occurrences Weighting Technique for Arabic Textual Information Retrieval

Information retrieval of documents is an important process in the current time, and the vector space retrieval model uses a term weighting scheme as a basic method for matching queries with documents. Term frequency-Inverse document frequency is a widely used and famous term weighting scheme, and many studies proved its effectiveness in information retrieval. However, this term weighting scheme has some drawbacks like retrieving irrelevant documents, which sometimes reduces effectiveness. From this point, a new term weighting scheme called Term Frequency with Average Term Occurrence was proposed and experienced in the English language to minimize retrieving unnecessary documents. In this paper, an information retrieval system is built for the Arabic language, and Open-Source Arabic Corpora was used to complete experiments. Calculations were made using two schemes which are traditional Term frequency-inverse Document Frequency and proposed Term Frequency with Average Term Occurrence. After that, comparisons of results were made using evaluation measures. With all obtained queries, four case studies with two approaches (stop word removal and stemming) are implemented. In English experiments, stop word removal was applied with another discriminative approach, which calculates the centroid of documents. After the analysis of the results, it was found that the proposed scheme is applicable on Arabic text and applied approaches enhance IR effectiveness if they are both implemented. Furthermore, it was found that stop word removal has a favorable effect on both schemes which was also proved in English experiments.

Association retrieval and recommendation of agricultural environmental sensing data based on knowledge map

When the current general methods are used to retrieve agricultural environmental sensing data, there is no knowledge map constructed, and there are often problems sunch as low recall rate, low precision rate and low retrieval efficiency. In order to retrieve agricultural environmental sensing data efficiently, accurately and comprehensively, the method of association retrieval and recommendation of agricultural environmental sensing data based on knowledge map was proposed. The NLP algorithm was used to associate knowledge concepts of various agricultural environmental sensing data to construct the knowledge map of agricultural environmental sensing data. The knowledge map was pruned and expanded to reduce the amount of related calculations, and improved the retrieval efficiency of agricultural environmental sensing data. On the basis of the knowledge map of agricultural environmental sensing data, the vector space retrieval model was further used to realize the association retrieval and recommendation of agricultural environmental sensing data. To compared with the data retrieval method based on the quantification of the product within the cluster and the agricultural environment before the optimization of the knowledge map Sensing data association retrieval and recommendation method, the relevant experimental results showed that when the number of retrieved data sets increased, the recall rate, precision rate, and retrieval efficiency of the agricultural environmental sensing data association retrieval and recommendation method based on the knowledge map optimized after the knowledge map constructed in this paper were higher than the other two retrieval methods, which had high practical applicability and development prospects.

Exploiting Wikipedia API for Hindi-english Cross-language Information Retrieval

The rapidly increasing demographics of the internet population and the abundance of multilingual content on the web increased the communication in multiple languages. Most of the people use their regional languages to express their needs and the language diversity becomes a great barrier. Cross-Language Information Retrieval (CLIR) provides a solution for that language barrier which allows a user to ask a query in the native language and get the relevant documents in the different language. In this paper, we proposed a Wikipedia API based query translation approach. Queries are tokenized and multi-words query terms are created using N-gram technique. Wikipedia title and inter-wiki link features are exploited for query translation. Target language documents are retrieved using vector space retrieval model and BM25 retrieval algorithm. Experiment results shows that the proposed approach achieves better results without exploiting any language resources.

Salton and Buckley’s Landmark Research in Experimental Text Information Retrieval

Salton and Buckley’s Landmark Research in Experimental Text Information Retrieval

A personalised information support system for searching portals and e‐resources

PurposeThe purpose of this paper is to describe the development of a personalised information support system to help faculty members to search various portals and e‐resources without typing the search terms in different interfaces and to obtain results re‐ordered without human intervention.Design/methodology/approachAfter a careful survey of various tools and techniques available for computerised client‐centred information services, the study selected to apply web usage mining, proxy level data collection and a vector space retrieval model to develop the personalised information support for teaching and research in a higher education institution.FindingsThere are practical constraints in the implementation stage. There is considerable difficulty in getting real and correct user interests and mapping them effectively into the products and services offered by the library. Also the interests of users change continuously. If multiple users share the same PC, it is difficult to identify the user as there is no one‐to‐one mapping between user and IP address.Research limitations/implicationsThe paper has not considered cases for all the faculty members due to time constraints. The results obtained from the system also need correlation with the sources actually used by the faculty to test its efficacy in a highly fluid research situation like higher academics.Practical implicationsA pragmatic client‐centred information support prototype described in this paper may find use in other institutions needing similar information support.Originality/valueThis paper demonstrates the pragmatic application of ICT for linking users and e‐resources in an academic library.

Meeting of the MINDS

Since its inception in the late 1950s, the field of Information Retrieval (IR) has developed tools that help people find, organize, and analyze information. The key early influences on the field are well-known. Among them are H. P. Luhn's pioneering work, the development of the vector space retrieval model by Salton and his students, Cleverdon's development of the Cranfield experimental methodology, Spärck Jones' development of idf , and a series of probabilistic retrieval models by Robertson and Croft. Until the development of the WorldWideWeb (Web), IR was of greatest interest to professional information analysts such as librarians, intelligence analysts, the legal community, and the pharmaceutical industry.

정보검색 성능 향상을 위한 단어 중의성 해소 모형에 관한 연구

This paper presents a semantic vector space retrieval model incorporating a word sense disambiguation algorithm in an attempt to improve retrieval effectiveness. Nine Korean homonyms are selected for the sense disambiguation and retrieval experiments. The total of approximately 120,000 news articles comprise the raw test collection and 18 queries including homonyms as query words are used for the retrieval experiments. A Naive Bayes classifier and EM algorithm representing supervised and unsupervised learning algorithms respectively are used for the disambiguation process. The Naive Bayes classifier achieved disambiguation accuracy. while the clustering performance of the EM algorithm is on the average. The retrieval effectiveness of the semantic vector space model incorporating the Naive Bayes classifier showed precision achieving about improvement. However, the retrieval effectiveness of the EM algorithm-based semantic retrieval is lower than the baseline retrieval without disambiguation. It is worth noting that the performances of disambiguation and retrieval depend on the distribution patterns of homonyms to be disambiguated as well as the characteristics of queries.

한국어-영어/일본어-영어 교차언어정보검색에서 클러스터 분석을 통한 성능 향상

본 논문에서는 교차언어정보검색에서 점진적 클러스터링을 통해서 모호성을 묵시적으로 해소하는 방법을 제안한다. 연구 목적은 질의 번역에서 모호성이 크게 증가된 상태에서 문서 클러스터가 문서 문맥 역할과 모호성 해소 역할을 하는지를 보고자 하는 것이다. 제안하는 방법은 한국어/일본어 질의를 사전을 이용하여 영어로 번역을 하고, 번역된 영어 질의에 대해서 벡터공간검색모델이나 확률검색모델에 의해서 문서를 검색한다 검색된 문서의 순위대로 점진적 클러스터를 동적으로 생성하고, 이 클러스터 정보를 질의에 반영해서 문서의 순위를 다시 결정하는 것이다. TREC 테스트컬렉션을 이용한 실험에서 모호성 해소를 하지 않은 질의에 대해서, 제안한 방법은 한국어-영어 교차언어정보검색에서는 벡터공간검색모델에서 39.41%의 성능향상, 확률검색모델에서 36.79%의 성능향상을 보였다. 일-영 교차언어정보검색에서는 각각 17.59%와 30.46%의 성능향상을 보였다. 적합성 피드백 방법과의 비교에서는 모호성 해소를 하지 않은 경우 확률검색모델에서 12.30%의 성능향상을 보였다. 이를 통해, 클러스터 분석은 질의 모호성 해소에 도움을 주어서 검색성능 향상에 기여하였음을 알 수 있다. This paper presents a method to implicitly resolve ambiguities using dynamic incremental clustering in Korean-to-English and Japanese-to-English cross-language information retrieval (CLIR). The main objective of this paper shows that document clusters can effectively resolve the ambiguities tremendously increased in translated queries as well as take into account the context of all the terms in a document. In the framework we propose, a query in Korean/Japanese is first translated into English by looking up bilingual dictionaries, then documents are retrieved for the translated query terms based on the vector space retrieval model or the probabilistic retrieval model. For the top-ranked retrieved documents, query-oriented document clusters are incrementally created and the weight of each retrieved document is re-calculated by using the clusters. In the experiment based on TREC test collection, our method achieved 39.41% and 36.79% improvement for translated queries without ambiguity resolution in Korean-to-English CLIR, and 17.89% and 30.46% improvements in Japanese-to-English CLIR, on the vector space retrieval and on the probabilistic retrieval, respectively. Our method achieved 12.30% improvements for all translation queries, compared with blind feedback in Korean-to-English CLIR. These results indicate that cluster analysis help to resolve ambiguity.

An entropy‐based interpretation of retrieval status value‐based retrieval, and its application to the computation of term and query discrimination value

AbstractThe concepts of Shannon information and entropy have been applied to a number of information retrieval tasks such as to formalize the probabilistic model, to design practical retrieval systems, to cluster documents, and to model texture in image retrieval. In this report, the concept of entropy is used for a different purpose. It is shown that any positive Retrieval Status Value (RSV)‐based retrieval system may be conceived as a special probability space in which the amount of the associated Shannon information is being reduced; in this view, the retrieval system is referred to as Uncertainty Decreasing Operation (UDO). The concept of UDO is then proposed as a theoretical background for term and query discrimination power, and it is applied to the computation of term and query discrimination values in the vector space retrieval model. Experimental evidence is given as regards such computation; the results obtained compare well to those obtained using vector‐based calculation of term discrimination values. The UDO‐based computation, however, presents advantages over the vector‐based calculation: It is faster, easier to assess and handle in practice, and its application is not restricted to the vector space model. Based on the ADI test collection, it is shown that the UDO‐based Term Discrimination Value (TDV) weighting scheme yields better retrieval effectiveness than using the vector‐based TDV weighting scheme. Also, experimental evidence is given to the intuition that the choice of an appropriate weighting scheme and similarity measure depends on collection properties, and thus the UDO approach may be used as a theoretical basis for this intuition.

Implicit ambiguity resolution using incremental clustering in cross-language information retrieval

This paper presents a method to implicitly resolve ambiguities using dynamic incremental clustering in cross-language information retrieval (CLIR) such as Korean-to-English and Japanese-to-English CLIR. The main objective of this paper shows that document clusters can effectively resolve the ambiguities tremendously increased in translated queries as well as take into account the context of all the terms in a document. In the framework we propose, a query in Korean/Japanese is first translated into English by looking up bilingual dictionaries, then documents are retrieved for the translated query terms based on the vector space retrieval model or the probabilistic retrieval model. For the top-ranked retrieved documents, query-oriented document clusters are incrementally created and the weight of each retrieved document is re-calculated by using the clusters. In the experiment based on TREC CLIR test collection, our method achieved 39.41% and 36.79% improvement for translated queries without ambiguity resolution in Korean-to-English CLIR, and 17.89% and 30.46% improvements in Japanese-to-English CLIR, on the vector space retrieval and on the probabilistic retrieval, respectively. Our method achieved 12.30% improvement for all translation queries, compared with blind feedback for the probabilistic retrieval in Korean-to-English CLIR. These results indicate that cluster analysis help to resolve ambiguity.

Automatic cataloguing and searching for retrospective data by use of OCR text

This article describes our efforts in supporting information retrieval from OCR degraded text. In particular, we report our approach to an automatic cataloging and searching contest for books in multiple languages. In this contest, 500 books in English, German, French, and Italian published during the 1770s to 1970s are scanned into images and OCRed to digital text. The goal is to use only automatic ways to extract information for sophisticated searching. We adopted the vector space retrieval model, an n-gram indexing method, and a special weighting scheme to tackle this problem. Although the performance by this approach is slightly inferior to the best approach, which is mainly based on regular expression match, one advantage of our approach is that it is less language dependent and less layout sensitive, thus is readily applicable to other languages and document collections. Problems of OCR text retrieval for some Asian languages are also discussed in this article, and solutions are suggested.

Automatic cataloguing and searching for retrospective data by use of OCR text

This article describes our efforts in supporting information retrieval from OCR degraded text. In particular, we report our approach to an automatic cataloging and searching contest for books in multiple languages. In this contest, 500 books in English, German, French, and Italian published during the 1770s to 1970s are scanned into images and OCRed to digital text. The goal is to use only automatic ways to extract information for sophisticated searching. We adopted the vector space retrieval model, an n-gram indexing method, and a special weighting scheme to tackle this problem. Although the performance by this approach is slightly inferior to the best approach, which is mainly based on regular expression match, one advantage of our approach is that it is less language dependent and less layout sensitive, thus is readily applicable to other languages and document collections. Problems of OCR text retrieval for some Asian languages are also discussed in this article, and solutions are suggested.

GlOSS

The dramatic growth of the Internet has created a new problem for users: location of the relevant sources of documents. This article presents a framework for (and experimentally analyzes a solution to) this problem, which we call the text-source discovery problem . Our approach consists of two phases. First, each text source exports its contents to a centralized service. Second, users present queries to the service, which returns an ordered list of promising text sources. This article describes GlOSS , Glossary of Servers Server, with two versions: bGlOSS , which provides a Boolean query retrieval model, and vGlOSS , which provides a vector-space retrieval model. We also present hGlOSS , which provides a decentralized version of the system. We extensively describe the methodology for measuring the retrieval effectiveness of these systems and provide experimental evidence, based on actual data, that all three systems are highly effective in determining promising text sources for a given query.