Document Retrieval Process Research Articles

Document clustering analysis with aid of adaptive Jaro Winkler with Jellyfish search clustering algorithm

In this research, document clustering is analyzed with the help of Adaptive Jaro Winkler with Jellyfish Search Clustering (AJWJSC) algorithm and Chimp Optimization Algorithm (COA). The major motive of the research is to compute the relevant topics with the easiest way in addition reduce the complexity of the domain analysis. The document retrieval process is analysis for the recent topic detection and identification. In the research, four stages are considered to analyze the documents, named as pre-processing, feature extraction, feature knowledge establishment, in addition document clustering phase. Initially, the documents can be pre-processed with the consideration on tokenization, stop-word removal and stemming methods. After that, the Relative Document-Term Frequency Difference (RD-TFD) technique can be utilized to extract the features. Based on the extracted feature set, the essential features are nominated with the help of Chimp Optimization Algorithm (COA). Afterwards, the document clustering process is computed using the Adaptive Jaro Winkler with Jellyfish Search Clustering (AJWJSC) algorithm. The novelty of the work is to document clustering from the documents for utilized in different applications. The proposed method is designed in the MATLAB platform and analyzed with the three datasets named as Reuter database, 20 Newsgroups and Topic detection and tracking (TDT2) database respectively. While analyzing the statistical measures of the research, various parameters is determined like precision, Recall, F-measure, accuracy, and the efficiency. Here, the clustered documents are validated based on their similarity and this is used for the document retrieval purpose. The projected technique can be contrasted with the k-means Clustering, Krill herd (KH) algorithm, and Moth Flame Optimization (MFO) Algorithm respectively.

Euclidean Distance Based Similarity Measurement and Ensuing Ranking Scheme for Document Search from Outsourced Cloud Data

In this paper, we propose the Euclidean Distance based Similarity Measurement and Ensuing Ranking (EDSMER) scheme to aid effective document search from outsourced cloud data. It is another attempt to find an alternative to binary based approaches. In this approach, the User or the Data owner needs to filter out the suitable keywords for the document and then the index is prepared. To provide security and privacy, both the data and the index are encrypted and moved to the cloud space. The application of Euclidean Distance based Similarity Measurement and Ensuing Ranking (EDSMER) scheme for document searching takes place after the authorized user requests for the documents through query terms. Initially the authorized user sends a query to Cloud Service Provider to retrieve all the documents which are mapped with the keywords provided by him. The proposed algorithm calculates the distance between the query terms and the index terms. The minimum the distance, the more it is closer towards each other and vice-versa. Our Euclidean Distance based Similarity Measurement and Ensuing Ranking (EDSMER) scheme greatly enhances the system functionality by sending the most relevant documents instead of transmitting all documents back. The experimental validations are performed on RFC and FIRE dataset. Through experimental analysis, we prove that our proposed approach is secure and efficient as well as exhibits better recall and precision rate in the IR system to deal with the document-retrieval process.

Document Retrieval and Cluster Based Indexing using Rider Spider Monkey Optimization Algorithm

Document retrieval process is more significant in the field of research community for retrieving the highly-relevant documents that fit for the user query. Even though various document retrieval methods are introduced, retrieving the exact document based on the indexing is a quite challenging task in the document retrieval framework. Thus, an effective document retrieval algorithm named Rider Spider Monkey Optimization Algorithm (RSOA) is proposed in this research. Initially, the documents are pre-processed by the stop word elimination and the stemming process, and the features are extracted to find the key words of the documents by applying the Term Frequency-Inverse Document Frequency (TF-IDF). The selected keywords are passed into the cluster-based indexing phase, where the cluster centroids are identified by using the proposed Rider Spider Monkey Optimization Algorithm. Moreover the query matching is carried out at two levels, at first, the query is forwarded and is matched to the entire cluster centroid to find the appropriate centroid. At the second level; the user query is matched based on the records present inside the matched centroid. Moreover, the query matching is progressed using the distance measure by the Bhattacharya distance to retrieve the documents. The performance is analyzed using the metrics, namely precision, F-measure, and recall and accuracy with the values of 90.141%, 91.876%, 91.178%, and 91.202%, respectively using 20 news group dataset .

A Novel Securable Fuzzy Logic Based Ranking Scheme for Document Searching on Outsourced Cloud Data

Most of the existing searchable encryption scheme allows the user to search for the encrypted data with the support of Boolean search and this scheme are not so much effective to meet the data requirements of the user in the presence of large amount of documents in cloud. In this paper, we propose an effective Securable Fuzzy Logic based Ranking mechanism for document searching on outsourced cloud data. Our approach employs ECC based encryption and a fuzzy logic based ranking scheme over the encrypted-data to retrieve the documents from the cloud. The newly developed fuzzy logic based ranking scheme adopts six query-expansion (QE) ‘terms selection’ methods for computing the degrees of all the unique-terms comprised in the top retrieved document. Further, our fuzzy logic based ranking mechanism greatly enhances the system functionality by sending top-most relevant documents based on the relevance scores obtained for the term selection methods and increases the document retrieval accuracy by sending alone the top-most relevant documents instead of transmitting all documents back. As a result, data security is increased by reducing the communication and computational overhead. The experimental validations are performed on RFC and FIRE dataset. Through experimental analysis, we prove that our proposed approach is highly secure and efficient as well as exhibits better recall and precision rate in the IR system to deal with the document-retrieval process.

Efficient Document Retrieval using Annotation, Searching and Ranking

is always difficult to find relevant information in unstructured text documents. In this paper we study the methods of fuzzy search, instant search and proximity ranking and how they can be used in the process of annotation of documents. These various methods can be integrated to give better search results and to achieve efficient space and time complexities. We propose a novel alternative approach which facilitates the generation of the structured metadata automatically using OpenNLP, methods of Instant-fuzzy search and Proximity ranking. It is done by identifying documents which are likely to contain the information of interest. And this information will be subsequently useful for querying the database. Fig 1: Document retrieval If a user wants an efficient document retrieval process then annotation, document searching methods and ranking methods play a vital role in whole retrieval process. Here we discuss what these techniques are and how these different techniques are used in this document retrieval system. Ranking: In the process of ranking every query answer is ranked based on its similarity or relevance to query, it is defined on various information pieces like co-occurrence of some keywords of query as a phrase in record and the query keywords frequencies in the record. Domain-specific features can play a vital role in ranking. E.g., for some publication, number of citations can be used as an indication in ranking because it is a good indicator of its impact. The Phrase matching effect in ranking gives better results. E.g., for the query q = bbrain, surgeryii, record containing the phrase brain surgery is more relevant than the record containing the

A method for collaborative recommendation using knowledge integration tools and hierarchical structure of user profiles

This paper proposes a new approach to collaborative profile recommendation using a hierarchical structure for user modeling. In an information retrieval system a hierarchical user profile, used to personalize the document retrieval process, is being recommended to a new user based on profiles of other, similar users. Using methodology from the Knowledge Integration domain, four criteria are defined and analyzed to complete the aim of recommendation: Reliability is required for maintaining the correct structure of the profile, O1 and O2 Optimality postulates are required to calculate the best output profile by minimizing distances to other profiles, and Conflict Solution is used to better represent situations inherent to profile recommendation. Based on those criteria, four algorithms are proposed: O1 and O2 algorithms and modified O1 and O2 algorithms. These algorithms are further analyzed to check if they provide good recommendation.

Soft clustering: An overview

Document clustering has been extensively investigated as a methodology for improving document retrieval process. In Traditional clustering algorithm each documents belongs to exactly one cluster & hence cannot detect the multiple themes of a document where as soft clustering algorithm each document can belong to multiple clusters. This paper gives a comparative study of hard clustering & soft clustering algorithm.

User keyword preference: the Nwords and Rwords experiments

Everyday, millions of people use some form of text-based interface to search inefficiently for information. This reflects a lack of penetration of key developments in Human Computer Interaction (HCI) designed to expedite document retrieval. In the context of document search, the value of textual language is self-evident for searching natural language documents. However, there has been little human factors research into general user preferences such as the number of words preferred to describe or search for a document. Such heuristics can help to balance processing overheads of tasks like clustering against user cognitive load in the realisation of, overall, more efficient document retrieval process. We report the results from two internet delivered experiments designed to obtain this heuristic information.

Text retrieval with more realistic concept matching and reinforcement learning

This paper reports our experimental investigation into the use of more realistic concepts as opposed to simple keywords for document retrieval, and reinforcement learning for improving document representations to help the retrieval of useful documents for relevant queries. The framework used for achieving this was based on the theory of Formal Concept Analysis (FCA) and Lattice Theory. Features or concepts of each document (and query), formulated according to FCA, are represented in a separate concept lattice and are weighted separately with respect to the individual documents they present. The document retrieval process is viewed as a continuous conversation between queries and documents, during which documents are allowed to learn a set of significant concepts to help their retrieval. The learning strategy used was based on relevance feedback information that makes the similarity of relevant documents stronger and non-relevant documents weaker. Test results obtained on the Cranfield collection show a significant increase in average precisions as the system learns from experience.

On recommending

AbstractThe core of any document retrieval system is a mechanism that ranks the documents in a large collection in order of the likelihood with which they match the preferences of any person who interacts with the system. Given a broader interpretation of “recommending” than is commonly accepted, such a preference ordering may be viewed as a recommendation, made by the system to the information seeker, that is itself typically derived through synthesis of multiple preference orderings expressed as recommendations by indexers, information seekers, and document authors. The ERIn (Evaluation–Recommendation–Information) model, a decision‐theoretic framework for understanding information‐related activity, highlights the centrality of recommending in the document retrieval process, and may be used to clarify the respects in which indexing, rating, and citation may be considered analogous, as well as to make explicit the points at which content‐based, collaboration‐based, and context‐based flavors of document retrieval systems vary.

Discovering Text Databases with Neural Nets

Since documents on the Web are naturally partitioned into many text databases, the efficient document retrieval process requires identifying the text databases that are most likely to provide relevant documents to the query and then searching for the identified text databases. In this paper, we propose a neural net based approach to such an efficient document retrieval. First, we present a neural net agent that learns about underlying text databases from the user's relevance feedback. For a given query, the neural net agent, which is sufficiently trained on the basis of the BPN learning mechanism, discovers the text databases associated with the relevant documents and retrieves those documents effectively. In order to scale our approach with the large number of text databases, we also propose the hierarchical organization of neural net agents which reduces the total training cost at the acceptable level. Finally, we evaluate the performance of our approach by comparing it to those of the conventional well-known approaches.

Incorporating fuzzy logic with neural networks for document retrieval

Classical methods such as Boolean searches, the vector space model and probabilistic retrieval have been applied to the field of document retrieval. However, these methods cannot handle the increasing demands of end-users in satisfying their needs. Recently, there has been specific interest in adapting advanced computer techniques in the field of document retrieval. The most recent attempt investigates the possibility of applying the neural-network paradigm as a means of providing end-users with a more powerful retrieval mechanism. Most of the work done thus far has been focused on incorporating neural networks into the existing classical retrieval methods. This paper demonstrates how to apply neural networks with fuzzy logic for document retrieval. In particular, the Fuzzy Kohonen Neural Network (FKNN) is used as an example to illustrate the versatility of fuzzy neural networks as applied to the document-retrieval process. In this paper, the training, pattern recall, ranking, relevance feedback and performance issues of the FKNN document-retrieval process are discussed.

Trends in research on information retrieval — The potential for improvements in conventional Boolean retrieval systems

Operational retrieval systems are firmly embedded within the pure Boolean framework, and the theoretical model underlying these systems is based on the implicit assumption that documents and user information needs can be precisely and completely characterized by sets of index terms and Boolean search request formulations, respectively. However, this assumption must be considered grossly inaccurate since uncertainty is intrinsic to the document retrieval process. The inability of the standard Boolean model to deal effectively with the inherent fallibility of retrieval decisions is the main reason for a number of serious deficiencies exhibited by present-day operational retrieval systems. This article reviews recent advances in information retrieval research and examines their practical potential for overcoming these deficiencies. The primary source for this review is the subsequent articles that comprise this special issue of Information Processing & Management, although earlier results published elsewhere have also been considered.

Document representation in probabilistic models of information retrieval

AbstractProbabilistic models of retrieval have provided insights into the document retrieval process and contain the basis for very effective search strategies. A major limitation of these models is that they assume that documents are represented by binary index terms. In many cases the index terms will be assigned weights, such as within‐document frequency weights, which are derived from the content of the documents by the indexing process. These weights, which are referred to here as term significance weights, indicate the relative importance of the terms in individual documents. This article describes how retrieval models which use either independence or dependence assumptions can be extended to include document representatives containing term significance weights. Comparison with other research indicates that search strategies based on models modified in this way can further improve the effectiveness of document retrieval systems.

Fuzzy set theoretical approach to document retrieval

The aim of a document retrieval system is to issue documents which contain the information needed by a given user of an information system. The process of retrieving documents in response to a given query is carried out by means of the search patterns of these documents and the query. It is thus clear that the quality of this process, i.e. the pertinence of the information system response to the information need of a given user depends on the degree of accuracy in which document and query contents are represented by their search patterns. It seems obvious that the weighting of descriptors entering document search patterns improves the quality of the document retrieval process. A mathematical apparatus which takes into consideration, in a natural manner, the fact that the grades of importance of the descriptors in document search patterns are of the continuum type, that is an apparatus adequate to the description of a retrieval system of documents indexed by weighted descriptors is—among known mathematical methods—the theory of fuzzy sets, formulated by L.A. Zadeh. It is the aim of this paper to present a new method of document retrieval based on the fundamental operations of the fuzzy set theory. We start by introducing basic notions, then the syntax and semantics of the proposed language for document retrieval will be given and an algorithm allocating documents to particular queries will be described and its properties discussed. The basic advantage of the use of the fuzzy set theory for document retrieval system description is that it takes into consideration, in a simple way, the differentiation of the importance of descriptors in document search patterns and the differentiation of the formal relevance grades of particular documents of an information system to a given query. Documents of the highest grades (in the given information system) of formal relevance to the given query may be retrieved by means of the application of simple operations of the fuzzy set theory.

Syracuse information retrieval experiment (SIRE)

Proponents of interactive online bibliographic retrieval systems frequently stress the potential of such systems to take advantage of both human and computer capabilities in the document retrieval process. Yet systems implemented to date have to a great extent placed constraints on both human and machine performance by decisions made at the design stage. To increase the flexibility of online systems, alternative modes of document representation, storage utilization, and query formulation should be considered.