Data Fusion In Information Retrieval Research Articles

Exploiting the Local Optima in Genetic Algorithm using Tabu Search

Objectives: To explores the process of selecting retrieval schemes along with their weights, and fusion function for data fusion in information retrieval. Methods/Statistical Analysis: This has been carried out using the hybrid Genetic Algorithm. The fusion function, retrieval schemes and their weights lead to a tremendous combination. Finding an optimal solution from this great combination is entirely based on the exploration. Findings: We used, odd and even point crossover as an exploration tool. This exploration tool suffers a setback of slow convergence. The convergence rate can be improved by merging Tabu search, a best local search, with the genetic algorithm. This Tabu GA is used to select the retrieval schemes, weights and fusion function. The outcome of the experiments conducted over the test data sets namely: 1. adi, 2. cisi, and 3. cranlooks promising. We achieved 6.89% of improvement in performance, and the significance of the result is tested statistically. The convergence rate is also improved. Application/Improvements: We achieved 6.89% of improvement in performance, and the significance of the result is tested statistically. The convergence rate is also improved. Keywords: Genetic Algorithm, Information Retrieval, Odd and Even Point Crossover, Tabu GA, Tabu Search

A geometric framework for data fusion in information retrieval

Data fusion in information retrieval has been investigated by many researchers and a number of data fusion methods have been proposed. However, problems such as why data fusion can increase effectiveness and favorable conditions for the use of data fusion methods are poorly resolved at best. In this paper, we formally describe data fusion under a geometric framework, in which each component result returned from an information retrieval system for a given query is represented as a point in a multi-dimensional space. The Euclidean distance is the measure by which the effectiveness and similarity of search results are judged. This allows us to explain all component results and fused results using geometrical principles. In such a framework, score-based data fusion becomes a deterministic problem. Several interesting features of the centroid-based data fusion method and the linear combination method are discussed. Nevertheless, in retrieval evaluation, ranking-based measures are the most popular. Therefore, this paper investigates the relation and correlation between the Euclidean distance and several typical ranking-based measures. We indeed find that a very strong correlation exists between these. It means that the theorems and observations obtained using the Euclidean distance remain valid when ranking-based measures are used. The proposed framework enables us to have a better understanding of score-based data fusion and use score-based data fusion methods more precisely and effectively in various ways.

A Novel Class-Based Data Fusion Technique for Information Retrieval

Data fusion in information retrieval combines the results from multiple retrieval models or document representations. The achievement of data fusion technique is dependent on the quality of the inputs; classical data fusion techniques fail to improve the retrieval if the quality of the retrieval results varies from low to high quality. In order to tackle this problem, in this paper we address the issue of high variation among the retrieval strategies or document representations which affect the combination of their outputs. Our investigation on the MALACH speech collection – in which different segment representations are available – shows that neither the classical data fusion (CombSUM) nor the weighted version (WCombSum) improve the retrieval. We propose a novel class-based data fusion technique to deal with this issue. The segments retrieved by models based on different document representations are classified according to the quality of the segment into three classes: high, intermediate, and low quality class; then the similarity scores of each segment are fused using the classical CombSUM. Our experimental results show that the new technique is significantly better than CombSUM or WCombSUM in combing results with high quality variation.

Data fusion according to the principle of polyrepresentation

AbstractWe report data fusion experiments carried out on the four best‐performing retrieval models from TREC 5. Three were conceptually/algorithmically very different from one another; one was algorithmically similar to one of the former. The objective of the test was to observe the performance of the 11 logical data fusion combinations compared to the performance of the four individual models and their intermediate fusions when following the principle of polyrepresentation. This principle is based on cognitive IR perspective (Ingwersen & Järvelin, 2005) and implies that each retrieval model is regarded as a representation of a unique interpretation of information retrieval (IR). It predicts that only fusions of very different, but equally good, IR models may outperform each constituent as well as their intermediate fusions. Two kinds of experiments were carried out. One tested restricted fusions, which entails that only the inner disjoint overlap documents between fused models are ranked. The second set of experiments was based on traditional data fusion methods. The experiments involved the 30 TREC 5 topics that contain more than 44 relevant documents. In all tests, the Borda and CombSUM scoring methods were used. Performance was measured by precision and recall, with document cutoff values (DCVs) at 100 and 15 documents, respectively. Results show that restricted fusions made of two, three, or four cognitively/algorithmically very different retrieval models perform significantly better than do the individual models at DCV100. At DCV15, however, the results of polyrepresentative fusion were less predictable. The traditional fusion method based on polyrepresentation principles demonstrates a clear picture of performance at both DCV levels and verifies the polyrepresentation predictions for data fusion in IR. Data fusion improves retrieval performance over their constituent IR models only if the models all are quite conceptually/algorithmically dissimilar and equally and well performing, in that order of importance.

Applying statistical principles to data fusion in information retrieval

Data fusion in information retrieval has been investigated by many researchers and quite a few data fusion methods have been proposed. However, their effect on effectiveness has not been well understood. In this paper, we apply statistical principles to data fusion and obtain some useful conclusions, which can be used as a guideline for data fusion methods. Based on that, CombSum, the linear combination methods, and the correlation methods can be justified in certain conditions. We also investigate how to improve the effectiveness of some existing data fusion methods such as CombSum and the linear combination method. Experimental results with TREC data are reported to support the conclusions.

Comparing Rank and Score Combination Methods for Data Fusion in Information Retrieval

Combination of multiple evidences (multiple query formulations, multiple retrieval schemes or systems) has been shown (mostly experimentally) to be effective in data fusion in information retrieval. However, the question of why and how combination should be done still remains largely unanswered. In this paper, we provide a model for simulation and a framework for analysis in the study of data fusion in the information retrieval domain. A rank/score function is defined and the concept of a Cayley graph is used in the design and analysis of our framework. The model and framework have led us to better understanding of the data fusion phenomena in information retrieval. In particular, by exploiting the graphical properties of the rank/score function, we have shown analytically and by simulation that combination using rank performs better than combination using score under certain conditions. Moreover, we demonstrated that the rank/score function might be used as a predictive variable for the effectiveness of combination of multiple evidences.