Probabilistic Information Retrieval Research Articles

A risk minimization framework for information retrieval

This paper presents a probabilistic information retrieval framework in which the retrieval problem is formally treated as a statistical decision problem. In this framework, queries and documents are modeled using statistical language models, user preferences are modeled through loss functions, and retrieval is cast as a risk minimization problem. We discuss how this framework can unify existing retrieval models and accommodate systematic development of new retrieval models. As an example of using the framework to model non-traditional retrieval problems, we derive retrieval models for subtopic retrieval, which is concerned with retrieving documents to cover many different subtopics of a general query topic. These new models differ from traditional retrieval models in that they relax the traditional assumption of independent relevance of documents.

Probabilistic information retrieval model for a dependency structured indexing system

Most previous information retrieval (IR) models assume that terms of queries and documents are statistically independent from each other. However, conditional independence assumption is obviously and openly understood to be wrong, so we present a new method of incorporating term dependence into a probabilistic retrieval model by adapting a dependency structured indexing system using a dependency parse tree and Chow Expansion to compensate the weakness of the assumption. In this paper, we describe a theoretic process to apply the Chow Expansion to the general probabilistic models and the state-of-the-art 2-Poisson model. Through experiments on document collections in English and Korean, we demonstrate that the incorporation of term dependences using Chow Expansion contributes to the improvement of performance in probabilistic IR systems.

An information retrieval model based on simple Bayesian networks

In this article a new probabilistic information retrieval (IR) model, based on Bayesian networks (BNs), is proposed. We first consider a basic model, which represents only direct relationships between the documents in the collection and the terms or keywords used to index them. Next, we study two versions of an extended model, which also represents direct relationships between documents. In either case the BNs are used to compute efficiently, by means of a new and exact propagation algorithm, the posterior probabilities of relevance of the documents in the collection given a query. The performance of the proposed retrieval models is tested through a series of experiments with several standard document collections. © 2003 Wiley Periodicals, Inc.

New informetric aspects of the Internet: some reflections - many problems

This paper poses more problems than it solves: it investigates the new (virtual) world of the Internet and the challenges that it offers for informetric analysis. The paper studies five different aspects. First of all, there is the increasing problem of data gathering in the Internet. The second topic is the Internet version of the informetric laws: are the same types of classical distributions valid or not? The third topic deals with scientometric aspects: can the clickable buttons (hyperlinks) in Web pages replace the role of classical references in scientific papers? It also contains a study of the Web impact factor and a discussion on ageing. The fourth topic discusses information retrieval (IR) aspects of search engines. It studies aspects of probabilistic IR as applied in these engines and poses the question of quantitative evaluation of IR (Web analogues of recall and precision). Lastly, aspects of complexity are discussed. The fractal nature of the Internet is highlighted and a modest attempt to measure it is given.

New informetric aspects of the Internet: some reflections - many problems

This paper poses more problems than it solves: it investigates the new (virtual) world of the Internet and the challenges that it offers for informetric analysis. The paper studies five different aspects. First of all, there is the increasing problem of data gathering in the Internet. The second topic is the Internet version of the informetric laws: are the same types of classical distributions valid or not? The third topic deals with scientometric aspects: can the clickable buttons (hyperlinks) in Web pages replace the role of classical references in scientific papers? It also contains a study of the Web impact factor and a discussion on ageing. The fourth topic discusses information retrieval (IR) aspects of search engines. It studies aspects of probabilistic IR as applied in these engines and poses the question of quantitative evaluation of IR (Web analogues of recall and precision). Lastly, aspects of complexity are discussed. The fractal nature of the Internet is highlighted and a modest attempt to measure it is given.

Indexing and retrieval of broadcast news

This paper describes a spoken document retrieval (SDR) system for British and North American Broadcast News. The system is based on a connectionist large vocabulary speech recognizer and a probabilistic information retrieval (IR) system. We discuss the development of a real-time Broadcast News speech recognizer, and its integration into an SDR system. Two advances were made for this task: automatic segmentation and statistical query expansion using a secondary corpus. Precision and recall results using the Text Retrieval Conference (TREC) SDR evaluation infrastructure are reported throughout the paper, and we discuss the application of these developments to a large scale SDR task based on an archive of British English broadcast news.

Probabilistic datalog: Implementing logical information retrieval for advanced applications

In the logical approach to information retrieval (IR), retrieval is considered as uncertain inference. Whereas classical IR models are based on propositional logic, we combine Datalog (function-free Horn clause predicate logic) with probability theory. Therefore, probabilistic weights may be attached to both facts and rules. The underlying semantics extends the well-founded semantics of modularly stratified Datalog to a possible worlds semantics. By using default independence assumptions with explicit specification of disjoint events, the inference process always yields point probabilities. We describe an evaluation method and present an implementation. This approach allows for easy formulation of specific retrieval models for arbitrary applications, and classical probabilistic IR models can be implemented by specifying the appropriate rules. In comparison to other approaches, the possibility of recursive rules allows for more powerful inferences, and predicate logic gives the expressiveness required for multimedia retrieval. Furthermore, probabilistic Datalog can be used as a query language for integrated information retrieval and database systems.

Testing the maximum entropy principle for information retrieval

A probabilistic information retrieval method using the Maximum Entropy Principle (MEP) was proposed by Cooper and Huizinga (1982). Several refinements of the MEP for information retrieval have been proposed by Kantor and Lee (1986, 1991), but the MEP has not been evaluated in any large database. This article examines the MEP retrieval method using the TREC5 database. The MEP is evaluated by several tests and compared with a naive ordering method and lexicographic ordering method. The MEP does not provide any startling improvement, and it works reasonably well only in the case of a small number of keys and a relatively small collection.

Comparing Boolean and probabilistic information retrieval systems across queries and disciplines

Whether using Boolean queries or ranking documents using document and term weights will result in better retrieval performance has been the subject of considerable discussion among document retrieval system users and researchers. We suggest a method that allows one to analytically compare the two approaches to retrieval and examine their relative merits. The performance of information retrieval systems may be determined either by using experimental simulation, or through the application of analytic techniques that directly estimate the retrieval performance, given values for query and database characteristics. Using these performance predicting techniques, sample performance figures are provided for queries using the Boolean and and or, as well as for probabilistic systems assuming statistical term independence or term dependence. The variation of performance across sublanguages (used in different academic disciplines) and queries is examined. The performance of models failing to meet statistical and other assumptions is examined.

Bayesian Belief Networks: Odds and Ends

In artificial intelligence research, the belief network framework for automated reasoning with uncertainty is rapidly gaining in popularity. The framework provides a powerful formalism for representing a joint probability distribution on a set of statistical variables. In addition, it offers algorithms for efficient probabilistic inference. At present, more and more knowledge-based systems employing the framework are being developed for various domains of application ranging from probabilistic information retrieval to medical diagnosis. This paper provides a tutorial introduction to the belief network framework and highlights some issues of ongoing research in applying the framework for real-life problem solving.

A network approach to probabilistic information retrieval

In this article we show how probabilistic information retrieval based on document components may be implemented as a feedforward (feedbackward) artificial neural network. The network supports adaptation of connection weights as well as the growing of new edges between queries and terms based on user relevance feedback data for training, and it reflects query modification and expansion in information retrieval. A learning rule is applied that can also be viewed as supporting sequential learning using a harmonic sequence learning rate. Experimental results with four standard small collections and a large Wall Street Journal collection (173,219 documents) show that performance of feedback improves substantially over no feedback, and further gains are obtained when queries are expanded with terms from the feedback documents. The effect is much more pronounced in small collections than in the large collection. Query expansion may be considered as a tool for both precision and recall enhancement. In particular, small query expansion levels of about 30 terms can achieve most of the gains at the low-recall high-precision region, while larger expansion levels continue to provide gains at the high-recall low-precision region of a precision recall curve.

TREC and TIPSTER experiments with inquery

INQUERY is a probabilistic information retrieval system based upon a Bayesian inference network model. This paper describes recent improvements to the system as a result of participation in the TIPSTER project and the TREC-2 conference. Improvements include transforming forms-based specifications of information needs into complex structured queries, automatic query expansion, automatic recognition of features in documents, relevance feedback, and simulated document routing. Experiments with one- and two-gigabyte document collections are also described.

Some inconsistencies and misidentified modeling assumptions in probabilistic information retrieval

Some inconsistencies and misidentified modeling assumptions in probabilistic information retrieval

Term dependence: Truncating the Bahadur Lazarsfeld expansion

The performance of probabilistic information retrieval systems is studied where differing statistical dependence assumptions are used when estimating the probabilities inherent in the retrieval model. Experimental results using the Bahadur Lazarsfeld expansion suggest that the greatest degree of performance increase is achieved by incorporating term dependence information in estimating Pr( d¦ rel). It is suggested that incorporating dependence in Pr( d¦ rel) to degree 3 be used; incorporating more dependence information results in relatively little increase in performance. Experiments examine the span of dependence in natural language text, the window of terms in which dependencies are computed, and their effect on information retrieval performance. Results provide additional support for the notion of a window of ± 3 to ± 5 terms in width; terms in this window may be most useful when computing dependence.

Probabilistic information retrieval as a combination of abstraction, inductive learning, and probabilistic assumptions

We show that former approaches in probabilistic information retrieval are based on one or two of the three concepts abstraction, inductive learning , and probabilistic assumptions , and we propose a new approach which combines all three concepts. This approach is illustrated for the case of indexing with a controlled vocabulary. For this purpose, we describe a new probabilistic model first, which is then combined with logistic regression, thus yielding a generalization of the original model. Experimental results for the pure theoretical model as well as for heuristic variants are given. Furthermore, linear and logistic regression are compared.

Probabilistic Models in Information Retrieval

In this paper, an introduction and survey over probabilistic information retrieval (IR) is given. First, the basic concepts of this approach are described: the probability-ranking principle shows that optimum retrieval quality can be achieved under certain assumptions; a conceptual model for IR along with the corresponding event space clarify the interpretation of the probabilistic parameters involved. For the estimation of these parameters, three different learning strategies are distinguished, namely query-related, document-related and description-related learning. As a representative for each of these strategies, a specific model is described. A new approach regards IR as uncertain inference; here, imaging is used as a new technique for estimating the probabilistic parameters, and probabilistic inference networks support more complex forms of inference. Finally, the more general problems of parameter estimations, query expansion and the development of models for advanced document representations are discussed.

A study of probabilistic information retrieval systems in the case of inconsistent expert judgments

The maximum entropy principle may be applied to the design of probabilistic retrieval systems. When there are inconsistent expert judgments, the resulting optimization problem cannot be solved. The inconsistency of the expert judgments can be revealed by solving a linear programming formulation. In the case of inconsistent judgment, four plausible schemes are proposed in order to find revised judgments which are consistent with the true data structure but still reflect the original expert judgment. These schemes are the Interactive, Minimum Distance, Minimum Cross-Entropy, and Path methods. Background and Purpose of the Study The maximum entropy principle (MEP) based on Shannon’s measure (Shannon, 1928) has been used with great success in many areas. Cooper and Huizinga (1982) and Cooper (1983) have applied the MEP to the design of probabilistic information retrieval systems. Specifically, we consider a collection of documents which are categorized into Boolean components by attributes. The MEP estimates the probability of “relevance” of each Boolean component by integrating expert judgments about the “relevance” of attributes with the observed distribution of the Boolean components. The MEP retrieval system, in response to a user’s request, provides an ordering of the Boolean components using this estimated probability of “relevance.” Cooper (1983) has noted the potential of the MEP retrieval system as follows

SAPHIRE—An information retrieval system featuring concept matching, automatic indexing, probabilistic retrieval, and hierarchical relationships

SAPHIRE (Semantic and Probabilistic Heuristic Information Retrieval Environment) is an experimental computer program designed to test new techniques in automated information retrieval in the biomedical domain. A main feature of the program is a concept-finding algorithm that processes free text to find canonical concepts. The algorithm is designed to handle a wide variety of synonyms and convert them to canonical form. This allows natural language to be used for query input and also serves as the basis for a new approach to automatic indexing based on a combination of probabilistic and linguistic methods.

Experiments with a component theory of probabilistic information retrieval based on single terms as document components

A component theory of information retrieval using single content terms as component for queries and documents was reviewed and experimented with. The theory has the advantages of being able to (1) bootstrap itself, that is, define initial term weights naturally based on the fact that items are self relevent; (2) make use of within-item term frequencies; (3) account for query-focused and document-focused indexing and retrieval strategies cooperatively; and (4) allow for component-specific feedback if such information is available. Retrieval results with four collections support the effectiveness of all the first three aspects, except for predictive retrieval. At the initial indexing stage, the retrieval theory performed much more consistantly across collections than croft's model and provided results comparable to Salton's tf*idf approach. An inverse collection term frequency (ICTF) formula was also tested that performed much better than the inverse document frequency (IDF). With full feedback retrospective retrieval, the component theory performed substantially better than Croft's, because of the highly specific nature of document-focused feedback. Repetitive retireval results with partial relevance feedback mirrored those for the retrospective. However, for the important case of predictive retrieval using residual ranking, results were not unequivocal.

A sensitivity analysis of a probabilistic information retrieval system

Results of a set of exploratory simulations to test the effects of errors in estimation of individual term probabilities on the performance of a probabilistic information retrieval system are presented. Searches were executed with various levels of term error on a test collection of probabilistically indexed information sources. The amount of error in the final probability of relevance used to rank sources introduced by these errors was analytically determined. The first simulation analyzed simulated rankings obtained by simulating probabilities of relevance according to the error distribution; in the second simulation measures of rank correlation between simulated rankings and the system's ranking were calculated; in the third, the effect of the error on retrieval performance using the measure expected search length was determined. It was found that substantial error was introduced into final probabilities of relevance, but that for low levels of term error the impact on ranking and retrieval performance was moderate, while even with high levels the actual ranking performed significantly better than a random ranking of retrieved sources. © 1990 John Wiley & Sons, Inc.