Adaptive Information Retrieval System Research Articles

Is exploratory search different? A comparison of information search behavior for exploratory and lookup tasks

Exploratory search is an increasingly important activity yet challenging for users. Although there exists an ample amount of research into understanding exploration, most of the major information retrieval (IR) systems do not provide tailored and adaptive support for such tasks. One reason is the lack of empirical knowledge on how to distinguish exploratory and lookup search behaviors in IR systems. The goal of this article is to investigate how to separate the 2 types of tasks in an IR system using easily measurable behaviors. In this article, we first review characteristics of exploratory search behavior. We then report on a controlled study of 6 search tasks with 3 exploratory—comparison, knowledge acquisition, planning—and 3 lookup tasks—fact‐finding, navigational, question answering. The results are encouraging, showing that IR systems can distinguish the 2 search categories in the course of a search session. The most distinctive indicators that characterize exploratory search behaviors are query length, maximum scroll depth, and task completion time. However, 2 tasks are borderline and exhibit mixed characteristics. We assess the applicability of this finding by reporting on several classification experiments. Our results have valuable implications for designing tailored and adaptive IR systems.

Predicting users' domain knowledge in information retrieval using multiple regression analysis of search behaviors

User domain knowledge affects search behaviors and search success. Predicting a user's knowledge level from implicit evidence such as search behaviors could allow an adaptive information retrieval system to better personalize its interaction with users. This study examines whether user domain knowledge can be predicted from search behaviors by applying a regression modeling analysis method. We identify behavioral features that contribute most to a successful prediction model. A user experiment was conducted with 40 participants searching on task topics in the domain of genomics. Participant domain knowledge level was assessed based on the users' familiarity with and expertise in the search topics and their knowledge of MeSH (Medical Subject Headings) terms in the categories that corresponded to the search topics. Users' search behaviors were captured by logging software, which includes querying behaviors, document selection behaviors, and general task interaction behaviors. Multiple regression analysis was run on the behavioral data using different variable selection methods. Four successful predictive models were identified, each involving a slightly different set of behavioral variables. The models were compared for the best on model fit, significance of the model, and contributions of individual predictors in each model. Each model was validated using the split sampling method. The final model highlights three behavioral variables as domain knowledge level predictors: the number of documents saved, the average query length, and the average ranking position of the documents opened. The results are discussed, study limitations are addressed, and future research directions are suggested.

Adaptive information retrieval system via modelling user behaviour

There has been an exponential growth in the volume and variety of information available on the Internet, similarly there has been a significant demand from users’ for accurate information that matches their interests, however, the two are often incompatible because of the effectiveness of retrieving the exact information the user requires. This paper addresses this problem with an adaptive agent-based modelling approach that relies on evolutionary user-modelling. The proposed information retrieval system learns user needs from user-provided relevance feedback. It is proposed that retrieval effectiveness can be improved by applying computational intelligence techniques for modelling information needs, through interactive reinforcement learning. The method combines qualitative (subjective) user relevance feedback with quantitative (algorithmic) measures of the relevance of retrieved documents. An adaptive information retrieval system is developed whose retrieval effectiveness is evaluated using traditional precision and recall.

User relevance criteria choices and the information search process

Relevance judgments occur within an information search process, where time, context and situation can impact the judgments. The determination of relevance is dependent on a number of factors and variables which include the criteria used to determine relevance. The relevance judgment process and the criteria used to make those judgments are manifestations of the cognitive changes which occur during the information search process. Understanding why these relevance criteria choices are made, and how they vary over the information search process can provide important information about the dynamic relevance judgment process. This information can be used to guide the development of more adaptive information retrieval systems which respond to the cognitive changes of users during the information search process. The research data analyzed here was collected in two separate studies which examined a subject’s relevance judgment over an information search process. Statistical analysis was used to examine these results and determine if there were relationships between criteria selections, relevance judgments, and the subject’s progression through the information search process. Findings confirm and extend findings of previous studies, providing strong statistical evidence of an association between the information search process and the choices of relevance criteria by users, and identifying specific changes in the user preferences for specific criteria over the course of the information search process.

The evaluation of adaptive and personalised information retrieval systems: a review

A current problem with the research of adaptive systems is the inconsistency of evaluation applied to the adaptive systems. However, evaluating an adaptive system is a difficult task due to the complexity of such systems. Evaluators need to ensure correct evaluation methods and measurement metrics are used. This paper reviews a variety of evaluation techniques applied in adaptive and user-adaptive systems. More specifically, it focuses on the user-centred evaluation of adaptive systems such as personalised recommender systems and adaptive information retrieval systems. The review tackles the question of i¾˜How have user-centred evaluations of adaptive and user-adaptive systems been conducted and how can these evaluation practices be improved?' Based on the analysed results of the: (a) evaluation approaches, (b) user-centred evaluation techniques, and (c) evaluation metrics, we propose an evaluation framework for end-user experience in evaluating adaptive systems (EFEx).

Usagers et recherche d'information. Modélisation des usagers pour la recherche d'information adaptative

User profiles are the main component of adaptive information retrieval systems. We are identifying in this paper main limits of the usage of user profiles. We are discussing about these limits in underlying some solutions and taking into account two points of view: the application point of view and the user point of view.

AIR 2006

Adaptive Information Retrieval (IR) systems are designed to optimize retrieval effectiveness and user interaction in an underlying search environment. In this article, we report on a workshop held at Glasgow, UK, in October 2006. The workshop program consisted of three invited talks, four provocative presentations, and a panel discussion. In addition, 15 poster presentations enriched the workshop themes. Important pointers to further development of adaptive IR systems and their evaluation were suggested.

Towards a belief-revision-based adaptive and context-sensitive information retrieval system

In an adaptive information retrieval (IR) setting, the information seekers' beliefs about which terms are relevant or nonrelevant will naturally fluctuate. This article investigates how the theory of belief revision can be used to model adaptive IR. More specifically, belief revision logic provides a rich representation scheme to formalize retrieval contexts so as to disambiguate vague user queries. In addition, belief revision theory underpins the development of an effective mechanism to revise user profiles in accordance with information seekers' changing information needs. It is argued that information retrieval contexts can be extracted by means of the information-flow text mining method so as to realize a highly autonomous adaptive IR system. The extra bonus of a belief-based IR model is that its retrieval behavior is more predictable and explanatory. Our initial experiments show that the belief-based adaptive IR system is as effective as a classical adaptive IR system. To our best knowledge, this is the first successful implementation and evaluation of a logic-based adaptive IR model which can efficiently process large IR collections.

An information retrieval system based on a user profile

In this era of information explosion, providing the right information to the right person within reasonable time duration is a very important goal for today’s information retrieval (IR) systems. Due to the characteristics of retrieving methods, the conventional IR models often suffer from inaccurate and incomplete queries as well as inconsistent document relevance. Moreover, each user has his/her own interpretation of the semantic meaning of query terms during the retrieving process. Thus, high accuracy of retrieved information is somewhat hard to achieve. However, accuracy can be improved by designing an IR model that can adapt to the diverse needs of an individual and can perform a personalized search. In this paper, an adaptive information retrieval system embedded in an intelligent feedback tuning mechanism is proposed to capture the personal notions of query terms. This mechanism together with a correlation table is used as a user profile to ‘simulate’ the user’s notions of terms. This correlation table includes the degrees of semantic relevance (SR) and co-occurrence (CO) among terms. By using the personal profile during retrieval, the system can obtain a better match between the information needs and the retrieved results. It is also shown that the recall/precision rates for our system are approximately 91% and 81%, respectively, on average.

Shifts of interactive intentions and information-seeking strategies in interactive information retrieval

Research has demonstrated that people engage in multiple types of information-seeking strategies when using information retrieval (IR) systems; unfortunately, current IR systems are designed to support only one type of information-seeking strategy: specifying queries. The limitation of the existing IR systems calls for the need to investigate how to support users as they shift from one information-seeking strategy to another in their attempts to achieve their information-seeking goals. The focus of this study is on the in-depth investigation of shifts in the micro-level of user goals—“interactive intention” and information-seeking strategies that users engage in within an information-seeking episode. Forty cases of library uses were selected from four different types of libraries for this study. The qualitative and quantitative analysis of the data identifies four types of shifts of interactive intentions and three types of information-seeking strategies. The results of the study are discussed to understand the nature of the interactive IR process, and to further suggest their implications for the design of adaptive IR systems.

Outline of a dynamic self-tuning and adaptive information retrieval system

A self-tuning adaptive information retrieval system as an extension of the concept of a "classical" document retrieval system, is outlined. This system accepts documents and search requests in natural language, as well as the system-proposals previously produced by the system itself or prepared by the system operator. It produces a system-proposal that consists of a list of documents ranked according to their relevance to the query.Incorporated into the system is a system valuation subsystem that uses weighted relevance judgements. This subsystem gives as output an effectiveness value and an efficiency value: both together measure the quality of an information retrieval system.The computation of the quality values and the values themselves are independent of a specific implementation. The retrieval process in this system consists of two parts, namely a query-document match and a query-query match.