Abstract

This paper presents work in progress toward the development of the information retrieval (IR) evaluation measure - Information Problem Shift- within a theoretical framework and model for conceptualizing and researching information retrieval (IR) within an information seeking context. The theoretical model consists of a set of situated actions by information-seekers within interactive search sessions with information retrieval (IR) systems over a period of time. First, the paper outlines the IR evaluation measure - Information Problem Shift - within the theoretical framework of the model. Second, the paper discusses work in progress in the form of two studies currently collecting data, including the proposed IR evaluation measure - one study of information-seekers using an IR system and another study of information-seekers using the Web. Overall, this research seeks to develop an integrated view of the interactive IR processes by information-seekers and an IR evaluation measure with utility for IR system researchers, designers and information-seekers themselves.

Highlights

  • This paper presents work in progress toward the development of a user-based information retrieval (IR) evaluation measure - Information Problem Shift – within the context of an integrated model of information seeking and retrieving

  • The relationship between interactive IR and HIB will further develop as both fields mature theoretically and intellectually, and the emergence of models and research findings from empirical studies of human information behavior and seeking (Wilson, 1997; Ellis, 1989; Kuhlthau, 1993) and interactive IR (Belkin, Cool, Stein & Theil, 1995; Ingwersen, 1992, 1996; Saracevic, 1996a, 1997)

  • We propose that a meaningful IR evaluation measure must be useful to IR researchers and designers, and should be meaningful and useful for people using IR systems by measuring what is important to those information-seekers in the form of a self-assessment tool

Read more

Summary

INTRODUCTION

This paper presents work in progress toward the development of a user-based IR evaluation measure - Information Problem Shift – within the context of an integrated model of information seeking and retrieving. The further integration of research and elements of both fields is emerging as an important area of research (Spink, 1998: Vakkari, 1998; Wilson, 1998), to the development of more effective Web and IR systems design and evaluation. The relationship between interactive IR and HIB will further develop as both fields mature theoretically and intellectually, and the emergence of models and research findings from empirical studies of human information behavior and seeking (Wilson, 1997; Ellis, 1989; Kuhlthau, 1993) and interactive IR (Belkin, Cool, Stein & Theil, 1995; Ingwersen, 1992, 1996; Saracevic, 1996a, 1997). This paper outlines our proposed IR evaluation measure – Information Problem Shift – including the limitations of existing measures, criteria for an IR evaluation measure and a theoretical framework and model as the basis of the IR evaluation measure

Limitations of Existing IR Evaluation Measures
Criteria for IR Evaluation Measures
Problem-Solving Time
INTERACTIVE SEARCH SESSIONS
SET OF SITUATED ACTIONS
Degrees of Relevance
Information Problem Shift
OPERATIONALIZATION STUDIES
CONCLUSIONS
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call