Experimental study of index terms and information access

Abstract

AbstractIn this paper we report on the methodology used in an experiment whose objective was to discover whether information seekers display experimentally demonstrable preferences for index terms with particular properties, and if so, to analyze those preferences. For the experiment, we merged three sets of terms: two sets were identified automatically and one manually. 23 subjects used the merged list of terms via ESBI (Experimental Searching and Browsing Interface) to answer a predetermined set of questions of varying degrees of difficulty. By design, the interface discourages direct searching and forces users to access content indirectly via index terms. Our results provide strong evidence that it is indeed possible to measure human preference for index terms: subjects displayed a strong preference for the manual index terms, which on average were significantly longer and structurally more complex than the automatically identified terms. However, the automatically identified index terms were selected often enough to suggest that they too are of value. We discuss the design of the computer interface, the process of question selection, the preparation of the merged list of index terms, the techniques for assessment of question difficulty, the experimental protocol, and the methods we used for assessing the validity of the experimental results.