The impact of MeSH (Medical Subject Headings) terms on information seeking effectiveness

Abstract

To what extent do MeSH (Medical Subject Headings) terms improve search effectiveness of different kinds of users? We observed four different kinds of searchers using an experimental information retrieval (IR) system: (1) search novices; (2) domain experts; (3) search experts and (4) medical librarians. The information needs were a subset of the relatively difficult topics originally created for the Text REtrieval Conference (TREC). By experimental design, we used 20 search topics in an IR user experiment to alleviate search topic variability. Effectiveness of retrieval was based on the relevance judgments set provided by TREC. Thirty-two participants searched either using a version of the system in which abstracts and MeSH terms were displayed or another version in which they had to formulate their own terms based only on the display of abstracts. We found that MeSH terms were more useful for domain experts than for search experts in terms of the precision measure, even though domain experts did not perceive that MeSH terms were useful. We speculate that because of the technical topics, only the domain experts had the knowledge to understand and therefore make use of the MeSH terms. The primary contributions of this research are: (1) assessment of relative impact of searchers characteristics of domain knowledge and search training on search effectiveness and (2) design and methodology for assessing the usefulness of controlled vocabulary. The effort to create MeSH terms is worthwhile for domain experts' searches on technical topics.