Automatic subject indexing using an associative neural network

Abstract

The global growth in popularity of the World Wide Web has been enabled in part by the availability of browser based search tools which in turn have led to an increased demand for indexing techniques and technologies. As the amount of globally accessible information in community repositories grows, it is no longer cost-effective for such repositories to be indexed by professional indexers who have been trained to be consistent in subject assignment from controlled vocabulary lists. The era of amateur indexers is thus upon us, and the information infrastructure needs to provide support for such indexing if search of the Net is to produce useful results. In this paper, we propose the {\em Concept Assigner}, an automatic subject indexing system based on a variant of the Hopfield network \cite{Hopfield82}. In the application discussed herein, a collection of documents is used to automatically create a subset of a thesaurus termed a {\em Concept Space} \cite{ChenSchatz97}. To automatically index an individual document, concepts extracted from the given document become the input pattern to a Concept Space represented as a Hopfield network. The Hopfield net parallel spreading activation process produces another set of concepts that are strongly related to the concepts of the input document. Such concepts are suitable for use in an interactive indexing environment. A prototype of our automatic subject indexing system has been implemented as part of the {\em Interspace}, a semantic indexing and retrieval environment which supports statistically-based semantic indexing in a persistent object environment.