Experiments in automatic extracting and indexing

Abstract

This article reports on several experiments in automatic extracting and on one experiment in automatic indexing. Nine chapters, each from a different technical book, were used as the text corpus for all the experiments. In the first experiment, an attempt was made to construct a sentence dictionary of syntactic sentence types, for distinguishing extract-worthy sentences, but it proved unrewarding. Nevertheless, the results indicated that sentence typing might be used in a screening process in conjunction with other extracting techniques. The later attempts to combine syntactic and statistical criteria in the choice of extract sentences and index phrases proved more rewarding. The sentences selected by the extracting algorithm were representative and are presented for the reader to peruse. The noun phrases selected by the indexing algorithm compared favorably with the back-of-the-book index phrases. There is every indication that satisfactory back-of-the-book indexes could be produced automatically, with post-editing to delete superfluous items.