Classifying disease outbreak reports using n-grams and semantic features

Abstract

Introduction This paper explores the benefits of using n-grams and semantic features for the classification of disease outbreak reports, in the context of the BioCaster disease outbreak report text mining system. A novel feature of this work is the use of a general purpose semantic tagger – the USAS tagger – to generate features. Background We outline the application context for this work (the BioCaster epidemiological text mining system), before going on to describe the experimental data used in our classification experiments (the 1000 document BioCaster corpus). Feature sets Three broad groups of features are used in this work: Named Entity based features, n-gram features, and features derived from the USAS semantic tagger. Methodology Three standard machine learning algorithms – Naïve Bayes, the Support Vector Machine algorithm, and the C4.5 decision tree algorithm – were used for classifying experimental data (that is, the BioCaster corpus). Feature selection was performed using the χ 2 feature selection algorithm. Standard text classification performance metrics – Accuracy, Precision, Recall, Specificity and F-score – are reported. Results A feature representation composed of unigrams, bigrams, trigrams and features derived from a semantic tagger, in conjunction with the Naïve Bayes algorithm and feature selection yielded the highest classification accuracy (and F-score). This result was statistically significant compared to a baseline unigram representation and to previous work on the same task. However, it was feature selection rather than semantic tagging that contributed most to the improved performance. Conclusion This study has shown that for the classification of disease outbreak reports, a combination of bag-of-words, n-grams and semantic features, in conjunction with feature selection, increases classification accuracy at a statistically significant level compared to previous work in this domain.