Automatic classification of sentences to support Evidence Based Medicine

Su Nam Kim,Lars Yencken,David Martinez,Lawrence Cavedon

doi:10.1186/1471-2105-12-s2-s5

Su Nam Kim, Lars Yencken + Show 2 more

Open Access

https://doi.org/10.1186/1471-2105-12-s2-s5

Copy DOI

Abstract

AimGiven a set of pre-defined medical categories used in Evidence Based Medicine, we aim to automatically annotate sentences in medical abstracts with these labels.MethodWe constructed a corpus of 1,000 medical abstracts annotated by hand with specified medical categories (e.g. Intervention, Outcome). We explored the use of various features based on lexical, semantic, structural, and sequential information in the data, using Conditional Random Fields (CRF) for classification.ResultsFor the classification tasks over all labels, our systems achieved micro-averaged f-scores of 80.9% and 66.9% over datasets of structured and unstructured abstracts respectively, using sequential features. In labeling only the key sentences, our systems produced f-scores of 89.3% and 74.0% over structured and unstructured abstracts respectively, using the same sequential features. The results over an external dataset were lower (f-scores of 63.1% for all labels, and 83.8% for key sentences).ConclusionsOf the features we used, the best for classifying any given sentence in an abstract were based on unigrams, section headings, and sequential information from preceding sentences. These features resulted in improved performance over a simple bag-of-words approach, and outperformed feature sets used in previous work.

Highlights

Evidence Based Medicine (EBM) is an approach to clinical practice whereby medical decisions are informed by primary evidence, such as the results of randomized control trials (RCTs)
Of the features we used, the best for classifying any given sentence in an abstract were based on unigrams, section headings, and sequential information from preceding sentences
These features resulted in improved performance over a simple bag-of-words approach, and outperformed feature sets used in previous work

Summary

Introduction

Evidence Based Medicine (EBM) is an approach to clinical practice whereby medical decisions are informed by primary evidence, such as the results of randomized control trials (RCTs). Evidence-based practitioners use specific criteria when judging whether an RCT is relevant to a given question They generally follow the PICO criterion [1]: Population (P) (i.e., participants in a study); Intervention (I); Comparison (C) (if appropriate); and Outcome (O) (of an Intervention). The classes Background and Outcome are the most confused: 496 errors (23% of the total) when Outcome is the goldstandard label, and 272 errors (13% of the total) when Background is the goldstandard label. This seems to indicate that the structure from the abstracts is helpful in avoiding these types of errors. The label Other is responsible for a high proportion of errors, being confused with Outcome 245 times (12% of the total)

Objectives

Methods

Results

Conclusion