An Augmented Intelligence Model to Extract Pragmatic Markers from Large Content Repositories

Abstract

This paper presents a novel methodology for automatically extracting pragmatic markers from large streams of texts and repositories of documents. Pragmatic markers typically are implications, innuendos, suggestions, contradictions, sarcasms or references that are difficult to define objectively, but that are subjectively evident. Our methodology uses a two-stage augmented learning model applied to a specific use case, extracting from a repository of over 1500 Article IV country reports prepared for government officials by International Monetary Fund (IMF) staff. The model uses principles of evidence theory to train a machine to decipher the textual patterns of suggested actions for government officials and to extract those suggestions from the country reports at scale. We demonstrate the effectiveness of the model with impressive precision and recall metrics that over time outperform even the human trainers.