Learning Personal Human Biases and Representations for Subjective Tasks in Natural Language Processing

Abstract

Many tasks in natural language processing like offensive, toxic, or emotional text classification are subjective by nature. Humans tend to perceive textual content in their own individual way. Existing methods commonly rely on the agreed output values, the same for all consumers. Here, we propose personalized solutions to subjective tasks. Our four new deep learning models take into account not only the content but also the specificity of a given human. The models represent different approaches to learning the representation and processing data about text readers. The experiments were carried out on four datasets: Wikipedia discussion texts labelled with attack, aggression, and toxicity, as well as opinions annotated with ten numerical emotional categories. Emotional data was considered as multivariate regression (multitask), whereas Wikipedia data as independent classifications. All our models based on human biases and their representations significantly improve the prediction quality in subjective tasks evaluated from the individual’s perspective.