Integrating Deep Learning for NLP in Romanian Psychology

Abstract

With the emergence of efficient word embeddings for free text, there has been a bloom of Natural Language Processing (NLP) breakthroughs using deep learning techniques. However, the literature is skewed towards the languages that offer large corpuses, and little research has been done in the direction of Romanian text. In this paper, we propose a Deep Learning (DL)-based system for classifying free sentences in the context of psychological surveys, automatically discovering whether respondees are talking about the expected subject in their answers (thoughts, emotions or behaviors) or not. We test several new architectures of convolutional and recurrent neural networks using pre-trained word embeddings from a very large corpus consisting of Wikipedia and Common Crawl data sets. We also present the benefits of transfer learning applied to NLP, allowing general language models trained on large data sets to be applied to problems that have a small amount of data, as in our case, allowing applications of these techniques in many fields.