Benchmarking Shallow and Deep Neural Networks for Contextual Representation of Social Data

Abstract

Representing the underlying context in text data is a much-explored research domain, where language model construction for the sizeable unstructured corpus is the central premise. To date, several deep language embedding representation techniques have been put forth for context-aware modelling of text data, focusing on word, sentence and document-level representations for specific tasks. In this paper, we experiment with shallow and deep embedding representation techniques for social media text data to predict Atherosclerotic Heart Disease (AHD) mortality rate. We employed Word2Vec, Doc2Vec, and LSTM based embedding techniques for this experimentation and analyzed the performance on standard datasets. Experimental evaluation evidence suggests that Doc2Vec, a shallow network, outperforms deep neural networks by attaining a Pearson correlation value of 0.8199 for tuned hyper-parameters, exceeding Word2Vec and Bi-LSTM models by a margin of 60 per cent.