Classifying patient portal messages using Convolutional Neural Networks

Abstract

ObjectivePatients communicate with healthcare providers via secure messaging in patient portals. As patient portal adoption increases, growing messaging volumes may overwhelm providers. Prior research has demonstrated promise in automating classification of patient portal messages into communication types to support message triage or answering. This paper examines if using semantic features and word context improves portal message classification. Materials and methodsPortal messages were classified into the following categories: informational, medical, social, and logistical. We constructed features from portal messages including bag of words, bag of phrases, graph representations, and word embeddings. We trained one-versus-all random forest and logistic regression classifiers, and convolutional neural network (CNN) with a softmax output. We evaluated each classifier’s performance using Area Under the Curve (AUC). ResultsRepresenting the messages using bag of words, the random forest detected informational, medical, social, and logistical communications in patient portal messages with AUCs: 0.803, 0.884, 0.828, and 0.928, respectively. Graph representations of messages outperformed simpler features with AUCs: 0.837, 0.914, 0.846, 0.884 for informational, medical, social, and logistical communication, respectively. Representing words with Word2Vec embeddings, and mapping features using a CNN had the best performance with AUCs: 0.908 for informational, 0.917 for medical, 0.935 for social, and 0.943 for logistical categories. Discussion and conclusionWord2Vec and graph representations improved the accuracy of classifying portal messages compared to features that lacked semantic information such as bag of words, and bag of phrases. Furthermore, using Word2Vec along with a CNN model, which provide a higher order representation, improved the classification of portal messages.