Distributional Semantics of Clinical Words

Abstract

Word embeddings are the distributed representation of the words in numerical form. Recent research in word embeddings shows the importance of using them in deep learning algorithms. Word embeddings are commonly leveraged as feature inputs to many deep learning models. A lot of research in word embeddings helped in various open source releases of embeddings to the larger deep learning community. However, these embeddings are trained on generic corpus which limits their use for domain specific tasks. In our paper, we propose a transfer learning based approach to train the skip gram model for medical terms. In addition to pre-trained embeddings, we also added customized clinical knowledge for each term in our training data. We have compared our word embeddings with Google pre-trained word embeddings. Both the pre-trained embeddings and embeddings from our approach are used for training a Named entity recognition classifier and follow-up detection use case on radiology reports. We are able to achieve improvement in the Fl-score and also observed faster convergence in respective NLP classification tasks.