Transfer Learning in Classifying Prescriptions and Keyword-based Medical Notes

Abstract

Medical text classification is one of the primary steps of health care automation. Diagnosing disease at the right time, and going to the right doctor is important for patients. To do that, two types of medical texts were classified into some medical specialties in this study. The first one is the keywords-based medical notes and the second one is the prescriptions. There are many methods and techniques to classify texts from any domain. But, textual resources of a specific domain can be inadequate to build a sustainable and accurate classifier. This problem can be solved by incorporating transfer learning. The objective of this study is to analyze the prospects of transfer learning in medical text classification. To do that, a transfer learning system has been created for classification tasks by fine-tuning Bidirectional Encoder Representations from Transformers aka the BERT language model, and its performance has been compared with three deep learning models - multi-layer perceptron, long short-term memory, and convolutional neural network. The fine-tuned BERT model has shown the best performance among all the other models in both classification tasks. It has 0.84 and 0.96 weighted f1-score in classifying medical notes and prescriptions respectively. This study has proved that transfer learning can be used in medical text classification, and significant improvement in performance can be achieved through it.