Improved Identification of Imbalanced Multiple Annotation Intent Labels with a Hybrid BLSTM and CNN Model and Hybrid Loss Function

Abstract

Payment or fund transfer transactions can be annotated by users when they are made through a mobile banking app, for example, SCB Easy app—a mobile banking app by Siam Commercial Bank—allows users to annotate transactions with 40 character texts. The AI\(^2\) framework was used to identify user intentions with the transactions, so that the bank can offer the right product to the right customer at the right time. The framework employed Long Short-Term Memory (LSTM). Commonly, one annotated sample can be interpreted as representing multiple intents, thus we had a multiple label classification problem. However, the original model did not consider the class imbalance, that caused the model to bias toward the majority class. We introduced a new hybrid Bidirectional LSTM and Convolutional Neural Network model in conjunction with a new hybrid loss function to tackle the imbalance. Our model with hybrid loss function performed better than the AI\(^2\) framework with a 4.5% improvement in \(F_1\)-score. Moreover, our hybrid loss function enabled the model to classify minority classes better, when the imbalance ratio became higher, compared with a conventional cross-entropy loss function. In other words, our hybrid loss function made the model to be more efficient in real-world multiple label imbalance problem.