Year-end Sale % OFF 
Abstract
User intent classification is a vital component of a question-answering system or a task-based dialogue system. In order to understand the goals of users’ questions or discourses, the system categorizes user text into a set of pre-defined user intent categories. User questions or discourses are usually short in length and lack sufficient context; thus, it is difficult to extract deep semantic information from these types of text and the accuracy of user intent classification may be affected. To better identify user intents, this paper proposes a BERT-Cap hybrid neural network model with focal loss for user intent classification to capture user intents in dialogue. The model uses multiple transformer encoder blocks to encode user utterances and initializes encoder parameters with a pre-trained BERT. Then, it extracts essential features using a capsule network with dynamic routing after utterances encoding. Experiment results on four publicly available datasets show that our model BERT-Cap achieves a F1 score of 0.967 and an accuracy of 0.967, outperforming a number of baseline methods, indicating its effectiveness in user intent classification.
Highlights
In question-answering systems and task-driven dialogue systems, the classification of user intent is an essential task to understand the target of user questions or discourses
For the datasets with uneven distribution of categories, we focus on samples that are difficult to classify and improve the accuracy of user intent classification with focal loss
A Bidirectional Encoder Representations from Transformers (BERT)-Cap hybrid model with focal loss based on pretrained BERT and capsule network is newly proposed for user intent classification. e BERT-Cap model consists of four modules: input embedding, sequence encoding, feature extraction, and intent classification. e architecture of our model is shown Figure 1
Summary
In question-answering systems and task-driven dialogue systems, the classification of user intent is an essential task to understand the target of user questions or discourses. Spoken language understanding usually involves two sub-tasks, namely, user intent classification and semantic slot filling [2]. Deep neural network methods are frequently used to extract user short sentence features and classify users’ hidden intentions. Focal loss is an improved loss function based on the softmax function to improve the accuracy of classification task for uneven distribution datasets It is initially used in image detection tasks and has a positive effect on solving the imbalance of category distribution [13]. To further study user intent classification, a model BERT-Cap is proposed in this paper combining focal loss to solve the problem of uneven distribution of data. (2) Capsule network captures deep features of sentence representation obtained by the encoder and transfers iteratively important information from the lower-level capsule to the higher-level capsule through the dynamic routing mechanism.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
