A Hybrid Neural Network RBERT-C Based on Pre-trained RoBERTa and CNN for User Intent Classification

Abstract

User intent classification plays a critical role in identifying the interests of users in question-answering and spoken dialog systems. The question texts of these systems are usually short and their conveyed semantic information are frequently insufficient. Therefore, the accuracy of user intent classification related to user satisfaction may be affected. To address the problem, this paper proposes a hybrid neural network named RBERT-C for text classification to capture user intent. The network uses the Chinese pre-trained RoBERTa to initialize representation layer parameters. Then, it obtains question representations through a bidirectional transformer structure and extracts essential features using a Convolutional Neural Network after question representation modeling. The evaluation is based on the publicly available dataset ECDT containing 3736 labeled sentences. Experimental result indicates that our model RBERT-C achieves a F1 score of 0.96 and an accuracy of 0.96, outperforming a number of baseline methods.