Two End-to-End Quantum-Inspired Deep Neural Networks for Text Classification

Jinjing Shi,Ronghua Shi,Yanyan Feng,Shichao Zhang,Wei Lai,Fangfang Li,Zhenhuan Li

doi:10.1109/tkde.2021.3130598

Abstract

In linguistics, the uncertainty of context due to polysemy is widespread, which attracts much attention. Quantum-inspired complex word embedding based on Hilbert space plays an important role in natural language processing (NLP), which fully leverages the similarity between quantum states and word tokens. A word containing multiple meanings could correspond to a single quantum particle which may exist in several possible states, and a sentence could be analogous to the quantum system where particles interfere with each other. Motivated by quantum-inspired complex word embedding, interpretable complex-valued word embedding (ICWE) is proposed to design two end-to-end quantum-inspired deep neural networks (ICWE-QNN and CICWE-QNN representing convolutional complex-valued neural network based on ICWE) for binary text classification. They have the proven feasibility and effectiveness in the application of NLP and can solve the problem of text information loss in CE-Mix [1] model caused by neglecting the important linguistic features of text, since linguistic feature extraction is presented in our model with deep learning algorithms, in which gated recurrent unit (GRU) extracts the sequence information of sentences, attention mechanism makes the model focus on important words in sentences and convolutional layer captures the local features of projected matrix. The model ICWE-QNN can avoid random combination of word tokens and CICWE-QNN fully considers textual features of the projected matrix. Experiments conducted on five benchmarking classification datasets demonstrate our proposed models have higher accuracy than the compared traditional models including CaptionRep BOW, DictRep BOW and Paragram-Phrase, and they also have great performance on F1-score. Eespecially, CICWE-QNN model has higher accuracy than the quantum-inspired model CE-Mix as well for four datasets including SST, SUBJ, CR and MPQA. It is a meaningful and effictive exploration to design quantum-inspired deep neural networks to promote the performance of text classification.

Highlights

T EXT classification [2], [3], [4] as a basic task in natural language processing (NLP) has been researched for a long time, where a large number of superior deep neural networks like AC-BLSTM [5] are applied with remarkable performance on experiments
There is a common problem for lack of sufficient interpretability in above text classification schemes with deep neural networks, which means that it is hard to explain how the ”black box” works in classification tasks, and how to enhance the model interpretability is a recognized question worth exploring in recent years [11]
The second model called CICWE-QNN we propose is presented for more remarkable performance on classification task by applying convolutional layer [22] on the projected matrix for considering the complete information and capturing local textual features, motivated by the extraction for the joint representation of question-answer pairs [14]

Summary

Introduction

T EXT classification [2], [3], [4] as a basic task in NLP has been researched for a long time, where a large number of superior deep neural networks like AC-BLSTM [5] are applied with remarkable performance on experiments. It benefits from the powerful capabilities of deep learning methods such as recurrent neural network (RNN) [6] and convolution neural network (CNN) [7] for feature extraction. There is a common problem for lack of sufficient interpretability in above text classification schemes with deep neural networks, which means that it is hard to explain how the ”black box” works in classification tasks, and how to enhance the model interpretability is a recognized question worth exploring in recent years [11]

Methods

Results

Conclusion