Enhanced attentive convolutional neural networks for sentence pair modeling

Abstract

Listen

Translate article

Sentence Pair Modeling is a critical and challenging problem in natural language processing (NLP). It aims to seek the underlying semantic relationship between two sentences. Inspired by human learning, attention mechanisms are widely used in NLP. Previous attentive CNNs mainly focus on attentive pooling which only compute the matching scores between sentence pairs with the same filter size and lack substantive interactive context. In this paper, we propose Enhanced Attentive Convolutional Neural Networks (EACNNs) for modeling sentence pairs to make full use of the characteristics of convolution. Enhanced attention mechanisms help strengthen the interaction between sentences via adding alignment context into local context in convolution operation and combining multi-grained similarity features in different filter sizes. We exploit two attention schemes: (i) attention before representation to capture the interaction information of sentence pairs by attentive convolution (AC) and multi-window advanced attention (MWA), and (ii) attention after representation to emphasize different importance of each word by multi-view similarity measurement layer (MVS). All the enhanced attention mechanisms can make our EACNNs outperform existing attentive CNN models. Furthermore, the combinations of these attention have strong competition in complicated LSTMs and show great advantage in training efficiency.