Kernel compositional embedding and its application in linguistic structured data classification

Abstract

In many applications such as natural language processing, speech recognition, and computer vision, there are inputs with hierarchical compositional structure and long relation among their subcomponents. Introducing this information in the definition of a model can improve its performance in dealing with this type of data. On the other side, the high generalization power of kernel methods is proven in traditional machine learning problems. If we can employ some idea of these methods on handling structured objects, we can benefit from improving the performance and the generalization capability of compositional models. Accordingly, a new approach is introduced in this paper to realize the idea of simultaneously leveraging advantages of both kernel methods and compositional embedding to provide powerful representations and classifiers for structured data. Based on this approach, which is named Kernel Compositional Embedding (KCE), we propose two methods: Direct KCE (DKCE), and Indirect KCE (IKCE). In the DKCE method, we directly deal with a potentially infinite dimensional embedding space, i.e., the embeddings are used implicitly in the classification task. In IKCE method, instead of implicitly performing all operations inside embedding space, only some elements of one or more reproducing kernel Hilbert spaces are used to embed structured objects into low-dimensional Euclidean spaces. To evaluate the performance of the proposed methods, we apply them on two common computational linguistic tasks, i.e, sentiment analysis and natural language inference. The experimental results illustrate that the classification performance of the proposed methods is higher than or competitive to some well-known competing methods.