Cross-lingual Word Embeddings Research Articles

Borrow from rich cousin: transfer learning for emotion detection using cross lingual embedding

Performance of any natural language processing (NLP) system greatly depends on the amount of resources and tools available in a particular language or domain. Therefore, while solving any problem in low-resource setting, it is important to investigate techniques to leverage the resources and tools available in resource-rich languages. In this paper we propose an efficient technique to mitigate the problem of resource scarcity for emotion detection in Hindi by leveraging information from a resource-rich language like English. Our method follows a deep transfer learning framework which efficiently captures relevant information through the shared space of two languages, showing significantly better performance compared to the monolingual scenario that learns in the vector space of only one language. As base learning models, we use Convolution Neural Network (CNN) and Bi-Directional Long Short Term Memory (Bi-LSTM). As there are no available emotion labeled dataset for Hindi, we create a new dataset for emotion detection in disaster domain by annotating sentences of news documents with nine different classes based on Plutchikâ;;s wheel of emotions. To improve the performance of emotion classification in Hindi, we employ transfer learning to exploit the resources available in the related domains. The core of our approach lies in generating a cross-lingual word embedding representation of words in the shared embedding space. The neural networks are trained on the existing datasets, and then weights are fine-tuned following the four different transfer learning strategies for emotion classification in Hindi. We obtain a significant performance gain in our our proposed transfer learning techniques, achieving an F1-score of 0.53 (compared to 0.47)-thereby implying that knowledge from a resource-rich language can be transferred across language and domains.11codes and data available at https://github.com/zishanahmad1821cs18/crosslingual_transfer

Cross-lingual similar document retrieval methods

In this paper, we compare different methods for cross-lingual similar document retrieval. We focus on Russian-English language pair. We compare well-known methods like Cross Lingual Explicit Semantic Analysis (CL-ESA) with methods based on cross-lingual embeddings. We use approximate nearest neighbor (ANN) search to retrieve documents based entirely on distances between learned document embeddings. Also we employ a more traditional approach with usage of inverted index, with extra step of mapping top keywords from one language to other with the help of cross-lingual word embeddings. We use Russian-English aligned Wikipedia articles to evaluate all approaches. Conducted experiments show that an approach with inverted index achieves better performance in terms of recall and MAP than other methods.

Improving Vector Space Word Representations Via Kernel Canonical Correlation Analysis

Cross-lingual word embeddings are representations for vocabularies of two or more languages in one common continuous vector space and are widely used in various natural language processing tasks. A state-of-the-art way to generate cross-lingual word embeddings is to learn a linear mapping, with an assumption that the vector representations of similar words in different languages are related by a linear relationship. However, this assumption does not always hold true, especially for substantially different languages. We therefore propose to use kernel canonical correlation analysis to capture a non-linear relationship between word embeddings of two languages. By extensively evaluating the learned word embeddings on three tasks (word similarity, cross-lingual dictionary induction, and cross-lingual document classification) across five language pairs, we demonstrate that our proposed approach achieves essentially better performances than previous linear methods on all of the three tasks, especially for language pairs with substantial typological difference.