Source Words Research Articles

Syntax-Directed Attention for Neural Machine Translation

Attention mechanism, including global attention and local attention, plays a key role in neural machine translation (NMT). Global attention attends to all source words for word prediction. In comparison, local attention selectively looks at fixed-window source words. However, alignment weights for the current target word often decrease to the left and right by linear distance centering on the aligned source position and neglect syntax distance constraints. In this paper, we extend the local attention with syntax-distance constraint, which focuses on syntactically related source words with the predicted target word to learning a more effective context vector for predicting translation. Moreover, we further propose a double context NMT architecture, which consists of a global context vector and a syntax-directed context vector from the global attention, to provide more translation performance for NMT from source representation. The experiments on the large-scale Chinese-to-English and English-to-German translation tasks show that the proposed approach achieves a substantial and significant improvement over the baseline system.

A comparison of graph-based word sense induction clustering algorithms in a pseudoword evaluation framework

This article presents a comparison of different Word Sense Induction (wsi) clustering algorithms on two novel pseudoword data sets of semantic-similarity and co-occurrence-based word graphs, with a special focus on the detection of homonymic polysemy. We follow the original definition of a pseudoword as the combination of two monosemous terms and their contexts to simulate a polysemous word. The evaluation is performed comparing the algorithm’s output on a pseudoword’s ego word graph (i.e., a graph that represents the pseudoword’s context in the corpus) with the known subdivision given by the components corresponding to the monosemous source words forming the pseudoword. The main contribution of this article is to present a self-sufficient pseudoword-based evaluation framework for wsi graph-based clustering algorithms, thereby defining a new evaluation measure (top2) and a secondary clustering process (hyperclustering). To our knowledge, we are the first to conduct and discuss a large-scale systematic pseudoword evaluation targeting the induction of coarse-grained homonymous word senses across a large number of graph clustering algorithms.

A dataset on human navigation strategies in foreign networked systems

Humans are involved in various real-life networked systems. The most obvious examples are social and collaboration networks but the language and the related mental lexicon they use, or the physical map of their territory can also be interpreted as networks. How do they find paths between endpoints in these networks? How do they obtain information about a foreign networked world they find themselves in, how they build mental model for it and how well they succeed in using it? Large, open datasets allowing the exploration of such questions are hard to find. Here we report a dataset collected by a smartphone application, in which players navigate between fixed length source and destination English words step-by-step by changing only one letter at a time. The paths reflect how the players master their navigation skills in such a foreign networked world. The dataset can be used in the study of human mental models for the world around us, or in a broader scope to investigate the navigation strategies in complex networked systems.

A Neural Approach to Source Dependence Based Context Model for Statistical Machine Translation

In statistical machine translation, translation prediction considers not only the aligned source word itself but also its source contextual information. Learning context representation is a promising method for improving translation results, particularly through neural networks. Most of the existing methods process context words sequentially and neglect source long-distance dependencies. In this paper, we propose a novel neural approach to source dependence-based context representation for translation prediction. The proposed model is capable of not only encoding source long-distance dependencies but also capturing functional similarities to better predict translations (i.e., word form translations and ambiguous word translations). To verify our method, the proposed mode is incorporated into phrase-based and hierarchical phrase-based translation models, respectively. Experiments on large-scale Chinese-to-English and English-to-German translation tasks show that the proposed approach achieves significant improvement over the baseline systems and outperforms several existing context-enhanced methods.

Fine-grained attention mechanism for neural machine translation

Abstract Neural machine translation (NMT) has been a new paradigm in machine translation, and the attention mechanism has become the dominant approach with the state-of-the-art records in many language pairs. While there are variants of the attention mechanism, all of them use only temporal attention where one scalar value is assigned to one context vector corresponding to a source word. In this paper, we propose a fine-grained (or 2D) attention mechanism where each dimension of a context vector will receive a separate attention score. In experiments with the task of En-De and En-Fi translation, the fine-grained attention method improves the translation quality in terms of BLEU score. In addition, our alignment analysis reveals how the fine-grained attention mechanism exploits the internal structure of context vectors.

Topic and Sentiment Words Extraction in Cross-Domain Product Reviews

Sentiment analysis is very popular in natural language processing and text mining. The traditional sentiment analysis methods use supervised and unsupervised classifiers in a single domain and achieve good results. When training data and test data come from different domains, these methods become poor. The problem of cross-domain opinion analysis is that it is not easy to get a large number of tagged data sets and it is impossible to tag all the data in the interesting domains. We propose an extraction method for topic and sentiment words based on conditional random field and syntactic structure to analyze the sentiment orientation of Chinese product reviews. We aim to extract topic and sentiment words from target domain and identify their sentiment orientation with one or a few topic and sentiment words being tagged in the source domain and words in the target domain without any tagged information. Our experimental results show that our method is effective in cross-domain sentiment analysis.

Emerging hybrid Spanish–English blend structures: ‘Summergete con socketines’

This study focuses on the analysis of morphologically induced new hybrid blending patterns in Spanish due to the influence and contact with English. Our data-driven approach does not separate word creation from grammatical phenomena but considers hybrid blends – a completely new emerging pattern itself in Spanish – as the output of speakers’ or users’ word creativity and manipulation that may be subject to morphological laws. However, the limited number of existing examples as well as the understandable lack of records for such recent data led us to analyze them in terms of tendencies and probabilities. We pay special attention to the ordering of the source words according to foreign or native origin; the presence of full forms and the distribution of full words and splinters, clipping and overlapping of the source words, as well as the semantic patterns and relations between the source words.

A Polynomial-Time Dynamic Programming Algorithm for Phrase-Based Decoding with a Fixed Distortion Limit

Decoding of phrase-based translation models in the general case is known to be NP-complete, by a reduction from the traveling salesman problem (Knight, 1999). In practice, phrase-based systems often impose a hard distortion limit that limits the movement of phrases during translation. However, the impact on complexity after imposing such a constraint is not well studied. In this paper, we describe a dynamic programming algorithm for phrase-based decoding with a fixed distortion limit. The runtime of the algorithm is O( nd! lh d+1) where n is the sentence length, d is the distortion limit, l is a bound on the number of phrases starting at any position in the sentence, and h is related to the maximum number of target language translations for any source word. The algorithm makes use of a novel representation that gives a new perspective on decoding of phrase-based models.

Phonological properties of word classes and directionality in conversion

In the study of the word-formation process of conversion, one particularly difficult task is to determine the directionality of the process, that is, to decide which word represents the base and which the derived word. One possibility to inform this decision that has received only limited attention is to capitalize on word-class-specific phonological properties. This paper empirically investigates this option for English noun-verb conversion by building on recent findings on phonological differences between these two word classes. A large-scale study of phonological properties is carried out on CELEX data, employing the quantitative techniques of conditional inference trees and random forests. An important result of this analysis is that the accuracy of phonological cues varies widely across different subsamples in the data. Essentially this means that phonological cues can be used as a criterion to determine the directionality of words that are at least two syllables in length. When restricted to this part of the lexicon, phonological properties represent a fairly accurate indicator of source word class and are therefore a useful addition to the linguist's toolkit for determining directionality in conversion. Based on this result, the paper also discusses the relations of phonological properties to other criteria commonly employed to determine directionality.

Context-dependent word representation for neural machine translation

We first observe a potential weakness of continuous vector representations of symbols in neural machine translation. That is, the continuous vector representation, or a word embedding vector, of a symbol encodes multiple dimensions of similarity, equivalent to encoding more than one meaning of the word. This has the consequence that the encoder and decoder recurrent networks in neural machine translation need to spend substantial amount of their capacity in disambiguating source and target words based on the context which is defined by a source sentence. Based on this observation, in this paper we propose to contextualize the word embedding vectors using a nonlinear bag-of-words representation of the source sentence. Additionally, we propose to represent special tokens (such as numbers, proper nouns and acronyms) with typed symbols to facilitate translating those words that are not well-suited to be translated via continuous vectors. The experiments on En–Fr and En–De reveal that the proposed approaches of contextualization and symbolization improves the translation quality of neural machine translation systems significantly.

Neural Machine Translation with Reconstruction

Although end-to-end Neural Machine Translation (NMT) has achieved remarkable progress in the past two years, it suffers from a major drawback: translations generated by NMT systems often lack of adequacy. It has been widely observed that NMT tends to repeatedly translate some source words while mistakenly ignoring other words. To alleviate this problem, we propose a novel encoder-decoder-reconstructor framework for NMT. The reconstructor, incorporated into the NMT model, manages to reconstruct the input source sentence from the hidden layer of the output target sentence, to ensure that the information in the source side is transformed to the target side as much as possible. Experiments show that the proposed framework significantly improves the adequacy of NMT output and achieves superior translation result over state-of-the-art NMT and statistical MT systems.

A Dependency-Based Neural Reordering Model for Statistical Machine Translation

In machine translation (MT) that involves translating between two languages with significant differences in word order, determining the correct word order of translated words is a major challenge. The dependency parse tree of a source sentence can help to determine the correct word order of the translated words. In this paper, we present a novel reordering approach utilizing a neural network and dependency-based embeddings to predict whether the translations of two source words linked by a dependency relation should remain in the same order or should be swapped in the translated sentence. Experiments on Chinese-to-English translation show that our approach yields a statistically significant improvement of 0.57 BLEU point on benchmark NIST test sets, compared to our prior state-of-the-art statistical MT system that uses sparse dependency-based reordering features.

Interfixes in Igbo morphology

ABSTRACTThis article uses synchronic evidence to show that the interfix in Igbo does not always divide its host into two equal parts. It recognises the differences in the form classes of the source words and their co-occurrence potentials due to semantic or syntactic peculiarities as the major determinant of the morphological environment of the interfix. The theory of lexicalist morphology was used as the theoretical framework in data analysis. The article's findings show that lgbo interfixes serve to link outputs of two distinct morphological rules: reduplication and prefixation, and that only reduplication-motivated interfixes divide their host into two equal parts because they are found between a stem and its reduplicant. The structure of the interfixes is also found to be basically a consonant segment which is phonologically motivated to fulfil a basic lgbo syllabification requirement after the rule of word formation has taken place in the lexicon.

ON SOME HUMOROUS ASPECTS OF CONCEPTUAL BLENDING IN POLITICIZED DISCOURSE

In this paper, a very dynamic linguistic phenomenon, humour, is analysed on the basis of the Theory of Conceptual Blending developed by Gilles Fauconnier and Mark Turner (2002). It was discovered that in Lithuanian politicized discourse during and after the period of elections (2012, 2016) the majority of political events and personalities were given an evaluative implicit assessment through hu­mour/irony, which can be seen as an emergent result of blending processes. In the paper, the tendency to ridicule politicians and political events on news websites is discussed focusing exceptionally on the processes of blending at the lexical level. The research reveals that the coinage of new lexical units by fusing at least two source words or splinters follows certain patterns. The structural analysis is based on the works of Anita Lehrer, Michael Kelly, Stefan Gries, and Ekaterine Bakaradze. The main claims of the paper are supported by 362 blends collected from the headlines and comments of delfi.lt, lrytas. lt, and blogs, during the period of 2012–2016.

Intensifiers and image schemas: Schema type determines intensifier type

This paper develops a theory whereby the type of intensifier that a source word grammaticalizes as is determined by the semantics of the source word. Three intensifiers (way, thoroughly, and overly) are looked at in-depth and it is shown that their semantics as intensifiers follow from certain features, namely path and endpoint, of the image schemas depicting the meanings of their source words. It is possible that this theory can be extended to all intensifiers, especially given that many intensifiers grammaticalize out of source words that have spatial interpretations.

When Unnecessary Repairs Become Necessary: The Case of Nasal Insertion in Standard Mandarin Loanwords

[Abstract not available]

An Investigation into the Processing of Lexicalized English Blend Words: Evidence from Lexical Decisions and Eye Movements During Reading.

New words enter the language through several word formation processes [see Simonini (Engl J 55:752-757, 1966)]. One such process, blending, occurs when two source words are combined to represent a new concept (e.g., SMOG, BRUNCH, BLOG, and INFOMERCIAL). While there have been examinations of the structure of blends [see Gries (Linguistics 42:639-667, 2004) and Lehrer (Am Speech 73:3-28, 1998)], relatively little attention has been given to how lexicalized blends are recognized and if this process differs from other types of words. In the present study, blend words were matched to non-blend control words on length, familiarity, and frequency. Two tasks were used to examine blend processing: lexical decision and sentence reading. The results demonstrated that blend words were processed differently than non-blend control words. However, the nature of the effect varied as a function of task demands. Blends were recognized slower than control words in the lexical decision task but received shorter fixation durations when embedded in sentences.

Learning local word reorderings for hierarchical phrase-based statistical machine translation

Statistical models for reordering source words have been used to enhance hierarchical phrase-based statistical machine translation. There are existing word-reordering models that learn reorderings for any two source words in a sentence or only for two contiguous words. This paper proposes a series of separate sub-models to learn reorderings for word pairs with different distances. Our experiments demonstrate that reordering sub-models for word pairs with distances less than a specific threshold are useful to improve translation quality. Compared with previous work, our method more effectively and efficiently exploits helpful word-reordering information; it improves a basic hierarchical phrase-based system by 2.4-3.1 BLEU points and keeps the average time of translating one sentence under 10 s.

Improved Neural Machine Translation with SMT Features

Neural machine translation (NMT) conducts end-to-end translation with a source language encoder and a target language decoder, making promising translation performance. However, as a newly emerged approach, the method has some limitations. An NMT system usually has to apply a vocabulary of certain size to avoid the time-consuming training and decoding, thus it causes a serious out-of-vocabulary problem. Furthermore, the decoder lacks a mechanism to guarantee all the source words to be translated and usually favors short translations, resulting in fluent but inadequate translations. In order to solve the above problems, we incorporate statistical machine translation (SMT) features, such as a translation model and an n-gram language model, with the NMT model under the log-linear framework. Our experiments show that the proposed method significantly improves the translation quality of the state-ofthe-art NMT system on Chinese-to-English translation tasks. Our method produces a gain of up to 2.33 BLEU score on NIST open test sets.

How Translation Alters Sentiment

Sentiment analysis research has predominantly been on English texts. Thus there exist many sentiment resources for English, but less so for other languages. Approaches to improve sentiment analysis in a resource-poor focus language include: (a) translate the focus language text into a resource-rich language such as English, and apply a powerful English sentiment analysis system on the text, and (b) translate resources such as sentiment labeled corpora and sentiment lexicons from English into the focus language, and use them as additional resources in the focus-language sentiment analysis system. In this paper we systematically examine both options. We use Arabic social media posts as stand-in for the focus language text. We show that sentiment analysis of English translations of Arabic texts produces competitive results, w.r.t. Arabic sentiment analysis. We show that Arabic sentiment analysis systems benefit from the use of automatically translated English sentiment lexicons. We also conduct manual annotation studies to examine why the sentiment of a translation is different from the sentiment of the source word or text. This is especially relevant for building better automatic translation systems. In the process, we create a state-of-the-art Arabic sentiment analysis system, a new dialectal Arabic sentiment lexicon, and the first Arabic-English parallel corpus that is independently annotated for sentiment by Arabic and English speakers.