Word Alignment Research Articles

Two approaches to compilation of bilingual multi-word terminology lists from lexical resources

AbstractIn this paper, we present two approaches and the implemented system for bilingual terminology extraction that rely on an aligned bilingual domain corpus, a terminology extractor for a target language, and a tool for chunk alignment. The two approaches differ in the way terminology for the source language is obtained: the first relies on an existing domain terminology lexicon, while the second one uses a term extraction tool. For both approaches, four experiments were performed with two parameters being varied. In the experiments presented in this paper, the source language was English, and the target language Serbian, and a selected domain was Library and Information Science, for which an aligned corpus exists, as well as a bilingual terminological dictionary. For term extraction, we used the FlexiTerm tool for the source language and a shallow parser for the target language, while for word alignment we used GIZA++. The evaluation results show that for the first approach the F1 score varies from 29.43% to 51.15%, while for the second it varies from 61.03% to 71.03%. On the basis of the evaluation results, we developed a binary classifier that decides whether a candidate pair, composed of aligned source and target terms, is valid. We trained and evaluated different classifiers on a list of manually labeled candidate pairs obtained after the implementation of our extraction system. The best results in a fivefold cross-validation setting were achieved with the Radial Basis Function Support Vector Machine classifier, giving a F1 score of 82.09% and accuracy of 78.49%.

Words and deeds – Experimental evidence on leading-by-example

The paper studies how leadership recommendations and leading-by-example shape group behavior. The experimental results from an intergroup contest show that the leaders are the key drivers of contest expenditure. They increase their expenditure substantially once they make recommendations to the followers. The leaders' expenditure recommendations have a highly significant impact on the investments of followers. The additional impact of leading by example confirms the relevance of the alignment of words and deeds. A second experiment replicates the treatments in a non-competitive voluntary contribution game. In this setting, the opportunity for making recommendations has an even stronger impact on the leader's expenditure. For a given level of leader expenditure, the behavior of followers remains remarkably consistent across the two experiments but the additional impact of leading by example is not significant anymore.

Towards Better Word Alignment in Transformer

While neural models based on the Transformer architecture achieve the State-of-the-Art translation performance, it is well known that the learned target-to-source attentions do not correlate well with word alignment. There is an increasing interest in inducing accurate word alignment in Transformer, due to its important role in practical applications such as dictionary-guided translation and interactive translation. In this article, we extend and improve the recent work on unsupervised learning of word alignment in Transformer on two dimensions: a) parameter initialization from a pre-trained cross-lingual language model to leverage large amounts of monolingual data for learning robust contextualized word representations, and b) regularization of the training objective to directly model characteristics of word alignments which results in favorable word alignments receiving more concentrated probabilities. Experiments on benchmark data sets of three language pairs show that the proposed methods can significantly reduce alignment error rate (AER) by at least 3.7 to 7.7 points on each language pair over two recent works on improving the Transformer's word alignment. Moreover, our methods can achieve better alignment results than GIZA++ on certain test sets.

Multi-Information Spatial–Temporal LSTM Fusion Continuous Sign Language Neural Machine Translation

There are two basic problems in sign language recognition (SLR): (a) isolated word SLR and (b) continuous SLR. Most of the existing continuous SLR methods are extensions of the isolated word SLR methods. These methods use the isolated word SLR results as the basic module and obtain the sentence recognition results through sentence segmentation and word alignment. However, sentence segmentation and word alignment are often not accurate, resulting in a low sentence recognition accuracy. At the same time, continuous SLR usually requires strict sample labels, leading to the difficult task of manual labeling and limited training data availability. To address these challenges, this paper proposes a bidirectional spatial–temporal LSTM fusion attention network (Bi-ST-LSTM-A) for continuous SLR. This approach avoids problems such as sentence segmentation, word alignment, and tedious manual labeling. Our contributions are summarized as follows: (1) we proposed a sign language video feature representation method using a convolutional neural network (CNN) and spatial–temporal LSTM (ST-LSTM) information fusion technology; and (2) we constructed a uniform neural machine translation framework that can be used for complex continuous SLR and gesture recognition of nonspecific people in nonspecific environments. Experiments were carried out on some large continuous sign language datasets. The sign language recognition accuracy reached 81.22% on the 500 CSL dataset, 76.12% on the RWTH-PHOENIX-Weather dataset and 75.32% on the RWTH-PHOENIX-Weather-2014T dataset, thereby illustrating the effectiveness of the proposed framework.

Improving English-Arabic statistical machine translation with morpho-syntactic and semantic word class

In this paper, we present a new method for the extraction and integrating of morpho-syntactic and semantic word classes in a statistical machine translation (SMT) context to improve the quality of English-Arabic translation. It can be applied across different statistical machine translations and with languages that have complicated morphological paradigms. In our method, we first identify morpho-syntactic word classes to build up our statistical language model. Then, we apply a semantic word clustering algorithm for English. The obtained semantic word classes are projected from the English side to the featured Arabic side. This projection is based on available word alignment provided by the alignment step using GIZA++ tool. Finally, we apply a new process to incorporate semantic classes in order to improve the SMT quality. We show its efficacy on small and larger English to Arabic translation tasks. The experimental results show that introducing morpho-syntactic and semantic word classes achieves 7.7% of relative improvement on the BLEU score.

Future-Aware Knowledge Distillation for Neural Machine Translation

Although future context is widely regarded useful for word prediction in machine translation, it is quite difficult in practice to incorporate it into neural machine translation. In this paper, we propose a future-aware knowledge distillation framework (FKD) to address this issue. In the FKD framework, we learn to distill future knowledge from a backward neural language model (teacher) to future-aware vectors (student) during the training phase. The future-aware vector for each word position is computed in a bridge network and optimized towards the corresponding hidden state in the backward neural language model via a knowledge distillation mechanism. We further propose an algorithm to jointly train the neural machine translation model, neural language model and knowledge distillation module end-to-end. The learned future-aware vectors are incorporated into the attention layer of the decoder to provide full-range context information during the decoding phase. Experiments on the NIST Chinese-English and WMT English-German translation tasks show that the proposed method significantly improves translation quality and word alignment.

A Deep Neural Network Framework for English Hindi Question Answering

In this article, we propose a unified deep neural network framework for multilingual question answering (QA). The proposed network deals with the multilingual questions and answers snippets. The input to the network is a pair of factoid question and snippet in the multilingual environment (English and Hindi), and output is the relevant answer from the snippet. We begin by generating the snippet using a graph-based language-independent algorithm, which exploits the lexico-semantic similarity between the sentences. The soft alignment of the question words from the English and Hindi languages has been used to learn the shared representation of the question. The learned shared representation of question and attention-based snippet representation are passed as an input to the answer extraction layer of the network, which extracts the answer span from the snippet. Evaluation on a standard multilingual QA dataset shows the state-of-the-art performance with 39.44 Exact Match (EM) and 44.97 F1 values. Similarly, we achieve the performance of 50.11 Exact Match (EM) and 53.77 F1 values on Translated SQuAD dataset.

Topic sensitive image descriptions

The objective of description models is to generate image captions to elaborate contents. Despite recent advancements in machine learning and computer vision, generating discriminative captions still remains a challenging problem. Traditional approaches imitate frequent language patterns without considering the semantic alignment of words. In this work, an image captioning framework is proposed that generates topic sensitive descriptions. The model captures the semantic relation and polysemous nature of the words that describe the images and resultantly generates superior descriptions for the target images. The efficacy of the proposed model is indicated by the evaluation on the state-of-the-art captioning datasets. The model shows promising performance compared to the existing description models proposed in the recent literature.

An Automatic and a Machine-assisted Method to Clean Bilingual Corpus

Two different methods of corpus cleaning are presented in this article. One is a machine-assisted technique, which is good to clean small-sized parallel corpus, and the other is an automatic method, which is suitable for cleaning large-sized parallel corpus. A baseline SMT (MOSES) system is used to evaluate these methods. The machine-assisted technique used two features: word alignment and length of the source and target language sentence. These features are used to detect mistranslations in the corpus, which are then handled by a human translator. Experiments of this method are conducted on the English-to-Indian Language Machine Translation (EILMT) corpus (English-Hindi). The Bilingual Evaluation Understudy (BLEU) score is improved by 0.47% for the clean corpus. Automatic method of corpus cleaning uses a combination of two features. One feature is length of source and target language sentence and the second feature is Viterbi alignment score generated by Hidden Markov Model for each sentence pair. Two different threshold values are used for these two features. These values are decided by using a small-sized manually annotated parallel corpus of 206 sentence pairs. Experiments of this method are conducted on the HindEnCorp corpus, released in the workshop of the Association of Computational Linguistics (ACL 2014). The BLEU score is improved by 0.6% on clean corpus. A comparison of the two methods is also presented on EILMT corpus.

병렬코퍼스 활용과 근대 한일 어휘 대조연구의 확장

In this paper, I suggest that we need contrastive studies through building parallel corpora in order to fully describe the differences of vocabularies of Korean and Japanese. And I show some researches dealing with actually built parallel corpora. I demonstrate that these researches are possible only through parallel corpora or they show that the research method of using parallel corpora is an efficient one. Furthermore, I argue that in order to expand the purview of the comparative studies of Korean and Japanese modern vocabularies and its method, we need to build not just parallel corpora for sentence alignment, but also parallel corpora for word alignment, and also we need to solve many problems including developing morphological analyzer of modern Korean.

Issues In Word Alignment From Hindi-English Languages

The paper discusses the various methods and issues related to word alignment. This paper focus on the main problem arises in word alignment because Hindi language is based on subject object verb “SVO” and for English language is subject verb objects “SOV”. Hindi is morphology rich language, therefore correct alignment of word order from Hindi to English language is quite difficult. The paper presents survey on for foreign and Indian language of word alignment in the application of machine translation

Identifying word evolution by incorporating PoS and avoiding alignment of temporal words

Identifying word evolution is important for understanding cultural and societal changes. The key to accurate identification of word evolution is to distinguish word semantics. Recently methods based on low-dimensional embedding representation for words were proposed but they require the alignment of word embeddings across different time periods. This process is computationally expensive, prohibitively time consuming and suffering from contextual variability. In this paper, we propose a method to learn low-dimensional time-aware embeddings using both statistical and POS (part-of-speech) tagging information of words. Besides, the proposed method bypasses the computationally expensive step of aligning the word embeddings by tagging each word with a time prefix and encoding them into the common vector space. The learnt temporal embeddings better reveal semantic changes over time. A comprehensive experiment is conducted on Google Books N-gram (throughout 100 years). When compared with other three top-performing temporal embedding methods (PPMI, SVD, SGNS), our method achieves state-of-the-art in terms of time complexity, precision, recall, F1-score and the number of words identified to have changed in meaning, respectively. Additionally, we provide an intuitive illustration of the semantic evolution of the interesting words identified with our method.

Bilingual lexical extraction based on word alignment for improving corpus search

Purpose This paper aims to describe the structure of an aligned Serbian-German literary corpus (SrpNemKor) contained in a digital library Bibliša. The goal of the research was to create a benchmark Serbian-German annotated corpus searchable with various query expansions. Design/methodology/approach The presented research is particularly focused on the enhancement of bilingual search queries in a full-text search of aligned SrpNemKor collection. The enhancement is based on using existing lexical resources such as Serbian morphological electronic dictionaries and the bilingual lexical database Termi. Findings For the purpose of this research, the lexical database Termi is enriched with a bilingual list of German-Serbian translated pairs of lexical units. The list of correct translation pairs was extracted from SrpNemKor, evaluated and integrated into Termi. Also, Serbian morphological e-dictionaries are updated with new entries extracted from the Serbian part of the corpus. Originality/value A bilingual search of SrpNemKor in Bibliša is available within the user-friendly platform. The enriched database Termi enables semantic enhancement and refinement of user’s search query based on synonyms both in Serbian and German at a very high level. Serbian morphological e-dictionaries facilitate the morphological expansion of search queries in Serbian, thereby enabling the analysis of concepts and concept structures by identifying terms assigned to the concept, and by establishing relations between terms in Serbian and German which makes Bibliša a valuable Web tool that can support research and analysis of SrpNemKor.

Annotated Guidelines and Building Reference Corpus for Myanmar-English Word Alignment

Reference corpus for word alignment is an important resource for developing and evaluating word alignment methods. For Myanmar-English language pairs, there is no reference corpus to evaluate the word alignment tasks. Therefore, we created the guidelines for Myanmar-English word alignment annotation between two languages over contrastive learning and built the Myanmar-English reference corpus consisting of verified alignments from Myanmar ALT of the Asian Language Treebank (ALT). This reference corpus contains confident labels sure (S) and possible (P) for word alignments which are used to test for the purpose of evaluation of the word alignments tasks. We discuss the most linking ambiguities to define consistent and systematic instructions to align manual words. We evaluated the results of annotators agreement using our reference corpus in terms of alignment error rate (AER) in word alignment tasks and discuss the words relationships in terms of BLEU scores.

Research on statistical machine translation model based on deep neural network

With the increase of translation demand, the advancement of information technology, the development of linguistic theories and the progress of natural language understanding models in artificial intelligence research, machine translation has gradually gained worldwide attention. However, at present, machine translation research still has problems such as insufficient bilingual data and lack of effective feature representation, which affects the further improvement of key modules of machine translation such as word alignment, sequence adjustment and translation modelling. The effect of machine translation is still unsatisfactory. As a new machine learning method, deep neural network can automatically learn abstract feature representation and establish a complex mapping relationship between input and output signals, which provides a new idea for statistical machine translation research. Firstly, the multi-layer neural network and the undirected probability graph model are combined, and the similarity and context information of vocabulary are effectively utilized to model the word alignment more fully, and the word alignment model named NNWAM is constructed. Secondly, the low dimension will be used. The feature representation is combined with other features into a linearly ordered pre-ordering model to construct the pre-ordering model named NNPR. Finally, the word alignment model and the pre-ordering model are combined in the same deep neural network framework to form DNNAPM, a statistical machine translation model based on deep neural networks. The experimental results show that the statistical machine translation model based on deep neural network has better effect, faster convergence and better reliability than the comparison model algorithm.

Permainan Bahasa dalam Meda Sosial

The title of the article "In Social Media is" Language Games "(?)" Departing from the purpose of proof that the existence of Social Media is a source of data "language games". Such statements are reasonable, because the actors, adolescents or who youthful, can present the desired content of the message regardless of the rules of the binding language.The application of theory combines the linguistic theory of semantics with sociolinguistic (theory), because between them are empirical sciences. The existence of both of them complement each other. The assessment phase starts from: 1. provision of data, 2. classification and analysis of data, and 3. preparation / writing.The data finding is more secondary, because the data is obtained from written sources, that is, as the facts of the text in Social Media. The findings of the form of "language games" are based on: 1. text typology, and 2. text content (messages).The results of the study based on the typology of the text produce: a. type of word alignment (in sentence form), b. dialog type, and c. type of poetry. While the results of the study based on the contents (message) of the text produce: a. `satire ', b. `invitation ', c. `warning, and d. `harassment and / or punching`.

GBI Treebanks as a Resource for New Applications

Global Bible Initiative (GBI) have developed Hebrew OT treebanks and Greek NT syntactic treebanks. The treebanks were first generated with a parser using computerized Hebrew and Greek grammars and then proofed verse by verse by Hebrew and Greek Scholars. All the corrections made by the scholars were kept as disambiguation data.
 The phrase structures in the trees have been used to build interlinears, concordances, and translation memories which operate not only on the word level, but on the phrase and clause levels as well. The syntactic relations (dependencies) in the trees have also been used to do smart search where we can find texts that are different in form but similar in meaning.
 Recently, we have also used the trees to improve the accuracy of automatic word alignment and explore tree-based interactive machine translation of the Bible. The auto aligner can be used to the Hebrew and Greek texts to translations in various languages. The interactive machine translation will speed up Bible translation without compromising quality by providing real time suggestions and checking.
 We have already contributed two sets of Greek trees to Creative Commons, the Nestle 1904 version and the SBLGNT version. We also have trees for NA27 and NA28, but we do not own the texts. The Hebrew OT treebank we developed was owned by the Groves Center. We are also capable of creating new treebanks with the parser, grammar, and disambiguation data we own if we are given a text that is morphologically tagged.

TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts

Scientific documents rely on both mathematics and text to communicate ideas. Inspired by the topical correspondence between mathematical equations and word contexts observed in scientific texts, we propose a novel topic model that jointly generates mathematical equations and their surrounding text (TopicEq). Using an extension of the correlated topic model, the context is generated from a mixture of latent topics, and the equation is generated by an RNN that depends on the latent topic activations. To experiment with this model, we create a corpus of 400K equation-context pairs extracted from a range of scientific articles from arXiv, and fit the model using a variational autoencoder approach. Experimental results show that this joint model significantly outperforms existing topic models and equation models for scientific texts. Moreover, we qualitatively show that the model effectively captures the relationship between topics and mathematics, enabling novel applications such as topic-aware equation generation, equation topic inference, and topic-aware alignment of mathematical symbols and words.

Holistic word context does not influence holistic processing of artificial objects in an interleaved composite task.

Holistic processing, a hallmark of expert processing, has been shown for written words, signaled by the word composite effect, similar to the face composite effect: fluent readers find it difficult to focus on just one half of a written word while ignoring the other half, especially when the two word halves are aligned rather than misaligned. This effect is signaled by a significant interaction between alignment and congruency of the two word parts. Face and visual word recognition, however, involve different neural mechanisms with an opposite hemispheric lateralization. It is then possible that faces and words can both involve holistic processing in their own separate face and word processing systems, but by using different mechanisms. In the present study, we replicated with words a previous study done with faces (Richler, Bukach, & Gauthier, 2009, Experiment 3). In a first experiment we showed that in a composite task with aligned artificial objects, no congruency effects are found. In a second experiment, using an interleaved task, a congruency effect for Ziggerins was induced in trials in which a word was first encoded, but more strongly when it was aligned. However, in a stricter test, we found no differences between the congruency effect for Ziggerins induced by aligned words versus pseudowords. Our results demonstrate that different mechanisms can underlie holistic processing in different expertise domains.

FAST: Fast and Accurate Synoptic Texts

Abstract Many classical texts are available in multiple versions that almost always differ from each other due to transcription error and editorial discretion. One of the central challenges in the study of such texts is the preparation of a ‘synoptic’ text: an aligned presentation of the various versions in which corresponding words or phrases, even if not identical, are mapped to each other. Multiple text alignment of this sort must take into account orthographic and conceptual relationships between words. In this article, we define this text alignment problem as an optimization problem by providing a formal measure of alignment quality. Unlike previous measures, our measure uses word embeddings to take into account conceptual similarity between aligned words. We propose an efficient and scalable alignment method in accordance with the proposed criteria. This method splits the texts to be aligned into smaller subtexts, thus improving both efficiency and accuracy. Empirical comparisons on sample data indicate our method is significantly faster than existing methods, often rendering intractable problems tractable, and that the alignment obtained by our method is considerably better than that obtained by other methods.