Language Technology Tools Research Articles

Unearthing the latent assumptions inscribed into language tools: the cross-cultural benefits of applying a reflexive lens in co-design

ABSTRACT Language technology tools provide a promising way to teach, share, retain, and curate under-resourced language learning materials in community. The inclusion of language teachers working with communities increases the potential for designed tools to be adopted by those groups. However, there is little research concerning the adaptation of tools designed with one community to other languages. To identify the implications for such scalability, we ran workshops with the ‘Record and Write’ tool, developed as a versatile format for collection, curation, and use of under-resourced language learning materials in community. The process enabled teachers of languages with varying availability of teaching materials to reflect on some of the embedded complexities of adapting the tool to their context. This paper critically reflects on the design process of the tool and design lessons learned relating to language governance, the reflection of culture in database tools, conversational learning support, and differentiated needs for grammatical accuracy and annotation. Methodologically, the paper proposes a reflexive lens on co-design in cross-cultural contexts, identifying some of the latent assumptions embedded in technologies that emerge when tools are transposed to different language and learning contexts.

Mit Wortschatz und lexikografischen Ressourcen handeln: kritische Überlegungen zur Anwendung lexikalischer, lexikografischer und digitaler Kompetenzen im virtuellen Raum beim akademischen Schreiben im DaF-Bereich

Acting with Lexicon and Lexicographical Resources: Critical Reflec­tions on the Use of Lexical, Lexicographic and Digital Literacy in Vir­tual Environments in Academic Writing in GfL. The paper deals with vocabulary acquisi­tion in the academic writing process in advanced GfL in interaction with the development of appropriate language awareness and wide-ranging up-to-date digital literacy. The starting point is the observation that in the academic L2 writing process the extensive information potential of mono­lingual, bilingual and multilingual lexicographic and lexico-grammatical online resources as well as other language technology tools is not adequately known. Accordingly, the many digital language resources are not optimally used for methods of teaching vocabulary and acting with it, e.g. in academic writing. On the other hand, digital media occupy more and more space in our everyday life and also provide highly developed tools based on linguistic technology for various L2 competence areas, such as translation. The aim of this paper is therefore both to show ways of making sensible use of existing internet resources and to outline entirely new ways based on artifi­cial intelligence. Selected resources will be used as examples to demonstrate the extensive infor­ma­tion potential and to discuss possibilities for use in academic writing. An outlook points to the necessity of adapting to revolutionary technological changes in the field of foreign language didac­tics. Keywords: learner's dictionaries, vocabulary acquisition, academic writing, multiliteracies, online-lexicography, language technologies, german as foreign language, corpuslinguistics, writing assistant, artifi­cial intelligence

Universal Dependency Treebank for Santali Language

A major effort is currently underway to develop a large-scale treebank for Indian low resource Languages (ILRLs). Apart from that, a rich and large-scale treebank can be an essential resource for linguistic investigations. This paper presents the first publicly available treebank of Santali low resource Indian language. The treebank contains 307 tokens (51 sentences) in the Santali language. All the selected sentences are manually annotated following the Universal Dependency guidelines. The morphological analysis of the Santali treebank was performed using machine learning techniques. The Santali annotated treebank will enrich the Santali language resource and will help in building language technology tools for cross-lingual learning and typological research. We also built a preliminary Santali parser using a machine learning approach. Finally, the paper briefly discusses the linguistic analysis of the Santali UD treebank.

Infraestrutura de Investigação para a Ciência e Tecnologia da Linguagem - PORTULAN CLARIN

This paper presents the PORTULAN CLARIN Research Infrastructure for the Science and Technology of Language, which is part of the European research infrastructure CLARIN ERIC as its Portuguese national node, and belongs to the Portuguese National Roadmap of Research Infrastructures of Strategic Relevance. The PORTULAN CLARIN includes a helpdesk, a repository, where resources, such as corpora, lexicons and processing tools are deposited for long-term archiving and can be searched and retrieved, and a workbench, where Language Technology tools and applications are made readily available online and can be used in different types of interfaces. Its goal is to contribute to the technological development of natural languages and for their preparation for the digital age, with a special focus on the Portuguese language in all its varieties and modalities.

Hierarchical self attention based sequential labelling model for Bhojpuri, Maithili and Magahi languages

Sequential labelling plays a vital role in solving numerous Natural Language Processing (NLP) applications such as Machine Translation and Information Extraction etc. One of these is Part-of-Speech (POS) tagging, which assigns a sequence of grammatical categories to the given sentence, and Chunking which groups them into ‘chunks’ or what can be called minimal phrases. Bhojpuri, Maithili and Magahi are low resource languages and widely spoken in central north-eastern India, belonging to the Indo-Aryan language family. The creation of an annotated corpus for POS tagging and Chunking, and then building an initial automatic tool for these problems is the first attempt towards building language technology tools for these languages. The annotated corpus used to develop POS Taggers and Chunkers, based on various machine learning algorithms (TnT, CRF, MEMM and Structured SVM) and state-of-the-art LSTM-CNN-CRF model, and then these compared with the obtained results on two new proposed deep learning-based models, Self-Attention Hierarchical Bi-LSTM CRF (SAHBiLC) and a fine-tuned version of it, Fine-SAHBiLC. The SAHBiLC and Fine-SAHBiLC models outperform on Bhojpuri (Accuracy for POS and Chunking is 0.86% and 0.94%, respectively) and Maithili (Accuracy for POS and Chunking is 0.86% and 0.95%, respectively) and Magahi (Accuracy for POS is 0.86%).

Spoken word corpus and dictionary definition for an African language

The preservation of languages is critical to maintaining and strengthening the cultures and identities of communities, and this is especially true for under-resourced languages with a predominantly oral culture. Most African languages have a relatively short literary past, and as such the task of dictionary making cannot rely on textual corpora as has been the standard practice in lexicography. This paper emphasizes the significance of the spoken word and the oral tradition as repositories of vocabulary, and argues that spoken word corpora greatly outweigh the value of printed texts for lexicography. We describe a methodology for creating a digital dialectal dictionary for the Igbo language from such a spoken word corpus. We also highlight the language technology tools and resources that have been created to support the transcription of thousands of hours of Igbo speech and the subsequent compilation of these transcriptions into an XML-encoded textual corpus of Igbo dialects. The methodology described in this paper can serve as a blueprint that can be adopted for other under-resourced languages that have predominantly oral cultures.

Towards Kikamba Computational Grammar

The under-resourced Kikamba language has few language technology tools since the more efficient and popular data driven approaches for developing them suffer from data sparseness due to lack of digitized corpora. To address this challenge, we have developed a computational grammar for the Kikamba language within the multilingual Grammatical Framework (GF) toolkit. GF uses the Interlingua rule-based translation approach. To develop the grammar, we used the morphology driven strategy. Therefore, we first developed regular expressions for morphology inflection and thereafter developed the syntax rules. Evaluation of the grammar was done using one hundred sentences in both English and Kikamba languages. The results were an encouraging four n-gram BLEU score of 83.05% and the Position independent error rate (PER) of 10.96%. Finally, we have made a contribution to the language technology resources for Kikamba including multilingual machine translation, a morphology analyzer, a computational grammar which provides a platform for development of multilingual applications and the ability to generate a variety of bilingual corpora for Kikamba for all languages currently defined in GF, making it easier to experiment with data driven approaches.

Lithuanian-Latvian, Latvian-Lithuanian Parallel Corpus (LILA)

The paper presents a new linguistic resource, LILA, which is the Lithuanian-Latvian-Lithuanian parallel corpus aligned on paragraph and sentence level.The total size of the LILA corpus is 9 m words. So far it is a unique resource for this language pair. The corpus contains metadata with bibliographicalinformation (title, author, year of publishing, etc.). The corpus contains the structural annotation, which includes boundaries of aligned segments,paragraphs, and sentences. The alignment of paragraphs and sentences has been done by the semi-automatic alignment tool Aligner 2.0.6.7. The corpuswas compiled during 2011-2012 by scientists of the Vytautas Magnus University’s Centre of Computational Linguistics (VMU CCL) and the LatvianUniversity’s Mathematical and Informatics Institute’s Laboratory of Artificial Intelligence (LU MII). The paper describes problems and challenges thatneed to be solved, when a parallel corpus for two small languages is created. The limited choice of appropriate parallel material poses the most difficultobstacle, as then it is difficult to compile a corpus of desired size. The paper presents: the conception and structure of the LILA corpus, phases of itscompilation, the alignment tool, the query system, and examples of usage. The corpus is especially useful for teaching and learning languages, forcomparing languages, for compilation of dictionaries, and for developing language technology tools (e. g. statistical machine translation systems). DOI: http://dx.doi.org/10.5755/j01.sal.0.23.4582

Historical Slovenian Language Resources

EXTENDED ABSTRACT:The paper presents three language resources enabling better full-text access to digitised printed historical Slovenian texts: a hand-annotated corpus, a hand-annotated lexicon of historical words and a collection of transcribed texts. The aim of the resources is twofold: on one hand they support empirical linguistic research (corpus, collection) and represent a reference tool for the research of historical Slovenian (lexicon) while on the other hand they may serve as training data for the development of Human Language Technologies enabling better full-text search in digital libraries containing Slovenian written cultural heritage, modernisation of historical texts, and the development of better technological solutions for text recognition and scanning. The hand annotated corpus of historical Slovenian contains the text from 1,000 pages sampled from the years 1750 to 1900, two texts date to the end of the 16th or 17th century. The corpus contains a little more than 250,000 word tokens; each of them being annotated with hand validated linguistic features: modernised form, lemma or base form, and morhpo-syntactic description. Thus the word token »ajfram« is annotated with the normalised form »ajfrom«, by the lemma »ajfer« and morphosyntactic description »Som« or »Samostalnik« (noun), »občni« (common), »moški« (masculine) and a modernised form »gorečnost« (fervour). At first the corpus was annotated automatically and then manually verified and corrected. The lexicon was created automatically from the hand-annotated corpus. It contains only attested word-forms and examples of use. The word-forms are ordered under their modern equivalents. All the modern forms of a particular word constitute a dictionary entry, defined by its lemma with conjoint information i.e. the morpho-syntactic description and the closest contemporary synonyms. Thus the entry »ajfrer/Som/gorečnost« is annotated by two modernised words »ajfra « and »ajfrom« and their archaic forms »ajfram« and »aifram« and by attestattion: »…shaz noi frihtei tu shebranje karbo sdei udrukono is velzhim aifram noi is flisam inu is andohtjo 3 vezhiere saporedama …« (Tapravi inu tazieli Colemone-Shegen, 1800, p. 183). At present, the lexicon contains over 25,000 entries (including modern words in archaic texts), 50,000 word-forms and 70,000 archaic forms. The third resource is represented by an extensive collection of digitised texts similar to the corpus. The difference is that the words are annotated automatically by a tool developed to process historical Slovenian text named ToTrTaLe. The tool implements a pipeline, where it first tokenises the text and then attempts to transcribe the archaic words to their modern day equivalents. Then, the text is tagged and lemmatised using the models for modern Slovenian language. It contains about 5 million words of hand-corrected transcriptions from the following digitised texts: • Slovenian books and editions of the newspaper »Kmetijske in rokodelske novice«, digitised by the National University Library (NUK) in the frame of the EU project IMPACT (5000 pages); • Digital library AHLib,1 comprising Slovenian books translated from German (100 books); • A selection of Slovenian books2 All three resources (corpus, lexicon, collection) are encoded according to the Text Encoding Initiative Guidelines TEI P5, which enable the definition of XML schemas for encoding texts for scholarly purposes. The home page of the project at http://nl.ijs.si/imp/ enables access to the resources. The collection and the lexicon are available for on-line browsing, the corpus and the automatically annotated collection for linguistics searches via a concordancer, while all the resources can be also downloaded in their source XML form under the Creative Commons Attribution Licence. In future we expect to extend the resources, however, even their present scope is sufficient for corpus based diachronic studies of historical Slovenian language and for developing useful language technology tools for processing cultural heritage texts.

INTERVIEW: Knowledge and Terminology Management at Crisplant

Margrethe H. Møller interviews Lisbeth Kjeldgaard Almsten(translator/coauthor: Birthe Toft)“If you think that terminology work is simply a matter of buying terminology management software and getting started, you are in for trouble” At Crisplant, we have been doing terminology management for the past 20 years. Today, term bases are used not just for terminology-oriented term management. Recording other types of master data needed by all kinds of professionals in the enterprise is equally important. Within the past year, Crisplant has been acquired by the German BEUMER group, which means that the terminological resources of the two enterprises are in the process of being integrated. The challenges presented by this process demonstrate the importance of adhering to terminological principles when recording terminology resources, while at the same time reminding us what an essential discipline terminology management really is, in enterprise practice as well as in education. Lisbeth Kjeldgaard AlmstenProject Manager & Linguistic Coordinator at Crisplant.Lisbeth Kjeldgaard Almsten is dedicated to language technology, and she plays the role of coordinator vis-à-vis her colleagues in the BEUMER Group, who use the same language technology tools as the ones she works with.

Characteristics of Finnish and Swedish intensive care nursing narratives: a comparative analysis to support the development of clinical language technologies

BackgroundFree text is helpful for entering information into electronic health records, but reusing it is a challenge. The need for language technology for processing Finnish and Swedish healthcare text is therefore evident; however, Finnish and Swedish are linguistically very dissimilar. In this paper we present a comparison of characteristics in Finnish and Swedish free-text nursing narratives from intensive care. This creates a framework for characterising and comparing clinical text and lays the groundwork for developing clinical language technologies.MethodsOur material included daily nursing narratives from one intensive care unit in Finland and one in Sweden. Inclusion criteria for patients were an inpatient period of least five days and an age of at least 16 years. We performed a comparative analysis as part of a collaborative effort between Finnish- and Swedish-speaking healthcare and language technology professionals that included both qualitative and quantitative aspects. The qualitative analysis addressed the content and structure of three average-sized health records from each country. In the quantitative analysis 514 Finnish and 379 Swedish health records were studied using various language technology tools.ResultsAlthough the two languages are not closely related, nursing narratives in Finland and Sweden had many properties in common. Both made use of specialised jargon and their content was very similar. However, many of these characteristics were challenging regarding development of language technology to support producing and using clinical documentation.ConclusionsThe way Finnish and Swedish intensive care nursing was documented, was not country or language dependent, but shared a common context, principles and structural features and even similar vocabulary elements. Technology solutions are therefore likely to be applicable to a wider range of natural languages, but they need linguistic tailoring.AvailabilityThe Finnish and Swedish data can be found at: http://www.dsv.su.se/hexanord/data/.

El projecte Atlantis: recursos digitals per a les llengües minoritzades de la UE, recursos per a l'ensenyament del català

The Atlantis Observatory (acronym for Academic Training, Language Acquisition and New Technologies in the Information Society) is the outcome of a project co-funded by the Directorate-General for Education and Culture of the European Commission, and carried out by a consortium consisting of the Humanities and Philology Studies of the UOC, the Research Centre on Multilingualism (Katholieke Universiteit Brussel), and the Foundation for European Research (Wales). It has a database which can be accessed on-line through the Internet, containing information on more than 1000 digital resources in so-called minority languages in the EU. The study is an attempt to see to what extent each language community is availing itself of the opportunities that the information society presents. Some of the smaller communities have virtually no digital resources, but multiple resources have been located in Catalan, Welsh and Basque, particularly, including language technology developments. The database has six sections, covering Learning platforms; Language technology tools and resources; Information and communication technologies (including regional development plans, software and Internet tools); Digitised cultural resources; Broadcasting; and Electronic publishing. Each of these sections has a final report on the website, in English.

Compilation and Exploitation of Parallel Corpora

With more and more text being available in electronic form, it is becoming relatively easy to obtain digital texts together with their translations. The paper presents the processing steps necessary to compile such texts into parallel corpora, an extremely useful language resource. Parallel corpora can be used as a translation aid for second-language learners, for translators and lexicographers, or as a data-source for various language technology tools. We present our work in this direction, which is characterised by the use of open standards for text annotation, the use of publicly available third-party tools and wide availability of the produced resources. Explained is the corpus annotation chain involving normalisation, tokenisation, segmentation, alignment, word-class syntactic tagging, and lemmatisation. Two exploitation results over our annotated corpora are also presented, namely aWeb concordancer and the extraction of bi-lingual lexica.