Part-of-speech Tagging Research Articles

MedLexSp – a medical lexicon for Spanish medical natural language processing

BackgroundMedical lexicons enable the natural language processing (NLP) of health texts. Lexicons gather terms and concepts from thesauri and ontologies, and linguistic data for part-of-speech (PoS) tagging, lemmatization or natural language generation. To date, there is no such type of resource for Spanish.Construction and contentThis article describes an unified medical lexicon for Medical Natural Language Processing in Spanish. MedLexSp includes terms and inflected word forms with PoS information and Unified Medical Language System^{circledR } (UMLS) semantic types, groups and Concept Unique Identifiers (CUIs). To create it, we used NLP techniques and domain corpora (e.g. MedlinePlus). We also collected terms from the Dictionary of Medical Terms from the Spanish Royal Academy of Medicine, the Medical Subject Headings (MeSH), the Systematized Nomenclature of Medicine - Clinical Terms (SNOMED-CT), the Medical Dictionary for Regulatory Activities Terminology (MedDRA), the International Classification of Diseases vs. 10, the Anatomical Therapeutic Chemical Classification, the National Cancer Institute (NCI) Dictionary, the Online Mendelian Inheritance in Man (OMIM) and OrphaData. Terms related to COVID-19 were assembled by applying a similarity-based approach with word embeddings trained on a large corpus. MedLexSp includes 100 887 lemmas, 302 543 inflected forms (conjugated verbs, and number/gender variants), and 42 958 UMLS CUIs. We report two use cases of MedLexSp. First, applying the lexicon to pre-annotate a corpus of 1200 texts related to clinical trials. Second, PoS tagging and lemmatizing texts about clinical cases. MedLexSp improved the scores for PoS tagging and lemmatization compared to the default Spacy and Stanza python libraries.ConclusionsThe lexicon is distributed in a delimiter-separated value file; an XML file with the Lexical Markup Framework; a lemmatizer module for the Spacy and Stanza libraries; and complementary Lexical Record (LR) files. The embeddings and code to extract COVID-19 terms, and the Spacy and Stanza lemmatizers enriched with medical terms are provided in a public repository.

COVID-19 INFODEMIC – UNDERSTANDING CONTENT FEATURES IN DETECTING FAKE NEWS USING A MACHINE LEARNING APPROACH

The use of content features, particularly textual and linguistic for fake news detection is under-researched, despite empirical evidence showing the features could contribute to differentiating real and fake news. To this end, this study investigates a selection of content features such as word bigrams, part of speech distribution etc. to improve fake news detection. We performed a series of experiments on a new dataset gathered during the COVID-19 pandemic using Decision Tree, K-Nearest Neighbor, Logistic Regression, Support Vector Machine and Random Forest. Random Forest yielded the best results, followed closely by Support Vector Machine, across all setups. In general, both the textual and linguistic features were found to improve fake news detection when used separately, however, combining them into a single model did not improve the detection significantly. Differences were also noted between the use of bigrams and part of speech tags. The study shows that textual and linguistic features can be used successfully in detecting fake news using the traditional machine learning approach as opposed to deep learning.

A deep learning approach to building a framework for Urdu POS and NER

The study examines various studies on Named Entity Recognition (NER) and Part of Speech (POS) tagging for the Urdu language conducted by academics and researchers. POS and NER tagging for Urdu still faces obstacles in terms of increasing accuracy while lowering false-positive rates and labelling unknown terms, despite the efforts of numerous researchers. In addition, ambiguity exists when tagging terms with different contextual meanings within a sentence. Due to the fact that Urdu is an inflectional, derivational, morphologically rich, and context-sensitive language, the existing models, such as Linguistic rule application, N-gram Markov model, Tree Tagger, random forest (RF) tagger, etc., were unable to produce accurate experimental results on Urdu language data. The significance of this study is that it fills a gap in the literature concerning the lack of POS and NER tagging for the Urdu language. For Urdu POS and NER tagging, we propose a deep learning model with a well-balanced set of language-independent features as well as a survey of important Urdu POS/NER techniques. In addition, this is the first study to use residual biDirectional residual Long short-term memory (residual biLSTM) architecture trained on the Urmono dataset in conjunction with the randomly initialised word2vec, fastText and mBERT embeddings are utilised to generate word or character vectors.For each experiment, the paper also employs the evaluation methods of Macro-F1, precision, precision, and recall. The proposed method with mbert embedding as word vectors provides best results of F1 score for POS and NER at 91.11% and 99.11% respectively. Also, the accuracy, precision and recall for POS are reported at 94.85%, 91.79% and 90.77%. Similarly, the accuracy, precision and recall for NER of the proposed model are reported at 99.77%, 98.78% and 99.45% respectively, which are higher than baseline models.

Improving Low-Resource Cross-lingual Parsing with Expected Statistic Regularization

Abstract We present Expected Statistic Regulariza tion (ESR), a novel regularization technique that utilizes low-order multi-task structural statistics to shape model distributions for semi- supervised learning on low-resource datasets. We study ESR in the context of cross-lingual transfer for syntactic analysis (POS tagging and labeled dependency parsing) and present several classes of low-order statistic functions that bear on model behavior. Experimentally, we evaluate the proposed statistics with ESR for unsupervised transfer on 5 diverse target languages and show that all statistics, when estimated accurately, yield improvements to both POS and LAS, with the best statistic improving POS by +7.0 and LAS by +8.5 on average. We also present semi-supervised transfer and learning curve experiments that show ESR provides significant gains over strong cross-lingual-transfer-plus-fine-tuning baselines for modest amounts of label data. These results indicate that ESR is a promising and complementary approach to model-transfer approaches for cross-lingual parsing.1

Tracing the translator's voice: A corpus-based study of six English translations of Daxue.

The Translator's voice is the "second" voice in a translated narrative. By tracing a set of textual cues, the translator's voice embedded in the discourse may come to light. This study is based on a self-constructed bilingual parallel corpus which adopts both quantitative and qualitative methods to reveal the translator's voice in six English versions of the Chinese classic Daxue. WordSmith 8.0, CLAWS POS Tagger, and Readability Analyzer were used to analyze the data and investigate the lexical, syntactic, and textual features of the translations of Daxue by David Collie, James Legge, Ku Hungming, Ezra Pound, Chan Wing-Tsit, and Andrew Plaks, respectively. Paratexts of translation-prefaces, introductions, translator's notes, footnotes, and so on, as well as representative examples, were also analyzed by close reading. The findings suggest that each translator's cultural identity, historical background, and motives for producing the translation are made manifest through various linguistic and non-linguistic choices. This study demonstrates that the translator's voice, which is always present along with the author's voice and may take various forms, is deeply associated with the cultural and ideological constructs in which the act of translation is embedded. It is hoped that the results of this research will contribute to an enhanced understanding of the translation and overseas dissemination of the Chinese classic Daxue, and further the study of the translator's voice with the aid of corpus technology.

Comparative Analysis of Hidden Markov Model and Bidirectional Long Short-Term Memory for POS Tagging in Eastern Armenian

This scientific article introduces a novel POS tagging system specifically developed for the Eastern Armenian language. The primary objective of the study is to conduct a comparative analysis of two well-established methods for part-of-speech (POS) tagging: the hidden Markov model (HMM) coupled with the Viterbi algorithm, and an artificial neural network in the form of a recurrent neural network. The study places particular emphasis on the Eastern Armenian language and employs the ArmSpeech-POS Eastern Armenian part-of-speech tagged corpus for conducting comprehensive experiments and evaluations. POS tagging is a fundamental task in natural language processing (NLP) that involves assigning grammatical tags to words in a given text. Accurate POS tagging is crucial for various NLP applications, including machine translation, information retrieval, and sentiment analysis. The Viterbi algorithm is a well-established probabilistic method that utilizes a hidden Markov model (HMM) to determine the most likely sequence of POS tags. On the other hand, RNNs, a type of deep learning model, can capture complex patterns and dependencies in sequential data. Experimental results indicate that both methods achieve reasonable accuracy in POS tagging for Eastern Armenian. However, the RNN outperforms the Viterbi algorithm, exhibiting higher accuracy rates. This can be attributed to the RNN’s ability to capture long-range dependencies and learn intricate linguistic patterns. The article concludes by discussing the implications of the study’s findings and potential areas for further research. It emphasizes the significance of accurate POS tagging for improving NLP applications in Eastern Armenian and suggests exploring advanced neural network architectures and incorporating linguistic features to enhance POS tagging performance.

ArmSpeech-POS: Eastern Armenian Part-of-Speech Tagged Corpus

Text chunking, Part-of-speech (POS) tagging, and named entity recognition (NER) are fundamental tasks in natural language processing (NLP). Part-of-speech (POS) tagging involves assigning grammatical labels to words in a sentence. Research shows that Armenian is a low-resourced language and there are not enough materials for developing higher accurate part-of-speech tagging systems in the Armenian language. This paper presents a fresh dataset for POS tagging in Armenian that follows the naming conventions of both Penn Treebank and Universal Dependencies tagsets, with two versions available. The dataset consists of 6081 sentences that were automatically annotated and then manually verified. The data was sourced from Armenian news websites, focusing on topics such as culture, medicine, and lifestyle, as well as 22 Armenian fairytales. The reason for having two versions of the POS tagset was to ensure compatibility and integration with all-natural language processing tools and models that use these standards. By standardizing the tagset, it becomes easier to compare and evaluate the effectiveness of different POS tagging models. The paper also describes data collection, cleaning, preprocessing, and processing steps. The ISMA translator was used for the annotation of the dataset, which not only performs machine translation but also conducts a syntactic and semantic analysis of the text and assigns a POS tag for each word in the sentence. The final corpus contains 13 groups of part-of-speech tags and a total of 57160 tagged tokens including the distinction between singular and plural parts of speech.

POS Tagging and Lemmatization of Historical Varieties of Languages. The Challenge of Old Italian

The paper discusses the challenges of POS tagging and lemmatization of historical varieties of Italian, and reports for both tasks the results of experiments carried out in a classical supervised domain adaptation scenario using the diachronic and typologically differentiated corpus built for the "Vocabolario Dinamico dell’Italiano Moderno" (VoDIM). For what concerns POS tagging, the effectiveness of retrained models is illustrated and substantiated with quantitative data, with a specific view to linguistic annotation results obtained with respect to specific language evolution stages, domains and textual genres. For lemmatization, different customized models have been developed, including lexicon-assisted ones and models retrained with historical annotated texts. In both cases, a detailed error analysis is provided.

Exploring the Performance of Farasa and CAMeL Taggers for Arabic Dialect Tweets

In Natural Language Processing (NLP), Part Of Speech (POS) tagging is an important step; it is a fundamental requirement for many applications, such as information extraction, machine translation, and grammar checking. Successful POS taggers have been developed for many languages, including Arabic. Currently, the spread of social media has increased the diversity of dialects as people use them in their online communications. Therefore, it has become more difficult for researchers to classify some words that are understood by humans but not computers. In addition, most Arabic POS research focuses on Modern Standard Arabic (MSA), while Dialect Arabic (DA) receives less attention. This paper aims to evaluate the performance of two Arabic taggers when used on dialect Arabic tweets and determine which tagger is the appropriate one, which will accordingly help to improve the existent taggers for dialect Arabic tweets. We used the Farasa and CAMeL taggers, which are commonly used to analyze Arabic texts and are considered the best taggers for Arabic. The results indicate that CAMeL tagger performed better than Farasa tagger, with accuracies of 92% and 83% respectively. In other words, a hybrid POS tagger trained with MSA and DA returns better results than the one trained on MSA.

Aspect-Based Sentiment Analysis for Social Multimedia: A Hybrid Computational Framework

People utilize microblogs and other social media platforms to express their thoughts and feelings regarding current events, public products and the latest affairs. People share their thoughts and feelings about various topics, including products, news, blogs, etc. In user reviews and tweets, sentiment analysis is used to discover opinions and feelings. Sentiment polarity is a term used to describe how sentiment is represented. Positive, neutral and negative are all examples of it. This area is still in its infancy and needs several critical upgrades. Slang and hidden emotions can detract from the accuracy of traditional techniques. Existing methods only evaluate the polarity strength of the sentiment words when dividing them into positive and negative categories. Some existing strategies are domain-specific. The proposed model incorporates aspect extraction, association rule mining and the deep learning technique Bidirectional Encoder Representations from Transformers (BERT). Aspects are extracted using Part of Speech Tagger and association rule mining is used to associate aspects with opinion words. Later, classification was performed using BER. The proposed approach attained an average of 89.45% accuracy, 88.45% precision and 85.98% recall on different datasets of products and Twitter. The results showed that the proposed technique achieved better than state-of-the-art sentiment analysis techniques.

Neural Attention Model for Abstractive Text Summarization Using Linguistic Feature Space

Summarization generates a brief and concise summary which portrays the main idea of the source text. There are two forms of summarization: abstractive and extractive. Extractive summarization chooses important sentences from the text to form a summary whereas abstractive summarization paraphrase using advanced and nearer-to human explanation by adding novel words or phrases. For a human annotator, producing summary of a document is time consuming and expensive because it requires going through the long document and composing a short summary. An automatic feature-rich model for text summarization is proposed that can reduce the amount of labor and produce a quick summary by using both extractive and abstractive approach. A feature-rich extractor highlights the important sentences in the text and linguistic characteristics are used to enhance results. The extracted summary is then fed to an abstracter to further provide information using features such as named entity tags, part of speech tags and term weights. Furthermore, a loss function is introduced to normalize the inconsistency between word-level and sentence-level attentions. The proposed two-staged network achieved a ROUGE score of 37.76% on the benchmark CNN/DailyMail dataset, outperforming the earlier work. Human evaluation is also conducted to measure the comprehensiveness, conciseness and informativeness of the generated summary.

Intelligent Part of Speech tagger for Hindi

English Part of Speech like noun, verb, adverb, adjective, pronoun, preposition, interjection, conjunction is somewhat similar in Hindi but not exactly the same. Hindi grammar has different Part of Speech (POS) based on its morphological features and the occurrence of a word/lexeme in a sentence. The existing techniques used in English language for POS tagging may not work properly for Indian language like Hindi. It is because the grammatical structure of the relatively free word order language like Hindi differs from English. Stochastic taggers may not give good performance as morphological information is not taken into account. The available Hindi word corpora usually have less frequency for individual tags. As a result, a larger size corpus having diversity in the type of sentences can provide better results. But, even after using smoothing techniques most these taggers fail to provide correct results in the presence of unknown words. Considering these aspects, this paper proposes an Intelligent POS tagger for Hindi language based on VITERBI and K-Nearest Neighbour, capable of providing more accurate results than VITERBI in the presence of unknown words.

Language-based transfer learning approaches for part-of-speech tagging on Saint Petersburg Corpus of Hagiographic texts (SKAT)

The article describes an experiment about training a part-of-speech tagger using artificial neural networks on the St. Petersburg Corpus of Hagiographic Texts (SKAT), which is being developed at the Department of Mathematical Linguistics of St. Petersburg State University. The corpus includes the texts of 23 manuscripts dating from the 15th–18th centuries with about 190,000 words usages, four of which were labelled manually. The bi-LSTM, distilled RuBERTtiny2 and RuBERT models were used to train a POS tagger. All of them were trained on modern Russian corpora and further fine-tuned to label Old Russian texts using a technique called language transfer. To fine-tune transformer-based language models it was necessary to tokenize the texts using byte pair encoding and map tokens from the original Russian-language tokenizer to the new one based on indices. Then the model was fine-tuned for the token classification task. To fine-tune the model, a tagged subcorpus of three hagiographical texts was used, which included 35,603 tokens and 2,885 sentences. The experiment took into account only the tags of the parts of speech, the classification included seventeen tags, thirteen of which corresponded to parts of speech, and the remaining four marked punctuation marks. To evaluate the quality of the model, the standard metrics F1 and Accuracy were used. According to automatic evaluation metrics, the RuBERT model showed the best result. Most of the errors were related to incorrect generalization of linear position patterns or to the similarity of word forms in both the extreme left and extreme right positions.

An Hybrid Part of Speech Tagger for Setswana Language using a Voting Method

Part-of Speech (PoS) tagging is a corpus linguistics that deals with assigning appropriate lexical categories to each word in a sentence. To effectively address challenges associated with PoS tagging, several Natural Language Processing (NLP) tasks modelling techniques have been employed, including Conditional Random Fields (CRF), Support Vector Machines (SVM), and Decision Trees in diverse languages. These PoS taggers implement the process of associating the correct PoS (nouns, verbs, adjectives, adverbs, etc.) with each word in a sentence. However, creating language resources is an expensive process for many languages, including the indigenous languages of South Africa that are classified as resource-scarce. Therefore, using Setswana as a language with limited resources, this study explores and applies methods to increase the utilization of existing resources and tagger accuracy. This is done using Setswana's two PoS taggers: a Maximum Entropy (MaxEnt) and an SVM, which achieved an accurateness of 94.4 per cent and 95.59 per cent respectively. To find errors in the taggers, an error analysis is carried out. The Setswana PoS Tagger was then built using a voting algorithm to improve results and attain 97.06 per cent accuracy. The combination of taggers reduces the error rate by up to 2.01 per cent.

Exploring Disciplinary Variation in Pakistani Academic Writing: A Corpus-Based Research on Doctoral Dissertations

It was a corpus-based research and aimed to explore disciplinary variations (in the use of phrasal features) in academic writing (AW) produced by L2 writers of English across disciplines. Corpus of this research (comprising 80 dissertations written by Pakistani doctoral level candidates from two academic disciplines i.e. Arts and Humanities and Life Sciences) was analyzed through AntConc after tagging through POS and MAT Taggers. The results revealed Pakistani AW (across academic disciplines) containing nouns more than any other phrasal feature. In addition, results also revealed the order of the highest and lowest frequencies to be similar both inter-, and intra-disciplines. It meant that Pakistani AW contained phrasal features. However, due to the frequent use of same features (i.e. nouns) in all of the eight sub-disciplines of Arts and Humanities and Life Sciences, disciplinary variation (except some inter- and intra-disciplinary variations in the frequencies of the use of the said features) was not significantly observed in Pakistani AW. On the basis of these results, it was concluded that Pakistani AW did not reflect significant disciplinary variation (a characteristic feature of L1 and L2 AW) in the use of phrasal features.

Exploring the decomposition of epics using natural language processing

Agile user requirements are typically givens as user stories written using natural language and they come in different forms. The most complex form of stories to work with are epics. If epics are poorly understood, they can contribute to threats regarding the sprints or projects becoming behind schedule. It can be attributed to the epic's complexity. The research aimed to explore and attempt the use of Stanza from the Stanford NLP group in the decomposition of epics by creating a text generative model. We have also utilised the chunking technique to formulate the tasks from the generated user stories by identifying the linguistic structure through the aid of a POS tagger. The obtained results illustrate that the stanza can be utilised in the requirements engineering domain such as Sprint backlog grooming. The benefits of this research work are enormous considering that sprint backlog grooming takes considerable time and is always in iterative mode. Agile teams will also benefit from this work by efficiently using sprint timeboxes with minimal sprint planning effort. This will enable agile teams to spend more time delivering the right solutions with reduced sprint planning time and effort.

Neural POS tagging of shahmukhi by using contextualized word representations

Part of Speech (POS) tagging has a preliminary role in building natural language processing applications. This paper presents the development and evaluation of the first POS tagged corpus along with a Bi-directional long-short memory (BiLSTM) network based POS tagger for Shahmukhi (Western Punjabi) at this scale. A balanced corpus of 0.13 million words has been annotated which contains text from 14 different text domains. A Shahmukhi POS tagset has been devised by studying the applicability of the CLE Urdu POS tagset and tagging guidelines have also been designed for annotation. A multi-step corpus evaluation process has been employed for tagged corpus including grammar-based and n-gram based consistency evaluations. The average inter-annotator agreement for all domains is 95.35% along with an average Kappa coefficient of 0.94. The performance of the BiLSTM POS tagger has been compared with the well-known language independent TreeTagger and the Stanford POS tagger. The accuracy of the tagger has been further improved by employing transfer learning by training context-free (Word2Vec) and contextualized (ELMo) word representations on a corpus of 14.9 Shahmukhi words which has been collected from World Wide Web. The tagger performed with an f-score of 96.11 and the accuracy of 96.12%. For a morphologically-rich and low-resourced language, these POS tagging results are quite promising.

Is word length inaccurate for authorship attribution?

Abstract Word length refers to a feature that is extracted from texts and used to characterize authorial style; it was quantitatively demonstrated by Mendenhall (Mendenhall, T. C., 1887, The characteristics curves of composition. Science, IX: 237–49). Many similar features for describing authorial style have been proposed; however, research indicates that compared with other features, word length identifies authors with lower accuracy. This study proposes a feature, referred to as c-wordL, to improve the accuracy of authorship attribution in texts through the classification of words into several types by following the part-of-speech (POS) tags and combining these types with the word length data. The proposed method was tested using 200 literary texts from ten different authors in Japanese, English, and Chinese. The results indicated that c-wordL was more accurate than the existing word length-based features and provided useful information that word unigrams and POS tag bigrams could not measure. In addition, the ease of interpretation of different types of features was discussed. In summary, c-wordL outperformed the existing superior features in explaining the distinct writing styles and identifying the authors.

Bilingual Corpus-based Hybrid POS Tagger for Low Resource Tamil Language: A Statistical approach

In India, most of the Science and Technology resources available are in English. Developing an Automatic Language Translation Engine from English (source language) to Tamil (target language) is very essential for the people who need to get technical resources in their native language. The challenges in designing such engines using Natural Language Processing (NLP) tools include Lexical, Structural, and Syntax level ambiguity. To solve these challenges, the development of a Part-Of-Speech (POS) tagger is essential. The Verb-Framed languages like Tamil, Japanese, and many languages in Romance, Semitic, and Mayan languages families have high morphological richness but lack either a large volume of annotated corpora or manually constructed linguistic resources for building POS tagger. Moreover, the Tamil Language has a low resource, high word sense ambiguity, and word-free order form giving rise to challenges in designing Tamil POS taggers. In this paper, we postulate a Hybrid POS tagger algorithm for Tamil Language using Cross-Lingual Transformation Learning Techniques. It is a novel Mining-based algorithm (MT), which finds equivalent words of Tamil in English on less volume of English-Tamil bilingual unannotated parallel corpus. To enhance the performance of MT, we developed Tamil language-specific auxiliary algorithms such as Keyword-based tagging algorithm (KT) and Verb pattern-based tagging algorithm (VT). We also developed a Unique pair occurrence-tagging algorithm (UT) to find the one-time occurrence of Tamil-English pair words. Our experiments show that by improving Context-based Bilingual Corpus to Bilingual parallel corpus and after leaving one-time occurrence words, the proposed Hybrid POS tagger can predict 81.15% words, with 73.51% accuracy and 90.50% precision. Evaluations prove our algorithms can generate language resources, which can improve the performance of NLP tasks in Tamil.

Modified EDA and Backtranslation Augmentation in Deep Learning Models for Indonesian Aspect-Based Sentiment Analysis

In the process of developing a business, aspect-based sentiment analysis (ABSA) could help extract customers' opinions on different aspects of the business from online reviews. Researchers have found great prospective in deep learning approaches to solving ABSA tasks. Furthermore, studies have also explored the implementation of text augmentation, such as Easy Data Augmentation (EDA), to improve the deep learning models’ performance using only simple operations. However, when implementing EDA to ABSA, there will be high chances that the augmented sentences could lose important aspects or sentiment-related words (target words) critical for training. Corresponding to that, another study has made adjustments to EDA for English aspect-based sentiment data provided with the target words tag. However, the solution still needs additional modifications in the case of non-tagged data. Hence, in this work, we will focus on modifying EDA that integrates POS tagging and word similarity to not only understand the context of the words but also extract the target words directly from non-tagged sentences. Additionally, the modified EDA is combined with the backtranslation method, as the latter has also shown quite a significant contribution to the model’s performance in several research studies. The proposed method is then evaluated on a small Indonesian ABSA dataset using baseline deep learning models. Results show that the augmentation method could increase the model’s performance on a limited dataset problem. In general, the best performance for aspect classification is achieved by implementing the proposed method, which increases the macro-accuracy and F1, respectively, on Long Short-Term Memory (LSTM) and Bidirectional LSTM models compared to the original EDA. The proposed method also obtained the best performance for sentiment classification using a convolutional neural network, increasing the overall accuracy by 2.2% and F1 by 3.2%. Doi: 10.28991/ESJ-2023-07-01-018 Full Text: PDF