Part-of-speech Tagging Research Articles

Comparative Analysis of Hidden Markov Model and Bidirectional Long Short-Term Memory for POS Tagging in Eastern Armenian

This scientific article introduces a novel POS tagging system specifically developed for the Eastern Armenian language. The primary objective of the study is to conduct a comparative analysis of two well-established methods for part-of-speech (POS) tagging: the hidden Markov model (HMM) coupled with the Viterbi algorithm, and an artificial neural network in the form of a recurrent neural network. The study places particular emphasis on the Eastern Armenian language and employs the ArmSpeech-POS Eastern Armenian part-of-speech tagged corpus for conducting comprehensive experiments and evaluations. POS tagging is a fundamental task in natural language processing (NLP) that involves assigning grammatical tags to words in a given text. Accurate POS tagging is crucial for various NLP applications, including machine translation, information retrieval, and sentiment analysis. The Viterbi algorithm is a well-established probabilistic method that utilizes a hidden Markov model (HMM) to determine the most likely sequence of POS tags. On the other hand, RNNs, a type of deep learning model, can capture complex patterns and dependencies in sequential data. Experimental results indicate that both methods achieve reasonable accuracy in POS tagging for Eastern Armenian. However, the RNN outperforms the Viterbi algorithm, exhibiting higher accuracy rates. This can be attributed to the RNN’s ability to capture long-range dependencies and learn intricate linguistic patterns. The article concludes by discussing the implications of the study’s findings and potential areas for further research. It emphasizes the significance of accurate POS tagging for improving NLP applications in Eastern Armenian and suggests exploring advanced neural network architectures and incorporating linguistic features to enhance POS tagging performance.

ArmSpeech-POS: Eastern Armenian Part-of-Speech Tagged Corpus

Text chunking, Part-of-speech (POS) tagging, and named entity recognition (NER) are fundamental tasks in natural language processing (NLP). Part-of-speech (POS) tagging involves assigning grammatical labels to words in a sentence. Research shows that Armenian is a low-resourced language and there are not enough materials for developing higher accurate part-of-speech tagging systems in the Armenian language. This paper presents a fresh dataset for POS tagging in Armenian that follows the naming conventions of both Penn Treebank and Universal Dependencies tagsets, with two versions available. The dataset consists of 6081 sentences that were automatically annotated and then manually verified. The data was sourced from Armenian news websites, focusing on topics such as culture, medicine, and lifestyle, as well as 22 Armenian fairytales. The reason for having two versions of the POS tagset was to ensure compatibility and integration with all-natural language processing tools and models that use these standards. By standardizing the tagset, it becomes easier to compare and evaluate the effectiveness of different POS tagging models. The paper also describes data collection, cleaning, preprocessing, and processing steps. The ISMA translator was used for the annotation of the dataset, which not only performs machine translation but also conducts a syntactic and semantic analysis of the text and assigns a POS tag for each word in the sentence. The final corpus contains 13 groups of part-of-speech tags and a total of 57160 tagged tokens including the distinction between singular and plural parts of speech.

POS Tagging and Lemmatization of Historical Varieties of Languages. The Challenge of Old Italian

The paper discusses the challenges of POS tagging and lemmatization of historical varieties of Italian, and reports for both tasks the results of experiments carried out in a classical supervised domain adaptation scenario using the diachronic and typologically differentiated corpus built for the "Vocabolario Dinamico dell’Italiano Moderno" (VoDIM). For what concerns POS tagging, the effectiveness of retrained models is illustrated and substantiated with quantitative data, with a specific view to linguistic annotation results obtained with respect to specific language evolution stages, domains and textual genres. For lemmatization, different customized models have been developed, including lexicon-assisted ones and models retrained with historical annotated texts. In both cases, a detailed error analysis is provided.

Exploring the Performance of Farasa and CAMeL Taggers for Arabic Dialect Tweets

In Natural Language Processing (NLP), Part Of Speech (POS) tagging is an important step; it is a fundamental requirement for many applications, such as information extraction, machine translation, and grammar checking. Successful POS taggers have been developed for many languages, including Arabic. Currently, the spread of social media has increased the diversity of dialects as people use them in their online communications. Therefore, it has become more difficult for researchers to classify some words that are understood by humans but not computers. In addition, most Arabic POS research focuses on Modern Standard Arabic (MSA), while Dialect Arabic (DA) receives less attention. This paper aims to evaluate the performance of two Arabic taggers when used on dialect Arabic tweets and determine which tagger is the appropriate one, which will accordingly help to improve the existent taggers for dialect Arabic tweets. We used the Farasa and CAMeL taggers, which are commonly used to analyze Arabic texts and are considered the best taggers for Arabic. The results indicate that CAMeL tagger performed better than Farasa tagger, with accuracies of 92% and 83% respectively. In other words, a hybrid POS tagger trained with MSA and DA returns better results than the one trained on MSA.

Aspect-Based Sentiment Analysis for Social Multimedia: A Hybrid Computational Framework

People utilize microblogs and other social media platforms to express their thoughts and feelings regarding current events, public products and the latest affairs. People share their thoughts and feelings about various topics, including products, news, blogs, etc. In user reviews and tweets, sentiment analysis is used to discover opinions and feelings. Sentiment polarity is a term used to describe how sentiment is represented. Positive, neutral and negative are all examples of it. This area is still in its infancy and needs several critical upgrades. Slang and hidden emotions can detract from the accuracy of traditional techniques. Existing methods only evaluate the polarity strength of the sentiment words when dividing them into positive and negative categories. Some existing strategies are domain-specific. The proposed model incorporates aspect extraction, association rule mining and the deep learning technique Bidirectional Encoder Representations from Transformers (BERT). Aspects are extracted using Part of Speech Tagger and association rule mining is used to associate aspects with opinion words. Later, classification was performed using BER. The proposed approach attained an average of 89.45% accuracy, 88.45% precision and 85.98% recall on different datasets of products and Twitter. The results showed that the proposed technique achieved better than state-of-the-art sentiment analysis techniques.

Neural Attention Model for Abstractive Text Summarization Using Linguistic Feature Space

Summarization generates a brief and concise summary which portrays the main idea of the source text. There are two forms of summarization: abstractive and extractive. Extractive summarization chooses important sentences from the text to form a summary whereas abstractive summarization paraphrase using advanced and nearer-to human explanation by adding novel words or phrases. For a human annotator, producing summary of a document is time consuming and expensive because it requires going through the long document and composing a short summary. An automatic feature-rich model for text summarization is proposed that can reduce the amount of labor and produce a quick summary by using both extractive and abstractive approach. A feature-rich extractor highlights the important sentences in the text and linguistic characteristics are used to enhance results. The extracted summary is then fed to an abstracter to further provide information using features such as named entity tags, part of speech tags and term weights. Furthermore, a loss function is introduced to normalize the inconsistency between word-level and sentence-level attentions. The proposed two-staged network achieved a ROUGE score of 37.76% on the benchmark CNN/DailyMail dataset, outperforming the earlier work. Human evaluation is also conducted to measure the comprehensiveness, conciseness and informativeness of the generated summary.

Intelligent Part of Speech tagger for Hindi

English Part of Speech like noun, verb, adverb, adjective, pronoun, preposition, interjection, conjunction is somewhat similar in Hindi but not exactly the same. Hindi grammar has different Part of Speech (POS) based on its morphological features and the occurrence of a word/lexeme in a sentence. The existing techniques used in English language for POS tagging may not work properly for Indian language like Hindi. It is because the grammatical structure of the relatively free word order language like Hindi differs from English. Stochastic taggers may not give good performance as morphological information is not taken into account. The available Hindi word corpora usually have less frequency for individual tags. As a result, a larger size corpus having diversity in the type of sentences can provide better results. But, even after using smoothing techniques most these taggers fail to provide correct results in the presence of unknown words. Considering these aspects, this paper proposes an Intelligent POS tagger for Hindi language based on VITERBI and K-Nearest Neighbour, capable of providing more accurate results than VITERBI in the presence of unknown words.

Language-based transfer learning approaches for part-of-speech tagging on Saint Petersburg Corpus of Hagiographic texts (SKAT)

The article describes an experiment about training a part-of-speech tagger using artificial neural networks on the St. Petersburg Corpus of Hagiographic Texts (SKAT), which is being developed at the Department of Mathematical Linguistics of St. Petersburg State University. The corpus includes the texts of 23 manuscripts dating from the 15th–18th centuries with about 190,000 words usages, four of which were labelled manually. The bi-LSTM, distilled RuBERTtiny2 and RuBERT models were used to train a POS tagger. All of them were trained on modern Russian corpora and further fine-tuned to label Old Russian texts using a technique called language transfer. To fine-tune transformer-based language models it was necessary to tokenize the texts using byte pair encoding and map tokens from the original Russian-language tokenizer to the new one based on indices. Then the model was fine-tuned for the token classification task. To fine-tune the model, a tagged subcorpus of three hagiographical texts was used, which included 35,603 tokens and 2,885 sentences. The experiment took into account only the tags of the parts of speech, the classification included seventeen tags, thirteen of which corresponded to parts of speech, and the remaining four marked punctuation marks. To evaluate the quality of the model, the standard metrics F1 and Accuracy were used. According to automatic evaluation metrics, the RuBERT model showed the best result. Most of the errors were related to incorrect generalization of linear position patterns or to the similarity of word forms in both the extreme left and extreme right positions.

An Hybrid Part of Speech Tagger for Setswana Language using a Voting Method

Part-of Speech (PoS) tagging is a corpus linguistics that deals with assigning appropriate lexical categories to each word in a sentence. To effectively address challenges associated with PoS tagging, several Natural Language Processing (NLP) tasks modelling techniques have been employed, including Conditional Random Fields (CRF), Support Vector Machines (SVM), and Decision Trees in diverse languages. These PoS taggers implement the process of associating the correct PoS (nouns, verbs, adjectives, adverbs, etc.) with each word in a sentence. However, creating language resources is an expensive process for many languages, including the indigenous languages of South Africa that are classified as resource-scarce. Therefore, using Setswana as a language with limited resources, this study explores and applies methods to increase the utilization of existing resources and tagger accuracy. This is done using Setswana's two PoS taggers: a Maximum Entropy (MaxEnt) and an SVM, which achieved an accurateness of 94.4 per cent and 95.59 per cent respectively. To find errors in the taggers, an error analysis is carried out. The Setswana PoS Tagger was then built using a voting algorithm to improve results and attain 97.06 per cent accuracy. The combination of taggers reduces the error rate by up to 2.01 per cent.

Exploring Disciplinary Variation in Pakistani Academic Writing: A Corpus-Based Research on Doctoral Dissertations

It was a corpus-based research and aimed to explore disciplinary variations (in the use of phrasal features) in academic writing (AW) produced by L2 writers of English across disciplines. Corpus of this research (comprising 80 dissertations written by Pakistani doctoral level candidates from two academic disciplines i.e. Arts and Humanities and Life Sciences) was analyzed through AntConc after tagging through POS and MAT Taggers. The results revealed Pakistani AW (across academic disciplines) containing nouns more than any other phrasal feature. In addition, results also revealed the order of the highest and lowest frequencies to be similar both inter-, and intra-disciplines. It meant that Pakistani AW contained phrasal features. However, due to the frequent use of same features (i.e. nouns) in all of the eight sub-disciplines of Arts and Humanities and Life Sciences, disciplinary variation (except some inter- and intra-disciplinary variations in the frequencies of the use of the said features) was not significantly observed in Pakistani AW. On the basis of these results, it was concluded that Pakistani AW did not reflect significant disciplinary variation (a characteristic feature of L1 and L2 AW) in the use of phrasal features.

Exploring the decomposition of epics using natural language processing

Agile user requirements are typically givens as user stories written using natural language and they come in different forms. The most complex form of stories to work with are epics. If epics are poorly understood, they can contribute to threats regarding the sprints or projects becoming behind schedule. It can be attributed to the epic's complexity. The research aimed to explore and attempt the use of Stanza from the Stanford NLP group in the decomposition of epics by creating a text generative model. We have also utilised the chunking technique to formulate the tasks from the generated user stories by identifying the linguistic structure through the aid of a POS tagger. The obtained results illustrate that the stanza can be utilised in the requirements engineering domain such as Sprint backlog grooming. The benefits of this research work are enormous considering that sprint backlog grooming takes considerable time and is always in iterative mode. Agile teams will also benefit from this work by efficiently using sprint timeboxes with minimal sprint planning effort. This will enable agile teams to spend more time delivering the right solutions with reduced sprint planning time and effort.

Neural POS tagging of shahmukhi by using contextualized word representations

Part of Speech (POS) tagging has a preliminary role in building natural language processing applications. This paper presents the development and evaluation of the first POS tagged corpus along with a Bi-directional long-short memory (BiLSTM) network based POS tagger for Shahmukhi (Western Punjabi) at this scale. A balanced corpus of 0.13 million words has been annotated which contains text from 14 different text domains. A Shahmukhi POS tagset has been devised by studying the applicability of the CLE Urdu POS tagset and tagging guidelines have also been designed for annotation. A multi-step corpus evaluation process has been employed for tagged corpus including grammar-based and n-gram based consistency evaluations. The average inter-annotator agreement for all domains is 95.35% along with an average Kappa coefficient of 0.94. The performance of the BiLSTM POS tagger has been compared with the well-known language independent TreeTagger and the Stanford POS tagger. The accuracy of the tagger has been further improved by employing transfer learning by training context-free (Word2Vec) and contextualized (ELMo) word representations on a corpus of 14.9 Shahmukhi words which has been collected from World Wide Web. The tagger performed with an f-score of 96.11 and the accuracy of 96.12%. For a morphologically-rich and low-resourced language, these POS tagging results are quite promising.

Is word length inaccurate for authorship attribution?

Abstract Word length refers to a feature that is extracted from texts and used to characterize authorial style; it was quantitatively demonstrated by Mendenhall (Mendenhall, T. C., 1887, The characteristics curves of composition. Science, IX: 237–49). Many similar features for describing authorial style have been proposed; however, research indicates that compared with other features, word length identifies authors with lower accuracy. This study proposes a feature, referred to as c-wordL, to improve the accuracy of authorship attribution in texts through the classification of words into several types by following the part-of-speech (POS) tags and combining these types with the word length data. The proposed method was tested using 200 literary texts from ten different authors in Japanese, English, and Chinese. The results indicated that c-wordL was more accurate than the existing word length-based features and provided useful information that word unigrams and POS tag bigrams could not measure. In addition, the ease of interpretation of different types of features was discussed. In summary, c-wordL outperformed the existing superior features in explaining the distinct writing styles and identifying the authors.

Bilingual Corpus-based Hybrid POS Tagger for Low Resource Tamil Language: A Statistical approach

In India, most of the Science and Technology resources available are in English. Developing an Automatic Language Translation Engine from English (source language) to Tamil (target language) is very essential for the people who need to get technical resources in their native language. The challenges in designing such engines using Natural Language Processing (NLP) tools include Lexical, Structural, and Syntax level ambiguity. To solve these challenges, the development of a Part-Of-Speech (POS) tagger is essential. The Verb-Framed languages like Tamil, Japanese, and many languages in Romance, Semitic, and Mayan languages families have high morphological richness but lack either a large volume of annotated corpora or manually constructed linguistic resources for building POS tagger. Moreover, the Tamil Language has a low resource, high word sense ambiguity, and word-free order form giving rise to challenges in designing Tamil POS taggers. In this paper, we postulate a Hybrid POS tagger algorithm for Tamil Language using Cross-Lingual Transformation Learning Techniques. It is a novel Mining-based algorithm (MT), which finds equivalent words of Tamil in English on less volume of English-Tamil bilingual unannotated parallel corpus. To enhance the performance of MT, we developed Tamil language-specific auxiliary algorithms such as Keyword-based tagging algorithm (KT) and Verb pattern-based tagging algorithm (VT). We also developed a Unique pair occurrence-tagging algorithm (UT) to find the one-time occurrence of Tamil-English pair words. Our experiments show that by improving Context-based Bilingual Corpus to Bilingual parallel corpus and after leaving one-time occurrence words, the proposed Hybrid POS tagger can predict 81.15% words, with 73.51% accuracy and 90.50% precision. Evaluations prove our algorithms can generate language resources, which can improve the performance of NLP tasks in Tamil.

Modified EDA and Backtranslation Augmentation in Deep Learning Models for Indonesian Aspect-Based Sentiment Analysis

In the process of developing a business, aspect-based sentiment analysis (ABSA) could help extract customers' opinions on different aspects of the business from online reviews. Researchers have found great prospective in deep learning approaches to solving ABSA tasks. Furthermore, studies have also explored the implementation of text augmentation, such as Easy Data Augmentation (EDA), to improve the deep learning models’ performance using only simple operations. However, when implementing EDA to ABSA, there will be high chances that the augmented sentences could lose important aspects or sentiment-related words (target words) critical for training. Corresponding to that, another study has made adjustments to EDA for English aspect-based sentiment data provided with the target words tag. However, the solution still needs additional modifications in the case of non-tagged data. Hence, in this work, we will focus on modifying EDA that integrates POS tagging and word similarity to not only understand the context of the words but also extract the target words directly from non-tagged sentences. Additionally, the modified EDA is combined with the backtranslation method, as the latter has also shown quite a significant contribution to the model’s performance in several research studies. The proposed method is then evaluated on a small Indonesian ABSA dataset using baseline deep learning models. Results show that the augmentation method could increase the model’s performance on a limited dataset problem. In general, the best performance for aspect classification is achieved by implementing the proposed method, which increases the macro-accuracy and F1, respectively, on Long Short-Term Memory (LSTM) and Bidirectional LSTM models compared to the original EDA. The proposed method also obtained the best performance for sentiment classification using a convolutional neural network, increasing the overall accuracy by 2.2% and F1 by 3.2%. Doi: 10.28991/ESJ-2023-07-01-018 Full Text: PDF

Aspect based Nepali Text Sentiment Analysis using BiLSTM

Aspect-Based Sentiment Analysis is a text analysis technique that classifies the sentiment of a specific aspect in a text. With a rise in internet and social media access, people tend to decide to buy a product or to visit a restaurant based on comments and reviews. Categorical reviews can play a significant role in improving the business of a restaurant if a business entity can capture insights of those reviews and use productively. Since Nepali is a low resource language there is no significant numbers of the dataset for restaurant reviews. Therefore a dataset is created in the work to model a target aspect-based sentiment analysis in restaurant review domain. The dataset comprises of comments extracted from social media sites, websites, and manually prepared data. Data are cleaned, pre-processed then aspects and sentiments are extracted using POS tagging. Sentiments and target categories are classified using word embedding and the deep learning BiLSTM model. The sentiment classification model classifies the sentiments into positive or negative. The target classification model classifies text into 5 target categories Food (F), Price (P), Service (S), Hygiene (H), and Experience (E). The result was found to be good compared to similar models in other domains. The sentiment prediction model and target category prediction model attains the F1 Score of 0.88 and 0.76 respectively.

Aspect Based Opinion Mining & Sentiment Analysis

Opinion mining is a relatively new field that refers to the practice of collecting feedback in the form of online reviews and ratings left by users on various topics. Researchers are now able to monitor the states of consciousness of individuals in real-time because to this development. Just lately, a number of research papers for sentiment analysis were implemented, each of which was based on a unique categorization and ranking procedure. However, the amount of time necessary for the newline performing class has not decreased in any way. Sentiment Sensitivity newline word list SST was provided as a solution to the problem of function mismatch in the go-domain sentiment class across the source area and the target domain; however, achieving improved accuracy and identifying distributional similarities of words became less effective as time went on. Hidden Markov’s persistent development may be seen at the beginning. Cosine In order to achieve more effective and clean pre-processing, a method that is conceptually quite similar to HM-CPCS has been devised. The HM-CPCS methodology, which has recently been suggested, makes use of the POS tagger, a variant of which is based on the Hidden Markov algorithm. Evaluations are created using data from a wide variety of different domains. Similar to a newline, the tags that come before and after it compute the possibility of transitions and the existence of the term newline among the tags in order to increase capability. This is done in order to improve capability.

Analysis of Telkom University News Subjects on Popular Indonesian News Portals Using a Combination of Hidden Markov Model (HMM) and Rule Based Methods

News media are often found in everyday life as a means of information for the public about something that is happening. In news articles, it is common to see several sentences that support the object to increase its popularity by being promoted by the subject. Part of Speech Tagging can determine the class of words in the sentence according to Tagsets provided by the corpus. That way, the search for the subject in the news article can be found from the word class obtained from a corpus. This research was focused on finding the subject "who" repeatedly spreading the news about Telkom University by using Part of Speech Tagging with the Hidden Markov Model and Rule Based on a news dataset from popular news portals about Telkom University. The process is taking all news about Telkom University on popular news portals and classifying it using the Hidden Markov Model and Rule-Based. We conducted to enhance the research results by changing the probability estimator on Hidden Markov Model. After running some scenarios, the best results obtained by the Hidden Markov Model and Rule-Based are the Accuracy of 94.96%, the Precision of 94.99%, the Recall of 94.96%, and the F1-Score of 94.95%.

Transitivity Analysis Annotation Scheme and Its Application to CQPweb: A Study on Stephen King’s Carrie

In this study, I aim to create a Transitivity Analysis Annotation Scheme (TAAS) within which, process types and clause patterns are taken into account. Once created, TAAS is applied to a corpus data, obtained from a novel entitled Carrie [8]. The scheme is manually implemented using XML format. The XML formatted corpus is converted to a format accepted in CQPweb [6], a corpus query tool. The corpus is tokenised using the Perl tokenizer, in TreeTagger [10]. Once tokenised, the corpus is tagged using English TreeTagger [10], which supplies two grammatical annotations (Part of Speech and Lemma). The annotated corpus is then indexed to CQPweb. Once indexed, users are able to search the annotated corpus using transitivity analytic labels, POS tags and lemma, in CQPweb. Keywords: transitivity analysis; annotation; corpus linguistics; CQPweb

Sentiment analysis on social media tweets using dimensionality reduction and natural language processing

AbstractSocial media has been embraced by different people as a convenient and official medium of communication. People write or share messages and attach images and videos on Twitter, Facebook and other social media platforms. It therefore generates a lot of data that is rich in sentiments. Sentiment analysis has been used to determine the opinions of clients, for instance, relating to a particular product or company. Lexicon and machine learning approaches are the strategies that have been used to analyze these sentiments. The performance of sentiment analysis is, however, distorted by noise, the curse of dimensionality, the data domains and the size of data used for training and testing. This article aims at developing a model for sentiment analysis of social media data in which dimensionality reduction and natural language processing with part of speech tagging are incorporated. The model is tested using Naïve Bayes, support vector machine, and K‐nearest neighbor algorithms, and its performance compared with that of two other sentiment analysis models. Experimental results show that the model improves sentiment analysis performance using machine learning techniques.