Part-of-speech Tagging Research Articles

Improving part-of-speech tagging in Amharic language using deep neural network

To date, several POS taggers have been introduced to facilitate the success of semantic analysis for different languages. However, the task of POS tagging becomes a bit intricate in morphologically complex languages, like Amharic. In this paper, we evaluated different models such as bidirectional long short term memory, convolutional neural network in combination with bidirectional long short term memory, and conditional random field for Amharic POS tagging. Various features, both language-dependent and -independent, have been explored in a conditional random field model. Besides, word-level and character-level features are analyzed in deep neural network models. A convolutional neural network is utilized for encoding features at the word and character level. Each model's performance has evaluated on the dataset that contained 321 K tokens and manually tagged with 31 POS tags. Lastly, the best performance obtained by an end-to-end deep neural network model, convolutional neural network in combination with bidirectional long term short memory and conditional random field, is 97.23% accuracy. This is the highest accuracy for Amharic POS tagging task and is competent with contemporary taggers currently existing in different languages.

Keyword/ Keyphrase Extraction from Text of Indian Election Domain

In recent years, there has been a tremendous increase in the amount of data generated from various sources including social media, news articles, and blogs. With the rise of social media platforms, people are expressing their opinions more freely than ever before. This has led to an explosion of data in the Indian Election domain, where people express their views on various political parties and candidates. In order to extract meaningful information from this vast amount of data, it is important to identify and extract relevant keywords and phrases. Keyword and phrase extraction is the process of automatically identifying important words and phrases from a piece of text. This process is crucial for various natural language processing tasks such as text mining, sentiment analysis, topic modeling, and text classification. In this research paper, we focus on the task of keyword and phrase extraction from Indian Election domain text. We aim to extract relevant keywords and phrases that are most commonly used in the context of Indian elections. This research is important as it can help in understanding the key issues and concerns of Indian voters during the election season. We use various natural language processing techniques and machine learning algorithms to extract keywords and phrases from a large corpus of Indian Election domain text. Our approach involves pre-processing the text, including tokenization, stop-word removal, stemming, and POS tagging. We then use various statistical and machine learning models to identify the most relevant keywords and phrases.

Robust Multi-task Learning-based Korean POS Tagging to Overcome Word Spacing Errors

End-to-end neural network-based approaches have recently demonstrated significant improvements in natural language processing (NLP). However, in the NLP application such as assistant systems, NLP components are still processed to extract results using a pipeline paradigm. The pipeline-based concept has issues with error propagation. In Korean, morphological analysis and part-of-speech (POS) tagging step, incorrectly analyzing POS tags for a sentence containing spacing errors negatively affects other modules behind the POS module. Hence, we present a multi-task learning-based POS tagging neural model for Korean with word spacing challenges. When we apply this model to the Korean morphological analysis and POS tagging, we get findings that are robust to word spacing errors. We adopt syllable-level input and output formats, as well as a simple structure for ELECTRA and RNN-CRF models for multi-task learning, and we achieve a good performance 98.30 of F1, better than previous studies on the Sejong corpus test set.

Part-of-Speech Tagging of Odia Language Using Statistical and Deep Learning Based Approaches

Automatic part-of-speech (POS) tagging is a preprocessing step of many natural language processing tasks, such as named entity recognition, speech processing, information extraction, word sense disambiguation, and machine translation. It has already gained promising results in English and European languages. However, in Indian languages, particularly in the Odia language, it is not yet well explored because of the lack of supporting tools, resources, and morphological richness of the language. Unfortunately, we were unable to locate an open source POS tagger for the Odia language, and only a handful of attempts have been made to develop POS taggers for the Odia language. The main contribution of this research work is to present statistical approaches such as the maximum entropy Markov model and conditional random field (CRF), as well as deep learning based approaches, including the convolutional neural network (CNN) and bidirectional long short-term memory (Bi-LSTM) to develop the Odia POS tagger. A publicly accessible corpus annotated with the Bureau of Indian Standards (BIS) tagset is used in our work. However, most of the languages around the globe have used the dataset annotated with the Universal Dependencies (UD) tagset. Hence, to maintain uniformity, the Odia dataset should use the same tagset. Thus, following the BIS and UD guidelines, we constructed a mapping from the BIS tagset to the UD tagset. The maximum entropy Markov model, CRF, Bi-LSTM, and CNN models are trained using the Indian Languages Corpora Initiative corpus with the BIS and UD tagsets. We have experimented with various feature sets as input to the statistical models to prepare a baseline system and observed the impact of constructed feature sets. The deep learning based model includes the Bi-LSTM network, the CNN network, the CRF layer, character sequence information, and a pre-trained word vector. Seven different combinations of neural sequence labeling models are implemented, and their performance measures are investigated. It has been observed that the Bi-LSTM model with the character sequence feature and pre-trained word vector achieved a result with 94.58% accuracy.

The Impact of Arabic Diacritization on Word Embeddings

Word embedding is used to represent words for text analysis. It plays an essential role in many Natural Language Processing (NLP) studies and has hugely contributed to the extraordinary developments in the field in the last few years. In Arabic, diacritic marks are a vital feature for the readability and understandability of the language. Current Arabic word embeddings are non-diacritized. In this article, we aim to develop and compare word embedding models based on diacritized and non-diacritized corpora to study the impact of Arabic diacritization on word embeddings. We propose evaluating the models in four different ways: clustering of the nearest words; morphological semantic analysis; part-of-speech tagging; and semantic analysis. For a better evaluation, we took the challenge to create three new datasets from scratch for the three downstream tasks. We conducted the downstream tasks with eight machine learning algorithms and two deep learning algorithms. Experimental results show that the diacritized model exhibits a better ability to capture syntactic and semantic relations and in clustering words of similar categories. Overall, the diacritized model outperforms the non-diacritized model. We obtained some more interesting findings. For example, from the morphological semantics analysis, we found that with the increase in the number of target words, the advantages of the diacritized model are also more obvious, and the diacritic marks have more significance in POS tagging than in other tasks.

Agglutinativ tillar uchun pos teglash va stemming masalasi (turk, uyg‘ur, o‘zbek tillari misolida)

The number of possible word forms in agglutinative languages is theoretically unlimited. This, in turn, creates the problem of POS tagging (part-of-speech) of out-of-vocabulary (OOV) words in agglutinative lan-guages. In agglutinative languages, words are formed by combining stems and suffixes. Because phonetic harmony and disharmony occur when suf-fixes are added to the root, it is necessary to analyze both phonetic and morphological changes. When solving many NLP tasks, it is necessary to reduce word forms to their root (stemming). Removing all inflectional affixes from a word and lemmatizing the rest of the word is considered one of the important tasks of natural language processing (NLP), and this process is called stemming. The stemming process is important in infor-mation retrieval (IR) systems.

Context-aware Answer Selection in Community Question Answering Exploiting Spatial Temporal Bidirectional Long Short-Term Memory

Community Question Answering (CQA) sites provide knowledge sharing facility as the users can post questions and other users can share their answers. The selection of top-quality answers from the set of answers in a thread is a significant and challenging task in Natural Language Processing (NLP). To address this issue, we propose a deep learning based spatial temporal Bidirectional Long Short-Term Memory (Bi-LSTM) algorithm. The existing studies mainly focus only computing semantic similarity between questions and answers using votes given by the users. The proposed hybrid approach, based on both forward and backward, consider question to answer and answer to answer similarity. The forward LSTM captures the spatial impact of the answer to estimate the relevancy, whereas the backward LSTM learns temporal features with the answer to predict the best quality answer. Moreover, spatial Bi-LSTM captures past and future dependencies for a better understanding of context and to improve the effectiveness of answer selection. For extracting meaningful information from noisy text data, data is preprocessed following standard steps such as tokenization, parsing, lemmatization, stop words removal, part of speech tagging and entities extraction. Word embeddings-based Paragraph to vector (par2vec) has additional input nodes to represent paragraph information in vector for context understanding. The empirical analysis carried out on the SemEval CQA dataset shows that the proposed model outperforms state-of-art answer selection approaches.

Slovak Language Models for Basic Preprocessing Tasks in Python

Abstract We propose a Slovak language model for the spaCy library in Python. These models are easy-to-use for basic natural language processing tasks in a single package. The package contains several components for basic preprocessing tasks, such as tokenization, sentence boundary detection, syntactic parsing, lemmatization, named entity recognition, morphology analysis, and word vectors. It is based on the state-of-the-art monolingual SlovakBERT model. Named entity recognition is trained on a separate, publicly available WikiAnn database. The other statistical classifiers use a Slovak Dependency Treebank corpus. Morphological tags are compatible with the conventions of the Slovak National Corpus. The part of speech tags use conventions of the Universal Dependencies framework. We trained a separate word vector model on a web-based corpus. The training uses fastText with Floret modification. We present a series of experiments that confirm that the model performs similarly to other languages for all tasks. Training scripts and data are publicly available.

Conversion of NLP to SQL Using Machine Learning Approach

Abstract: Now days data is increasing rapidly. There are so many new database tools and technologies are rowing, therefore we can store large data, but the problem is that the technology or an interface which can process data and display the data as per the user request is not familiarized with many of the people. It means many people don’t have proper knowledge of handling database. Actually, NLP to SQL conversion is part of machine learning. The main idea of NLP to SQL conversion is to generate SQL query from natural language. This technique is useful for accessing data from database without having prior knowledge of SQL. This technique can be used by many common people. In this system input is simple English text and query is generated by using POS tagger in python. In this project we are going to implement a system which will generate SQL query from natural language.

Morphology & word sense disambiguation embedded multimodal neural machine translation system between Sanskrit and Malayalam

Machine translation is the process of conversion from one language to another using computers. In spite of the existence of several traditional machine translation systems like rule-based, statistical etc., the development of neural machine translation has significantly improved translation quality. However, the morphological richness and agglutinative nature of Indian languages hampers the usage of neural machine translation for translation among Indian languages despite its potential to overcome the difficulties posed by conventional translation systems. Hence the combination of neural machine translation with morphology, part of speech tagger and word sense disambiguation is proposed to address these challenges and adapt it for bidirectional translation between Sanskrit and Malayalam. Conventional neural machine translations are unimodal systems that utilize textual data for translation. Translation quality of conventional neural machine translation is enhanced with a novel multimodal bidirectional neural machine translation system with text & speech as the two modalities based on parallel text corpus and speech database. The best performance score is obtained for two level fusion, the feature vectors extracted from speech signals using different transforms (Wavelet transform, 1-D sequency mapped real transform & 1-D GCD based mapped real transform) are fused and which is further fused with a context vector from the text modality. Automatic and manual review methods are employed to assess quality of translation. Findings from the experiments reveal that addition of speech modality improved the overall quality of translation. BLEU scores of 43.89 and 42.72 are obtained for Sanskrit to Malayalam and Malayalam to Sanskrit multimodal translation respectively.

Multi-lingual opinion mining for social media discourses: an approach using deep learning based hybrid fine-tuned smith algorithm with adam optimizer

Multi-lingual opinion mining has become an important research area in natural language processing, particularly in the context of social media, as it provides valuable insights from user-generated content. In this paper, a deep learning approach based on a hybrid fine-tuned Smith algorithm with Adam optimizer (HFS-AO) is proposed for multi-lingual opinion mining. The approach can process three different languages, namely Marathi, Hindi, and English, which are collected from various web resources using a web scraping algorithm. The collected data is annotated using the Zero-shot instance-weighting technique and pre-processed to remove unnecessary noises and symbols, including data transformation, data cleaning and reduction, stop-word removal, tokenization, lemmatization, stemming, and Parts of Speech Tagging (POS). Naive Bayes vectorization with Laplace smoothing is used for text vectorization, followed by feature selection and classification using the hybrid fine-tuned Smith algorithm with Adam optimizer for polarity classification.The proposed method of multi-lingual opinion mining using a deep learning approach based on a hybrid fine-tuned Smith algorithm with Adam optimizer is effective in extracting valuable insights from social media data in different languages. The approach has demonstrated high accuracy, precision, recall, and F1-Score values, making it a useful tool for improving performance and customer satisfaction by identifying patterns and trends in public opinion.Additionally, the proposed method outperforms other existing methods, including PGM, MCM, CNN, and NBi-LSTM, in terms of computational time and performance making it a valuable contribution to the field of information management.

Public Hospital Review on Map Service with Part of Speech Tagging and Biterm Topic Modeling

Public Hospital Review on Map Service with Part of Speech Tagging and Biterm Topic Modeling

Developing an Urdu Lemmatizer Using a Dictionary-Based Lookup Approach

Lemmatization aims at returning the root form of a word. The lemmatizer is envisioned as a vital instrument that can assist in many Natural Language Processing (NLP) tasks. These tasks include Information Retrieval, Word Sense Disambiguation, Machine Translation, Text Reuse, and Plagiarism Detection. Previous studies in the literature have focused on developing lemmatizers using rule-based approaches for English and other highly-resourced languages. However, there have been no thorough efforts for the development of a lemmatizer for most South Asian languages, specifically Urdu. Urdu is a morphologically rich language with many inflectional and derivational forms. This makes the development of an efficient Urdu lemmatizer a challenging task. A standardized lemmatizer would contribute towards establishing much-needed methodological resources for this low-resourced language, which are required to boost the performance of many Urdu NLP applications. This paper presents a lemmatization system for the Urdu language, based on a novel dictionary lookup approach. The contributions made through this research are the following: (1) the development of a large benchmark corpus for the Urdu language, (2) the exploration of the relationship between parts of speech tags and the lemmatizer, and (3) the development of standard approaches for an Urdu lemmatizer. Furthermore, we experimented with the impact of Part of Speech (PoS) on our proposed dictionary lookup approach. The empirical results showed that we achieved the best accuracy score of 76.44% through the proposed dictionary lookup approach.

Indonesian perceptions on online learning amidst COVID-19: a Twitter sentiment analysis

The objective of this research is to uncover Indonesians' perceptions of online learning during the COVID-19 pandemic by determining the polarity of language texts (positive, neutral, or negative) compiled from Twitter. The data required to reveal the Indonesian people's opinion on online learning during the COVID-19 pandemic is a tweet on Twitter with the hashtag #Pembelajaran daring (Online learning); #Pembelajaran jarak jauh (distance learning); #Belajar dari rumah (learning from home); #Belajar di rumah (learning in the home) (learning at home). The time frame for collecting these tweets is March 2020 to November 2021. The data was then analyzed using lexicon analysis and analytical tools that used Part of Speech Tagging. According to the results, 77.58% of the tweets are positive, 17.97% are negative, and the remainder are neutral. People prefer to refer to learning support, teachers, schools, education, students, and distance learning. Distance learning is the most positively received category among online learning. However, learning support is the most widely discussed topic among the general public. The overwhelming positive sentiment across all categories suggests that the majority of Indonesians have high hopes for online learning during the pandemic.

Transfer knowledge for punctuation prediction via adversarial training

Previous studies demonstrate that part-of-speech (POS) tags are helpful for punctuation restoration tasks. However, extra computation cost will be needed during decoding, due to POS tags are provided by an external POS tagger. This paper proposes to transfer knowledge via adversarial training and orthogonality constraints to fill in the gap. Adversarial multi-task learning is introduced to learn task invariant knowledge from the extra POS tagging task for a punctuation prediction task. Furthermore, orthogonality constraints are used to make private and shared features dissimilar. Only the punctuation predicting task is used during decoding. So extra computation is not needed. Experiments are conducted on IWSLT2011 datasets. The results show that the punctuation predicting models trained with adversarial learning obtain performance gains over the baseline models on test sets. The results also demonstrate that the models trained with orthogonality constraints further obtain performance improvement.

Exploring the efficacy and reliability of automatic text summarisation systems: Arabic texts in focus

This study compared the salient features of the three basic types of automatic text summarisation methods (ATSMs)—extractive, abstractive, and real-time—along with the available approaches used for each type. The data set comprised 12 reports on the current issues on automatic text summarisation methods and techniques across languages, with a special focus on Arabic whose structure has been largely claimed to be problematic in most ATSMs. Three main summarizers were compared: TAAM, OTExtSum, and OntoRealSumm. Further to this, a humanoid version of the summary of the data set was prepared, and then compared to the automatically generated summary. A 10-item questionnaire was built to help with the assessment of the target ATSMs. Also, Rouge analysis was performed to assess the efficacy of all techniques in minimising the redundancy of the data set. Findings showed that the precision of the target summarizers differed considerably, as 80% of the data set has been proven to be aware of the problems underlying ATSMS. The remaining parameters were in the normal range (65–75%). In light of the equations-based assessment of ATSMS, the highest range was noted with the removal of stop word, the least range was noted with POS tagging, stem weight, and stem collection. Regarding Arabic, the statistical analysis has been proven to be the most effective summarisation method (accuracy = 57.59%; reminiscence = 58.79%; F-Value = 57.99%). Further research is required to explore how the lexicogrammatical nature of languages and generic text structure would affect the text summarisation process.

Detecting and Correcting Contextual Mistakes in Sentences Using Part of Speech Tags

A grammar checker is a tool to check each sentence in a text to see whether it conforms to the grammar. In case it finds a structure that conflicts with the conformity to the grammar, it would give suggestions for alternatives. The grammar checkers for European languages and some Indic languages are well developed. However, perhaps, owing to Tamil being a morphologically rich and agglutinative language this has been a challenging task. An approach to detecting and correcting grammatical mistakes due to subject and finite-verb disagreement with regard to person, number and/or gender and due to disagreement in tense aspects in Tamil sentences is proposed in this paper. A method has been proposed that uses hierarchical part-of-speech tags of words to detect the grammatical mistakes in subject and finite-verb agreement and mistakes in tense aspects in Tamil sentences. Two sets of Tamil grammar rules are used to generate suggestions for the grammatical mistakes. Test results show that the proposed grammatical mistake detection and correction system performs well.

Abstractive Summarization of Text Document in Malayalam Language: Enhancing Attention Model Using POS Tagging Feature

Over the past few years, researchers are showing huge interest in sentiment analysis and summarization of documents. The primary reason being that huge volumes of information are available in textual format, and this data has proven helpful for real-world applications and challenges. The sentiment analysis of a document will help the user comprehend the content’s emotional intent. Abstractive summarization algorithms generate a condensed version of the text, which can then be used to determine the emotion represented in the text using sentiment analysis. Recent research in abstractive summarization concentrates on neural network-based models, rather than conjunctions-based approaches, which might improve the overall efficiency. Neural network models like attention mechanism are tried out to handle complex works with promising results. The proposed work aims to present a novel framework that incorporates the part of speech tagging feature to the word embedding layer, which is then used as the input to the attention mechanism. With POS feature being part of the input layer, this framework is capable of dealing with words containing contextual and morphological information. The relevance of POS tagging here is due to its strong reliance on the language’s syntactic, contextual, and morphological information. The three main elements in the work are pre-processing, POS tagging feature in the embedding phase, and the incorporation of it into the attention mechanism. The word embedding provides the semantic concept about the word, while the POS tags give an idea about how significant the words are in the context of the content, which corresponds to the syntactic information. The proposed work was carried out in Malayalam, one of the prominent Indian languages. A widely used and accepted dataset from the English language was translated to Malayalam for conducting the experiments. The proposed framework gives a ROUGE score of 28, which outperformed the baseline models.

Actualization of Evaluativity as a Qualitative Category in the Discourse of English Internet Film Review

Introduction. The article presents the results of the analysis aimed at identifying actualization methods of evaluativity as a qualitative category within the frames of the evaluative subgenres of expert and professional film reviews. The author assumes that evaluativity serves as a method of explicating subject's attitude towards the content of the film. The relevance of the study is determined by a lack of research dedicated to the analysis of the methods expressing author’s position within the evaluative genre of internet film review. The interconnection of evaluativity category, subject factor and modus as a method of explicating subjective meanings of the statement are analyzed.Methodology and sources. The methodology of the research is based on the classification of evaluative meanings developed by N. Arutyunova. The research methodology includes PoS tagging of the texts, identification of evaluative units including qualitative adjectives and adverbs, as well as description and comparison of leading evaluation types that actualize author's position. The methodology was studied using the example of a text corpora consisting of expert and professional film reviews extracted from the web archive of the journal of Film Criticism and the website metacritic.com.Results and discussion. With the implementation of PoS tagging and quantitative method of analysis leading evaluation categories within the corpora of the professional and expert film reviews were described and compared. The results have shown that the leading type of evaluation within the frames of the referred evaluative subgenres is represented by the emotional evaluation category. The results have also indicated that the leading evaluation types of expert evaluative discourse include prescriptive and ethical evaluation, whereas professional evaluative discourse is characterized by the prevalence of generalized evaluative units.Conclusion. The analysis allowed the author to conclude that the referred genres are characterized mainly by increased expressivity and emotiveness. However, author’s position actualization in the discourse of expert film review implicates detailed analysis of structural contents of the presented norms, as well as content of the film, whereas the discourse of professional film review is characterized by increased generalization tendency, which implicates low degree of authors’ competency.

Syllable-Based Multi-POSMORPH Annotation for Korean Morphological Analysis and Part-of-Speech Tagging

Various research approaches have attempted to solve the length difference problem between the surface form and the base form of words in the Korean morphological analysis and part-of-speech (POS) tagging task. The compound POS tagging method is a popular approach, which tackles the problem using annotation tags. However, a dictionary is required for the post-processing to recover the base form and to dissolve the ambiguity of compound POS tags, which degrades the system performance. In this study, we propose a novel syllable-based multi-POSMORPH annotation method to solve the length difference problem within one framework, without using a dictionary for the post-processing. A multi-POSMORPH tag is created by combining POS tags and morpheme syllables for the simultaneous POS tagging and morpheme recovery. The model is implemented with a two-layer transformer encoder, which is lighter than the existing models based on large language models. Nonetheless, the experiments demonstrate that the performance of the proposed model is comparable to, or better than, that of previous models.