Part-of-speech Tagging Research Articles

Human language technology tools for indigenous South African languages and their potential use

Human language technology (HLT) contributes to the development of languages by providing various avenues through which languages can be interrogated. Through HLT, diverse questions can be raised and answered scientifically and objectively. In the context of South African indigenous languages (SAIL), several HLT tools support these languages. However, it seems that some language users are unaware of the availability and capabilities of these tools, which contributes to their underutilisation. This study aims to identify and describe briefly some of the HLT tools that support and analyse SAIL. It presents an overview of the open access HLT tools, namely part-of-speech (POS) taggers, morphological decomposers (MDs), morphological analysers (MAs), isiZulu.net, ZulMorph and Google Translate (GT). These tools are crucial in analysing and understanding SAIL, as well as for advancing these languages in the field of HLT. In this study, the researchers anticipate that by raising awareness of the existence of these tools, more users of indigenous languages will be eager to use them.Contribution: This study fills the practical gap in the use of HLT to perform linguistic functions for SAIL. It seems that there is underutilisation of existing HLT tools for SAIL, which might be attributed to language users being unaware of these tools. Therefore, the study aims to identify and describe some HLT tools that support and analyse SAIL. It presents an overview of the open access HLT tools, namely POS taggers, MD, MA, isiZulu.net, ZulMorph and GT. The researchers intend to demonstrate the use of these tools and to raise awareness about their existence.

AI-Powered Keyword Extraction System Using NLP Techniques for Contextual Insights and Document Accessibility

This study presents the development of an AI-powered keyword extraction system using Python, aimed at efficiently identifying significant keywords from unstructured text data. Leveraging advanced natural language processing (NLP) techniques, the system integrates multiple methodologies, including Term Frequency-Inverse Document Frequency (TF-IDF), TextRank, Latent Semantic Analysis (LSA), and Part-of-Speech (POS) tagging. The incorporation of POS tagging enhances the accuracy of keyword identification by focusing on essential parts of speech, such as nouns and verbs, thereby filtering out irrelevant terms and ensuring the extraction of contextually meaningful keywords. To enhance user experience, the system includes innovative features such as a mobile document transfer facility that allows users to transfer documents via QR code and an option to download the extracted keywords as a document or PDF. These functionalities are designed to streamline accessibility and usability for diverse user groups. Performance evaluation was conducted using a dataset of research articles, demonstrating the system's effectiveness in accurately reflecting document content through keyword extraction. The results highlight its potential applications across various domains, including academia, marketing, and data science. This project offers a robust, intuitive, and user-friendly solution to make complex text data more accessible and actionable, contributing to the growing demand for efficient keyword extraction tools. Keywords- ai-powered keyword extraction, natural language processing (nlp), term frequency- inverse document frequency (tf-idf), textrank algorithm, latent semantic analysis (lsa), part-of- speech (pos) tagging

Tasaheel-v2: Development of Innovative Textual Analysis tool with Advanced Features

We introduce Tasaheel-v2, an automated tool specifically developed for Arabic Natural Language Processing (NLP)and textual analysis tasks. This work is an extension to the first version, Tasaheel-v1, comprised of traditional NLP tasksincluding stemming, segmentation, normalization, named entity recognition, and part of speech tagging. Furthermore, itincluded cutting-edge analytic methods, such as specific emotion, polarity, linguistics, and domain-specific word tagging. In thisnew innovative version, Tasaheel-v2, we introduce additional benefiting utilities designed to provide assistance for the Arabicresearch community. We specifically integrate another Arabic-specific POS tagger, a sentiment analyzer, and English-to-Arabictranslation functions. We leverage the utilities provided in Tasaheel to develop a machine-learning model designed to identifyArabic phishing emails and provide a thorough textual analysis to capture deceptive cues used to detect phishing linguisticpatterns. This tool contributes to the Arabic research domain by providing assistive NLP functions and textual analysis featuresall in one tool

Pedagogic natural language processing resources for L2 education: Teachers’ perceptions and beliefs

Pedagogic natural language processing resources (P-NLPRs) are a group of online technologies that aid teaching practices and hold the potential to enable Data-Driven Learning approaches by providing teachers and students with linguistic information. This study explores the perspectives of L2 educators on the potential implementation of P-NLPRs in their teaching practices. A training module was designed to provide information on the potential applications of different P-NLPRs, from which quantitative data was gathered (n=77) at PRE- and POST-test. Additionally, individual interviews were carried out with some of the participants (n=4) five years later to assess long-term P-NLPR uptake. Results offer insight into educators’ perception towards adopting P-NLPRs for their language teaching. Their perspectives seem to differentiate three main groups: a) tools to help learners learn (i.e. online dictionaries, text-to-speech technologies); b) tools to help teachers teach (i.e. automatic summarization tools, lexical profilers), and c) tools to help expand linguistic knowledge (corpora, POS taggers, lemmatizers).

A conditional random field based approach for high-accuracy part-of-speech tagging using language-independent features.

Part-of-speech (POS) tagging is the process of assigning tags or labels to each word of a text based on the grammatical category. It provides the ability to understand the grammatical structure of a text and plays an important role in many natural language processing tasks like syntax understanding, semantic analysis, text processing, information retrieval, machine translation, and named entity recognition. The POS tagging involves sequential nature, context dependency, and labeling of each word. Therefore it is a sequence labeling task. The challenges faced in Urdu text processing including resource scarcity, morphological richness, free word order, absence of capitalization, agglutinative nature, spelling variations, and multipurpose usage of words raise the demand for the development of machine learning automatic POS tagging systems for Urdu. Therefore, a conditional random field (CRF) based supervised POS classifier has been developed for 33 different Urdu POS categories using the language-independent features of Urdu text for the Urdu news dataset MM-POST containing 119,276 tokens of seven different domains including Entertainment, Finance, General, Health, Politics, Science and Sports. An analysis of the proposed approach is presented, proving it superior to other Urdu POS tagging research for using a simpler strategy by employing fewer word-level features as context windows together with the word length. The effective utilization of these features for the POS tagging of Urdu text resulted in the state-of-the-art performance of the CRF model, achieving an overall classification accuracy of 96.1%.

Using deep learning and word embeddings for predicting human agreeableness behavior

The latest advancements of deep learning have resulted in a new era of natural language processing. The machines now possess an unparallel ability to interpret and engage with various tasks such as text classification, content generation and natural language understanding. This development extended to the analysis of human behavior, where deep learning models are used to decode human personality. Due to the rise of social media, generating huge amounts of textual data that reshaped communication patterns. Understanding personality traits is a challenging topic which helps us to explore the patterns of thoughts, feelings and behaviors which are helpful for recruitment, career counselling and consumers’ behavior for marketing, etc. In this research study, the main aim is to predict the human personality trait of agreeableness showing whether a person is emotional who feels a lot or thinker who is logical and has rational thinking. This behavior leads to analyzing them as cooperative, friendly and respecting difference of views. For comprehensive empirical analysis, shallow machine learning models, ensemble models, and deep learning technique including state of the art transformer-based models are applied on real-world dataset of MBTI. For feature engineering, textual features of TF-IDF and POS tagging and word embeddings such as word2vec, glove and sentence embeddings are explored. The results analysis shows the highest performance 91.57% with sentence embeddings utilizing Bi-LSTM algorithm that highlights the power of this study as compared to existing studies in the relevant literature.

Conceptual sphere of the lexemes svoi (English our own), chuzhoi (English alien), inoi (English other) in the Russian language (corpus research experience)

The relevance of studying linguistic means used to represent the categories of community/otherness is justified by the significance of this social phenomenon constructed by society and the individual in accordance with the norms of a particular (i. e. Russian) culture. The attention of modern linguistics to the "person in language" issue has determined the anthropocentric approach to language study, which implies a rethinking of linguistic objectives when examining the category of identity. The focus of the study is the linguistic means of the Russian language representing the category of community/otherness with the lexemes svoi (English our own) / inoi (English other) / chuzhoi (English alien). The article aims to study the conceptual sphere of the lexemes svoi, chuzhoi, inoi within the framework of Russian linguoculture. The research methodology includes several stages: 1) statistical data analysis; 2) linguo-cognitive analysis of the use of the lexemes; 3) creation of an experimental microtext corpus. In the experimental part, we conducted a corpus study with a subsequent analysis of the statistical indicators of the occurrences of the lexemes in texts, as well as semantic analysis. Linguo-cognitive analysis of the use of the lexemes in proverbs and clichéd expressions enabled us to determine the conceptual meanings and environment of each lexeme. During the work, a research corpus was created. It contained contexts with the phrase we do not leave our own obtained from the Russian mass media. Using the PoS tagger RNNTagger, we processed and analysed the texts, and then the material was quantitatively and statistically characterised. The study results enabled us to draw conclusions about the frequency and conceptual meanings of the lexemes svoi, chuzhoi, inoi. Moreover, they expanded the understanding of the conceptual sphere of these linguistic units and made a certain contribution to the scientific understanding of Russian linguoculture.

Part-of-Speech (POS) Tagging of Low-Resource Language (Limbu) with Deep learning

POS tagging is a basic Natural Language Processing (NLP) task that tags the words in an input text according to its grammatical values. Although POS Tagging is a fundamental application for very resourced languages, such as Limbu, is still unknown due to only few tagged datasets and linguistic resources. This research project uses deep learning techniques, transfer learning, and the BiLSTM-CRF model to develop an accurate POS-tagging system for the Limbu language. Using annotated and unannotated language data, we progress in achieving a small yet informative dataset of Limbu text. Skilled multilingual tutoring was modified to enhance success on low-resource language tests. The model as propose attains 90% accuracy, which is very much better than traditional rule-based and machine learning methods for Limbu POS tagging. The results indicate that deep learning methods can address linguistic issues facing low-resource languages even with limited data. In turn, this study provides a cornerstone for follow up NLP-based applications of Limbu and similar low-resource languages, demonstrating how deep learning can fill the gap where data is scarce.

Analysis of the Arab singer Shadia’s lyrics

AbstractIn this work, we analyze the lyrics of one of the most famous and influential Arab artists in the twentieth century, namely "Image missing" (Shadia). Lyrics analysis provides a deep insight into the artist’s career evolution and her interactions with the surrounding environment including the social, political, and economic conditions. In order to perform such analysis, we had to collect and compile the lyrics of Shadia accompanied with the necessary meta-data into an organized and structured form. The data are preprocessed by removing stop words and doing some normalization operations over the songs prose. We did not perform any lemmatization or stemming as the original form of the tokens conveys much more information than the source words. We performed a lexical analysis in order to study both the lexical diversity and density over the course of Shadia’s career life. We have as well studied the most significant words, idioms, and terms played in the songs using tools such as word clouds and more quantitative measures such as term frequency–inverse document frequency. We have divided the career life of Shadia into sub-decades of length 5 years, and all analyses are done both in a yearly fashion and more coarsely over such sub-decades. Our quantitative analyses show strong correlations between the artistic lyrical work of Shadia and the state of affairs in Egypt and the Arab World during her time. In particular, Shadia’s lyrics reflect the radical changes in all aspects of the social, political, and economic conditions. This is especially relevant knowing that Shadia is very much truly considered the daughter of the generation of the 1952 revolution in Egypt. The significance of Shadia and her lyrics stem essentially from being contemporaneous to radical changes in Egypt across all sectors including political (support of liberation movements across the world and the conflict with Israel) and socioeconomic (especially changing the social class structure in Egypt). We also investigated the potential effectiveness of PoS (Part-of-Speech) tagging in genre analysis and classification.

Classification of Arabic User Requirements: A Semi-Automated Approach using NLP Tools

In this paper, we present a new Semi-Automated classification approach of Arabic user requirements into functional and non-functional requirements using natural language processing (NLP) tools, namely CAMeL Tools. We proposed a set of heuristics based on basic constructs of Arabic sentences in order to extract information from software requirements written in Arabic to classify the requirements into functional and non-functional requirements. CAMeL tools are used to generate tokens, PoS tags, and lemmas of the parsed user requirements, then a set of proposed heuristic rules are applied to CAMeL outputs to determine the closest tagging class for each statement. We implemented the proposed approach using Python code with using CAMeL Tools 1.3.1, under Ubuntu 20.04 LTS. The proposed approach is validated using a set of experiments involving a set of real cases evaluated by a group of experts, graduate, and undergraduate students who are familiar with software requirements. The results showed that the proposed approach achieved better results in the classification of Arabic software requirements than classifications by students with an accuracy of 90%.

NATURAL LANGUAGE PROCESSING ALGORITHMS AND PERFORMANCE COMPARISON

Natural language processing (NLP) is the general name for the methods and algorithms developed for computers to understand, interpret and produce human language. NLP plays a critical role in many fields, from social media analyses to customer service, from language translation to healthcare. This paper provides a comprehensive overview of the basic concepts of NLP, popular algorithms and models, performance comparisons, and various application areas. Key concepts of NLP include language models, tokenisation, lemmatisation, stemming, POS tagging, NER and syntactic parsing. These concepts are critical for processing, analysing and making sense of texts. Language models include popular methods such as N-gram, Word2Vec, GloVe and BERT. NLP algorithms are classified as rule-based methods, machine learning methods and deep learning methods. Rule-based methods are based on grammatical rules, while machine learning methods work on the principle of learning from data. Deep learning methods, on the other hand, achieve high accuracy results by using large datasets and powerful computational resources. In the performance comparison section, it is stated that the algorithms are evaluated with metrics such as accuracy, precision, recall and F1 score. Advanced models such as BERT and GPT-3 show superior performance in many NLP tasks. In conclusion, the field of NLP is rapidly evolving, with significant advancements anticipated in several key areas. These include the creation of more effective and efficient models, efforts to reduce biases, enhanced privacy protection, the growth of multilingual and cross-cultural models, and the development of explainable artificial intelligence techniques. This paper provides a comprehensive overview to understand the current status and future directions of NLP technologies.

Principle and Application for Rumination Computing Algorithms

To fully analyze, mine, and utilize the information and knowledge implied in problem resolving use cases, this paper proposed the autonomous learning method based on machine inducting, hypothesis formulating, and result verifying, which was similar to the biological process of cows ruminating, called rumination computing. Firstly, after inducting and summarizing over 1080 mathematic application problem, the system architecture and general algorithm for humanoid automatic resolving mathematic application problems were represented, which typically included functional modules such as commonsense knowledge base, domain knowledge base, and local knowledge base, preprocessing, word segmentation and part of speech tagging, semantic framework matching, global semantic analyzing, thinking mechanism implementing, etc. Secondly, after the use case solutions were approved, three typical rumination computing modes, including vocabulary sequence, semantic relationship, and computing action, were introduced based on the correct results, resolving steps, and basic rumination actions. The rumination computing step plan was formulated, new knowledge was obtained from the commonsense and results verification, so the continuous autonomous learning loop for machine thinking was formed. Detailed explanations were provided for the three core algorithms implemented (rumination framework algorithm, rumination semantic algorithm, rumination action algorithm). Then, by specific mathematic application problem humanoid resolving user cases, the above three types of rumination computing modes were illumined.

Tree hierarchical deep convolutional neural network optimized with sheep flock optimization algorithm for sentiment classification of Twitter data

ABSTRACT The increasing volume of online reviews and tweets poses significant challenges for sentiment classification because of the difficulty in obtaining annotated training data. This paper aims to enhance sentiment classification of Twitter data by developing a robust model that improves classification accuracy and computational efficiency. The proposed method named Tree Hierarchical Deep Convolutional Neural Network optimized with Sheep Flock Optimization Algorithm for Sentiment Classification of Twitter Data (SCTD-THDCNN-SFOA) utilizes the Stanford Sentiment Treebank dataset. The process begins with pre-processing steps including Tokenization, Stop words Elimination, Filtering, Hashtag Removal, and Multiword Grouping. The Gray Level Co-occurrence Matrix Window Adaptive Algorithm is employed to extract features, such as emoticon counts, punctuation counts, gazetteer word existence, n-grams, and part of speech tags. These features are selected using Entropy–Kurtosis-based Feature Selection approach. Finally, the Tree Hierarchical Deep Convolutional Neural Network enhanced by the Sheep Flock Optimization Algorithm is used to categorize the Twitter data as positive, negative, and neutral sentiments. The proposed SCTD-THDCNN-SFOA method demonstrates superior performance, achieving higher accuracy and lesser computation time than the existing models, respectively. The SCTD-THDCNN-SFOA framework significantly improves the accuracy and efficiency of sentiment classification for Twitter data.

Automatic Semantic Annotation of Indonesian Language Phrase Using N-Gram Language Model

Building semantic data populations in unstructured data or text is challenging. In this type of data, several problems can be raised, some of which are difficult to analyze. Some groups of words or expressions cannot be defined according to their meaning and can be a source of ambiguity. It can have a different meaning depending on the context of its use. This work aims to automatically annotate Indonesian Language text, especially phrases, with the existing knowledge base. The result is text with semantic markup. Machines can automatically process this type of text because it describes its meaning. This work applies an n-gram language model to identify meaningful phrases and defines them as a unit so that every existing word or phrase is automatically semantically tagged. This work uses the DBpedia and schema.org knowledge base. The percentage of successfully labeled data in this job was 78% with 84.95% accuracy using DBpedia and 5.9% with 97.46% accuracy using schema .org. Some factors affect the accuracy score, including the availability of the required data with the data contained in the knowledge base, the system's ability in the POS tagging process, and many new terminology and local cultures that have not yet been contained in the knowledge bases, especially schema.org that is utilized as a standard for all search engines. This work will help the machine understand the semantics of text data. All pages obtained will be semantically tagged and, therefore, will be understood by machines. This ability will support the following processes.

Reliable Part-of-Speech Tagging of Historical Corpora through Set-Valued Prediction

Syntactic annotation of corpora in the form of part-of-speech (POS) tags is a key requirement for both linguistic research and subsequent automated natural language processing (NLP) tasks. This problem is commonly tackled using machine learning methods, i.e., by training a POS tagger on a sufficiently large corpus of labeled data. While the problem of POS tagging can essentially be considered as solved for modern languages, historical corpora turn out to be much more difficult, especially due to the lack of native speakers and sparsity of training data. Moreover, most texts have no sentences as we know them today, nor a common orthography. These irregularities render the task of automated POS tagging more difficult and error-prone. Under these circumstances, instead of forcing the POS tagger to predict and commit to a single tag, it should be enabled to express its uncertainty. In this paper, we consider POS tagging within the framework of set-valued prediction, which allows the POS tagger to express its uncertainty via predicting a set of candidate POS tags instead of guessing a single one. The goal is to guarantee a high confidence that the correct POS tag is included while keeping the number of candidates small. In our experimental study, we find that extending state-of-the-art POS taggers to set-valued prediction yields more precise and robust taggings, especially for unknown words, i.e., words not occurring in the training data.

Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM.

Understanding spoken language is crucial for conversational agents, with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally, these tasks have been addressed individually, but recent studies have highlighted their interconnection, suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However, the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally, traditional models face difficulties in capturing both the semantic and syntactic nuances of language, which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information, part-of-speech (POS) tag embeddings for syntactic features, and contextual embeddings for deeper contextual understanding. Specifically, we utilized word2vec for non-contextual embeddings, one-hot vectors for POS tags, and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network, followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models, achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset, and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems, and paving the way for more accurate and efficient natural language understanding in real-world applications.

Role of Parts of Speech in Utterance Length for Mandarin-Speaking Children Aged 3–6 Years

We investigated the effect of part of speech adoption on the utterance length of Mandarin-speaking children. A total of 209 typically developing Taiwanese children aged 3–6 years participated in the study. They included 90 boys and 119 girls recruited from preschools in Miaoli City, New Taipei City, and Taipei City. We collected children’s language samples in four contexts: conversation (school life), story retelling (“Little Red Riding Hood”), free play, and conversation (daily life). One-on-one conversation with participants comprised the main interaction form. Language samples collected from the participants were transcribed and analyzed; all words in the samples were coded with part of speech tagging. Part of speech factors that could predict utterance length were examined using stepwise regression. The results showed that prepositions and conjunctions had positive effects on utterance length while interjections had a negative effect. Increasing utterance length may be a teaching or clinical intervention goal for some children. Educators and clinicians can consider guiding such children to use prepositions and conjunctions to increase their utterance length.

A Hybrid Approach to Ontology Construction for the Badini Kurdish Language

Semantic ontologies have been widely utilized as crucial tools within natural language processing, underpinning applications such as knowledge extraction, question answering, machine translation, text comprehension, information retrieval, and text summarization. While the Kurdish language, a low-resource language, has been the subject of some ontological research in other dialects, a semantic web ontology for the Badini dialect remains conspicuously absent. This paper addresses this gap by presenting a methodology for constructing and utilizing a semantic web ontology for the Badini dialect of the Kurdish language. A Badini annotated corpus (UOZBDN) was created and manually annotated with part-of-speech (POS) tags. Subsequently, an HMM-based POS tagger model was developed using the UOZBDN corpus and applied to annotate additional text for ontology extraction. Ontology extraction was performed by employing predefined rules to identify nouns and verbs from the model-annotated corpus and subsequently forming semantic predicates. Robust methodologies were adopted for ontology development, resulting in a high degree of precision. The POS tagging model attained an accuracy of 95.04% when applied to the UOZBDN corpus. Furthermore, a manual evaluation conducted by Badini Kurdish language experts yielded a 97.42% accuracy rate for the extracted ontology.

A survey on NLP tasks, resources and techniques for low-resource Telugu-English code-mixed text

With the proliferation of informal content on various social media platforms in the form of posts, comments, and feedback, the importance of analyzing text in code-mixed form is gaining importance. Telugu, a low-resource Indian language, has a lot of online content being generated in code-mixed form. However, the lack of large corpora, annotated data and Natural Language Processing (NLP) resources are impeding research on Telugu-English code-mixed data. This paper provides a survey of existing literature on Telugu-English code-mixed text in the areas of resources, POS tagging, Named Entity Recognition, language identification, sentiment analysis, application tasks, dialog systems, and Question-Answering. Various datasets being used by the researchers in the field, along with methods applied to them are detailed. Research gaps are identified to provide future direction for researchers working in this field.

Javanese part-of-speech tagging using cross-lingual transfer learning

<span dir="ltr">Large datasets that are publicly available for POS tagging do not always exist </span><span dir="ltr">for some languages. </span><span dir="ltr">One of those languages is Javanese, a local language in </span><span dir="ltr">Indonesia, which is considered as a low-resource language. This research aims </span><span dir="ltr">to examine the effectiveness of cross-lingual transfer learning for Javanese POS </span><span dir="ltr">tagging by fine-tuning the state-of-the-art Transformer-based models (such as </span><span dir="ltr">IndoBERT, mBERT, and XLM-RoBERTa) using different kinds of source lan</span><span dir="ltr">guages that have a higher resource (such as Indonesian, English, Uyghur, Latin, </span><span dir="ltr">and Hungarian languages), and then fine-tuning it again using the Javanese lan</span><span dir="ltr">guage as the target language. We found that the models using cross-lingual trans</span><span dir="ltr">fer learning can increase the accuracy of the models without using cross-lingual </span><span dir="ltr">transfer learning by 14.3%–15.3% over LSTM-based models, and by 0.21%–</span><span dir="ltr">3.95% over Transformer-based models. Our results show that the most accurate </span><span dir="ltr">Javanese POS tagger model is XLM-RoBERTa that is fine-tuned in two stages </span><span dir="ltr">(the first one using Indonesian language as the source language, and the second </span><span dir="ltr">one using Javanese language as the target language), capable of achieving an </span><span dir="ltr">accuracy of 87.65%</span>