Part-of-speech Tagging Research Articles

Part-of-Speech (POS) Tagging of Low-Resource Language (Limbu) with Deep learning

POS tagging is a basic Natural Language Processing (NLP) task that tags the words in an input text according to its grammatical values. Although POS Tagging is a fundamental application for very resourced languages, such as Limbu, is still unknown due to only few tagged datasets and linguistic resources. This research project uses deep learning techniques, transfer learning, and the BiLSTM-CRF model to develop an accurate POS-tagging system for the Limbu language. Using annotated and unannotated language data, we progress in achieving a small yet informative dataset of Limbu text. Skilled multilingual tutoring was modified to enhance success on low-resource language tests. The model as propose attains 90% accuracy, which is very much better than traditional rule-based and machine learning methods for Limbu POS tagging. The results indicate that deep learning methods can address linguistic issues facing low-resource languages even with limited data. In turn, this study provides a cornerstone for follow up NLP-based applications of Limbu and similar low-resource languages, demonstrating how deep learning can fill the gap where data is scarce.

Analysis of the Arab singer Shadia’s lyrics

AbstractIn this work, we analyze the lyrics of one of the most famous and influential Arab artists in the twentieth century, namely "Image missing" (Shadia). Lyrics analysis provides a deep insight into the artist’s career evolution and her interactions with the surrounding environment including the social, political, and economic conditions. In order to perform such analysis, we had to collect and compile the lyrics of Shadia accompanied with the necessary meta-data into an organized and structured form. The data are preprocessed by removing stop words and doing some normalization operations over the songs prose. We did not perform any lemmatization or stemming as the original form of the tokens conveys much more information than the source words. We performed a lexical analysis in order to study both the lexical diversity and density over the course of Shadia’s career life. We have as well studied the most significant words, idioms, and terms played in the songs using tools such as word clouds and more quantitative measures such as term frequency–inverse document frequency. We have divided the career life of Shadia into sub-decades of length 5 years, and all analyses are done both in a yearly fashion and more coarsely over such sub-decades. Our quantitative analyses show strong correlations between the artistic lyrical work of Shadia and the state of affairs in Egypt and the Arab World during her time. In particular, Shadia’s lyrics reflect the radical changes in all aspects of the social, political, and economic conditions. This is especially relevant knowing that Shadia is very much truly considered the daughter of the generation of the 1952 revolution in Egypt. The significance of Shadia and her lyrics stem essentially from being contemporaneous to radical changes in Egypt across all sectors including political (support of liberation movements across the world and the conflict with Israel) and socioeconomic (especially changing the social class structure in Egypt). We also investigated the potential effectiveness of PoS (Part-of-Speech) tagging in genre analysis and classification.

Classification of Arabic User Requirements: A Semi-Automated Approach using NLP Tools

In this paper, we present a new Semi-Automated classification approach of Arabic user requirements into functional and non-functional requirements using natural language processing (NLP) tools, namely CAMeL Tools. We proposed a set of heuristics based on basic constructs of Arabic sentences in order to extract information from software requirements written in Arabic to classify the requirements into functional and non-functional requirements. CAMeL tools are used to generate tokens, PoS tags, and lemmas of the parsed user requirements, then a set of proposed heuristic rules are applied to CAMeL outputs to determine the closest tagging class for each statement. We implemented the proposed approach using Python code with using CAMeL Tools 1.3.1, under Ubuntu 20.04 LTS. The proposed approach is validated using a set of experiments involving a set of real cases evaluated by a group of experts, graduate, and undergraduate students who are familiar with software requirements. The results showed that the proposed approach achieved better results in the classification of Arabic software requirements than classifications by students with an accuracy of 90%.

Principle and Application for Rumination Computing Algorithms

To fully analyze, mine, and utilize the information and knowledge implied in problem resolving use cases, this paper proposed the autonomous learning method based on machine inducting, hypothesis formulating, and result verifying, which was similar to the biological process of cows ruminating, called rumination computing. Firstly, after inducting and summarizing over 1080 mathematic application problem, the system architecture and general algorithm for humanoid automatic resolving mathematic application problems were represented, which typically included functional modules such as commonsense knowledge base, domain knowledge base, and local knowledge base, preprocessing, word segmentation and part of speech tagging, semantic framework matching, global semantic analyzing, thinking mechanism implementing, etc. Secondly, after the use case solutions were approved, three typical rumination computing modes, including vocabulary sequence, semantic relationship, and computing action, were introduced based on the correct results, resolving steps, and basic rumination actions. The rumination computing step plan was formulated, new knowledge was obtained from the commonsense and results verification, so the continuous autonomous learning loop for machine thinking was formed. Detailed explanations were provided for the three core algorithms implemented (rumination framework algorithm, rumination semantic algorithm, rumination action algorithm). Then, by specific mathematic application problem humanoid resolving user cases, the above three types of rumination computing modes were illumined.

Tree hierarchical deep convolutional neural network optimized with sheep flock optimization algorithm for sentiment classification of Twitter data

ABSTRACT The increasing volume of online reviews and tweets poses significant challenges for sentiment classification because of the difficulty in obtaining annotated training data. This paper aims to enhance sentiment classification of Twitter data by developing a robust model that improves classification accuracy and computational efficiency. The proposed method named Tree Hierarchical Deep Convolutional Neural Network optimized with Sheep Flock Optimization Algorithm for Sentiment Classification of Twitter Data (SCTD-THDCNN-SFOA) utilizes the Stanford Sentiment Treebank dataset. The process begins with pre-processing steps including Tokenization, Stop words Elimination, Filtering, Hashtag Removal, and Multiword Grouping. The Gray Level Co-occurrence Matrix Window Adaptive Algorithm is employed to extract features, such as emoticon counts, punctuation counts, gazetteer word existence, n-grams, and part of speech tags. These features are selected using Entropy–Kurtosis-based Feature Selection approach. Finally, the Tree Hierarchical Deep Convolutional Neural Network enhanced by the Sheep Flock Optimization Algorithm is used to categorize the Twitter data as positive, negative, and neutral sentiments. The proposed SCTD-THDCNN-SFOA method demonstrates superior performance, achieving higher accuracy and lesser computation time than the existing models, respectively. The SCTD-THDCNN-SFOA framework significantly improves the accuracy and efficiency of sentiment classification for Twitter data.

Automatic Semantic Annotation of Indonesian Language Phrase Using N-Gram Language Model

Building semantic data populations in unstructured data or text is challenging. In this type of data, several problems can be raised, some of which are difficult to analyze. Some groups of words or expressions cannot be defined according to their meaning and can be a source of ambiguity. It can have a different meaning depending on the context of its use. This work aims to automatically annotate Indonesian Language text, especially phrases, with the existing knowledge base. The result is text with semantic markup. Machines can automatically process this type of text because it describes its meaning. This work applies an n-gram language model to identify meaningful phrases and defines them as a unit so that every existing word or phrase is automatically semantically tagged. This work uses the DBpedia and schema.org knowledge base. The percentage of successfully labeled data in this job was 78% with 84.95% accuracy using DBpedia and 5.9% with 97.46% accuracy using schema .org. Some factors affect the accuracy score, including the availability of the required data with the data contained in the knowledge base, the system's ability in the POS tagging process, and many new terminology and local cultures that have not yet been contained in the knowledge bases, especially schema.org that is utilized as a standard for all search engines. This work will help the machine understand the semantics of text data. All pages obtained will be semantically tagged and, therefore, will be understood by machines. This ability will support the following processes.

Reliable Part-of-Speech Tagging of Historical Corpora through Set-Valued Prediction

Syntactic annotation of corpora in the form of part-of-speech (POS) tags is a key requirement for both linguistic research and subsequent automated natural language processing (NLP) tasks. This problem is commonly tackled using machine learning methods, i.e., by training a POS tagger on a sufficiently large corpus of labeled data. While the problem of POS tagging can essentially be considered as solved for modern languages, historical corpora turn out to be much more difficult, especially due to the lack of native speakers and sparsity of training data. Moreover, most texts have no sentences as we know them today, nor a common orthography. These irregularities render the task of automated POS tagging more difficult and error-prone. Under these circumstances, instead of forcing the POS tagger to predict and commit to a single tag, it should be enabled to express its uncertainty. In this paper, we consider POS tagging within the framework of set-valued prediction, which allows the POS tagger to express its uncertainty via predicting a set of candidate POS tags instead of guessing a single one. The goal is to guarantee a high confidence that the correct POS tag is included while keeping the number of candidates small. In our experimental study, we find that extending state-of-the-art POS taggers to set-valued prediction yields more precise and robust taggings, especially for unknown words, i.e., words not occurring in the training data.

Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

Understanding spoken language is crucial for conversational agents, with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally, these tasks have been addressed individually, but recent studies have highlighted their interconnection, suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However, the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally, traditional models face difficulties in capturing both the semantic and syntactic nuances of language, which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information, part-of-speech (POS) tag embeddings for syntactic features, and contextual embeddings for deeper contextual understanding. Specifically, we utilized word2vec for non-contextual embeddings, one-hot vectors for POS tags, and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network, followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models, achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset, and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems, and paving the way for more accurate and efficient natural language understanding in real-world applications.

Role of Parts of Speech in Utterance Length for Mandarin-Speaking Children Aged 3–6 Years

We investigated the effect of part of speech adoption on the utterance length of Mandarin-speaking children. A total of 209 typically developing Taiwanese children aged 3–6 years participated in the study. They included 90 boys and 119 girls recruited from preschools in Miaoli City, New Taipei City, and Taipei City. We collected children’s language samples in four contexts: conversation (school life), story retelling (“Little Red Riding Hood”), free play, and conversation (daily life). One-on-one conversation with participants comprised the main interaction form. Language samples collected from the participants were transcribed and analyzed; all words in the samples were coded with part of speech tagging. Part of speech factors that could predict utterance length were examined using stepwise regression. The results showed that prepositions and conjunctions had positive effects on utterance length while interjections had a negative effect. Increasing utterance length may be a teaching or clinical intervention goal for some children. Educators and clinicians can consider guiding such children to use prepositions and conjunctions to increase their utterance length.

A Hybrid Approach to Ontology Construction for the Badini Kurdish Language

Semantic ontologies have been widely utilized as crucial tools within natural language processing, underpinning applications such as knowledge extraction, question answering, machine translation, text comprehension, information retrieval, and text summarization. While the Kurdish language, a low-resource language, has been the subject of some ontological research in other dialects, a semantic web ontology for the Badini dialect remains conspicuously absent. This paper addresses this gap by presenting a methodology for constructing and utilizing a semantic web ontology for the Badini dialect of the Kurdish language. A Badini annotated corpus (UOZBDN) was created and manually annotated with part-of-speech (POS) tags. Subsequently, an HMM-based POS tagger model was developed using the UOZBDN corpus and applied to annotate additional text for ontology extraction. Ontology extraction was performed by employing predefined rules to identify nouns and verbs from the model-annotated corpus and subsequently forming semantic predicates. Robust methodologies were adopted for ontology development, resulting in a high degree of precision. The POS tagging model attained an accuracy of 95.04% when applied to the UOZBDN corpus. Furthermore, a manual evaluation conducted by Badini Kurdish language experts yielded a 97.42% accuracy rate for the extracted ontology.

A survey on NLP tasks, resources and techniques for low-resource Telugu-English code-mixed text

With the proliferation of informal content on various social media platforms in the form of posts, comments, and feedback, the importance of analyzing text in code-mixed form is gaining importance. Telugu, a low-resource Indian language, has a lot of online content being generated in code-mixed form. However, the lack of large corpora, annotated data and Natural Language Processing (NLP) resources are impeding research on Telugu-English code-mixed data. This paper provides a survey of existing literature on Telugu-English code-mixed text in the areas of resources, POS tagging, Named Entity Recognition, language identification, sentiment analysis, application tasks, dialog systems, and Question-Answering. Various datasets being used by the researchers in the field, along with methods applied to them are detailed. Research gaps are identified to provide future direction for researchers working in this field.

NATURAL LANGUAGE PROCESSING ALGORITHMS AND PERFORMANCE COMPARISON

Natural language processing (NLP) is the general name for the methods and algorithms developed for computers to understand, interpret and produce human language. NLP plays a critical role in many fields, from social media analyses to customer service, from language translation to healthcare. This paper provides a comprehensive overview of the basic concepts of NLP, popular algorithms and models, performance comparisons, and various application areas. Key concepts of NLP include language models, tokenisation, lemmatisation, stemming, POS tagging, NER and syntactic parsing. These concepts are critical for processing, analysing and making sense of texts. Language models include popular methods such as N-gram, Word2Vec, GloVe and BERT. NLP algorithms are classified as rule-based methods, machine learning methods and deep learning methods. Rule-based methods are based on grammatical rules, while machine learning methods work on the principle of learning from data. Deep learning methods, on the other hand, achieve high accuracy results by using large datasets and powerful computational resources. In the performance comparison section, it is stated that the algorithms are evaluated with metrics such as accuracy, precision, recall and F1 score. Advanced models such as BERT and GPT-3 show superior performance in many NLP tasks. In conclusion, the field of NLP is rapidly evolving, with significant advancements anticipated in several key areas. These include the creation of more effective and efficient models, efforts to reduce biases, enhanced privacy protection, the growth of multilingual and cross-cultural models, and the development of explainable artificial intelligence techniques. This paper provides a comprehensive overview to understand the current status and future directions of NLP technologies.

Javanese part-of-speech tagging using cross-lingual transfer learning

<span dir="ltr">Large datasets that are publicly available for POS tagging do not always exist </span><span dir="ltr">for some languages. </span><span dir="ltr">One of those languages is Javanese, a local language in </span><span dir="ltr">Indonesia, which is considered as a low-resource language. This research aims </span><span dir="ltr">to examine the effectiveness of cross-lingual transfer learning for Javanese POS </span><span dir="ltr">tagging by fine-tuning the state-of-the-art Transformer-based models (such as </span><span dir="ltr">IndoBERT, mBERT, and XLM-RoBERTa) using different kinds of source lan</span><span dir="ltr">guages that have a higher resource (such as Indonesian, English, Uyghur, Latin, </span><span dir="ltr">and Hungarian languages), and then fine-tuning it again using the Javanese lan</span><span dir="ltr">guage as the target language. We found that the models using cross-lingual trans</span><span dir="ltr">fer learning can increase the accuracy of the models without using cross-lingual </span><span dir="ltr">transfer learning by 14.3%–15.3% over LSTM-based models, and by 0.21%–</span><span dir="ltr">3.95% over Transformer-based models. Our results show that the most accurate </span><span dir="ltr">Javanese POS tagger model is XLM-RoBERTa that is fine-tuned in two stages </span><span dir="ltr">(the first one using Indonesian language as the source language, and the second </span><span dir="ltr">one using Javanese language as the target language), capable of achieving an </span><span dir="ltr">accuracy of 87.65%</span>

A hybrid approach to extract conceptual diagram from software requirements

Employing rules for the automatic extraction of conceptual diagrams from software requirements has been in practice for some time. However, considering only rules for extraction makes the system complex to handle. Moreover, the rules are predominantly based on the syntactic structure such as Part of Speech tags along with Dependency Grammar of sentences and rarely on semantics. In this paper, we propose to use a probabilistic approach in configuration with the rule-based technique and the Word embeddings to preserve the semantics of the sentence. Hence, reduces the complexity of the extraction procedure. Further, we advocate the use of a divide-and-conquer policy of extraction instead of extracting classes for one entire use case description. We extract the class diagram from small use cases and then merge it to obtain the class diagram. As generated class diagram corresponding to small use cases can be utilized in another similar software design, thus, it increases the scalability and decreases the extraction time. The proposed hybrid approach integrates the knowledge from the experiences. Thus, the proposed approach achieved 90% as F1-score whereas the F1-Score for the existing methods ranged between 79-88%. The proposed hybrid approach also shows a 19.44% reduction in terms of the number of iterations performed to carry out extraction procedures for individual use cases. Hence, reduces the extraction procedure complexity.

Beyond a Buzzword: Student Perspectives on what Contributes to Engaging Educators and Classroom Experiences

In a changing educational landscape, student engagement remains a prominent issue in research and practice. This study explores engagement from the student’s perspective across multiple disciplines, year levels and delivery modes. It draws on 13,125 Student Evaluation of Teaching survey comments where students have used the word “engage” (or derivative) in response to individual educators. Part of speech tagging was used to identify surrounding words that helped to contextualise the inclusion of the term. Through content analysis, key words were then grouped into six themes (approachability; synonyms for good; clarity; enjoyment and enthusiasm; relevance; and subject matter expertise) that highlighted both individual educator attributes and the overall learning experience. This research provides practical considerations for educators and administrators in relation to what students find engaging and how to support such positive environments. Additionally, it reinforces the role students play as major stakeholders in their learning and ongoing discussions around engagement.

Identification of Sentence Stems Characteristic of Chinese Learner English Writing

Phraseological units in academic English texts have been a central focus in recent corpus linguistic research. This paper describes a special category of clause-level phraseological units, namely, Characteristic Sentence Stems (CSSs), with a view to describing their identifying criteria and their extraction method. CSSs are contiguous lexico-grammatical sequences which contain a subject-predicate structure and which are frame expressions characteristic of academic writing. The extraction method of a CSS consists of six steps: POS tagging, n-gram segmentation, structure identification, significance of occurrence calculation, text range calculation, and overlapping sequence reduction. The significance of occurrence calculation is the crux of this method. It includes the computing of both the internal association and the boundary independence of a CSS, and it tests the occurring significance of the CSS from both the inside and the outside perspectives. Our methods and results suggest that CSSs can be statistically defined and extracted from corpora and can employed in large-scale studies to more fully account for the phraseological features of non-native English academic writing.

A detailed analysis of the interpretability of Convolutional Neural Networks for text classification

Abstract Convolutional Neural Networks (CNNs) have become ubiquitous in many NLP tasks. However, understanding its process is still an area with much to be done. In this paper, we introduce a method to study the interpretability of CNNs when used for text classification. More specifically, we work on the interpretability of the convolutional filters in the context of sentiment analysis. The framework used in this paper has allowed the understanding of the mechanics of the network when applied to the task of sentiment analysis. The results of the experiments reveal how certain parts of speech (POS) tags are more relevant than others to the classification of a sentence. Furthermore, we also observed a preference for shorter $n$-grams when classifying negative sentiment sequences. Additionally, we detected a certain amount of redundancy among the convolutional filters, thus allowing us to conclude that smaller architectures would have worked for this particular task. This was achieved by computing the relevant metrics to measure the influence of the pertinent property on the desired class. In our case, the concepts treated were POS tags that carry semantic information and properties were related to the convolutional filters.

Spoken Spanish PoS tagging: gold standard dataset

AbstractThe development of a benchmark for part-of-speech (PoS) tagging of spoken dialectal European Spanish is presented, which will serve as the foundation for a future treebank. The benchmark is constructed using transcriptions of the Corpus Oral y Sonoro del Español Rural (COSER;“Audible corpus of spoken rural Spanish”) and follows the Universal Dependencies project guidelines. We describe the methodology used to create a gold standard, which serves to evaluate different state-of-the-art PoS taggers (spaCy, Stanza NLP, and UDPipe), originally trained on written data and to fine-tune and evaluate a model for spoken Spanish. It is shown that the accuracy of these taggers drops from 0.98$$-$$ - 0.99 to 0.94$$-$$ - 0.95 when tested on spoken data. Of these three taggers, the spaCy’s trf (transformers) and Stanza NLP models performed the best. Finally, the spaCy trf model is fine-tuned using our gold standard, which resulted in an accuracy of 0.98 for coarse-grained tags (UPOS) and 0.97 for fine-grained tags (FEATS). Our benchmark will enable the development of more accurate PoS taggers for spoken Spanish and facilitate the construction of a treebank for European Spanish varieties.

Parts-of-speech tagger for Sindhi language using deep neural network architecture

Language is a fundamental medium for human communication, encompassing spoken and written forms, each governed by grammatical rules. Sindhi, one of the oldest languages, is characterized by its rich morphology and grammatical structure. Part-of-speech (POS) tagging, a crucial process in natural language processing, involves assigning grammatical tags to words. This research presents a novel approach to POS tagging for Sindhi text using deep learning techniques. We developed a POS tagger employing Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models, with LSTM demonstrating superior effectiveness. This study represents the first application of these deep learning methods for POS tagging in Sindhi. Utilizing fastText, we trained 79,959 Sindhi word vectors, derived from a corpus compiled from diverse sources including Sindhi books, stories, and poetry. The corpus comprises 1,459 sentences and 10,584 unique words, split into 80% for training and 20% for validation. Our results indicate that the LSTM model achieved an accuracy of 85.80%, outperforming the GRU model, which achieved 80.77%, by a margin of 5%. This work's novelty lies in the application of deep learning techniques to enhance POS tagging accuracy in the Sindhi language corpus.

Design of English Lexical Analysis System for Machine Translation Based on Multimodal Neural Network

Information resources have become a very important source of wealth, and due to differences in language and writing, technological information exchange between countries has become difficult. In this article, we propose a data augmentation method based on statistical models. A framework for a multimodal output model is proposed, taking into account the semantic relevance and importance between different languages. This framework is based on text sequence to sequence framework, decoupled, and a network architecture based on dual stream attention mechanism is designed. A multimodal interactive neural network layer was added between the encoder and decoder, achieving the fusion of multilingual domain information. It compensates for the influence of text on traditional translation generation systems in the process of English Chinese translation, and solves the problems of inaccurate translation results and low similarity with the original text in English Chinese machine translation. The system proposed in this article mainly consists of preprocessing module, lexical analysis and segmentation module, part of speech tagging and phrase analysis module, translation rule construction module, decoding module, translation generation module, etc. The experimental results show that using an improved generation system, compared with traditional generation systems, improves the accuracy of translation generation, has certain advantages, and is more practical.