Part-of-speech Tagging Research Articles

Principle and Application for Rumination Computing Algorithms

To fully analyze, mine, and utilize the information and knowledge implied in problem resolving use cases, this paper proposed the autonomous learning method based on machine inducting, hypothesis formulating, and result verifying, which was similar to the biological process of cows ruminating, called rumination computing. Firstly, after inducting and summarizing over 1080 mathematic application problem, the system architecture and general algorithm for humanoid automatic resolving mathematic application problems were represented, which typically included functional modules such as commonsense knowledge base, domain knowledge base, and local knowledge base, preprocessing, word segmentation and part of speech tagging, semantic framework matching, global semantic analyzing, thinking mechanism implementing, etc. Secondly, after the use case solutions were approved, three typical rumination computing modes, including vocabulary sequence, semantic relationship, and computing action, were introduced based on the correct results, resolving steps, and basic rumination actions. The rumination computing step plan was formulated, new knowledge was obtained from the commonsense and results verification, so the continuous autonomous learning loop for machine thinking was formed. Detailed explanations were provided for the three core algorithms implemented (rumination framework algorithm, rumination semantic algorithm, rumination action algorithm). Then, by specific mathematic application problem humanoid resolving user cases, the above three types of rumination computing modes were illumined.

Tree hierarchical deep convolutional neural network optimized with sheep flock optimization algorithm for sentiment classification of Twitter data

ABSTRACT The increasing volume of online reviews and tweets poses significant challenges for sentiment classification because of the difficulty in obtaining annotated training data. This paper aims to enhance sentiment classification of Twitter data by developing a robust model that improves classification accuracy and computational efficiency. The proposed method named Tree Hierarchical Deep Convolutional Neural Network optimized with Sheep Flock Optimization Algorithm for Sentiment Classification of Twitter Data (SCTD-THDCNN-SFOA) utilizes the Stanford Sentiment Treebank dataset. The process begins with pre-processing steps including Tokenization, Stop words Elimination, Filtering, Hashtag Removal, and Multiword Grouping. The Gray Level Co-occurrence Matrix Window Adaptive Algorithm is employed to extract features, such as emoticon counts, punctuation counts, gazetteer word existence, n-grams, and part of speech tags. These features are selected using Entropy–Kurtosis-based Feature Selection approach. Finally, the Tree Hierarchical Deep Convolutional Neural Network enhanced by the Sheep Flock Optimization Algorithm is used to categorize the Twitter data as positive, negative, and neutral sentiments. The proposed SCTD-THDCNN-SFOA method demonstrates superior performance, achieving higher accuracy and lesser computation time than the existing models, respectively. The SCTD-THDCNN-SFOA framework significantly improves the accuracy and efficiency of sentiment classification for Twitter data.

Automatic Semantic Annotation of Indonesian Language Phrase Using N-Gram Language Model

Building semantic data populations in unstructured data or text is challenging. In this type of data, several problems can be raised, some of which are difficult to analyze. Some groups of words or expressions cannot be defined according to their meaning and can be a source of ambiguity. It can have a different meaning depending on the context of its use. This work aims to automatically annotate Indonesian Language text, especially phrases, with the existing knowledge base. The result is text with semantic markup. Machines can automatically process this type of text because it describes its meaning. This work applies an n-gram language model to identify meaningful phrases and defines them as a unit so that every existing word or phrase is automatically semantically tagged. This work uses the DBpedia and schema.org knowledge base. The percentage of successfully labeled data in this job was 78% with 84.95% accuracy using DBpedia and 5.9% with 97.46% accuracy using schema .org. Some factors affect the accuracy score, including the availability of the required data with the data contained in the knowledge base, the system's ability in the POS tagging process, and many new terminology and local cultures that have not yet been contained in the knowledge bases, especially schema.org that is utilized as a standard for all search engines. This work will help the machine understand the semantics of text data. All pages obtained will be semantically tagged and, therefore, will be understood by machines. This ability will support the following processes.

Reliable Part-of-Speech Tagging of Historical Corpora through Set-Valued Prediction

Syntactic annotation of corpora in the form of part-of-speech (POS) tags is a key requirement for both linguistic research and subsequent automated natural language processing (NLP) tasks. This problem is commonly tackled using machine learning methods, i.e., by training a POS tagger on a sufficiently large corpus of labeled data. While the problem of POS tagging can essentially be considered as solved for modern languages, historical corpora turn out to be much more difficult, especially due to the lack of native speakers and sparsity of training data. Moreover, most texts have no sentences as we know them today, nor a common orthography. These irregularities render the task of automated POS tagging more difficult and error-prone. Under these circumstances, instead of forcing the POS tagger to predict and commit to a single tag, it should be enabled to express its uncertainty. In this paper, we consider POS tagging within the framework of set-valued prediction, which allows the POS tagger to express its uncertainty via predicting a set of candidate POS tags instead of guessing a single one. The goal is to guarantee a high confidence that the correct POS tag is included while keeping the number of candidates small. In our experimental study, we find that extending state-of-the-art POS taggers to set-valued prediction yields more precise and robust taggings, especially for unknown words, i.e., words not occurring in the training data.

Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

Understanding spoken language is crucial for conversational agents, with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally, these tasks have been addressed individually, but recent studies have highlighted their interconnection, suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However, the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally, traditional models face difficulties in capturing both the semantic and syntactic nuances of language, which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information, part-of-speech (POS) tag embeddings for syntactic features, and contextual embeddings for deeper contextual understanding. Specifically, we utilized word2vec for non-contextual embeddings, one-hot vectors for POS tags, and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network, followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models, achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset, and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems, and paving the way for more accurate and efficient natural language understanding in real-world applications.

A Hybrid Approach to Ontology Construction for the Badini Kurdish Language

Semantic ontologies have been widely utilized as crucial tools within natural language processing, underpinning applications such as knowledge extraction, question answering, machine translation, text comprehension, information retrieval, and text summarization. While the Kurdish language, a low-resource language, has been the subject of some ontological research in other dialects, a semantic web ontology for the Badini dialect remains conspicuously absent. This paper addresses this gap by presenting a methodology for constructing and utilizing a semantic web ontology for the Badini dialect of the Kurdish language. A Badini annotated corpus (UOZBDN) was created and manually annotated with part-of-speech (POS) tags. Subsequently, an HMM-based POS tagger model was developed using the UOZBDN corpus and applied to annotate additional text for ontology extraction. Ontology extraction was performed by employing predefined rules to identify nouns and verbs from the model-annotated corpus and subsequently forming semantic predicates. Robust methodologies were adopted for ontology development, resulting in a high degree of precision. The POS tagging model attained an accuracy of 95.04% when applied to the UOZBDN corpus. Furthermore, a manual evaluation conducted by Badini Kurdish language experts yielded a 97.42% accuracy rate for the extracted ontology.

A survey on NLP tasks, resources and techniques for low-resource Telugu-English code-mixed text

With the proliferation of informal content on various social media platforms in the form of posts, comments, and feedback, the importance of analyzing text in code-mixed form is gaining importance. Telugu, a low-resource Indian language, has a lot of online content being generated in code-mixed form. However, the lack of large corpora, annotated data and Natural Language Processing (NLP) resources are impeding research on Telugu-English code-mixed data. This paper provides a survey of existing literature on Telugu-English code-mixed text in the areas of resources, POS tagging, Named Entity Recognition, language identification, sentiment analysis, application tasks, dialog systems, and Question-Answering. Various datasets being used by the researchers in the field, along with methods applied to them are detailed. Research gaps are identified to provide future direction for researchers working in this field.

NATURAL LANGUAGE PROCESSING ALGORITHMS AND PERFORMANCE COMPARISON

Natural language processing (NLP) is the general name for the methods and algorithms developed for computers to understand, interpret and produce human language. NLP plays a critical role in many fields, from social media analyses to customer service, from language translation to healthcare. This paper provides a comprehensive overview of the basic concepts of NLP, popular algorithms and models, performance comparisons, and various application areas. Key concepts of NLP include language models, tokenisation, lemmatisation, stemming, POS tagging, NER and syntactic parsing. These concepts are critical for processing, analysing and making sense of texts. Language models include popular methods such as N-gram, Word2Vec, GloVe and BERT. NLP algorithms are classified as rule-based methods, machine learning methods and deep learning methods. Rule-based methods are based on grammatical rules, while machine learning methods work on the principle of learning from data. Deep learning methods, on the other hand, achieve high accuracy results by using large datasets and powerful computational resources. In the performance comparison section, it is stated that the algorithms are evaluated with metrics such as accuracy, precision, recall and F1 score. Advanced models such as BERT and GPT-3 show superior performance in many NLP tasks. In conclusion, the field of NLP is rapidly evolving, with significant advancements anticipated in several key areas. These include the creation of more effective and efficient models, efforts to reduce biases, enhanced privacy protection, the growth of multilingual and cross-cultural models, and the development of explainable artificial intelligence techniques. This paper provides a comprehensive overview to understand the current status and future directions of NLP technologies.

Javanese part-of-speech tagging using cross-lingual transfer learning

<span dir="ltr">Large datasets that are publicly available for POS tagging do not always exist </span><span dir="ltr">for some languages. </span><span dir="ltr">One of those languages is Javanese, a local language in </span><span dir="ltr">Indonesia, which is considered as a low-resource language. This research aims </span><span dir="ltr">to examine the effectiveness of cross-lingual transfer learning for Javanese POS </span><span dir="ltr">tagging by fine-tuning the state-of-the-art Transformer-based models (such as </span><span dir="ltr">IndoBERT, mBERT, and XLM-RoBERTa) using different kinds of source lan</span><span dir="ltr">guages that have a higher resource (such as Indonesian, English, Uyghur, Latin, </span><span dir="ltr">and Hungarian languages), and then fine-tuning it again using the Javanese lan</span><span dir="ltr">guage as the target language. We found that the models using cross-lingual trans</span><span dir="ltr">fer learning can increase the accuracy of the models without using cross-lingual </span><span dir="ltr">transfer learning by 14.3%–15.3% over LSTM-based models, and by 0.21%–</span><span dir="ltr">3.95% over Transformer-based models. Our results show that the most accurate </span><span dir="ltr">Javanese POS tagger model is XLM-RoBERTa that is fine-tuned in two stages </span><span dir="ltr">(the first one using Indonesian language as the source language, and the second </span><span dir="ltr">one using Javanese language as the target language), capable of achieving an </span><span dir="ltr">accuracy of 87.65%</span>

A hybrid approach to extract conceptual diagram from software requirements

Employing rules for the automatic extraction of conceptual diagrams from software requirements has been in practice for some time. However, considering only rules for extraction makes the system complex to handle. Moreover, the rules are predominantly based on the syntactic structure such as Part of Speech tags along with Dependency Grammar of sentences and rarely on semantics. In this paper, we propose to use a probabilistic approach in configuration with the rule-based technique and the Word embeddings to preserve the semantics of the sentence. Hence, reduces the complexity of the extraction procedure. Further, we advocate the use of a divide-and-conquer policy of extraction instead of extracting classes for one entire use case description. We extract the class diagram from small use cases and then merge it to obtain the class diagram. As generated class diagram corresponding to small use cases can be utilized in another similar software design, thus, it increases the scalability and decreases the extraction time. The proposed hybrid approach integrates the knowledge from the experiences. Thus, the proposed approach achieved 90% as F1-score whereas the F1-Score for the existing methods ranged between 79-88%. The proposed hybrid approach also shows a 19.44% reduction in terms of the number of iterations performed to carry out extraction procedures for individual use cases. Hence, reduces the extraction procedure complexity.

Beyond a Buzzword: Student Perspectives on what Contributes to Engaging Educators and Classroom Experiences

In a changing educational landscape, student engagement remains a prominent issue in research and practice. This study explores engagement from the student’s perspective across multiple disciplines, year levels and delivery modes. It draws on 13,125 Student Evaluation of Teaching survey comments where students have used the word “engage” (or derivative) in response to individual educators. Part of speech tagging was used to identify surrounding words that helped to contextualise the inclusion of the term. Through content analysis, key words were then grouped into six themes (approachability; synonyms for good; clarity; enjoyment and enthusiasm; relevance; and subject matter expertise) that highlighted both individual educator attributes and the overall learning experience. This research provides practical considerations for educators and administrators in relation to what students find engaging and how to support such positive environments. Additionally, it reinforces the role students play as major stakeholders in their learning and ongoing discussions around engagement.

Identification of Sentence Stems Characteristic of Chinese Learner English Writing

Phraseological units in academic English texts have been a central focus in recent corpus linguistic research. This paper describes a special category of clause-level phraseological units, namely, Characteristic Sentence Stems (CSSs), with a view to describing their identifying criteria and their extraction method. CSSs are contiguous lexico-grammatical sequences which contain a subject-predicate structure and which are frame expressions characteristic of academic writing. The extraction method of a CSS consists of six steps: POS tagging, n-gram segmentation, structure identification, significance of occurrence calculation, text range calculation, and overlapping sequence reduction. The significance of occurrence calculation is the crux of this method. It includes the computing of both the internal association and the boundary independence of a CSS, and it tests the occurring significance of the CSS from both the inside and the outside perspectives. Our methods and results suggest that CSSs can be statistically defined and extracted from corpora and can employed in large-scale studies to more fully account for the phraseological features of non-native English academic writing.

A detailed analysis of the interpretability of Convolutional Neural Networks for text classification

Abstract Convolutional Neural Networks (CNNs) have become ubiquitous in many NLP tasks. However, understanding its process is still an area with much to be done. In this paper, we introduce a method to study the interpretability of CNNs when used for text classification. More specifically, we work on the interpretability of the convolutional filters in the context of sentiment analysis. The framework used in this paper has allowed the understanding of the mechanics of the network when applied to the task of sentiment analysis. The results of the experiments reveal how certain parts of speech (POS) tags are more relevant than others to the classification of a sentence. Furthermore, we also observed a preference for shorter $n$-grams when classifying negative sentiment sequences. Additionally, we detected a certain amount of redundancy among the convolutional filters, thus allowing us to conclude that smaller architectures would have worked for this particular task. This was achieved by computing the relevant metrics to measure the influence of the pertinent property on the desired class. In our case, the concepts treated were POS tags that carry semantic information and properties were related to the convolutional filters.

Spoken Spanish PoS tagging: gold standard dataset

AbstractThe development of a benchmark for part-of-speech (PoS) tagging of spoken dialectal European Spanish is presented, which will serve as the foundation for a future treebank. The benchmark is constructed using transcriptions of the Corpus Oral y Sonoro del Español Rural (COSER;“Audible corpus of spoken rural Spanish”) and follows the Universal Dependencies project guidelines. We describe the methodology used to create a gold standard, which serves to evaluate different state-of-the-art PoS taggers (spaCy, Stanza NLP, and UDPipe), originally trained on written data and to fine-tune and evaluate a model for spoken Spanish. It is shown that the accuracy of these taggers drops from 0.98$$-$$ - 0.99 to 0.94$$-$$ - 0.95 when tested on spoken data. Of these three taggers, the spaCy’s trf (transformers) and Stanza NLP models performed the best. Finally, the spaCy trf model is fine-tuned using our gold standard, which resulted in an accuracy of 0.98 for coarse-grained tags (UPOS) and 0.97 for fine-grained tags (FEATS). Our benchmark will enable the development of more accurate PoS taggers for spoken Spanish and facilitate the construction of a treebank for European Spanish varieties.

Parts-of-speech tagger for Sindhi language using deep neural network architecture

Language is a fundamental medium for human communication, encompassing spoken and written forms, each governed by grammatical rules. Sindhi, one of the oldest languages, is characterized by its rich morphology and grammatical structure. Part-of-speech (POS) tagging, a crucial process in natural language processing, involves assigning grammatical tags to words. This research presents a novel approach to POS tagging for Sindhi text using deep learning techniques. We developed a POS tagger employing Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models, with LSTM demonstrating superior effectiveness. This study represents the first application of these deep learning methods for POS tagging in Sindhi. Utilizing fastText, we trained 79,959 Sindhi word vectors, derived from a corpus compiled from diverse sources including Sindhi books, stories, and poetry. The corpus comprises 1,459 sentences and 10,584 unique words, split into 80% for training and 20% for validation. Our results indicate that the LSTM model achieved an accuracy of 85.80%, outperforming the GRU model, which achieved 80.77%, by a margin of 5%. This work's novelty lies in the application of deep learning techniques to enhance POS tagging accuracy in the Sindhi language corpus.

Design of English Lexical Analysis System for Machine Translation Based on Multimodal Neural Network

Information resources have become a very important source of wealth, and due to differences in language and writing, technological information exchange between countries has become difficult. In this article, we propose a data augmentation method based on statistical models. A framework for a multimodal output model is proposed, taking into account the semantic relevance and importance between different languages. This framework is based on text sequence to sequence framework, decoupled, and a network architecture based on dual stream attention mechanism is designed. A multimodal interactive neural network layer was added between the encoder and decoder, achieving the fusion of multilingual domain information. It compensates for the influence of text on traditional translation generation systems in the process of English Chinese translation, and solves the problems of inaccurate translation results and low similarity with the original text in English Chinese machine translation. The system proposed in this article mainly consists of preprocessing module, lexical analysis and segmentation module, part of speech tagging and phrase analysis module, translation rule construction module, decoding module, translation generation module, etc. The experimental results show that using an improved generation system, compared with traditional generation systems, improves the accuracy of translation generation, has certain advantages, and is more practical.

Ga-RBP: A Rule-based Parser for the Syntactic Analysis of the Ga Language of Ghana

This research presents a Parts of Speech (POS) corpus and a static rule-based technique, which we refer to as Ga Rule-based Parser (Ga-RBP), for the syntactic analysis and the parsing of sentences for the Ga language of Ghana. The technique is developed to parse sentences by utilising the POS tagged corpus; the corpus was developed by manually tagging the Ga words with their corresponding POS tags following a standard Ga-English dictionary and custom Tagset for the language. The syntax rules were computationally defined using production rules, which establish how a word should follow the other in the right sequence to form a correct grammatical statement based on their POS. The model generally analyses the sentence structure of the language to assert its syntactic state for correctness or otherwise.

Part-of-speech tagger for Bodo language using deep learning approach

Abstract Language processing systems such as part-of-speech (POS) tagging, named entity recognition, machine translation, speech recognition, and language modeling have been well-studied in high-resource languages. Nevertheless, research on these systems for several low-resource languages, including Bodo, Mizo, Nagamese, and others, is either yet to commence or is in its nascent stages. Language model (LM) plays a vital role in the downstream tasks of modern natural language processing. Extensive studies are carried out on LMs for high-resource languages. However, these low-resource languages are still underreprese. In this study, we first present BodoBERT, an LM for the Bodo language. To the best of our knowledge, this work is the first such effort to develop an LM for Bodo. Second, we present an ensemble deep learning-based POS tagging model for Bodo. The POS tagging model is based on combinations of BiLSTM with conditional random field and stacked embedding of BodoBERT with BytePairEmbeddings. We cover several LMs in the experiment to see how well they work in POS tagging tasks. The best-performing model achieves an F1 score of 0.8041. A comparative experiment was also conducted on Assamese POS taggers, considering that the language is spoken in the same region as Bodo.

Comparative Analysis of Deep Learning Models for Part of Speech Tagging in the Malay Language

Despite the widespread use of Malay, under-resourced languages like Malay face challenges in Natural Language Processing (NLP), particularly in Part-of-Speech (POS) tagging. The scarcity of annotated corpora poses a primary obstacle to POS tagging in Malay. This study aims to enhance the effectiveness and reliability of POS tagging models explicitly tailored for under-resourced languages within the field of NLP, focusing on Malay. Existing models, which rely on Conditional Random Fields and Hidden Markov Models, exhibit limitations, underscoring the need for more robust approaches. The research conducts a comparative analysis of various deep-learning models with different encoders for POS tagging in Malay sentences. The experimental analysis demonstrates that the Bidirectional Long Short-Term Memory (Bi-LSTM) model, leveraging a pre-trained Bidirectional Encoder Representations from Transformers (BERT) embedding model, achieves exceptional accuracy, precision, recall, and F1 scores in predicting tags. Notably, the BERT + Bi-LSTM model, boasting an accuracy of 98.82%, outperforms other models, showcasing superior performance across all evaluated metrics. Additionally, this combined model effectively handles known and unknown words, yielding highly accurate POS tagging results for Malay sentences. Doi: 10.28991/HIJ-2024-05-02-04 Full Text: PDF

Comparison of Sentences using POS Tagging Tool under Subjective Examination

The Question answering system is used to generate the correct result that is asked by humans in natural language. In an online examination system, most of the work has been done but still, problems occur in preprocessing i.e. Part Of Speech (POS). POS tagger is used to properly tag each word in the sentences. In this paper, we used two datasets i.e. TREC DATA and data collected from the student. We apply the POS tagger to both datasets and compare the result. For generating the POS tagger we used NLTK and spaCy libraries for comparison. We observed that using those libraries the same word has a different tag. Using both tools, we computed the difference between the words and assigned the count to the POS tagging on that result, we calculate the accuracy of both libraries. The result shows that the spaCy library is best for POS tagging because it generates more correct results as compared to NLTK.