Part-of-speech Tagging Research Articles

Analyzing fuzzy semantics of reviews for multi-criteria recommendations

Hotel reviews play a vital role in tourism recommender system. They should be analyzed effectively to enhance the accuracy of recommendations which can be generated either from crisp ratings on a fixed scale or real sentiments of reviews. But crisp ratings cannot represent the actual feelings of reviewers. Existing tourism recommender systems mostly recommend hotels on the basis of vague and sparse ratings resulting in inaccurate recommendations or preferences for online users. This paper presents a semantic approach to analyze the online reviews being crawled from tripadvisor.in. It discovers the underlying fuzzy semantics of reviews with respect to the multiple criteria of hotels rather than using the crisp ratings. The crawled reviews are preprocessed via data cleaning such as stopword and punctuation removal, tokenization, lemmatization, pos tagging to understand the semantics efficiently. Nouns representing frequent features of hotels are extracted from pre-processed reviews which are further used to identify opinion phrases. Fuzzy weights are derived from normalized frequency of frequent nouns and combined with sentiment score of all the synonyms of adjectives in the identified opinion phrases. This results in fuzzy semantics which form an ideal representation of reviews for a multi-criteria tourism recommender system. The proposed work is implemented in python by crawling the recent reviews of Jaipur hotels from TripAdvisor and analyzing their semantics. The resultant fuzzy semantics form a manually tagged dataset of reviews tagged with sentiments of identified aspects, respectively. Experimental results show improved sentiment score while considering all the synonyms of adjectives. The results are further used to fine-tune BERT models to form encodings for a query-based recommender system. The proposed approach can help tourism and hospitality service providers to take advantage of such sentiment analysis to examine the negative comments or unpleasant experiences of tourists and making appropriate improvements. Moreover, it will help online users to get better recommendations while planning their trips.

APPLICATION OF HYBRID APPROACH FOR WOLAITA LANGUAGE PART OF SPEECH TAGGING

The main purpose of this study is to develop part-of-speech tagger for Wolaita Language using hybrid approach. Part of speech tagger is one of the subtasks in NLP application which is important for other Natural Language Processing (NLP) applications, like parser, machine translator, speech recognizer and search engines. PoST is a process of tagging a corresponding part of speech tag for a word that tag defines how the word is used in a sentence. The PoST for Wolaita language is not enough yet to be used as one vital module in other natural language processing applications. In this study, the development of PoS tagger using hybrid approach that combines HMM and rule based approaches was conducted for Wolaita language. In general HMM model need large data to increase the performance and the rule based model learner rule based on the language features. The HMM tagger, tags the words based on the optimal path for a given sequence of words and transformation based learning (TBL) is a rule based approaches that learns rule directly from the training corpus without expert knowledge. The developed hybrid approach of Wolaita language PoS tagger uses HMM tagger as initial annotators and rule based tagger as a corrector based on fixed threshold values. For implementation and experiment purpose the researcher used python programming and NLTK. For training and testing the models, 1256 sentences or 15,268 words are collected from three different categories (Bible, Social media in Wolaita language (Wogetta FM 96.6 ) and Wolaita language department) and annotated data manually. For tagging purpose 26 PoS tag are identified. From entire corpus, 90% for training and the remaining of entire corpus for testing purpose. The performance of the taggers, are tested by using different experiments. After experiment the researcher found that the performance of HMM, rule based and hybrid taggers shows 88.14%, 93.19% and 94.82% respectively. Generally, hybrid approach showed the better performance to assigning part of speech tag for Wolaita language.

Incorporating knowledge for joint Chinese word segmentation and part-of-speech tagging with SynSemGCN

PurposeRecent trends have shown the integration of Chinese word segmentation (CWS) and part-of-speech (POS) tagging to enhance syntactic and semantic parsing. However, the potential utility of hierarchical and structural information in these tasks remains underexplored. This study aims to leverage multiple external knowledge sources (e.g. syntactic and semantic features, lexicons) through various modules for the joint task.Design/methodology/approachWe introduce a novel learning framework for the joint CWS and POS tagging task, utilizing graph convolutional networks (GCNs) to encode syntactic structure and semantic features. The framework also incorporates a pre-defined lexicon through a lexicon attention module. We evaluate our model on a range of public corpora, including CTB5, PKU and UD, the novel ZX dataset and the comprehensive CTB9 dataset.FindingsExperimental results on these benchmark corpora demonstrate the effectiveness of our model in improving the performance of the joint task. Notably, we find that syntax information significantly enhances performance, while lexicon information helps mitigate the issue of out-of-vocabulary (OOV) words.Originality/valueThis study introduces a comprehensive approach to the joint CWS and POS tagging task by combining multiple features. Moreover, the proposed framework offers potential adaptability to other sequence labeling tasks, such as named entity recognition (NER).

Multi Task Learning Based Shallow Parsing for Indian Languages

Shallow Parsing is an important step for many Natural Language Processing tasks. Although shallow parsing has a rich history for resource rich languages, it is not the case for most Indian languages. Shallow Parsing consists of POS Tagging and Chunking. Our study focuses on developing shallow parsers for Indian languages. As part of shallow parsing we included morph analysis as well. For the study, we first consolidated available shallow parsing corpora for 7 Indian Languages (Hindi, Kannada, Bangla, Malayalam, Marathi, Urdu, Telugu) for which treebanks are publicly available. We then trained models to achieve state of the art performance for shallow parsing in these languages for multiple domains. Since analyzing the performance of model predictions at sentence level is more realistic, we report the performance of these shallow parsers not only at the token level, but also at the sentence level. We also present machine learning techniques for multitask shallow parsing. Our experiments show that fine-tuned contextual embedding with multi-task learning improves the performance of multiple as well as individual shallow parsing tasks across different domains. We show the transfer learning capability of these models by creating shallow parsers (only with POS and Chunk) for Gujarati, Odia, and Punjabi for which no treebanks are available. As a part of this work, we will be releasing the Indian Languages Shallow Linguistic (ILSL) benchmarks for 10 Indian languages including both the major language families Indo-Aryan and Dravidian as common building blocks that can be used to evaluate and understand various linguistic phenomena found in Indian languages and how well newer approaches can tackle them.

Named Entity Recognition Using Conditional Random Fields for Flood Detection In Gerbang Kertosusila Based Twitter Data

The national strategic area Gerbang Kertosusila East Java should be aware of floods. One of the existing efforts is to place flood sensors at several flood-prone points. However, that way is constrained by the need for more equipment to handle the many needy areas. So it is necessary to develop technology for the dissemination of flood information. Dissemination of flood information was quickly obtained from social media Twitter. One way is to use Twitter's text data source for a Named Entity Recognition model to help detect flood events and their locations. The Named Entity Recognition (NER) model was constructed using the Conditional Random Fields (CRFs) method to achieve research objectives. This research adds slang word handling at the preprocessing stage to improve model performance and the use of the BIO format in the labeling process and POS Tagging in the Feature Extraction process. Evaluation results with five Kfolds, 80% training data, and 20% test data show that the NER CRFs model performs excellently with a Precision of 0.981, Recall of 0.926, and f-measure of 0.950 so that these results can help the community and government regarding the information on the distribution of floods.

Advancements in Neural Machine Translation: Methodological Innovations and Empirical Insights for Cross-Linguistic Discourse Preservation

Abstract: Natural Language Processing (NLP), in the form recognizable today, really began to take hold in the 1980s, when machine learning helped propel it to soaring heights. However, due to a lack of processing power, machine learning, and to an extent, NLP, started slowing down in innovation and ideas and had almost ground to a relative halt, until the last decade, when a sudden increase in both productivity and interest in the machine learning helped increase the amount of knowledge in the space itself. This review provides several different case studies using different methodologies. The first paper was a deep analysis on how researchers were able to use Tesseract and Google Vision in tandem with automatic data mining methods to enrich the Cherokee language database in order to preserve it from extinction. The second paper takes a query translation-based approach toward translating English to Indian languages and utilizes a Multilingual Cross-Language Information Retrieval (MLCIR) system with tools such as Part of Speech Tagger (POST), Stop-Word, and Porter Stemmer. The third paper presents CoVe, which transfers knowledge from machine translation to improve performance on NLP tasks like sentiment analysis and question answering by using contextualized word vectors along with word embeddings, achieving new state-of-the-art results on some datasets. The fourth paper aims to translate English to Pakistan Sign Language (PSL) and also uses POST and goes through dependency analysis, sentence classification, and PSL using PLS trees. The fifth paper uses a Multilingual Neural Machine Translation (NMT) system for LowResource languages and incorporates two main models: a recurrent NMT and a transformer NMT. The sixth paper analyzes how a fine-tuned transformer model seems to work better than transformer models trained from scratch on high-resource languages, while vice-versa seems to occur for low-resource languages. The seventh paper adds to this by talking about how multilingual translation seems to work better than a back-translation model. Given the diverse array of approaches that could be used, we aim to identify the most efficient and correct methodology for future researchers to use in their work, based on the papers in this literature review.

Towards automatic question generation using pre-trained model in academic field for Bahasa Indonesia

AbstractExam evaluations are essential to assessing students’ knowledge and progress in a subject or course. To meet learning objectives and assess student performance, questions must be themed. Automatic Question Generation (AQG) is our novel approach to this problem. A comprehensive process for autonomously generating Bahasa Indonesia text questions is shown. This paper suggests using a decoder to generate text from deep learning models’ tokens. The suggested technique pre-processes Vectorized Corpus, Token IDs, and Features Tensor. The tensors are embedded to increase each token, and attention is masked to separate padding tokens from context-containing tokens. An encoder processes the encoded tokens and attention masks to create a contextual understanding memory that the decoder uses to generate text. Our work uses the Sequence-to-Sequence Learning architecture of BiGRU, BiLSTM, Transformer, BERT, BART, and GPT. Implementing these models optimizes computational resources while extensively exploring the research issue. The model uses context sentences as input and question sentences as output, incorporating linguistic elements like response placement, POS tags, answer masking, and named entities (NE) to improve comprehension and linguistic ability. Our approach includes two innovative models: IndoBERTFormer, which combines a BERT encoder with a Transformer decoder, and IndoBARTFormer, which decodes vectors like BERT. IndoTransGPT uses the Transformer as an encoder to improve understanding, extending the GPT model’s adaptability.

A framework for decision making to purchase the best product using feature-based opinions

As more people express their thoughts on products on various online shopping platforms, the feelings expressed in these opinions are becoming a significant source of information for marketers and buyers. These opinions have a big impact on consumers’ decision to buy the best quality product. When there are too many features or a small number of records to analyze, the decision-making process gets difficult. A recent stream of study has used the conventional quantitative star score ratings and textual content reviews in this context. In this research, a decision-making framework is proposed that relies on feature-based opinions to analyze the textual content of reviews and classify buyer’s opinions, thereby assisting consumers in making long-term purchases. The framework is proposed in this paper for product purchase decision making based on feature-based opinions and deep learning. Framework consists of four components: i) Pre-processing, ii) Feature extraction, iii) Feature-based opinion classification, and iv) Decision-making. Web scraping is used to obtain the dataset of Smartphone reviews, which is subsequently clean and pre-processed using tokenization and POS tagging. From the tagged dataset, noun labeled words are retrieved, and then the probable product’s features are extracted. These feature-based sentences or reviews are processed using a word embedding to generate review vectors that identify contextual information. These word vectors are used to construct hidden vectors at the word and sentence levels using a hierarchical attention method. With respect to each feature, reviews are divided into five classes: extremely positive, positive, extremely negative, negative, and neutral. The proposed method may readily detect a customer’s opinion on the quality of a product based on a certain attribute, which is beneficial in making a purchase choice.

Evolution of Natural Language Processing: A Review

Over the years, Natural Language Processing (NLP) has evolved dramatically, moving from early rule based systems to the current era dominated by advanced deep learning models. An overview of the significant turning points and patterns that have influenced the development of NLP is given in this study. In the early days of Natural Language Processing (NLP), rule based methods were the main focus. Linguists would manually create rules to analyze and comprehend human language. Although these systems were somewhat successful, they were unable to handle the complexity and unpredictability of spoken language. A major change was brought about by the introduction of probabilistic models and machine learning techniques with the emergence of statistical approaches. The development of methods like n gram models and hidden Markov models during this time allowed computers to handle linguistic patterns. Large scale linguistic resources like word embeddings and annotated corpora started to appear, which further accelerated the development of NLP. Machine learning algorithms have led to notable advancements in tasks such as machine translation, named entity recognition, and part of speech tagging. NLP has seen a revolution in recent years thanks to deep learning, which uses neural networks to learn intricate language representations. Sequential dependencies in language can now be better understood because of models like Long Short Term Memory Networks (LSTMs) and Recurrent Neural Networks (RNNs). The addition of attention mechanisms, as demonstrated by Transformer and other models, improves the model's ability to manage long range dependencies and perform better on a variety of NLP tasks. In the future, NLP will develop in ways that go beyond performance measurements, exploring interpretability, ethical issues, and the incorporation of multimodal data. As the area develops, it becomes increasingly important to eliminate biases, ensure ethical AI deployment, and improve user centric experiences. This introduction lays the groundwork for an in depth examination of the development of NLP, highlighting significant turning points, difficulties, and potential future directions in this vibrant and quickly developing subject.

English Vocabulary Learning Aid System Using Digital Twin Wasserstein Generative Adversarial Network Optimized With Jelly Fish Optimization Algorithm

ABSTRACT Natural Language Processing (NLP) is a technology that permits computers to recognize human languages. Words are the fundamental unit of analysis in deep-level grammatical and semantic analysis. The main goal of NLP is typically word segmentation. Since the machine learning techniques cannot be directly applied to the practical issue of significant structural disparities between various data modalities in a multi-modal context. In this paper, English Vocabulary Learning Aid System Using Digital Twin Wasserstein Generative Adversarial Network Optimized using Jelly Fish Optimization Algorithm is proposed. The problematic of multiple modal data heterogeneity is handled by the feature extraction of Parameterized Local Maximum Synchro squeezing Transform and extract the features such as Phonetic features, sentence length, word embedding’s, part of speech tags, word frequencies, N-grams. Then, the Digital twin Wasserstein generative adversarial network classifies the English vocabulary to easy words, intermediate words, and difficult words. The performance of the proposed EVLS-DtwinWGAN-NLP approach attains 3.101%, 7.12%, 7.73% higher accuracy, 24.13%, 13.04%, 29.51% lower computation Time and 2.292%, 5.365%, 1.551% higher AUC compared with existing methods like Feature extraction and analysis of natural language processing for deep learning English language (EVLS-BiLSTM-NLP), State of art for semantic analysis of natural language processing (EVLS-SA-NLP) respectively.

Deep Learning Model for Tamil Part-of-Speech Tagging

Abstract Part-of-Speech (POS) tagging is one of the popular Natural Language Processing (NLP) tasks. It is also considered to be a preliminary task for other NLP applications such as speech recognition, machine translation, and sentiment analysis. A few works have been published on POS tagging for the Tamil language. However, the performance of the POS tagger with unknown words is not explored in the literature. The appearance of unknown words is a frequently occurring problem in POS tagging and makes it a challenging task. In this paper, we propose a deep learning-based POS tagger for Tamil using Bi-directional Long Short Term Memory (BLSTM). The performance of the POS tagger was evaluated using known and unknown words. The POS tagger with regular word-level embeddings produces 99.83 and 92.46% accuracies for all known and 63.21% unknown words. It clearly shows that the accuracy decreases when the number of unknown words increases. To improve the performance of the POS tagger with unknown words, the proposed BLSTM model that uses word-level, character-level and pre-trained word embeddings. Test results of this model show a 2.57% improvement for 63.21% of unknown words, with an accuracy of 95.03%.

Enabling Natural Language Processing and AI Research in Low-Resource Languages: Development and Description of an Assamese UPoS Tagged Dataset

This paper describes in detail the Universal Parts of Speech (UPoS) tagged dataset for the Assamese language. PoS tagged dataset in a language is crucial for experimenting and creating resources for various Natural Language Processing (NLP) and AI research. With the growing usage of Universal Dependency standards, tagged dataset with Universal PoS tags are becoming very much essential for contemporary experiments in the NLP community. NLP research in Assamese, and Indo-Aryan language, is relatively new, and the language is considered a Low Resource language. The dataset of UPoS tagged Assamese text is created with an aim of contributing towards enriching this low resource language for NLP and AI tasks. The size of the dataset is 283506 tokens of Assamese vocabulary, against total 20280 sentences, tagged with 17 standard UPoS tags of core lexical categories. The raw data are taken from an open-source corpus originally tagged with BIS tagset. The original size of 453457 tokens against 29504 sentences, after subjected to data filtering, was reduced to this clean resource of 283506 tokens. Lexical categories mapping is done with linguistic expertise, from BIS to UPoS tagsets. Mapped pattern was used for a first-level conversion of BIS tags to UPoS tags. Linguistic validation is also performed with linguistic experts and inter annotator agreement/disagreements were recorded. Second level validation resulted in deciding on the agreement, producing the final version of the dataset. This Assamese UPoS tagged dataset is the first of its kind with UPoS annotations and shall serve a wider Assamese NLP research community for model training using Machine Learning/Deep Learning Techniques.

BioBBC: a multi-feature model that enhances the detection of biomedical entities

The rapid increase in biomedical publications necessitates efficient systems to automatically handle Biomedical Named Entity Recognition (BioNER) tasks in unstructured text. However, accurately detecting biomedical entities is quite challenging due to the complexity of their names and the frequent use of abbreviations. In this paper, we propose BioBBC, a deep learning (DL) model that utilizes multi-feature embeddings and is constructed based on the BERT-BiLSTM-CRF to address the BioNER task. BioBBC consists of three main layers; an embedding layer, a Long Short-Term Memory (Bi-LSTM) layer, and a Conditional Random Fields (CRF) layer. BioBBC takes sentences from the biomedical domain as input and identifies the biomedical entities mentioned within the text. The embedding layer generates enriched contextual representation vectors of the input by learning the text through four types of embeddings: part-of-speech tags (POS tags) embedding, char-level embedding, BERT embedding, and data-specific embedding. The BiLSTM layer produces additional syntactic and semantic feature representations. Finally, the CRF layer identifies the best possible tag sequence for the input sentence. Our model is well-constructed and well-optimized for detecting different types of biomedical entities. Based on experimental results, our model outperformed state-of-the-art (SOTA) models with significant improvements based on six benchmark BioNER datasets.

CoreNLP dependency parsing and pattern identification for enhanced opinion mining in aspect-based sentiment analysis

Aspect-Based Sentiment Analysis (ABSA) aims to identify the sentiment expressed towards a specific feature or aspect of a given text. Although certain ABSA techniques employ syntactic information to capture the connection between the opinion target and the sentiment word, they often do not incorporate data processing techniques such as dependency parsing, which can be beneficial in accurately capturing the sentiment expressed towards the opinion target. In this paper a method for ABSA that employs both syntax and semantic information and incorporates dependency parsing(Semantic-Syntactic Dependency Parsing (SSDP) Method) with Core Natural Language Processing (CoreNLP) which is a natural language processing library for processing the input text and identifying patterns effectively (according to the CoreNLP relations and part of speech tagging(POS)) to extract the critical relations that accurately reflect the sentiment conveyed regarding the opinion target is proposed. The results show that the proposed pattern captured approximately 75% of the data, and the rest were classified via Long Short-Term Memory (LSTM) based on semantic information. We illustrated the efficacy of SSDP, through experiments on the SemEval14,Semval15 and Semval16 datasets, which include two datasets (laptops and restaurants) carefully categorized by a human annotator into categories of positive, negative, or neutral attitudes. The experimental results reveal that SSDP is superior to the other state-of-the-art ABSA approaches, that use syntax information but do not utilize data processing techniques. Additionally, we highlight the limitations of ABSA methods that do not incorporate syntax information and the potential improvements that can be made through the use of data processing.

Exploring part of speech (pos) tag sequences in a large-scale learner corpus of L2 English: a developmental perspective

This research explores the pos tag sequences that shape the transition from upper intermediate (B2 cefr) to near-native proficiency (C2 cefr) in a corpus of essays ( n=32,410) from the Cambridge Learner Corpus. Gilquin (2018) and others have shown that pos tag sequences offer a holistic approach to extracting the most commonly used patterns without a starting point of an a priori set of words and word sequences. Using corpus linguistics informed by usage-based theories of language learning, this paper examines the frequency and distribution of 4-slot pos-tag sequences in L2 English writing, drawing on the taxonomy of pattern grammar ( Francis et al., 1996 , 1998 ; and Hunston and Francis, 2000 ). Findings point to the presence of both core and emergent pos-tag sequences in learner language in the two proficiency levels analysed. These sequences point to the presence of dynamic language restructuring processes as learners become more proficient and re-evaluate their understanding of frequency and distribution in English. This paper shows evidence of how language competence increases with proficiency. The research offers new evidence in our understanding of the development of L2 writing in efl contexts.

Toward accurate Amazigh part-of-speech tagging

<span lang="EN-US">Part-of-speech (POS) tagging is the process of assigning to each word in a text its corresponding grammatical information POS. It is an important pre-processing step in other natural language processing (NLP) tasks, so the objective of finding the most accurate one. The previous approaches were based on traditional machine learning algorithms, later with the development of deep learning, more POS taggers were adopted. If the accuracy of POS tagging reaches 97%, even with the traditional machine learning, for high resourced language like English, French, it’s far the case in low resource language like Amazigh. The most used approaches are traditional machine learning, and the results are far from those for rich language. In this paper, we present a new POS tagger based on bidirectional long short-term memory for Amazigh language and the experiments that have been done on real dataset shows that it outperforms the existing machine learning methods.</span>

Natural Language Processing For Requirement Elicitation In University Using Kmeans And Meanshift Algorithm

Data Driven Requirement Engineering (DDRE) represents a vision for a shift from the static traditional methods of doing requirements engineering to dynamic data-driven user-centered methods. Data available and the increasingly complex requirements of system software whose functions can adapt to changing needs to gain the trust of its users, an approach is needed in a continuous software engineering process. This need drives the emergence of new challenges in the discipline of requirements engineering to meet the required changes. The problem in this study was the method in data discrepancies which resulted in the needs elicitation process being hampered and in the end software development found discrepancies and could not meet the needs of stakeholders and the goals of the organization. The research objectives in this research to the process collected and integrating data from multiple sources and ensuring interoperability. Conclusion in this research is determining is the clustering algorithm help the collection data and elicitation process has a somewhat greater impact on the ratings provided by professionals for pairs that belong to the same cluster. However, the influence of POS tagging on the ratings given by professionals is relatively consistent for pairs within the same cluster and pairs in different clusters.

When Socialism Meets Terrorism: A Computer-Assisted Discursive News Values Analysis of Chinese Newspapers’ Coverage of Domestic and International Terrorist Attacks

ABSTRACT Terrorist attacks, as a particular type of negative social event, are a potential threat to the establishment of a “Socialist Harmonious Society” advocated by the Chinese government. To shed light on how Chinese journalists construct reports on terrorist attacks and, more broadly, how non-Western countries report on negative social issues, we compared newspaper coverage of one domestic attack with four international incidents. Starting from Bednarek and Caple’s framework, we further developed the computer-assisted Discursive News Values Analysis (DNVA) they proposed by adapting individual definitions of news values and dividing them into 15 subcategories to suit the Chinese context, the topic of terrorism, and the automatic processing of large-scale datasets. Based on our understanding, we developed an open-source list of Chinese lexical indicators by Part of Speech tagging, sentiment, collocation, and concordance analysis, with a view to providing resources to support future quantitative DNVA. Following this approach, we found that while market factors enabled Chinese newspapers to cover domestic terrorist attacks, media adopted specific media strategies to meet political needs, which differed from those used to cover international attacks. These different media strategies can be explained by the Chinese developmentalist ideology and harmonious cultural traditions.

Deep Learning-based POS Tagger and Chunker for Odia Language Using Pre-trained Transformers

Developing effective natural language processing (NLP) tools for low-resourced languages poses significant challenges. This article centers its attention on the task of Part-of-speech (POS) tagging and chunking, which pertains to the identification and categorization of linguistic units within sentences. POS tagging and Chunking have already produced positive results in English and other European languages. However, in Indian languages, particularly in Odia language, it is not yet well explored because of the lack of supporting tools, resources, and its complex linguistic morphology. This study presents the building of a manually annotated dataset for Odia phrase chunking task and the development of a deep learning-based model specifically tailored to accommodate the distinctive properties of the language. The process of annotating the Odia chunking corpus involved the utilization of inside-outside-begin labels, which were tagged by using designed Odia chunking tagset. We utilize the constructed Odia chunking dataset to build Odia chunker based on deep learning techniques, employing state-of-the-art architectures. Various techniques, such as Recurrent Neural Networks, Convolutional Neural Networks, and transformer-based models, are investigated to determine the most effective approach for Odia POS tagging and chunking. In addition, we conduct experiments utilizing diverse input representations, including Odia word embeddings, character-level representations, and sub-word units, to effectively capture the complex linguistic characteristics of the Odia language. Numerous experiments are conducted that evaluate the performance of our Odia POS tagger and chunker, employing standard evaluation metrics and making comparisons with existing approaches. The results demonstrate that our transformer-based tagger and chunker achieves superior accuracy and robustness in identifying and categorizing linguistic POS tags and chunks within Odia sentences. It outperforms existing work and exhibits consistent performance across diverse linguistic contexts and sentence structures. The developed Odia POS tagger and chunker have enormous potential for a variety of NLP applications, including information extraction, syntactic parsing, and machine translation, all of which are tailored to the low-resource Odia language. This work contributes to developing NLP tools and technologies for low-resource languages, thereby facilitating enhanced language processing capabilities in various linguistic contexts.

A Machine Learning–Based Readability Model for Gujarati Texts

This study aims to develop a machine learning–based model to predict the readability of Gujarati texts. The dataset was 50 prose passages from Gujarati literature. Fourteen lexical and syntactic readability text features were extracted from the dataset using a machine learning algorithm of the unigram parts of speech tagger and three Python programming scripts. Two samples of native Gujarati speaking secondary and higher education students rated the Gujarati texts for readability judgment on a 10-point scale of “easy” to “difficult” with the interrater agreement. After dimensionality reduction, seven text features as the independent variables and the mean readability rating as the dependent variable were used to train the readability model. As the students' level of education and gender were related to their readability rating, four readability models for school students, university students, male students, and female students were trained with a backward stepwise multiple linear regression algorithm of supervised machine learning. The trained model is comparable across the raters’ groups. The best model is the university students’ readability rating model. The model is cross-validated. It explains 91% and 88% of the variance in readability ratings at training and cross-validation, respectively, and its effect size and power are large and high.