Area Of Language Processing Research Articles

Aphasia Screening Test (TeRAp): Construction and Validation for European Portuguese.

Aphasia is a common acquired language disorder following stroke or other brain injuries. However, it is not always easy to make a differential diagnosis with another communication disorder. Communication assessment in acute phases of the stroke, when the patient is bedridden or when there is no time for a more in-depth assessment, needs to be done with a formal screening test that has normative data. The aim of this study was to develop the Aphasia Screening Test (TeRAp) in digital format (appWeb) and present its clinimetric values. A screening test (TeRAp) was built in appWeb format that assesses the main areas of language processing and automatically provides a diagnostic hypothesis. A group of people with aphasia was evaluated and their performance was compared with three control groups, one of healthy people and two groups of people with neurological conditions, one with dysarthria and the other with mild cognitive impairment. Ceiling values were obtained in all the tests. Sensitivity values of 1 and specificity values of 0.99 were obtained for the presence of aphasia. An online aphasia screening test was developed, with excellent sensitivity and specificity results, which can be used by any health professional.

A Review of Humor Recognition Research

Abstract. Humor recognition is a popular research area in Natural Language Processing (NLP), with the fundamental goal of detecting whether humor is present. With the rapid development of artificial intelligence and the growing demand for human-computer interaction, humor recognition has broad applications in areas such as social media analysis, human-computer interaction systems, and intelligent question-answering assistants. However, due to the highly subjective and complex nature of humor, teaching computers to understand and recognize humor in a manner similar to humans is an exceptionally challenging task. This paper reviews the major research progress in the field of humor recognition and the construction of humor corpora. It discusses humor recognition methods based on traditional machine learning, deep learning, and multimodal approaches, and then highlights the advantages and disadvantages of each. Based on these analyses, the paper identifies the main challenges and difficulties in current research and provides suggestions and prospects for the future development of humor recognition systems.

Editorial for Vol.32, No.2

The June 2024 (Vol. 32, No. 2) issue of CIT. Journal of Computing and Information Technology brings four papers from the areas of natural language processing, image processing, network security, and biomedical image classification.

From Sound to Meaning: Navigating Wernicke's Area in Language Processing.

Wernicke's area, a critical brain region associated with language comprehension, was first identified by Carl Wernicke in the late 19th century. Situated in the left hemisphere's posterior superior temporal gyrus, this area is essential for processing auditory and visual language inputs. It integrates semantic and syntactic information, playing a key role in meaningful communication. The development of Wernicke's area during infancy and childhood is marked by rapid growth and refinement influenced by early language exposure and environmental stimuli. Neuroplasticity, the brain's ability to reorganize and adapt, is crucial for recovery from language impairments such as Wernicke's aphasia. This capacity for reorganization includes synaptic plasticity and axonal sprouting, which facilitate recovery through targeted rehabilitation and enriched environments. Recent research utilizing advanced neuroimaging and neuroanatomical tracing techniques has elucidated the connectivity of Wernicke's area with other language-related regions, such as Broca's area. Functional studies have revealed its specialized roles in processing different aspects of language, including phonological, semantic, and syntactic features. Moreover, investigations into language disorders and potential therapeutic interventions underscore the importance of harnessing neuroplasticity for effective treatment. Emerging technologies, such as non-invasive brain stimulation and multimodal imaging, offer promising avenues for further exploration of Wernicke's area and its role in language functions. These innovations hold the potential to enhance our understanding of language processing and improve therapeutic strategies for language impairments. In conclusion, Wernicke's area is central to language comprehension, and genetic and environmental factors influence its development. Understanding neuroplasticity and leveraging advanced research technologies can significantly advance our ability to address language-related disorders and enhance patient outcomes.

Cross-lingual dependency parsing for a language with a unique script

Abstract Syntactic parsing is one of the areas in Natural Language Processing. The development of large-scale multilingual language models has enabled cross-lingual parsing approaches, which allows us to develop parsers for languages that do not have treebanks available. However, these approaches rely on the assumption that languages share orthographic representations and lexical entries. In this article, we investigate methods for developing a dependency parser for Xibe, a low-resource language that is written in a unique script. We first investigate lexicalized monolingual dependency parsing experiments to examine the effectiveness of word, part-of-speech, and character embeddings as well as pre-trained language models. Results show that character embeddings can significantly improve performance, while pre-trained language models decrease performance since they do not recognize the Xibe script. We also train delexicalized monolingual models, which yield competitive results to the best lexicalized model. Since the monolingual models are trained on a very small training set, we also investigate lexicalized and delexicalized cross-lingual models. We use six closely related languages as source language, which cover a wide range of scripts. In this setting, the delexicalized models achieve higher performance than lexicalized models. A final experiment shows that we can increase performance of the cross-lingual model by combining source languages and selecting the most similar sentences to Xibe as training set. However, all cross-lingual parsing results are still considerably lower than the monolingual model. We attribute the low performance of cross-lingual methods to syntactic and annotation differences as well as to the impoverished input of Universal Dependency Part-of-Speech tags that the delexicalized model has access to.

BERT-based models for classifying multi-dialect Arabic texts

<p><a name="_Hlk133997549"></a><a name="_Hlk127607325"></a><span lang="EN-US">The area of natural language processing (NLP) is presently a rapidly developing field characterized by innovation and research. Despite this progress, several </span><a name="_Hlk148089340"></a><span lang="EN-US">dialects of Arabic</span><span><span lang="EN-US"> (DA) are classified as low-resource languages, making it challenging for NLP systems to process DA data. One approach to address this issue is to train NLP models on social media data sets containing DA texts. </span><span lang="IN">Therefore, these open-access social media datasets, as outlined in our paper, can serve as a valuable resource for developers and researchers involved in the processing of DA.</span><span lang="EN-US">To create our multilingual corpus, we gathered data from various datasets containing different versions of DA. These datasets will be used to classify texts in terms of sentiment classification, topic classification, and dialect identification. Our study contributes to the automated analysis of the classification of Arabic dialects. We aim to investigate and assess various machine learning and deep learning techniques, with a specific focus on utilizing the BERT model. </span><span lang="EN-US">The results of our experiments on our datasets show that DarijaBERT and DziriBERT trained on a similar DA outperform traditional machine learning methods and previous more general pre-trained models that were trained on multiple dialects or languages.</span></span></p>

An Unsupervised Domain-Adaptive Framework for Chinese Spelling Checking

Chinese Spelling Check (CSC) is a meaningful task in the area of Natural Language Processing (NLP), which aims at detecting spelling errors in Chinese texts and then correcting these errors. Current typical Chinese Spelling Check models have shown impressive performance in general datasets with the help of pretrained language models such as BERT, but suffer great perform loss in downstream tasks with domain-specific terms because they are primarily trained on general corpora. To verify the cross-domain adaptation ability of these models, we build three new datasets with abundant domain-specific terms on financial, medical, and legal domains and conduct empirical investigations on them in the corresponding domain-specific test datasets to verify the cross-domain adaptation ability. In response to the poor performance of the existing models, we propose a framework named uChecker which utilizes unsupervised method in spelling error detection and correction. Experiment results prove that uChecker can perform well in domain-specific test datasets while not losing its performance in the general domain.

Leveraging ChatGPT for Personalized Learning: A systematic Review in Educational Settings

In today’s digital world, personalization is a leading point for each learner to strengthen their educational achievements. Considering innovations in artificial intelligence (AI), particularly in the area of natural language processing, Chat GPT has sparked awareness as a viable mechanism for supporting personalized learning. The aim of the systematic review is to identify the present-day status of research on using Chat GPT for independent learning in educational settings. The research effort combines works that study Chat GPT application in learning environments, summarizing the results of a broad survey of scholarly literature. The significance of the Chat GPT in the learning path was identified from the previous studies. Also, the study discusses the practical difficulties and challenges of this technology. The study highlights some potential improvements by the Chat GPT, like improving the individual learning experience, offering seamless feedback and error corrections, encouraging study, and finding study metrics. Also, the study identified some potential challenges, for example, data privacy, bias, accessibility of databases, and similarity between answers. There is a possibility for further research on this topic in advance to get more valuable ideas in the academic field. This comprehensive study examines a collection of AI-based personalized learning ideas to clearly enlighten the advantages and disadvantages of utilizing Chat GPT in an academic environment. Practitioners can gain a clear idea by referring to the article, which will maximize the usage of technology in the near future.

Performance Analysis of Embedding Methods for Deep Learning-Based Turkish Sentiment Analysis Models

AbstractThe complex syntactic structure of Turkish text makes sentiment analysis in natural language processing (NLP) a challenging task. Conventional sentiment analysis methods often fail to effectively identify attitudes in Turkish texts, creating an urgent need for more efficient approaches. To fill this need, our study investigates the effectiveness of embedding techniques including pre-trained Turkish models such as Word2Vec, GloVe, and FastText in addition to two character-level embedding methods, namely, character-integer embedding (CIE) and character one-hot encoding embedding (COE), in conjunction with deep learning models specifically long short-term memory (LSTM), convolution neural networks (CNNs), bidirectional LSTM (Bi-LSTM), and hybrid models, for Turkish short-texts sentiment analysis. DL-based models were investigated on two datasets (e.g., an original Twitter (X) dataset and an accessible hotel reviews dataset). In addition to providing an intensive performance analysis of different embedding strategies and assessing their efficacy in dealing with the linguistic intricacies of Turkish, this study proposed a previously unexplored method in Turkish text representation that relies on a character-level one-hot encoding technique. The obtained findings indicate positive progress using a novel approach utilizing a dual-pathway architecture for both character level and word level that constitutes a substantial contribution to the area of natural language processing (NLP), specifically in the context of complex morphological languages. By employing a hybrid strategy that combines character and word levels on Twitter (X) data, the LSTM model obtained anF1 score of$$0.835 \pm 0.005$$0.835±0.005concerning cross-validation while CNN-BiLSTM attained the highestF1 Score (0.8392) using holdout validation. This strategy consistently produced modest improvements across the second public dataset (hotel reviews dataset) by emerging as the runner-up embedding technique in effectiveness, surpassed only by FastText. Findings provide practical recommendations for practitioners on how to effectively use sentiment analysis to make informed decisions by introducing an extensive performance analysis of the use of embedding techniques and deep learning models for sentiment analysis in Turkish texts, which is crucial in the current age of data analysis.

Nonfactoid Question Answering as Query-Focused Summarization With Graph-Enhanced Multihop Inference.

Nonfactoid question answering (QA) is one of the most extensive yet challenging applications and research areas in natural language processing (NLP). Existing methods fall short of handling the long-distance and complex semantic relations between the question and the document sentences. In this work, we propose a novel query-focused summarization method, namely a graph-enhanced multihop query-focused summarizer (GMQS), to tackle the nonfactoid QA problem. Specifically, we leverage graph-enhanced reasoning techniques to elaborate the multihop inference process in nonfactoid QA. Three types of graphs with different semantic relations, namely semantic relevance, topical coherence, and coreference linking, are constructed for explicitly capturing the question-document and sentence-sentence interrelationships. Relational graph attention network (RGAT) is then developed to aggregate the multirelational information accordingly. In addition, the proposed method can be adapted to both extractive and abstractive applications as well as be mutually enhanced by joint learning. Experimental results show that the proposed method consistently outperforms both existing extractive and abstractive methods on two nonfactoid QA datasets, WikiHow and PubMedQA, and possesses the capability of performing explainable multihop reasoning.

Spatial entropy as an inductive bias for vision transformers

Recent work on Vision Transformers (VTs) showed that introducing a local inductive bias in the VT architecture helps reducing the number of samples necessary for training. However, the architecture modifications lead to a loss of generality of the Transformer backbone, partially contradicting the push towards the development of uniform architectures, shared, e.g., by both the Computer Vision and the Natural Language Processing areas. In this work, we propose a different and complementary direction, in which a local bias is introduced using an auxiliary self-supervised task, performed jointly with standard supervised training. Specifically, we exploit the observation that the attention maps of VTs, when trained with self-supervision, can contain a semantic segmentation structure which does not spontaneously emerge when training is supervised. Thus, we explicitly encourage the emergence of this spatial clustering as a form of training regularization. In more detail, we exploit the assumption that, in a given image, objects usually correspond to few connected regions, and we propose a spatial formulation of the information entropy to quantify this object-based inductive bias. By minimizing the proposed spatial entropy, we include an additional self-supervised signal during training. Using extensive experiments, we show that the proposed regularization leads to equivalent or better results than other VT proposals which include a local bias by changing the basic Transformer architecture, and it can drastically boost the VT final accuracy when using small-medium training sets. The code is available at https://github.com/helia95/SAR.

Text summarization based on semantic graphs: an abstract meaning representation graph-to-text deep learning approach

Nowadays, due to the constantly growing amount of textual information, automatic text summarization constitutes an important research area in natural language processing. In this work, we present a novel framework that combines semantic graph representations along with deep learning predictions to generate abstractive summaries of single documents, in an effort to utilize a semantic representation of the unstructured textual content in a machine-readable, structured, and concise manner. The overall framework is based on a well defined methodology for performing semantic graph parsing, graph construction, graph transformations for machine learning models and deep learning predictions. The employed semantic graph representation focuses on using the model of abstract meaning representation. Several combinations of graph construction and graph transformation methods are investigated to specify the most efficient of them for the machine learning models. Additionally, a range of deep learning architectures is examined, including a sequence-to-sequence attentive network, reinforcement learning, transformer-based architectures, and pre-trained neural language models. In this direction, a semantic graph representation of an original text is extracted, and then the present framework formulates the problem as a graph-to-summary learning problem to predict a summary of an original text. To the best of our knowledge, this formulation of graph-to-summary predictions in abstractive text summarization, without other intermediate steps in the machine learning phase, has not been presented in the relevant literature. Another important contribution is an introduction of a measure for assessing the factual consistency of the generated summaries in an effort to provide a qualitative evaluation. To assess the framework, an extensive experimental procedure is presented that uses popular datasets to evaluate key aspects of the proposed approach. The obtained results exhibit promising performance, validating the robustness of the proposed framework.

Comparison of Sentiment Analysis Algorithms with SMOTE Oversampling and TF-IDF Implementation on Google Reviews for Public Health Centers

Sentiment analysis, or opinion mining, is a key area of natural language processing that identifies sentiments in free text. As digital business services grow and user-generated content increases, analyzing sentiments in online reviews is vital for enhancing business operations and customer satisfaction. This study focuses on sentiment analysis of user reviews from Google Reviews for Public Health Centers (PHCs) in Bali, Indonesia, using five machine learning models: Logistic Regression, Support Vector Machine (SVM), XGBoost, Naive Bayes, and Random Forest. These models classified sentiments into positive and negative categories using a dataset balanced with SMOTE to improve accuracy. We divided a total of 1.834 reviews, using 20% for testing and 80% for training, to ensure a thorough evaluation under real-world conditions. Logistic Regression and Naive Bayes performed best, both achieving an accuracy of 0.89, with Logistic Regression providing a balanced precision and recall. The study enhances academic understanding of sentiment analysis in healthcare and offers insights for business administrators on handling online customer feedback. The findings stress the importance of choosing suitable machine learning techniques based on specific data characteristics and project requirements to optimize both technological and business outcomes.

Arabic Spam Tweets Classification: A Comprehensive Machine Learning Approach

Nowadays, one of the most common problems faced by Twitter (also known as X) users, including individuals as well as organizations, is dealing with spam tweets. The problem continues to proliferate due to the increasing popularity and number of users of social media platforms. Due to this overwhelming interest, spammers can post texts, images, and videos containing suspicious links that can be used to spread viruses, rumors, negative marketing, and sarcasm, and potentially hack the user’s information. Spam detection is among the hottest research areas in natural language processing (NLP) and cybersecurity. Several studies have been conducted in this regard, but they mainly focus on the English language. However, Arabic tweet spam detection still has a long way to go, especially emphasizing the diverse dialects other than modern standard Arabic (MSA), since, in the tweets, the standard dialect is seldom used. The situation demands an automated, robust, and efficient Arabic spam tweet detection approach. To address the issue, in this research, various machine learning and deep learning models have been investigated to detect spam tweets in Arabic, including Random Forest (RF), Support Vector Machine (SVM), Naive Bayes (NB) and Long-Short Term Memory (LSTM). In this regard, we have focused on the words as well as the meaning of the tweet text. Upon several experiments, the proposed models have produced promising results in contrast to the previous approaches for the same and diverse datasets. The results showed that the RF classifier achieved 96.78% and the LSTM classifier achieved 94.56%, followed by the SVM classifier that achieved 82% accuracy. Further, in terms of F1-score, there is an improvement of 21.38%, 19.16% and 5.2% using RF, LSTM and SVM classifiers compared to the schemes with same dataset.

A Systematic Survey of Automatic Image Description Generation Systems

Automatic image description generation is an active research area in Computer Vision and Natural Language Processing. Objects, their attributes, actions, and spatial relationships are identified in the image description generation system. Earlier, these systems used classical machine learning approaches. Later majority of these works follow deep learning strategies. The essential goal of these systems is to produce syntactically and semantically correct sentences. This review aims to synthesize the studies conducted from 2010 to 2023 to get a deeper view of various image description generation systems and their applications. The prominent contribution of this review is that it covers the different aspects of image captioning systems, such as the methods used, the various applied domains, evaluation measures, and the datasets used. A single synthesized study directs scholars regarding developing image captioning systems to date utilizing machine learning approaches. It also offers suggestions for researchers in this sector for the future. Image captioning is applied in many fields like natural images, medical images, remote sensing images, videos, etc. This review paper reviews the various taxonomies of image description generation systems. We also analyzed multiple methods used in the architecture of image captioning systems. The datasets and the evaluation metrics used in these systems are discussed. We studied the performance of the system under various circumstances.

Editorial for Vol.31, No.4

The December 2023 (Vol. 31, No. 4) issue of CIT. Journal of Computing and Information Technology brings four papers from the areas of cooperative networks, network security, and natural language processing.

The Thai Discourse Treebank: Annotating and Classifying Thai Discourse Connectives

Abstract Discourse analysis is a highly applicable area of natural language processing. In English and other languages, resources for discourse-based tasks are widely available. Thai, however, has hitherto lacked such resources. We present the Thai Discourse Treebank, the first, large Thai corpus annotated in the style of the Penn Discourse Treebank. The resulting corpus has over 10,000 sentences and 18,000 instances of connectives in 33 different relations. We release the corpus alongside our list of 148 potentially polysemous discourse connectives with a total of 340 form-sense pairs and their classification criteria to facilitate future research. We also develop models for connective identification and classification tasks. Our best models achieve an F1 of 0.96 in the identification task and 0.46 on the sense classification task. Our results serve as benchmarks for future models for Thai discourse tasks.

Improvement of Multimodal Emotion Recognition Based on Temporal-Aware Bi-Direction Multi-Scale Network and Multi-Head Attention Mechanisms

Emotion recognition is a crucial research area in natural language processing (NLP), aiming to identify emotional states such as happiness, anger, and sadness from various sources like speech, text, and facial expressions. In this paper, we propose an improved MMER (multimodal emotion recognition) method using TIM-Net (Temporal-Aware Bi-Direction Multi-Scale Network) and attention mechanisms. Firstly, we introduce the methods for extracting and fusing the multimodal features. Then, we present the TIM-Net and attention mechanisms, which are utilized to enhance the MMER algorithm. We evaluate our approach on the IEMOCAP and MELD datasets, and compared to existing methods, our approach demonstrates superior performance. The weighted accuracy recall (WAR) on the IEMOCAP dataset is 83.9%, and the weighted accuracy recall rate on the MELD dataset is 62.7%. Finally, the impact of the TIM-Net model and the attention mechanism on the emotion recognition performance is further investigated through ablation experiments.

Homonym Detection Using WordNet and Modified Lesk Approach for English Language

Word sense disambiguation (WSD) is a basic and persistent problem that has existed since its inception in the natural language processing (NLP) area. The process of determining the accurate meaning of a word within a specific context is referred to as word sense disambiguation, commonly known as WSD. In NLP, a single word can have two or more meanings, with each meaning being distinguished by its context. This is known as word polysemy. Its applications span a wide range of fields, such as question answering systems, machine translation, information retrieval (IR) etc. Ontology and NLP are still struggling with ambiguity. Homonyms, which are ubiquitous in most languages, are words that have the same spelling but a different meaning. This method's fundamental premise is to select the appropriate sense by comparing a word's context in a sentence to contexts generated from WordNet. The primary goal of this study is to employ WordNet and the Lesk algorithm for WSD. After the algorithm was put into practice and tested on a collection of sentences that included ambiguous words, the synset was able to determine the proper interpretation for most of the sentences. The Lesk algorithm relies on finding the highest number of shared words (maximum overlap) between a word’s context, prepositions and the definitions for its different meanings(glosses). This approach helps in identifying the most accurate interpretation for a given word within a specific context. According to experimental findings, the suggested strategy considerably boosts performance while identifying homonyms.

The Use of Syntactic Information in fake news detection: A Systematic Review

Fake news has been a critical problem for society, to the extent that its damaging effects can already be seen in several areas, such as democracy and health. However, as fake news grow in number, manual fact-checking becomes impractical for identifying them, which makes automatic detection a compelling alternative. In this sense, this study gathers multiple solutions for the problem of automatically detecting fake news, through the usage of both lexical and syntactic information. This study consists of a systematic review on fake news detection through linguistic patterns, focusing on the use of syntax to aid in the task. Solving complex problems by capturing linguistic patterns is mostly explored in the Natural Language Processing (NLP) area. In general, the use of shallow syntax representations, such as Parts of speech, only marginally increases the performance of classifiers in this task. However, relying on deeper syntactic representations, such as context-free grammars or syntactic dependency trees, present more promising results.