Field Of Natural Language Processing Research Articles

Initial Exploration of the Transformation of Technology Museums Empowered by Artificial Intelligence

Traditional science and technology museums are facing challenges in the rapid advancements in science and technology. These challenges include limited exhibition formats, outdated methods of conveying information, and inefficient resource management. However, with the rapid development of artificial intelligence technology, there is an opportunity to overcome these challenges. Artificial intelligence technology, specifically in the field of natural language processing, has made significant progress in improving multilingual understanding and generative models. This advancement can greatly enhance the learning experience for diverse audiences. In addition, the development of computer vision technology focuses on scene understanding, video analysis, and visual inference. These advancements can provide a more immersive and engaging exhibition experience at science and technology museums. This paper aims to explore how traditional science and technology museums can integrate and redesign artificial intelligence technology to better meet the needs of their audiences. By doing so, it can improve the effectiveness of science and technology communication and enable museums to adapt more flexibly to the rapidly changing science and technology landscape. The findings of this study can provide strategic considerations and guidance for the future development of science and technology museums.

Performance of Generative Pretrained Transformer on the National Medical Licensing Examination in Japan.

The remarkable performance of ChatGPT, launched in November 2022, has significantly impacted the field of natural language processing, inspiring the application of large language models as supportive tools in clinical practice and research worldwide. Although GPT-3.5 recently scored high on the United States Medical Licensing Examination, its performance on medical licensing examinations of other nations, especially non-English speaking nations, has not been sufficiently evaluated. This study assessed GPT's performance on the National Medical Licensing Examination (NMLE) in Japan and compared it with the actual minimal passing rate for this exam. In particular, the performances of both the GPT-3.5 and GPT-4 models were considered for the comparative analysis. We initially used the GPT models and several prompts for 290 questions without image data from the 116th NMLE (held in February 2022 in Japan) to maximize the performance for delivering correct answers and explanations of the questions. Thereafter, we tested the performance of the best GPT model (GPT-4) with optimized prompts on a dataset of 262 questions without images from the latest 117th NMLE (held in February 2023). The best model with the optimized prompts scored 82.7% for the essential questions and 77.2% for the basic and clinical questions, both of which sufficed the minimum passing scoring rates of 80.0% and 74.6%, respectively. After an exploratory analysis of 56 incorrect answers from the model, we identified the three major factors contributing to the generation of the incorrect answers-insufficient medical knowledge, information on Japan-specific medical system and guidelines, and mathematical errors. In conclusion, GPT-4 with our optimized prompts achieved a minimum passing scoring rate in the latest 117th NMLE in Japan. Beyond its original design of answering examination questions for humans, these artificial intelligence (AI) models can serve as one of the best "sidekicks" for solving problems and addressing the unmet needs in the medical and healthcare fields.

The Role of Typological Feature Prediction in NLP and Linguistics

Abstract Computational typology has gained traction in the field of Natural Language Processing (NLP) in recent years, as evidenced by the increasing number of papers on the topic and the establishment of a Special Interest Group on the topic (SIGTYP), including the organization of successful workshops and shared tasks. A considerable amount of work in this sub-field is concerned with prediction of typological features, for example, for databases such as the World Atlas of Language Structures (WALS) or Grambank. Prediction is argued to be useful either because (1) it allows for obtaining feature values for relatively undocumented languages, alleviating the sparseness in WALS, in turn argued to be useful for both NLP and linguistics; and (2) it allows us to probe models to see whether or not these typological features are encapsulated in, for example, language representations. In this article, we present a critical stance concerning prediction of typological features, investigating to what extent this line of research is aligned with purported needs—both from the perspective of NLP practitioners, and perhaps more importantly, from the perspective of linguists specialized in typology and language documentation. We provide evidence that this line of research in its current state suffers from a lack of interdisciplinary alignment. Based on an extensive survey of the linguistic typology community, we present concrete recommendations for future research in order to improve this alignment between linguists and NLP researchers, beyond the scope of typological feature prediction.

EGLR: Two-staged Explanation Generation and Language Reasoning framework for commonsense question answering

In the field of natural language processing, strong commonsense reasoning abilities are often required for intelligent systems to excel in the task of commonsense question answering (QA). To enhance the interpretability of QA systems, a natural approach is to generate textual explanations in addition to the predicted answers, ensuring the understandability of the results. Recent efforts address this by using a prompted language model (LM) with frozen parameters to generate explanations in a few-shot manner. These explanations are then used as additional context to guide a finetuned LM in making the final decision. However, these methods still underutilize the semantic information embedded in the explanatory text. Consequently, the reasoning models tend to rely on word co-occurrence and the knowledge stored in the model itself rather than fully exploiting the explanations. Therefore, we propose a two-staged Explanation Generation and Language Reasoning Framework (EGLR), Our framework takes advantage of the in-context learning capability of LMs to generate explanations and reformulates the reasoning task based on explanations as a semantic matching problem. Through joint prompting and training, our model can select the most appropriate explanation by comparing multiple explanations. Experimental results on three public datasets demonstrate that our framework achieves superior performance on the full dataset while maintaining performance on out-of-domain scenarios.

Ensemble Classifier for Hindi Hostile Content Detection

Detection of hostile content from social media posts (Facebook, Twitter, etc.) is a demanding task in the field of Natural Language Processing. The increase of hostile content in different electronic media has opened up new challenges in language understanding. It becomes more difficult in regional languages. AI-based solutions are required to identify hostile content on a large scale. Although a satisfactory amount of research has been carried out in the English language, finding hostile content in regional languages is still under development due to the unavailability of suitable datasets and tools. In terms of the number of speakers, Hindi ranks third in the world and first on the Indian subcontinent. The objective of this article is to design a hostile content detection system in Hindi using coarse-grained (binary) classification and fine-grained (multi-class, multi-label) classification. We note that different baseline learning methods with different pre-trained language models perform differently. Using the Constraint 2021 Hindi Dataset, this research proposes a Bidirectional Encoder Representations from Transformers–(BERT) based contextual embedding technique with a concatenation of emoji2vec embeddings to classify social media posts in Hindi Devanagari script as hostile or non-hostile. Additionally, for the fine-grained tasks where hostile posts are sub-categorized as defamation, fake, hate, and offensive, we develop an ensemble classifier varying different learning methods and embedding models. With an F1-Score of 0.9721, it is found that our proposed Indic-BERT+emoji model outperforms the baseline model and other existing models for the coarse-grained task. We have also observed that our proposed ensemble method provides better results than the existing models and the baseline model for the fine-grained tasks with F1-Scores of 0.43, 0.82, 0.58, and 0.62 for the defamation, fake, hate, and offensive classes, respectively. The code and the data are available at https://github.com/skarifahmed/hostile .

Evaluating machine translation of literature through rhetorical analysis

This paper looks at how well ChatGPT and DeepL, two AI tools, translate literary works. Not only can ChatGPT translate text, but it can also carry out other jobs. DeepL is a service that performs computer translation and uses neural networks. The paper looks at how ChatGPT and DeepL translate books, poems, and dialogues compared to translations done by humans. The paper also talks about the pros and cons of using machine translation for literary reasons, including issues of creativity, style, and adapting to different cultures. The paper uses both new and old studies on machine translation technologies and how they work with human translation. The paper comes to the conclusion that ChatGPT and DeepL are useful but imperfect tools for translating literature, and they require human review and improvement. The paper adds to the fields of machine translation and natural language processing by looking at how two cutting-edge AI tools, ChatGPT and DeepL, can be used to translate literary works. The paper also adds to literature studies and digital humanities by looking into what machine translation can and can't do for creative writing and dialog systems. The goal of the paper is to encourage researchers, translators, writers, and users from different fields to work together and talk to each other. ⁤

Analysis of Abstractive and Extractive Summarization Methods

This paper explains the existing approaches employed for (automatic) text summarization. The summarizing method is part of the natural language processing (NLP) field and is applied to the source document to produce a compact version that preserves its aggregate meaning and key concepts. On a broader scale, approaches for text-based summarization are categorized into two groups: abstractive and extractive. In abstractive summarization, the main contents of the input text are paraphrased, possibly using vocabulary that is not present in the source document, while in extractive summarization, the output summary is a subset of the input text and is generated by using the sentence ranking technique. In this paper, the main ideas behind the existing methods used for abstractive and extractive summarization are discussed broadly. A comparative study of these methods is also highlighted.

Explore the Development Status of Artificial Intelligence and the Application Analysis of Specific Fields

In the wave of scientific and technological development in the new era, artificial intelligence technology has increasingly become one of the hot scientific and technological fields of social development. Since the concept was proposed, the development of artificial intelligence has also virtually stimulated the market and business strategies in a variety of industry fields. Starting from the actual situation, this paper now divides the four development stages of artificial intelligence from the past to the future in detail and uses the small-range questionnaire survey method to understand the current Chinese society's understanding of and interest in artificial intelligence and its products. Then, literature analysis is used to search and read relevant materials and literature, and the development status, capabilities, and prospects of artificial intelligence in the field of natural language processing and machine learning large models and image and face recognition are analyzed and summarized. In the field of face recognition selected popular processing methods and material library, simple simulation of mainstream recognition options and means, through further examples to help readers understand the development of artificial intelligence status and direction. Understanding the information and relevant viewpoints and attitudes in the specific field, detailing the logic behind the inability of artificial intelligence to replace human beings and the technical difficulties and challenges faced, is of great value to improving the popularization of the concept of artificial intelligence and its development and progress in the future society.

Few-shot learning for automated content analysis: Efficient coding of arguments and claims in the debate on arms deliveries to Ukraine

Pre-trained language models (PLM) based on transformer neural networks developed in the field of natural language processing (NLP) offer great opportunities to improve automatic content analysis in communication science, especially for the coding of complex semantic categories in large datasets via supervised machine learning. However, three characteristics so far impeded the widespread adoption of the methods in the applying disciplines: the dominance of English language models in NLP research, the necessary computing resources, and the effort required to produce training data to fine-tune PLMs. In this study, we address these challenges by using a multilingual transformer model in combination with the adapter extension to transformers, and few-shot learning methods. We test our approach on a realistic use case from communication science to automatically detect claims and arguments together with their stance in the German news debate on arms deliveries to Ukraine. In three experiments, we evaluate (1) data preprocessing strategies and model variants for this task, (2) the performance of different few-shot learning methods, and (3) how well the best setup performs on varying training set sizes in terms of validity, reliability, replicability and reproducibility of the results. We find that our proposed combination of transformer adapters with pattern exploiting training provides a parameter-efficient and easily shareable alternative to fully fine-tuning PLMs. It performs on par in terms of validity, while overall, provides better properties for application in communication studies. The results also show that pre-fine-tuning for a task on a near-domain dataset leads to substantial improvement, in particular in the few-shot setting. Further, the results indicate that it is useful to bias the dataset away from the viewpoints of specific prominent individuals.

Word Embedding as a Semantic Feature Extraction Technique in Arabic Natural Language Processing: An Overview

Feature extraction has transformed the field of Natural Language Processing (NLP) by providing an effective way to represent linguistic features. Various techniques are utilised for feature extraction, such as word embedding. This latter has emerged as a powerful technique for semantic feature extraction in Arabic Natural Language Processing (ANLP). Notably, research on feature extraction in the Arabic language remains relatively limited compared to English. In this paper, we present a review of recent studies focusing on word embedding as a semantic feature extraction technique applied in Arabic NLP. The review primarily includes studies on word embedding techniques applied to the Arabic corpus. We collected and analysed a selection of journal papers published between 2018 and 2023 in this field. Through our analysis, we categorised the different feature extraction techniques, identified the Machine Learning (ML) and/or Deep Learning (DL) algorithms employed, and assessed the performance metrics utilised in these studies. We demonstrate the superiority of word embeddings as a semantic feature representation in ANLP. We compare their performance with other feature extraction techniques, highlighting the ability of word embeddings to capture semantic similarities, detect contextual associations, and facilitate a better understanding of Arabic text. Consequently, this article provides valuable insights into the current state of research in word embedding for Arabic NLP.

Quantum-Accelerated Hyperparameter Tuning for Dynamic NLP Models

In the rapidly evolving field of natural language processing (NLP), performance optimization of large-scale NLP models is crucial. Through the application of Quantum-Accelerated Hyperparameter Tuning (QAHT), this abstract introduces a novel approach to addressing this issue. Our proposed framework leverages quantum computing capabilities to dynamically optimize NLP model hyperparameters in real-time, catering to the ever-changing character of textual data streams. Traditional hyperparameter optimisation methods usually rely on laborious grid searches or random exploration, which may not be suitable for dynamic NLP jobs. Contrarily, QAHT uses Quantum Neural Network (QNN) architectures that have been specially designed for hyperparameter optimisation. These QNNs improve performance and efficacy by dynamically modifying and improving model configurations. This abstract discusses the key elements of the QAHT architecture, including real-time model deployment, adaptive learning, and continuous data stream processing. In addition to speeding up the hyperparameter optimisation process, QAHT makes sure that NLP models are still flexible and responsive to shifts in the sentiment of the data and its distribution. This method has applications beyond NLP since it provides a foundation for effectively optimising machine learning models in complex, real-time situations. As quantum computing develops, QAHT represents a promising future in machine learning, where quantum-enhanced capabilities satisfy the needs of contemporary data-driven applications.

English-Arabic Phonetic Dataset construction

In the field of natural language processing, the effectiveness of a semantic similarity task is significantly influenced by the presence of an extensive corpus. While numerous monolingual corpora exist, predominantly in English, the availability of multilingual resources remains quite restricted. In this study, we present a semi- automated framework designed for generating a multilingual phonetic English- Arabic corpus, specifically tailored for application in multilingual phonetically and semantic similarity tasks. The proposed model consists of four phases: data gathering, preprocessing and translation, extraction IPA representation, and manual correction. Four datasets were used one of them was constructed from many sources. A manual correction was used at all the levels of the system to produce a golden standard dataset. The final dataset was in the form (English Word, English Phonetic, equivalent Arabic Word, and Arabic Phonetic). Also, a deep learning approach was used for extracting International Phonetic Alphabet (IPA) phonetic representation where the results for 13400 samples show that the Phonetic Error Rate (PER) and accuracy were 11.96% and 88.04 % respectively which are good results for producing IPA representation for unknown English and Arabic names.

Test

Feature extraction has transformed the field of Natural Language Processing (NLP) by providing an effective way to represent linguistic features. Various techniques are utilised for feature extraction, such as word embedding. This latter has emerged as a powerful technique for semantic feature extraction in Arabic Natural Language Processing (ANLP). Notably, research on feature extraction in the Arabic language remains relatively limited compared to English. In this paper, we present a review of recent studies focusing on word embedding as a semantic feature extraction technique applied in Arabic NLP. The review primarily includes studies on word embedding techniques applied to the Arabic corpus. We collected and analysed a selection of journal papers published between 2018 and 2023 in this field. Through our analysis, we categorised the different feature extraction techniques, identified the Machine Learning (ML) and/or Deep Learning (DL) algorithms employed, and assessed the performance metrics utilised in these studies. We demonstrate the superiority of word embeddings as a semantic feature representation in ANLP. We compare their performance with other feature extraction techniques, highlighting the ability of word embeddings to capture semantic similarities, detect contextual associations, and facilitate a better understanding of Arabic text. Consequently, this article provides valuable insights into the current state of research in word embedding for Arabic NLP.

Word Embedding as a Semantic Feature Extraction Technique in Arabic Natural Language Processing: An Overview

Feature extraction has transformed the field of Natural Language Processing (NLP) by providing an effective way to represent linguistic features. Various techniques are utilised for feature extraction, such as word embedding. This latter has emerged as a powerful technique for semantic feature extraction in Arabic Natural Language Processing (ANLP). Notably, research on feature extraction in the Arabic language remains relatively limited compared to English. In this paper, we present a review of recent studies focusing on word embedding as a semantic feature extraction technique applied in Arabic NLP. The review primarily includes studies on word embedding techniques applied to the Arabic corpus. We collected and analysed a selection of journal papers published between 2018 and 2023 in this field. Through our analysis, we categorised the different feature extraction techniques, identified the Machine Learning (ML) and/or Deep Learning (DL) algorithms employed, and assessed the performance metrics utilised in these studies. We demonstrate the superiority of word embeddings as a semantic feature representation in ANLP. We compare their performance with other feature extraction techniques, highlighting the ability of word embeddings to capture semantic similarities, detect contextual associations, and facilitate a better understanding of Arabic text. Consequently, this article provides valuable insights into the current state of research in word embedding for Arabic NLP.

Hybrid Actor-Action Relation Extraction: A Machine Learning Approach

Software architects and developers face challenges while trying to explain the complex functional relationships between actors and systems in the process of system design. The field of natural language processing (NLP) and information extraction is always growing. One of the biggest problems that comes up time and time again is finding relationships between entities and actions in textual data. Actor-action relation extraction (AARE) is a hybrid model that combines NLP with rule-based and machine-learning models. It effectively analyzes unstructured data and takes into account different contextual factors, saving time and reducing errors. The proposed approach extracts Actors, actions, entities, and relationships (AARE) from natural language text more accurately and completely using machine learning techniques and rule-based systems. The hybrid model handles unstructured input and adapts to changing linguistic signals using machine learning. The hybrid approach uses Named Entity Recognition, rule-based extraction, and machine learning principles to convert unstructured data into structured format. It uses tokenization, part-of-speech tagging, nlp, and semantic role labeling for relationship categorization. The model has a 93% accuracy rate and is effective in extracting actor- action relations. Future research should focus on improving rule-based techniques, semantic learning, and addressing complexities in UML diagrams.

Context-Aware Attentive Multilevel Feature Fusion for Named Entity Recognition.

In the era of information explosion, named entity recognition (NER) has attracted widespread attention in the field of natural language processing, as it is fundamental to information extraction. Recently, methods of NER based on representation learning, e.g., character embedding and word embedding, have demonstrated promising recognition results. However, existing models only consider partial features derived from words or characters while failing to integrate semantic and syntactic information, e.g., capitalization, inter-word relations, keywords, and lexical phrases, from multilevel perspectives. Intuitively, multilevel features can be helpful when recognizing named entities from complex sentences. In this study, we propose a novel attentive multilevel feature fusion (AMFF) model for NER, which captures the multilevel features in the current context from various perspectives. It consists of four components to, respectively, capture the local character-level (CL), global character-level (CG), local word-level (WL), and global word-level (WG) features in the current context. In addition, we further define document-level features crafted from other sentences to enhance the representation learning of the current context. To this end, we introduce a novel context-aware attentive multilevel feature fusion (CAMFF) model based on AMFF, to fully leverage document-level features from all the previous inputs. The obtained multilevel features are then fused and fed into a bidirectional long short-term memory (BiLSTM)-conditional random field (CRF) network for the final sequence labeling. Extensive experiments on four benchmark datasets demonstrate that our proposed AMFF and CAMFF models outperform a set of state-of-the-art baseline methods and the features learned from multiple levels are complementary.

REVOLUTIONIZING INTERACTION: EXPLORING THE IMPACT OF CHATGPT IN EDUCATION

This study investigates the transformative influence of ChatGPT in education, exploring its impact on administrative processes and academic support. Utilizing the GPT-3.5 architecture, ChatGPT has emerged as a paradigm-shifting language model, reshaping interactions in the ever-evolving field of natural language processing. Delving into its applications, the research showcases ChatGPT's commendable performance in economic analyses while addressing challenges in mathematical domains. The core focus centers on its dynamic integration into the educational landscape and the paper navigates beyond foundational models to assess ChatGPT's practical manifestations in educational settings. A rapid review underscores its potential in elevating administrative services and academic support, presenting both opportunities and challenges that necessitate institutional adjustments. The study employs a mixed-methods approach, combining qualitative and quantitative methodologies, to holistically understand ChatGPT's effectiveness. Case studies from diverse educational contexts, including the United States, India, Singapore, and Australia, enrich the exploration, providing nuanced insights into the model's multifaceted role. Through this scholarly inquiry, the paper contributes to the discourse on integrating advanced language models like ChatGPT for the enhancement of educational processes globally.

A Reinforcement Learning Approach for Intelligent Conversational Chatbot For Enhancing Mental Health Therapy

Mental health therapy plays a crucial role in addressing the growing challenges of mental health disorders. However, barriers such as limited accessibility and social stigma often hinder individuals from seeking timely and effective support. Advancement in the field of Artificial Intelligence (AI) and Natural Language Processing (NLP) has enabled the development of Chatbots which can simulate counseling conversations and provide support to those who seek help. However, existing chatbots often lack contextual understanding, limit user responses to multiple-choice questions, and face challenges in generating human-like responses. In this work, we present an Intelligent Conversational Agent using Reinforcement Learning with Human Feedback (RLHF) for mental health therapy support. The proposed system employs reinforcement learning to optimize its responses and adapts its conversational strategies to better understand user's responses, analyze the emotions expressed, and provide personalized therapeutic interventions to enhance Mental Health Therapy.

A Novel Model Combining Transformer and Bi-LSTM for News Categorization

News categorization (NC), the aim of which is to identify distinct categories of news through analyzing the contents, has acquired substantial progress since deep learning was introduced into the natural language processing (NLP) field. As a state-of-art model, transformer’s classification performance is not satisfied compared with recurrent neural network (RNN) and convolutional neural network (CNN) if it does not get pretrained. Based on the transformer model, this article proposes a novel framework that combines bidirectional long short-term memory (Bi-LSTM) network and transformer to solve this problem. In the suggested framework, the self-attention mechanism is substituted with Bi-LSTM to capture the semantic information from sentences. Meanwhile, an attention mechanism model is applied to focus on those important words and adjust their weights to solve the problem of long-distance information loss. With pooling network, the network complexity can be reduced and the main features can be highlighted by halving the dimension of the hidden state. Finally, after acquiring the hidden representation by the above structures, we utilize a contraction network to further capture the long-range associations from a text. Experiments on three large-scale corpora were performed to evaluate the suggested framework, and the results demonstrate that our model outperforms other models such as deep pyramid CNN (DPCNN), transformer.

Pretrained Quantum-Inspired Deep Neural Network for Natural Language Processing.

Natural language processing (NLP) may face the inexplicable "black-box" problem of parameters and unreasonable modeling for lack of embedding of some characteristics of natural language, while the quantum-inspired models based on quantum theory may provide a potential solution. However, the essential prior knowledge and pretrained text features are often ignored at the early stage of the development of quantum-inspired models. To attacking the above challenges, a pretrained quantum-inspired deep neural network is proposed in this work, which is constructed based on quantum theory for carrying out strong performance and great interpretability in related NLP fields. Concretely, a quantum-inspired pretrained feature embedding (QPFE) method is first developed to model superposition states for words to embed more textual features. Then, a QPFE-ERNIE model is designed by merging the semantic features learned from the prevalent pretrained model ERNIE, which is verified with two NLP downstream tasks: 1) sentiment classification and 2) word sense disambiguation (WSD). In addition, schematic quantum circuit diagrams are provided, which has potential impetus for the future realization of quantum NLP with quantum device. Finally, the experiment results demonstrate QPFE-ERNIE is significantly better for sentiment classification than gated recurrent unit (GRU), BiLSTM, and TextCNN on five datasets in all metrics and achieves better results than ERNIE in accuracy, F1-score, and precision on two datasets (CR and SST), and it also has advantage for WSD over the classical models, including BERT (improves F1-score by 5.2 on average) and ERNIE (improves F1-score by 4.2 on average) and improves the F1-score by 8.7 on average compared with a previous quantum-inspired model QWSD. QPFE-ERNIE provides a novel pretrained quantum-inspired model for solving NLP problems, and it lays a foundation for exploring more quantum-inspired models in the future.