Field Of Natural Language Processing Research Articles

A comprehensive analysis of static word embeddings for Turkish

Word embeddings are fixed-length, dense and distributed word representations that are used in natural language processing (NLP) applications. There are basically two types of word embedding models which are non-contextual (static) models and contextual models. The former method generates a single embedding for a word regardless of its context, while the latter method produces distinct embeddings for a word based on the specific contexts in which it appears. There are plenty of works that compare contextual and non-contextual embedding models within their respective groups in different languages. However, the number of studies that compare the models in these two groups with each other is very few and there is no such study in Turkish. This process necessitates converting contextual embeddings into static embeddings. In this paper, we compare and evaluate the performance of several contextual and non-contextual models in both intrinsic and extrinsic evaluation settings for Turkish. We make a fine-grained comparison by analyzing the syntactic and semantic capabilities of the models separately. The results of the analyses provide insights about the suitability of different embedding models in different types of NLP tasks. We also build a Turkish word embedding repository comprising the embedding models used in this work, which may serve as a valuable resource for researchers and practitioners in the field of Turkish NLP. We make the word embeddings, scripts, and evaluation datasets publicly available.

AI Call Assistant – Extractive Summarization of Call Recordings

In today's fast-paced business landscape, effective communication is a key for success. With the increasing volume of phone calls in various industries, there is a growing need for efficient call management and analysis tools. The AI Call Assistant project aims to address this need by leveraging artificial intelligence (AI) technology to summarize call recordings, enhancing productivity and decision-making processes. The core functionality of this project revolves around its ability to process audio data from call recordings and extract key insights and information. Using advanced natural language processing (NLP) techniques, the system identifies important topics, sentiments, and actions discussed during the call. The call transcripts attained from call recording pose unique challenges that are not adequately addressed by most open-source automatic text summarizers. This project aims to contribute to the field of artificial intelligence by providing efficient and effective methods to recognize call recording and summarize, offering a valuable tool for extracting insights from spoken audio efficiently.Top of Form This research aligns with the broader fields of natural language processing (NLP), machine learning, and artificial intelligence (AI), particularly in the domain of speech recognition and understanding. It also intersects with communication technology and data analytics, focusing on optimizing call management processes and extracting actionable insights from conversational data. Moreover, it aligns with the goal of improving efficiency and decision-making in various industries through the application of AI-driven solutions. Overall, this project offers a novel approach to improving call management processes through AI-driven extractive summarization, contributing to advancements in natural language processing and communication technologies. Key Words: Artificial Intelligence, BERT, Textual Summarization, Transformers, Natural Language Processing, Speech Recognition, Speech-To-Text.

Text Summarization Based on Semantic Similarity

In the contemporary information age, the sheer volume of textual data poses a significant challenge for efficient comprehension and utilization. This project endeavors to address this challenge by developing a Text Summarization System grounded in semantic similarities. The primary goal is to create a robust and intuitive tool that extracts key information from large textual datasets, offering users a concise and meaningful summary. The proposed system employs advanced Natural Language Processing (NLP) techniques to analyze the semantic relationships within the text. Rather than relying solely on syntactic structures, the model identifies and leverages semantic similarities, such as shared concepts, themes, and contextual relationships, to distill the essential content. This approach enhances the summarization process by ensuring that the generated summaries reflect a deeper understanding of the underlying semantics, thereby capturing the core meaning of the text. Throughout the development of this project, the B.Tech student will delve into the intricacies of semantic analysis, exploring techniques to recognize and prioritize key concepts. The system's effectiveness will be evaluated through rigorous testing on diverse textual datasets, assessing its ability to generate coherent and relevant summaries across various domains. This project not only contributes to the field of NLP but also has practical applications in information retrieval, document summarization, and content curation. By providing an innovative solution to the challenges of information overload, the Text Summarization System based on semantic similarities offers a valuable tool for enhancing efficiency in information processing and decision-making. Index terms Text Summarization, Semantic Similarities, Natural Language Processing (NLP), Semantic Analysis, Information Retrieval, Document Summarization, Content Curation, Information Overload, Decision Making, Textual Data Analysis, Key Concept Recognition, Conceptual Relationships, Syntactic Structures, Semantic Understanding, Textual Datasets Evalution.

ADVANCMENTS IN TEXT SUMMARIZATION AND EXTRACTIVE QUESTION- ANSWERING : A MACHINE LEARNING APPROACH

In the era of social media platforms, the rapid expansion of data mining in the fields of information retrieval and natural language processing emphasizes the crucial need for automated text summarization. At the current time, pretrained word embedding techniques and sequence to sequence models can be effectively repurposed in the realm of social network summarization to efficiently condense significant information with strong encoding capabilities. However, dealing with the challenge of extended text dependency and efficiently utilizing latent topic mapping presents an increasingly significant obstacle for these computational models. In this document, we propose a topic- focused approach for both extractive and abstractive summarization, integrating Question Answering features by employing BERT and Pegasus pretrained models. A comprehensive analysis of the architectural complexities of the pre-existing models utilized in the creation of our model has been presented. The evaluation of our Text Summarization & Question Answering Model was conducted meticulously on a variety of datasets including Multi news, XSum, and CNN/Daily Mail, with experimental results indicating its achievement of state-of-the-art performance levels based on ROGUE scores. Following this, the verification of our results through human evaluation confirms that the summaries generated by our model are in alignment with the performance standards set by humans across various datasets. Furthermore, we have integrated an Extractive Question Answering task into our model, proposing an architecture leveraging BERT and conducting comparative analyses against alternative language models to gauge its efficacy. Keywords—Natural Language Processing (NLP), Text Summarization, Question & Answering (Q&A), Transformer Models, Pegasus, BERT (Bidirectional Encoder Representations from Transformers), Neural Topic Modeling, Extractive Summarization, Abstractive Summarization, Semantic Representation, Topic Embedding

A new framework for building agricultural domain-based ontologies from text documents using natural language processing and artificial intelligence techniques

The advancements in technology, particularly in the field of Natural Language Processing (NLP) and Artificial Intelligence (AI) can be advantageous for the agricultural sector to enhance the yield. Establishing an agricultural ontology as part of the development would spur the expansion of cross-domain agriculture. Semantic and syntactic knowledge of the domain data is required for building such a domain-based ontology. To process the data from text documents, a standard technique with syntactic and semantic features are needed because the availability of pre-determined agricultural domain-based data is insufficient. In this research work, an Agricultural Ontologies Construction framework (AOC) is proposed for creating the agricultural domain ontology from text documents using NLP techniques with Robustly Optimized BERT Approach (RoBERTa) model and Graph Convolutional Network (GCN). The anaphora present in the documents are resolved to produce precise ontology from the input data. In the proposed AOC work, the domain terms are extracted using the RoBERTa model with Regular Expressions (RE) and the relationships between the domain terms are retrieved by utilizing the GCN with RE. When compared to other current systems, the efficacy of the proposed AOC method achieves an exceptional result, with precision and recall of 99.6% and 99.1% respectively.

Human Language Data Processing using NLTK

Natural Language Toolkit (NLTK) is a comprehensive Python library designed to facilitate the exploration, analysis, and processing of human language data. With its extensive collection of tools, NLTK provides researchers, developers, and educators with a powerful platform for tasks ranging from basic text processing to advanced natural language understanding and machine learning. The toolkit includes modules for tokenization, stemming, lemmatization, part-of-speech tagging, named entity recognition, syntactic parsing, semantic analysis, and more. Furthermore, NLTK offers access to numerous linguistic resources such as corpora, lexicons, and treebanks, making it an invaluable resource for both learning and research in the field of natural language processing (NLP). NLTK serves as an indispensable tool for unlocking the complexities of human language

R-Cut: Enhancing Explainability in Vision Transformers with Relationship Weighted Out and Cut.

Transformer-based models have gained popularity in the field of natural language processing (NLP) and are extensively utilized in computer vision tasks and multi-modal models such as GPT4. This paper presents a novel method to enhance the explainability of transformer-based image classification models. Our method aims to improve trust in classification results and empower users to gain a deeper understanding of the model for downstream tasks by providing visualizations of class-specific maps. We introduce two modules: the "Relationship Weighted Out" and the "Cut" modules. The "Relationship Weighted Out" module focuses on extracting class-specific information from intermediate layers, enabling us to highlight relevant features. Additionally, the "Cut" module performs fine-grained feature decomposition, taking into account factors such as position, texture, and color. By integrating these modules, we generate dense class-specific visual explainability maps. We validate our method with extensive qualitative and quantitative experiments on the ImageNet dataset. Furthermore, we conduct a large number of experiments on the LRN dataset, which is specifically designed for automatic driving danger alerts, to evaluate the explainability of our method in scenarios with complex backgrounds. The results demonstrate a significant improvement over previous methods. Moreover, we conduct ablation experiments to validate the effectiveness of each module. Through these experiments, we are able to confirm the respective contributions of each module, thus solidifying the overall effectiveness of our proposed approach.

Research Paper Summarization and Recommendation

The exponential growth of academic literature has made it increasingly challenging for researchers to keep up with the latest developments in their respective fields. This paper presents an approach for summarizing and recommending research papers to help researchers efficiently manage and navigate the vast amount of academic literature. We propose using a transformer-based model for generating abstractive summaries of research papers and a cosine similarity-based recommendation system for suggesting similar papers to a given paper. The proposed approach is evaluated on a dataset of research papers, and the results show that it is effective in generating coherent and concise summaries and recommending relevant research papers. The findings of this study have important implications for researchers and practitioners in the field of natural language processing and information retrieval.

IDENTIFICATION OF EMOTIONAL WORDS IN TEXT COMMUNICATION

The study of the identification of emotions in text communication is an actual direction of research in the field of natural language processing and machine learning. The main goal of the research is to develop algorithms and models that can automatically determine a person's emotional state based on text messages. This work is devoted to the review of some models and an algorithm for improving data processing in the middle of text communication of users.

Using transformers for multimodal emotion recognition: Taxonomies and state of the art review

Emotion recognition is an aspect of human-computer interaction, affective computing, and social robotics. Conventional unimodal approaches for emotion recognition, depending on single data sources such as facial expressions or speech signals often fall short in capturing the complexity and context-dependent nature of emotions. Multimodal Emotion Recognition (MER), which integrates information from multiple modalities, has emerged as a promising solution to overcome these limitations. In recent years, Transformers-based approaches have gathered significant attention in the fields of natural language processing and computer vision, highlighting their ability to capture long-range dependencies and semantic representations. These models have rapidly achieved the MER state-of-the-art. However, current survey papers that cover MER lack a specific focus on Transformer-based techniques. To bridge this research gap, this review paper provides a comprehensive investigation of Transformers-based approaches for MER. It explores various Transformer architectures and proposes several scenarios for using Transformers at different stages of MER process. In addition, it examines datasets suitable for MER, discusses fusion mechanisms, and introduces novel taxonomies in both MER and Transformer technologies. The review also addresses challenges and future research directions. Through this review, we aim to provide researchers with an inclusive understanding of the current state-of-the-art in Transformers-based approaches for MER, paving the way for further advancements in this rapidly developing field.

A systematic literature review of hate speech identification on Arabic Twitter data: research challenges and future directions.

The automatic speech identification in Arabic tweets has generated substantial attention among academics in the fields of text mining and natural language processing (NLP). The quantity of studies done on this subject has experienced significant growth. This study aims to provide an overview of this field by conducting a systematic review of literature that focuses on automatic hate speech identification, particularly in the Arabic language. The goal is to examine the research trends in Arabic hate speech identification and offer guidance to researchers by highlighting the most significant studies published between 2018 and 2023. This systematic study addresses five specific research questions concerning the types of the Arabic language used, hate speech categories, classification techniques, feature engineering techniques, performance metrics, validation methods, existing challenges faced by researchers, and potential future research directions. Through a comprehensive search across nine academic databases, 24 studies that met the predefined inclusion criteria and quality assessment were identified. The review findings revealed the existence of many Arabic linguistic varieties used in hate speech on Twitter, with modern standard Arabic (MSA) being the most prominent. In identification techniques, machine learning categories are the most used technique for Arabic hate speech identification. The result also shows different feature engineering techniques used and indicates that N-gram and CBOW are the most used techniques. F1-score, precision, recall, and accuracy were also identified as the most used performance metric. The review also shows that the most used validation method is the train/test split method. Therefore, the findings of this study can serve as valuable guidance for researchers in enhancing the efficacy of their models in future investigations. Besides, algorithm development, policy rule regulation, community management, and legal and ethical consideration are other real-world applications that can be reaped from this research.

Weight Averaging and re-adjustment ensemble for QRCD

Question Answering (QA) is a prominent task in the field of Natural Language Processing (NLP) with extensive applications. Recently, there has been a notable surge in research interest concerning the development of QA systems for the Holy Qur’an, an Islamic religious text. The Qur’an Reading Comprehension Dataset (QRCD) Malhas and Elsayed (2020) is a highly commendable effort in this respect. It stands as the first benchmark dataset specifically designed for a set of directly answerable questions from the Qur’an. Each question in the dataset is meticulously labeled with all potential answers sourced from the Holy Qur’an. From our perspective, the main challenge in QRCD stems from the limited volume of training data it offers. As a solution we propose an innovative approach to build a Deep Neural Network (DNN) ensemble, centered around Ara-Electra model (Antoun et al., 2021), that we called Weight Averaging and Re-adjustment (WAR) model. The model is constructed by computing running averages of all model states that evolve during a single training session and ensuring that model weights are readjusted prior to each training epoch, in order to hold it back from over fitting the training data. The scheme results in a single standalone model that exhibits the benefits of multi-model ensembles. It is distinguished from other ensembles proposed for QRCD that accumulates outputs from multiple expert models and employs classic techniques like hard voting or score averaging on output probabilities to build unified results. Each expert model costs individual training time and compute resources. The WAR model outperforms existing systems with improved generalization over unseen data. It achieves F1, partial Reciprocal Rank (pRR), and exact-match (EM) scores of 0.567, 0.60 and 0.29 respectively, exceeding best reported QRCD scores by 3%, 1.5% and 0.69% respectively. Notably, we are comparing our results with the top scores from different models, highlighting our model’s consistent performance across all three metrics.

Discontinuous Arabic frozen expressions modelization and implementation

Frozen expressions hold significant importance in the field of natural language processing, attracting considerable attention from researchers across various languages in recent years. The Arabic language, in particular, boasts a wealth of frozen expressions inherited from the pre-Islamic and early Islamic periods, with persistent usage to the present day. This linguistic richness has motivated researchers to systematically collect, classify, and elucidate these expressions. Various classifications have emerged, addressing aspects such as continuity, discontinuity, allowance for variations, and restriction from variations. Our aim is to produce lexicon-grammar tables of discontinuous Arabic frozen expressions and implement them. Our approach involves the meticulous collection and study of these expressions, followed by the transformation of their lexicon-grammar tables into dictionaries and syntactic grammars within the NooJ platform. This methodology allows us to recognize and annotate these expressions in texts and corpora, even when they exhibit discontinuity. Such recognition has the potential to address several challenges in automatic natural language processing, including the area of automatic translation.

Enhancing Arabic offensive language detection with BERT-BiGRU model

With the advent of Web 2.0, various platforms and tools have been developed to allow internet users to express their opinions and thoughts on diverse topics and occurrences. Nevertheless, certain users misuse these platforms by sharing hateful and offensive speeches, which has a negative impact on the mental health of internet society. Thus, the detection of offensive language has become an active area of research in the field of natural language processing. Rapidly detecting offensive language on the internet and preventing it from spreading is of great practical significance in reducing cyberbullying and self-harm behaviors. Despite the crucial importance of this task, limited work has been done in this field for nonEnglish languages such as Arabic. Therefore, in this paper, we aim to improve the results of Arabic offensive language detection without the need for laborious preprocessing or feature engineering work. To achieve this, we combine the bidirectional encoder representations from transformers (BERT) model model with a bidirectional gated recurrent unit (BiGRU) layer to further enhance the extracted context and semantic features. The experiments were conducted on the Arabic dataset provided by the SemEval 2020 Task 12. The evaluation results show the effectiveness of our model compared to the baseline and related work models by achieving a macro F1- score of 93.16%.

Generating images using generative adversarial networks based on text descriptions

Modern developments in the fields of natural language processing (NLP) and computer vision (CV) emphasize the increasing importance of generating images from text descriptions. The presented article analyzes and compares two key methods in this area: generative adversarial network with conditional latent semantic analysis (GAN-CLS) and ultra-long transformer network (XLNet). The main components of GAN-CLS, including the generator, discriminator, and text encoder, are discussed in the context of their functional tasks—generating images from text inputs, assessing the realism of generated images, and converting text descriptions into latent spaces, respectively. A detailed comparative analysis of the performance of GAN-CLS and XLNet, the latter of which is widely used in the organic light-emitting diode (OEL) field, is carried out. The purpose of the study is to determine the effectiveness of each method in different scenarios and then provide valuable recommendations for selecting the best method for generating images from text descriptions, taking into account specific tasks and resources. Ultimately, our paper aims to be a valuable research resource by providing scientific guidance for NLP and CV experts.

Crosslingual Transfer Learning for Arabic Story Ending Generation

In the field of natural language processing, the task of generating story endings (SEG) requires not only a deep understanding of the narrative context but also the ability to formulate coherent conclusions. This study delves into the use of crosslingual transfer learning to address the challenges posed by the scarcity of Arabic data in SEG, proposing the utilization of extensive English story corpora as a solution. We evaluated the efficacy of multilingual models, such as mBART, mT5, and mT0, in generating Arabic story endings, assessing their performance in both zero-shot and few-shot scenarios. Despite the linguistic complexities of Arabic and the inherent challenges of crosslingual transfer, our findings demonstrate the potential of these multilingual models to transcend linguistic barriers, significantly contributing to the domain of natural language processing across different languages. This research has significant implications for generating creative text and improving multilingual natural language processing in resource-limited language contexts

Hyperparameter Tuning of Long Short-Term Memory Model for Clickbait Classification in News Headlines

Abstract. The information available on the internet nowadays is diverse and moves very quickly. Information is becoming easier to obtain by the general public with the numerous online media outlets, including news portals that provide up-to-date information insights. Various news portals earn revenue from advertising using pay-per-click methods that encourage article writers to use clickbait techniques to attract visitors. However, the negative effects of clickbait include a decrease in journalism quality and the spread of hoaxes. This problem can be prevented by using text classification to classify clickbait in news titles. One method that can be used for text classification is a neural network. Artificial neural networks use algorithms that can independently adjust input coefficient weights. This makes this algorithm highly effective for modeling non-linear statistical data. The artificial neural network algorithm, especially the Long Short-Term Memory (LSTM), has been widely used in various natural language processing fields with satisfying results, including text classification. To improve the performance of the neural network model, adjustments can be made to the model's hyperparameters. Hyperparameters are parameters that cannot be obtained through data and must be defined before the training process. In this research, the Long Short-Term Memory (LSTM) model was used in clickbait classification in news titles. Sixteen neural network models were trained with different hyperparameter configurations for each model. Hyperparameter tuning was carried out using the random search algorithm. The dataset used was the CLICK-ID dataset published by William & Sari, 2020[1], with a total of 15,000 annotated data. The research results show that the developed LSTM model has a validation accuracy of 0.8030, higher than William & Sari's research, and a validation loss of 0.4876. Using this model, researchers were able to classify clickbait in news titles with fairly good accuracy. Purpose: The study was to develop and evaluate a LSTM model with hyperparameter tuning for clickbait classification on news headlines. The thesis also aims to compare the performance of simple LSTM and bidirectional LSTM for this task. Methods: This study uses CLICK-ID dataset and applies different text preprocessing techniques. The dataset later was used to build and train 16 LSTM models with different hyperparameters and evaluates them using validation accuracy and loss. This study uses random search for hyperparameter tuning. Result: The results of the study show that the best model for clickbait classification on news headlines is a bidirectional LSTM model with one layer, 64 units, 0.2 dropout rate, and 0.001 learning rate. This model achieves a validation accuracy of 0.8030 and a validation loss of 0.4876. The results also show that hyperparameter tuning using random search can improve the performance of the LSTM models by avoiding zero probabilities and finding the optimal values for the hyperparameters. Novelty: This study compares and analyzes the different preprocessing methods on text and the different configurations of the models to find the best model for clickbait classification on news headlines. The study also uses hyperparameter tuning to tune the model into the best model and finding the optimal values for the hyperparameters.

Adoption and impacts of generative artificial intelligence: Theoretical underpinnings and research agenda

Large language models (LLMs) have received considerable interest in the field of natural language processing (NLP) owing to their remarkable ability to generate clear, consistent, and contextually relevant materials. Among the numerous LLMs, ChatGPT (Generative Pre-trained Transformer for Chatbots) is emerging as a prominent prospective tool for developing conversational agents such as chatbots. However, there is a need for a clear conceptual understanding of ChatGPT's potential implications for the industry and its role in marketing. This study explores the adoption of ChatGPT in marketing and examines theories that may influence its adoption by marketers and consumers, as well as its implications for marketers. This study discusses how ChatGPT may allow for more personalized and engaging content, better customer experience, and improved ROI. However, adoption also brings challenges, including ethical considerations and the need for new skill development. This study also discusses future research opportunities for the adoption of ChatGPT and other generative artificial intelligence technologies in marketing. The goal is to provide insights for organizations that consider implementing these technologies, and to contribute to the literature on the adoption of Artificial Intelligence (AI) and the use of Generative AI in marketing.

ADAPTIVE DOMAIN-SPECIFIC NAMED ENTITY RECOGNITION METHOD WITH LIMITED DATA

The ever-evolving volume of digital information requires the development of innovative search strategies aimed at obtaining the necessary data efficiently and economically feasible. The urgency of the problem is emphasized by the growing complexity of information landscapes and the need for fast data extraction methodologies. In the field of natural language processing, named entity recognition (NER) is an essential task for extracting useful information from unstructured text input for further classification into predefined categories. Nevertheless, conventional methods frequently encounter difficulties when confronted with a limited amount of labeled data, posing challenges in real-world scenarios where obtaining substantial annotated datasets is problematic or costly. In order to address the problem of domain-specific NER with limited data, this work investigates NER techniques that can overcome these constraints by continuously learning from newly collected information on pre-trained models. Several techniques are also used for making the greatest use of the limited labeled data, such as using active learning, exploiting unlabeled data, and integrating domain knowledge. Using domain-specific datasets with different levels of annotation scarcity, the fine-tuning process of pre-trained models, such as transformer-based models (TRF) and Toc2Vec (token-to-vector) models is investigated. The results show that, in general, expanding the volume of training data enhances most models' performance for NER, particularly for models with sufficient learning ability. Depending on the model architecture and the complexity of the entity label being learned, the effect of more data on the model's performance can change. After increasing the training data by 20%, the LT2V model shows the most balanced growth in accuracy overall by 11% recognizing 73% of entities and processing speed. Meanwhile, with consistent processing speed and the greatest F1-score, the Transformer-based model (TRF) shows promise for effective learning with less data, achieving 74% successful prediction and a 7% increase in performance after expanding the training data to 81%. Our results pave the way for the creation of more resilient and efficient NER systems suited to specialized domains and further the field of domain-specific NER with sparse data. We also shed light on the relative merits of various NER models and training strategies, and offer perspectives for future research.

“An Extensive study of Symantic and Syntatic Approaches to Automatic Text Summarization”

Automatic text summarization (ATS) has emerged as a crucial research domain in the discipline of natural language processing (NLP) and information retrieval. The exponential growth of digital content has necessitated the need for efficient techniques that can automatically generate concise and informative summaries from lengthy documents. This article provided a comprehensive recap of automatic text summarization, covering both abstractive and extractive methods. Using extractive techniques, prime phrases or keywords from the original text are identified and chosen, while abstractive methods involve producing summaries by paraphrasing and synthesizing content in a more human-like manner. Discussed the advantages and limitations of each approach, including the challenge of ATS, which arises when summarizing content from external sources. Furthermore, reviews common evaluation metrics used for assessing the quality of summaries and discusses recent advancements in neural network-based approaches for text summarization. This survey aims to provide an overview of automatic text summarization which acts as a useful resource for researchers and practitioners in the fields of information retrieval and NLP.