Natural Language Techniques Research Articles

Detection of news written by the ChatGPT through authorship attribution performed by a bidirectional LSTM model

The large language based-model chatbot ChatGPT gained a lot of popularity since its launch and has been used in a wide range of situations. This research centers around a particular situation, when the ChatGPT is used to produce news that will be consumed by the population, causing the facilitation in the production of fake news, spread of misinformation and lack of trust in news sources. Aware of these problems, this research aims to build an artificial intelligence model capable of performing authorship attribution on news articles, identifying the ones written by the ChatGPT. To achieve this goal, a dataset containing equal amounts of human and ChatGPT written news was assembled and different natural processing language techniques were used to extract features from it that were used to train, validate and test three models built with different techniques. The best performance was produced by the Bidirectional Long Short Term Memory (LSTM) Neural Network model, achieving 91.57% accuracy when tested against the data from the testing set.

Contextual embedded text summarizer system: A hybrid approach

AbstractSelecting crucial sentences from a document is a pivotal task in automatic text summarization systems. Abstractive summarization involves rephrasing key content through advanced natural language techniques, generating a concise, new text conveying critical information. Conversely, extractive summarization reproduces important material from the original text. In the proposed method, a hybrid ensemble approach combines BERTsum for extractive summarization and Longformer2Roberta for abstractive summarization for generating a contextual semantic rich summary for a huge collection of text. These proposed system‐generated summaries were evaluated against reference summaries using the ROUGE package at three rouge levels (Rouge‐1, Rouge‐2, and Rouge‐L). The Proposed contextual embedded hybrid text summarization model has shown significant performance improvement in multiple levels of Rouge score and word mover distance (WMD) of generated summary with a reference summary. The proposed hybrid model demonstrates superior performance over existing state‐of‐the‐art summarizing models on three distinct datasets CNN dataset, WikiSum, and Gigaword dataset. The proposed hybrid model as a text summarizer involves leveraging its capabilities to process longer sequences of text with domain‐specific contextual summaries. This transformers‐based text summarization model has great potential in developing expert systems in various research domains such as health decision support systems, the education sector, customer support chatbots, financial analysis investment recommendations, and financial assistance.

Deciphering 3'UTR Mediated Gene Regulation Using Interpretable Deep Representation Learning.

The 3' untranslated regions (3'UTRs) of messenger RNAs contain many important cis-regulatory elements that are under functional and evolutionary constraints. It is hypothesized that these constraints are similar to grammars and syntaxes in human languages and can be modeled by advanced natural language techniques such as Transformers, which has been very effective in modeling complex protein sequence and structures. Here 3UTRBERT is described, which implements an attention-based language model, i.e., Bidirectional Encoder Representations from Transformers (BERT). 3UTRBERT is pre-trained on aggregated 3'UTR sequences of human mRNAs in a task-agnostic manner; the pre-trained model is then fine-tuned for specific downstream tasks such as identifying RBP binding sites, m6A RNA modification sites, and predicting RNA sub-cellular localizations. Benchmark results show that 3UTRBERT generally outperformed other contemporary methods in each of these tasks. More importantly, the self-attention mechanism within 3UTRBERT allows direct visualization of the semantic relationship between sequence elements and effectively identifies regions with important regulatory potential. It is expected that 3UTRBERT model can serve as the foundational tool to analyze various sequence labeling tasks within the 3'UTR fields, thus enhancing the decipherability of post-transcriptional regulatorymechanisms.

On improving conversational interfaces in educational systems

Conversational Intelligent Tutoring Systems (CITS) have drawn increasing interest in education because of their capacity to tailor learning experiences, improve user engagement, and contribute to the effective transfer of knowledge. Conversational agents employ advanced natural language techniques to engage in a convincing human-like tutorial conversation. In solving math word problems, a significant challenge arises in enabling the system to understand user utterances and accurately map extracted entities to the essential problem quantities required for problem-solving, despite the inherent ambiguity of human natural language. In this study, we propose two possible approaches to enhance the performance of a particular CITS designed to teach learners to solve arithmetic–algebraic word problems. Firstly, we propose an ensemble approach to intent classification and entity extraction, which combines the predictions made by two distinct individual models that use constraints defined by human experts. This approach leverages the intertwined nature of the intents and entities to yield a comprehensive understanding of the user’s utterance, ultimately aiming to enhance semantic accuracy. Secondly, we introduce an adapted Term Frequency-Inverse Document Frequency technique to associate entities with problem quantity descriptions. The evaluation was conducted on the AWPS and MATH-HINTS datasets, containing conversational data and a collection of arithmetical and algebraic math problems, respectively. The results demonstrate that the proposed ensemble approach outperforms individual models, and the proposed method for entity–quantity matching surpasses the performance of typical text semantic embedding models.

COMPARISON OF DEEP LEARNING TECHNIQUES FOR TEXTUAL SENTIMENT ANALYSIS

Examining the opinions of the people might provide us with valuable knowledge. Sentiment analysis is a method for analyzing textual data that helps find subjective information, such as opinions and feelings that individuals or groups have expressed. It improves our understanding of human language using deep learning and natural language techniques. Several deep learning models, including RNNs, LSTMs, GRUs, and their bidirectional variants, are compared in this work. Three publicly available datasets - the imdb_reviews, Twitter Sentiment Dataset, and Emotions dataset were used in the investigation. Accuracy performance is evaluated for six deep learning models. According to experimental studies, bidirectional structures outperform their unidirectional counterparts in most cases. Across several datasets, the bidirectional models continuously produced the best accuracy.

Unveiling Gender Biases in Recruitment: A Natural Language Processing Approach

This paper investigates the potential of AI to identify gender biases in recruitment for senior management positions in businesses, dealing with many documents. It aims to unravel the impact of gender biases in job advertisements as a possible reason behind the underrepresentation of women in the corporate world. An innovative experiment that extracts and analyses 2.198 job offers published in February and September 2021 in the Financial Times newspaper is presented. Natural language techniques are used. These methods identify the most frequent terms and their appearance rate in the advertisements showing the gender biases they generate. By enabling the analysis of many documents, the method allows the accurate identification of gender biases. This use is unique in management studies. The results show a strong co-occurrence of terms associated with male roles in the studied sectors. The concept of agentic-communal role differentiation, rooted in the Identity, homosocial, and TM-TM theories, supports the findings. This knowledge will contribute to using NLP to discover gender biases in recruitment for high decision-making positions proposing action to improve the present situation. Should more women ascend to decision-making positions, selection processes should be improved to reflect a more neutral language. At the same time, cultural changes should be promoted in the corporate world toward more inclusive workplaces.

Sentimental analysis of audio based customer reviews without textual conversion

The current trends or procedures followed in the customer relation management system (CRM) are based on reviews, mails, and other textual data, gathered in the form of feedback from the customers. Sentiment analysis algorithms are deployed in order to gain polarity results, which can be used to improve customer services. But with evolving technologies, lately reviews or feedbacks are being dominated by audio data. As per literature, the audio contents are being translated to text and sentiments are analyzed using natural processing language techniques. However, these approaches can be time consuming. The proposed work focuses on analyzing the sentiments on the audio data itself without any textual conversion. The basic sentiment analysis polarities are mostly termed as positive, negative, and natural. But the focus is to make use of basic emotions as the base of deciding the polarity. The proposed model uses deep neural network and features such as Mel frequency cepstral coefficients (MFCC), Chroma and Mel Spectrogram on audio-based reviews.

Investigating Natural Language Techniques for Accurate Noun and Verb Extraction

Natural language processing (NLP) has witnessed significant advancements in recent decades. Automatically classifying parts of speech, like nouns and verbs, from textual input has transformed text analysis and language understanding. Using natural language processing techniques, we explore various methods for identifying noun and verb phrases automatically, with an emphasis on high accuracy. Our study explores rule-based, statistical, and Machine Learning (ML) approaches for determining the nouns and verbs from sentences. The effectiveness of these approaches is clearly evident, especially when NLP libraries such as SpaCy and the Natural Language Toolkit (NLTK) are used. As well as demonstrating their potential applications across diverse language processing tasks and industries, we conduct comparative research to showcase their advantages and disadvantages. The performance of these methods is also examined in terms of retrieving subject and action terms. SpaCy achieves an impressive accuracy of 95% in noun and verb extraction, while Part-Of-Speech (POS) technology tagging delivers an even higher accuracy of 96%. The results obtained with these methods illustrate how nouns, verbs, and names can be classified in text successfully.

Reusability report: Learning the transcriptional grammar in single-cell RNA-sequencing data using transformers

The rise of single-cell genomics is an attractive opportunity for data-hungry machine learning algorithms. The scBERT method, inspired by the success of BERT (‘bidirectional encoder representations from transformers’) in natural language processing, was recently introduced by Yang et al. as a data-driven tool to annotate cell types in single-cell genomics data. Analogous to contextual embedding in BERT, scBERT leverages pretraining and self-attention mechanisms to learn the ‘transcriptional grammar’ of cells. Here we investigate the reusability beyond the original datasets, assessing the generalizability of natural language techniques in single-cell genomics. The degree of imbalance in the cell-type distribution substantially influences the performance of scBERT. Anticipating an increased utilization of transformers, we highlight the necessity to consider data distribution carefully and introduce a subsampling technique to mitigate the influence of an imbalanced distribution. Our analysis serves as a stepping stone towards understanding and optimizing the use of transformers in single-cell genomics.

To Err is AI: Imperfect Interventions and Repair in a Conversational Agent Facilitating Group Chat Discussions

Conversational agents (CAs) can analyze online conversations using natural language techniques and effectively facilitate group discussions by sending supervisory messages. However, if a CA makes imperfect interventions, users may stop trusting the CA and discontinue using it. In this study, we demonstrate how inaccurate interventions of a CA and a conversational repair strategy can influence user acceptance of the CA, members' participation in the discussion, perceived discussion experience between the members, and group performance. We built a CA that encourages the participation of members with low contributions in an online chat discussion in which a small group (3-6 members) performs a decision-making task. Two types of errors can occur when detecting under-contributing members: 1) false-positive (FP) errors happen when the CA falsely identifies a member as under-contributing and 2) false-negative (FN) errors occur when the CA misses detecting an under-contributing member. We designed a conversational repair strategy that gives users a chance to contest the detection results and the agent sends a correctional message if an error is detected. Through an online study with 175 participants, we found that participants who received FN error messages reported higher acceptance of the CA and better discussion experience, but participated less compared to those who received FP error messages. The conversational repair strategy moderated the effect of errors such as improving the perceived discussion experience of participants who received FP error messages. Based on our findings, we offer design implications for which model should be selected by practitioners between high precision (i.e., fewer FP errors) and high recall (i.e., fewer FN errors) models depending on the desired effects. When frequent FP errors are expected, we suggest using the conversational repair strategy to improve the perceived discussion experience.

Generating textual emergency plans for unconventional emergencies — A natural language processing approach

An emergency plan is an emergency administrative document that specifies the course of actions taken to minimize the effects of a crisis or incident. Establishing high-quality emergency plans has been a fundamental task for various emergency administrative agencies. Traditionally, emergency plans are developed based on the experiences of handling past emergencies, thus may not be well applied to unconventional emergencies that arise in an unrepeatable and unpredictable manner. This work proposes a novel emergency plan generation approach to assist decision-making under unconventional emergent situations. This goal is achieved by leveraging deep-learning-based natural language techniques to explore the interrelationship between existing emergency plans developed for common emergencies and the target unconventional emergency. In particular, an emergency response knowledge base is constructed based on a large number of existing emergency plans, and the relevant part with respect to the target unconventional emergency is retrieved. Then the new emergency plan is formed by organizing the relevant knowledge guided by a pre-defined emergency plan template. Furthermore, a novel emergency plan evaluation approach is proposed to perform a comprehensive evaluation of the quality of generated emergency plans. Empirical results on a real-world unconventional emergency case verify the feasibility of our emergency plan generation approach.

Mapping Climate Themes From 2008-2021—An Analysis of Business News Using Topic Models

India and other developing economies are receiving more attention in the context of climate change due to their rapid rates of economic expansion and large populations. In terms of absolute emissions, India surpassed China and the U.S. in 2018 to become the third-largest emitter. Solving this wicked problem calls for climate action across the stakeholder spectrum involving governments, business communities, and citizens. While extant literature has focused significantly on the role of governments and individual perceptions, the business sector needs to be more represented. In this study, we consider business news media as a platform that reflects the industry engagement in climate change and as a source of information on climate change for business decision-makers. Hence, understanding the topic and themes in the nexus of climate and business is important to evaluate the business sector’s stance towards climate change and how it has evolved. This work explores business news related to climate change using natural language techniques. We first experiment with three topic-modeling techniques, such as LDA, NMF, and BERTopic, on the business news and two more benchmark news datasets. Our test data is derived from digital news archives of ’The Economic Times – India’s leading business news daily. We evaluate the performance based on quantitative metrics commonly used for topic models. We choose the algorithm that provides the highest precision for climate-specific information represented by the test dataset. We then apply the algorithm with the best performance, as evaluated by the experiment, to a large corpus of Indian climate news from E.T. spanning from 2008 - 2021. We present how different themes, including industry engagement, evolved over the last two decades. The results suggest that climate cooperation has the highest contribution in the corpus, with other themes on resource management, energy and business gaining traction in recent years.

Sentiment Analysis and Opinion Mining Applied to Scientific Paper Reviews

Abstract: Sentiment analysis and opinion mining is an area that has experienced considerable growth over the last decade.Todo this, natural language techniques and machine learning algorithms are used. This research attempts to determine the feelings, opinions, emotions, among other things, of people on something. A hybrid approach that combines an unsupervised machine learning algorithm with techniques from natural lan- guage processing is proposed to analyze reviews. The first aimof this analysis is to automatically determine the orientation of a review.This article discusses the problem of extracting sentiment and opinions from a collection of reviews on scientific articles conducted under an international conference on computing in northern Chile.

Locating Source Code Bugs in Software Information Systems Using Information Retrieval Techniques

Bug localization is the process through which the buggy source code files are located regarding a certain bug report. Bug localization is an overwhelming and time-consuming process. Automating bug localization is the key to help developers and increase their productivities. Expanding bug reports with more semantic and increasing software understanding using information retrieval and natural language techniques will be the way to locate the buggy source code file, in which the bug report works as a query and source code as search space. This research investigates the effect of segmenting open source files into executable code and comments, as they have a conflicting nature, seeks the effect of synonyms on the accuracy of bug localization, and examines the effect of “part-of-speech” techniques on reducing the manual inspection for appropriate synonyms. This research aims to approve that such methods improve the accuracy of bug localization tasks. The used approach was evaluated on three Java open source software, namely Eclipse 3.1, AspectJ 1.0, and SWT 3.1; we implement our dedicated Java tool to adopt our methodology and conduct several experiments on each software. The experimental results reveal a considerable improvement in recall and precision levels, and the developed methods display an accuracy improvement of 4–10% compared with the state-of-the-art approaches.

Sentimental Analysis of Industry 4.0 Perspectives Using a Graph-Based Bi-LSTM CNN Model

Today’s modern society mainly depends on Internet for every fundamental task in their life, like sharing thoughts, education, business, and Industry 4.0, etc. Internet has strengthened the base of society digitally. Searching for the reviews and comments for a particular product from the former or present customers has become compulsory for making decision or a purchase, helping them to make a fair deal of the product by view from social media on Industry 4.0. The increased use of data over Internet has led to rise of many e-commerce websites where people can buy things as per their requirement without even stepping out of their house using analysis of text using natural language techniques in Industry 4.0. The graph-based modelling of sentiment analysis classifier needs to divide the textual data into training dataset and testing dataset. First, preprocessing work is performed to improve the data reliability by removing unwanted information and fixing typing errors. To train the models, the whole dataset is used and to measure the classification performance of the categorized models 10-fold validation technique is being used. The paper proposes a combined hybrid model for sentiment analysis using CNN and independent bidirectional LSTM networks to enhance sentiment knowledge in order to address the issues mentioned for sentiment analysis. The proposed CNN model uses global max-pooling for retrieving context information and to downsample the dimensionality. Lastly, to acquire long term dependencies, a distinctive bidirectional LSTM is used. To emphasise each word's learning ability, parts-of-speech (PoS) are tagged in the LSTM layer. In addition, the regularization techniques, batch normalization, and dropout are used to prevent the overfitting issue. The proposed model is compared with a collaborative classifier with six classifiers and each of them predicts the sentiments separately, and the majority class prediction is taken under consideration. The proposed Bi-LSTM CNN model achieves an accuracy of 98.61% along with PoS tagging of the sentiments.

Theme extraction based on NLP

Theme Extraction is a method of identifying, evaluating, and understanding human perception about a product in form of key features or themes that we extract dynamically from a given set of reviews within a data set. These themes are then categories to form an opinion about a given feature inside of the product through which we can analyze the advantages as well as the short comings of a given product or organization. These key features are then displayed on to the user for them to make a wise decision based on their likes and dislikes which they can compare with the user base that have already formed a review. All of this happens seamlessly with the help of Natural language Techniques that enables us to dynamically extract features or themes and generalizes an opinion score alongside it to represent thousands of reviews in a small concise manner. To make this happen, we consider seven different steps: (i)Text Pre-Processing, (ii) Removal of Stop-Words, (iii) Vader Sentiment Analysis, (iv) Feature Extraction using HAC, (v) Classification of Key Features using MOS, (vi) Testing the Accuracy of the Score and (vii) Creation of Word-Cloud using Features.

Have You Heard?: How Gossip Flows Through Workplace Email

We spend a significant part of our lives chatting about other people. In other words, we all gossip. Although sometimes a contentious topic, various researchers have shown gossip to be fundamental to social life—from small groups to large, formal organizations. In this paper, we present the first study of gossip in a large CMC corpus. Adopting the Enron email dataset and natural language techniques, we arrive at four main findings. First, workplace gossip is common at all levels of the organizational hierarchy, with people most likely to gossip with their peers. Moreover, employees at the lowest level play a major role in circulating it. Second, gossip appears as often in personal exchanges as it does in formal business communication. Third, by deriving a power-law relation, we show that it is more likely for an email to contain gossip if targeted to a smaller audience. Finally, we explore the sentiment associated with gossip email, finding that gossip is in fact quite often negative: 2.7 times more frequent than positive gossip.

Creating Conversations: An Automated Dialog System

Online news sites often include a comments section where readers are allowed to leave their thoughts. These comments often contain interesting and insightful conversations between readers about the news article. However the richness of these conversations is often lost among other meaningless comments, and moreover all comments are found at the bottom of the web page. In this article, we discuss how our system inserts reader conversations into the news article to create a multimedia presentation called Shout Out. Shout Out features two virtual news anchors: one anchor reads the news and when appropriate the anchor pauses to have a conversation about the news with another anchor. This current iteration of Shout Out combines natural language techniques and reader conversations to create an engaging system.

Phony News Detection in Reddit Using Natural Language Techniques and Machine Learning Pipelines

Phony news or fake news spreads like a wildfire on social media causing loss to the society. Swift detection of fake news is a priority as it reduces harm to society. This paper developed a phony news detector for Reddit posts using popular machine learning techniques in conjunction with natural language processing techniques. Popular feature extraction algorithms like CountVectorizer (CV) and Term Frequency Inverse Document Frequency (TFIDF) were implemented. These features were fed to Multinomial Naive Bayes (MNB), Random Forest (RF), Support Vector Classifier (SVC), Logistic Regression (LR), AdaBoost, and XGBoost for classifying news as either genuine or phony. Finally, coefficient analysis was performed in order to interpret the best coefficients. The study revealed that the pipeline model of MNB and TFIDF achieved a best accuracy rate of 79.05% when compared to other pipeline models.

Question Answering techniques for detecting needs of people in crisis

The use of social networks has become one of the basics of daily life to follow the news, and also in the case of inquiries and requesting assistance. In case of a crisis, information about the event begins to spread on social networks with the aim of raising awareness and knowing all the important instructions and requesting assistance. This information can be used to respond to people with the required needs depending on the type of assistance or inquiry requested. In this paper, an approach was proposed that uses Question Answering techniques based on natural language techniques and neural networks to extract the needs of affected people during a crisis and provide the proper response. The approach was tested on twitter data from different types of crises. The results showed that the suggested approach can answer with suitable guidelines with a precision of 0.81, a recall of 0.76 and an f-score of 0.78.