Natural Language Text Research Articles

Detecting Propaganda in News Articles Using Large Language Models

The proliferation of media channels as a result of the information age has ushered in a new era of communication and access to information. However, this increased accessibility has also opened up new avenues for propaganda and the manipulation of public opinion. With the recent release of OpenAI's artificial intelligence chatbot, ChatGPT, users and the media are increasingly discovering and reporting on its range of novel capabilities. The most notable of these, such as answering technical questions, stem from its ability to perform advanced natural language processing and text generation. In this paper, we aim to assess the feasibility of using the underlying technology behind ChatGPT, Large Language Models (LLMs), to detect features of propaganda in news articles. The features we consider leverage the work of Martino et al., who define a list of 18 distinct propaganda techniques. For example, they outline the 'straw man' technique, which refers to the use of 'refuting an argument that was not presented' [1]. Based on these techniques, we develop a refined prompt that is coupled with news articles from Russia Today (RT), a prominent state-controlled news network, and from the labelled SemEval-2020 Task 11 dataset [2]. The prompt and article content are then sent to OpenAI’s gpt-3.5-turbo model to determine which propaganda techniques are present and to make a final judgement on whether the article is propaganda or not. We then qualitatively analyse a subset of the resulting output to determine whether LLMs can be used effectively in this way. With the results of the study, we aim to uncover whether such technologies show promise in detecting propaganda, and what sort of prompts lead to the most useful output. This has the potential to be useful for media consumers, for example, who could use our prompts to detect signs of propaganda in the articles they read.

Research on the model of automatic recognition and natural language question-answer system for traditional Chinese medicine tongue images based on LLMs

Large Language Models (LLMs) have recently demonstrated their potential in clinical applications by providing valuable medical knowledge and recommendations. Traditional Chinese tongue diagnosis, one of the "Four Diagnoses," is an essential method for traditional Chinese medicine diagnosis. This paper builds upon tongue image classification technology and utilizes natural language processing and image recognition techniques to enhance the discrimination and analysis of traditional Chinese tongue images through learning and inference. We propose a method to integrate LLMs into the tongue image automatic recognition model and use them in interactive question-answering by summarizing and reorganizing information in natural language text format.

Movie sentiment analysis based on Long Short-Term Memory Network

An important task in the study of Natural Language Processing (NLP) is the analysis of movie reviews. It finishes the task of classifying movie review texts into sentiment, such as positive, negative or neutral sentiment. Previous works mainly follow the pipeline of LSTM (Long Short-Term Memory Network). The network model is a variant of Recurrent Neural Network (RNN) and particularly suitable for processing natural language texts. Though existing LSTM-based works have improved the performance significantly, we argue that most of them deal with the problem of analyzing the sentiment of movie reviews while ignore analyze the model performance in different application scenarios, such as different lengths of the reviews and the frequency of sentiment adverbs in the reviews. To alleviate the above issue, in this paper, we constructed a simple LSTM model containing an embedding layer, a batch normalization layer, a dropout layer, a one-dimensional convolutional layer, a maximal pooling layer, a bi-directional LSTM layer and a fully connected layer. We used the existing IMDB movie review dataset to train the model, and selected two research scenarios of movie review length and frequency of occurrence of sentiment adverbs to test the model, respectively. From the experimental results, we proposed a model for the scenarios in which the LSTM model handles the problem of sentiment analysis with respect to the dataset construction, model stability and generalization ability, text fragment processing, data preprocessing and feature extraction, model optimization and improvement.

CR‐M‐SpanBERT: Multiple embedding‐based DNN coreference resolution using self‐attention SpanBERT

AbstractThis study introduces CR‐M‐SpanBERT, a coreference resolution (CR) model that utilizes multiple embedding‐based span bidirectional encoder representations from transformers, for antecedent recognition in natural language (NL) text. Information extraction studies aimed to extract knowledge from NL text autonomously and cost‐effectively. However, the extracted information may not represent knowledge accurately owing to the presence of ambiguous entities. Therefore, we propose a CR model that identifies mentions referring to the same entity in NL text. In the case of CR, it is necessary to understand both the syntax and semantics of the NL text simultaneously. Therefore, multiple embeddings are generated for CR, which can include syntactic and semantic information for each word. We evaluate the effectiveness of CR‐M‐SpanBERT by comparing it to a model that uses SpanBERT as the language model in CR studies. The results demonstrate that our proposed deep neural network model achieves high‐recognition accuracy for extracting antecedents from NL text. Additionally, it requires fewer epochs to achieve an average F1 accuracy greater than 75% compared with the conventional SpanBERT approach.

Exploring Customer Review of Local Agriculture Product Acceptance in Malaysia: A Concept Paper on Sentiment Mining

Online consumer reviews in e-commerce are one technique to gather consumer opinion and sentiment about a company's products and services. However, manual analysis is impractical due to natural language text's enormous volume and complexity. Text mining and sentiment analysis methods based on machine learning provide an opportunity to analyze data for marketing objectives by increasing sales, positive electronic word-of-mouth (e-WOM), and meeting consumer demands and wants through the enhancement of market offerings. Despite the numerous benefits of analyzing e-commerce reviews to assist a company's marketing strategy, very little research has focused on sentiment and acceptance for Malaysia’s local agriculture products due to mixed language (English-Malay language) processing challenges. This concept paper highlights the use of text mining techniques to extract valuable insights from e-commerce comments related to Malaysian local agriculture products. By leveraging text mining, the study aims to better understand consumer sentiments, preferences, and feedback regarding local products, thereby facilitating improved market analysis and decision-making processes.

Knowledge Graph-Based Hierarchical Text Semantic Representation

Document representation is the basis of language modeling. Its goal is to turn natural language text that flows into a structured form that can be stored and processed by a computer. The bag-of-words model is used by most of the text-representation methods that are currently available. And yet, they do not consider how phrases are used in the text, which hurts the performance of tasks that use natural language processing later on. Representing the meaning of text by phrases is a promising area of future research, but it is hard to do well because phrases are organized in a hierarchy and mining efficiency is low. In this paper, we put forward a method called hierarchical text semantic representation using the knowledge graph (HTSRKG), which uses syntactic structure features to find hierarchical phrases and knowledge graphs to improve how phrases are evaluated. First, we use CKY and PCFG to build the syntax tree sentence by sentence. Second, we walk through the parse tree using the hierarchical routing process to obtain the mixed phrase semantics in passages. Finally, the introduction of the knowledge graph improves the efficiency of text semantic extraction and the accuracy of text representation. This gives us a solid foundation for tasks involving natural language processing that come after. Extensive testing on actual datasets shows that HTSRKG surpasses baseline approaches with respect to text semantic representation, and the results of a recent benchmarking study support this.

Explainable text-based features in predictive models of crowdfunding campaigns

AbstractReward-Based Crowdfunding offers an opportunity for innovative ventures that would not be supported through traditional financing. A key problem for those seeking funding is understanding which features of a crowdfunding campaign will sway the decisions of a sufficient number of funders. Predictive models of fund-raising campaigns used in combination with Explainable AI methods promise to provide such insights. However, previous work on Explainable AI has largely focused on quantitative structured data. In this study, our aim is to construct explainable models of human decisions based on analysis of natural language text, thus contributing to a fast-growing body of research on the use of Explainable AI for text analytics. We propose a novel method to construct predictions based on text via semantic clustering of sentences, which, compared with traditional methods using individual words and phrases, allows complex meaning contained in the text to be operationalised. Using experimental evaluation, we compare our proposed method to keyword extraction and topic modelling, which have traditionally been used in similar applications. Our results demonstrate that the sentence clustering method produces features with significant predictive power, compared to keyword-based methods and topic models, but which are much easier to interpret for human raters. We furthermore conduct a SHAP analysis of the models incorporating sentence clusters, demonstrating concrete insights into the types of natural language content that influence the outcome of crowdfunding campaigns.

Mapping Insights from News Articles to Tackle Low Birth Rate and Parenthood in Finland

The aim of this study is to explore how parenthood and birth rate is manifested in Finnish society and citizens as revealed by automated mining of news articles from Finland News API. Several levels of analysis were conducted using natural language processing and text mining techniques to unfold relevant insights from the collected News API. This includes surface-level analysis, word frequency-based analysis, topic-based analysis, and health ontology mapping. In total, 1621 news articles were selected and analyzed from the collected dataset. The surface-level analysis revealed the capacity of imminent health researchers to gain public audience and interest. Content-based analysis revealed the importance of family, employment, and health issues. Topical analysis stressed on the dominance of family issues during corona time, followed by public services and employment issues. Finally, the health ontology mapping revealed the quasi dominance of mental health and development disorders concerns. The research work provides a general framework for analyzing unstructured text to extract useful insights that can help policymakers to positively impact the existing policy in health and social policy development.

Demonstration-Based and Attention-Enhanced Grid-Tagging Network for Mention Recognition

Concepts empower cognitive intelligence. Extracting flat, nested, and discontinuous name entities and concept mentions from natural language texts is significant for downstream tasks such as concept knowledge graphs. Among the algorithms that uniformly detect these types of name entities and concepts, Li et al. proposed a novel architecture by modeling the unified mention recognition as the classification of word–word relations, named W2NER, achieved state-of-the-art (SOTA) results in 2022. However, there is still room for improvement. This paper presents three improvements based on W2NER. We enhanced the grid-tagging network by demonstration learning and tag attention feature extraction, so our modified model is named DTaE. Firstly, addressing the issue of insufficient semantic information in short texts and the lack of annotated data, and inspired by the demonstration learning from GPT-3, a demonstration is searched during the training phase according to a certain strategy to enhance the input features and improve the model’s ability for few-shot learning. Secondly, to tackle the problem of W2NER’s subpar recognition accuracy problem for discontinuous entities and concepts, a multi-head attention mechanism is employed to capture attention scores for different positions based on grid tagging. Then, the tagging attention features are embedded into the model. Finally, to retain information about the sequence position, rotary position embedding is introduced to ensure robustness. We selected an authoritative Chinese dictionary and adopted a five-person annotation method to annotate multiple types of entities and concepts in the definitions. To validate the effectiveness of our enhanced model, experiments were conducted on the public dataset CADEC and our annotated Chinese dictionary dataset: on the CADEC dataset, with a slight decrease in recall rate, precision is improved by 2.78%, and the comprehensive metric F1 is increased by 0.89%; on the Chinese dictionary dataset, the precision is improved by 2.97%, the recall rate is increased by 2.35%, and the comprehensive metric F1 is improved by 2.66%.

An overview memristor based hardware accelerators for deep neural network

AbstractThe prevalence of artificial intelligence applications using artificial neural network architectures for functions such as natural language processing, text prediction, object detection, speech, and image recognition has significantly increased in today's world. The computational functions performed by artificial neural networks in classical applications require intensive and large‐scale data movement between memory and processing units. Various software and hardware efforts are being made to perform these operations more efficiently. Despite these efforts, latency in data traffic and the substantial amount of energy consumed in data processing emerge as bottleneck disadvantages of the Von Neumann architecture. To overcome this bottleneck problem, it is necessary to develop hardware units specific to artificial intelligence applications. For this purpose, neuro‐inspired computing chips are believed to provide an effective approach by designing and integrating a set of features inspired by neurobiological systems at the hardware level to address the problems arising in artificial intelligence applications. The most notable among these approaches is memristor‐based neuromorphic computing systems. Memristors are seen as promising devices for hardware‐level improvement in terms of speed and energy because they possess non‐volatile memory and exhibit analog behavior. They enable effective storage and processing of synaptic weights, offering solutions for hardware‐level development. Taking into account these advantages of memristors, this study examines the research conducted on artificial neural networks and hardware that can directly perform deep learning functions and mimic the biological brain, which is different from classical systems in today's context.

СТИСНЕННЯ ПРИРОДНОМОВНИХ ТЕКСТІВ РЕВЕРСНИМИ МУЛЬТИРОЗДІЛЬНИКОВИМИ КОДАМИ

We study a class of binary reverse multi-delimiter (RMD) data compression codes in application to natural language text compression. The RMD-codewords start with delimiters, i.e., prefixes of the form that cannot occur in other places of the codeword. The position of the delimiter in an RMD codeword differs from its position in “direct” multi-delimiter (MD) codes, where delimiters are codeword suffixes. RMD and MD codes possess many useful properties, such as unique decodability, completeness, universality, synchronizability, asymptotic densities, and finite automaton acceptability. For RMD-codes, we construct a monotonic mapping from the set of natural numbers to the set of codewords. For original MD-codes, hitherto, this was an open question. The discovered mapping and the byte quantification of a decoding automaton allow us to develop very fast byte-aligned algorithms for decoding and direct Boyer–Moore style search in compressed files. Compared with the known byte (SCDC) and Fibonacci codes, RMD codes demonstrate the best compression ratio on natural language texts (more than four times closer to the entropy bound than that of SCDC). Computer experiments demonstrate that RMD codes can be decoded almost as fast as SCDC and times faster than Fibonacci codes. In natural language text compression, we also practiced the RMD-encoding as a preprocessing tool, which improves the performance of the known modern powerful archivers.

Hybrid Actor-Action Relation Extraction: A Machine Learning Approach

Software architects and developers face challenges while trying to explain the complex functional relationships between actors and systems in the process of system design. The field of natural language processing (NLP) and information extraction is always growing. One of the biggest problems that comes up time and time again is finding relationships between entities and actions in textual data. Actor-action relation extraction (AARE) is a hybrid model that combines NLP with rule-based and machine-learning models. It effectively analyzes unstructured data and takes into account different contextual factors, saving time and reducing errors. The proposed approach extracts Actors, actions, entities, and relationships (AARE) from natural language text more accurately and completely using machine learning techniques and rule-based systems. The hybrid model handles unstructured input and adapts to changing linguistic signals using machine learning. The hybrid approach uses Named Entity Recognition, rule-based extraction, and machine learning principles to convert unstructured data into structured format. It uses tokenization, part-of-speech tagging, nlp, and semantic role labeling for relationship categorization. The model has a 93% accuracy rate and is effective in extracting actor- action relations. Future research should focus on improving rule-based techniques, semantic learning, and addressing complexities in UML diagrams.

Multi-Agent Systems and Foundation Models Enable Autonomous Supply Chains: Opportunities and Challenges

The demand for resilient and flexible supply chains has become even more apparent in the face of disruptions such as the COVID-pandemic and the ongoing Russia-Ukraine War. Whilst multi-agent system approaches have been proposed to increase resilience in supply chains for more than two decades, their development remains limited due to their difficulty of implementation and black box nature. The emergence of modern AI approaches including foundation models, enables the creation of generalist agents with multi-faceted decision-making capabilities. This opens up opportunities to create supply chain systems with self-orchestrating capabilities and heightened resilience, in a way that is human understandable, through natural language text. However, unlike areas such as healthcare and finance, the application of modern agent technology in the supply chain domain is under explored. This paper thus aims to conceptually explore this less studied domain, investigating the convergence between foundation models, multi-agent systems, and supply chain management. We discuss the opportunities and challenges arising from this convergence for enabling autonomous supply chains, and propose key future research topics to advance this convergence.

Predicting depression and suicidal tendencies by analyzing online activities using machine learning in android devices

Artificial Intelligence (AI) has brought about a profound transformation in the realm of technology, with Machine Learning (ML) within AI playing a crucial role in today's healthcare systems. Advanced systems with intellectual abilities resembling those of humans are being created and utilized to carry out intricate tasks. Applications like Object recognition, classification, Optical Character Recognition (OCR), Natural Language processing (NLP), among others, have started producing magnificent results with algorithms trained on humongous data readily available these days. Keeping in view the socio-economic implications of the pandemic threat posed to the world by COVID-19, this research aims at improving the quality of life of people suffering from mild depression by timely diagnosing the symptoms using AI in android devices, especially phones. In cases of severe depression, which is highly likely to lead to suicide, valuable lives can also be saved if adequate help can be dispatched to such patients within time. This can be achieved using automatic analysis of users’ data including text messages, emails, voice calls and internet search history, among other mobile phone activities, using Text mining/ text analytics which is the process of deriving meaningful information from natural language text. Machine Learning models analyse the users’ behaviour continuously from text and voice communications and data, thereby identifying if there are any negative tendencies in the behaviour over a certain period of time, and by using this information make inferences about the mental health state of the patient and instantly request appropriate healthcare before it is too late. In this research, an android application capable of performing the aforementioned tasks in real-time has been developed and tested for various performance features with an average accuracy of 95%.

APLICAÇÃO DE TÉCNICAS DE INTELIGÊNCIA ARTIFICIAL PARA CLASSIFICAÇÃO DE FUGA AO TEMA EM REDAÇÕES

ABSTRACT: The process of manual correction of essays causes some difficulties, among which we point out the time spent for correction and feedback to the student. For institutions such as elementary schools, universities, and the National High School Exam in Brazil (Enem), such activity demands time and cost for the evaluation of the texts produced. Going off-topic is one of the items evaluated in the Enem essay that can nullify the whole essay produced by the candidate. In this context, the automatic analysis of essays with the application of techniques and methods of Natural Language Processing, Text Mining, and other Artificial Intelligence techniques has shown to be promising in the process of automated evaluation of written language. The goal of this research is to compare different AI techniques for the classification of going off-topic in texts and identify the one with the best result to enable a smart correction system for essays. Therefore, computer experiments were carried out to classify these texts to normalize, identify patterns, and classify the essays in 1,320 Brazilian Portuguese essays on 119 different topics. The results indicate that the Convolutional Neural Network classifier obtained greater gain concerning the other classifiers analyzed, both in accuracy and about the results of false positives, the precision of metrics, Recall, and F1-Score. In conclusion, the solution validated in this research contributes to positively impacting the work of teachers and educational institutions, by reducing the time and costs associated with the essay evaluation process.

Cross-Domain Social Rumor-Propagation Model Based on Transfer Learning.

Rumors in different topic domains have different text characteristics but similar emotional tendencies. To resolve the scarce-data problem in some rumor-topic domains, this study proposes a cross-domain rumor-propagation model, which is based on transfer learning. First, given the diversity and complexity of the rumor-propagation landscape, this study introduces a novel method, User-Retweet-Rumor2vec (URR2vec), which leverages the power of representation learning to uncover latent features within rumor topics. It also displays the forwarding relationship between users and rumors, user node information, and rumor-topic information in low-dimensional space. To capture the impact of human emotional cognition during rumor spreading, we also introduce a deep-learning model based on the natural language texts of rumor topics, which analyzes the sentiment in the text and uncovers the emotional correlations among users. Furthermore, a rumor-propagation prediction model based on the text-sentiment analysis-graph convolutional network (TSA-GCN) is proposed and pre-trained on existing rumor-topic data to ensure its prediction accuracy. Finally, considering the data sparsity at a rumor-topic outbreak, the trained propagation model is transferred to the rumor topic for prediction. Meanwhile, the rumor topic in different domains has different edges and conditional distribution, similar emotional characteristics, and network structure among the rumor topics. After fine-tuning the parameter and adding a domain adaptation layer in TSA-GCN, a domain adaptation model based on parameter and graph-structure migration is obtained.

Evidence Mining for Interpretable Charge Prediction via Prompt Learning

Nowadays, more and more researchers are committed to applying artificial intelligence technology to the legal field to support decision-making. Charge prediction is a subtask of legal judgment prediction (LJP). Its purpose is to analyze the fact description of natural language text and predict a charge corresponding to a case. At present, most of the research takes charge prediction as a multiclass classification task, which leads to unsatisfactory interpretation due to the weak semantic correlation between the fact descriptions and charge labels. In order to solve this problem, we propose a method of generative evidence mining based on prompt learning. Specifically, in the training phase, we reformulate the charge labels into the prompt template that we design to enhance the semantic correlation between the charge labels and the fact descriptions. In the testing phase, the charge labels are generated via the model based on prompt learning. Meanwhile, we calculate the attention score of each sentence from the multihead self-attention in the transformer encoder and choose the sentence with the highest attention score as the evidence. Our experimental results on a real dataset show that our method is better than the traditional fine-tuning-based classification method.

Deep Learning for Technical Document Classification

In large technology companies, the requirements for managing and organizing technical documents created by engineers and managers have increased dramatically in recent years, which has led to a higher demand for more scalable, accurate, and automated document classification. Prior studies have only focused on processing text for classification, whereas technical documents often contain multimodal information. To leverage multimodal information for document classification to improve the model performance, this paper presents a novel multimodal deep learning architecture, TechDoc, which utilizes three types of information, including natural language texts and descriptive images within documents and the associations among the documents. The architecture synthesizes the convolutional neural network, recurrent neural network, and graph neural network through an integrated training process. We applied the architecture to a large multimodal technical document database and trained the model for classifying documents based on the hierarchical International Patent Classification system. Our results show that TechDoc presents a greater classification accuracy than the unimodal methods and other state-of-the-art benchmarks. The trained model can potentially be scaled to millions of real-world multimodal technical documents, which is useful for data and knowledge management in large technology companies and organizations.

인공지능(AI)을 활용한 테러 예방 프로그램 구축 방안

The study explores the development of an effective counterterrorism program utilizing Artificial Intelligence (AI) technologies. Emphasizing the integration of various technical elements such as big data analysis, natural language processing, image and video analysis, and the incorporation of Internet of Things (IoT) and sensor technologies, the research investigates comprehensive approaches. Firstly, the utilization of big data analysis aims to collect and analyze data from diverse sources, identifying patterns associated with terrorism. The application of natural language processing and text mining extracts meaningful information from textual data, while image and video analysis detect unusual patterns or behaviors in visual information. Moreover, the study highlights the use of IoT and sensor technologies to establish an intelligent security system, collecting real-time environmental data to detect anomalous indicators. The implementation of an automated alert and emergency response system is also suggested, facilitating swift and effective responses upon detecting abnormal signs. In conclusion, the integrated use of diverse technologies in the proposed AI-based counterterrorism program enables rapid and accurate threat detection, contributing to the enhancement of public safety and national security.

Ways of expressing the category of instrumentality in retranslated texts

The category of instrumentality remains a research niche due to the interrelation of many cognitive and linguistic factors during its implementation in language. This study can contribute to the improvement of methodology and tools for automated propositional analysis of natural language texts. The article aims to identify the types and frequency of instrumentality manifestations in oral immediate recalls. The dataset comprises two reading texts on Social Studies and 34 immediate text-based oral recalls (2779 words in total). The study involved 34 native Russian 5th graders aged 11-12. For the types and range of modifications in expressing the semantic roles of instrumentality, the recalls were analyzed individually and later contrasted with the original texts. The findings showed that the most typical modifications of Instruments in Russian recalls are diathesis shift, excorporation or dismantling of verbs integrated into constructions, and redistribution of semantic roles including replacement of Instrument with Causator, Theme and the Circumstants of Place. Based on the variety of modifications, all the means of manifesting the semantic role of instrumentality registered in our dataset fall into two groups. The first group comprises all the above types of modification, as well as Human Psychological State and Perception. The second group consists of Incorporated verbs and Instrumental actions that were reproduced by respondents using the “extraction” strategy with virtually no lexical loss. The findings expand our understanding of the varieties of expressing the category of instrumentality in oral speech, as well as the specifics of generating secondary texts. They can find application in the studies of receptive and productive speech activity as well as in automating propositional analysis of Russian texts.