Natural Language Research Articles

Human Versus Machine Intelligence: Assessing Natural Language Generation Models Through Complex Systems Theory.

The introduction of Transformer architectures - with the self-attention mechanism - in automatic Natural Language Generation (NLG) is a breakthrough in solving general task-oriented problems, such as the simple production of long text excerpts that resemble ones written by humans. While the performance of GPT-X architectures is there for all to see, many efforts are underway to penetrate the secrets of these black-boxes in terms of intelligent information processing whose output statistical distributions resemble that of natural language. In this work, through the complexity science framework, a comparative study of the stochastic processes underlying the texts produced by the English version of GPT-2 with respect to texts produced by human beings, notably novels in English and programming codes, is offered. The investigation, of a methodological nature, consists first of all of an analysis phase in which the Multifractal Detrended Fluctuation Analysis and the Recurrence Quantification Analysis - together with Zipf's law and approximate entropy - are adopted to characterize long-term correlations, regularities and recurrences in human and machine-produced texts. Results show several peculiarities and trends in terms of long-range correlations and recurrences in the last case. The synthesis phase, on the other hand, uses the complexity measures to build synthetic text descriptors - hence a suitable text embedding - which serve to constitute the features for feeding a machine learning system designed to operate feature selection through an evolutionary technique. Using multivariate analysis, it is then shown the grouping tendency of the three analyzed text types, allowing to place GTP-2 texts in between natural language texts and computer codes. Similarly, the classification task demonstrates that, given the high accuracy obtained in the automatic discrimination of text classes, the proposed set of complexity measures is highly informative. These interesting results allow us to add another piece to the theoretical understanding of the surprising results obtained by NLG systems based on deep learning and let us to improve the design of new informetrics or text mining systems for text classification, fake news detection, or even plagiarism detection.

Enhancing authenticity and trust in social media: an automated approach for detecting fake profiles

Enhancing authenticity and trust in social media: an automated approach for detecting fake profiles

Every Problem, Every Step, All in Focus: Learning to Solve Vision-Language Problems With Integrated Attention.

Integrating information from vision and language modalities has sparked interesting applications in the fields of computer vision and natural language processing. Existing methods, though promising in tasks like image captioning and visual question answering, face challenges in understanding real-life issues and offering step-by-step solutions. In particular, they typically limit their scope to solutions with a sequential structure, thus ignoring complex inter-step dependencies. To bridge this gap, we propose a graph-based approach to vision-language problem solving. It leverages a novel integrated attention mechanism that jointly considers the importance of features within each step as well as across multiple steps. Together with a graph neural network method, this attention mechanism can be progressively learned to predict sequential and non-sequential solution graphs depending on the characterization of the problem-solving process. To tightly couple attention with the problem-solving procedure, we further design new learning objectives with attention metrics that quantify this integrated attention, which better aligns visual and language information within steps, and more accurately captures information flow between steps. Experimental results on VisualHow, a comprehensive dataset of varying solution structures, show significant improvements in predicting steps and dependencies, demonstrating the effectiveness of our approach in tackling various vision-language problems.

Cloud-based machine learning algorithms for anomalies detection

Gradient boosting machines harnesses the inherent capabilities of decision trees and meticulously corrects their errors in a sequential fashion, culminating in remarkably precise predictions. Word2Vec, a prominent word embedding technique, occupies a pivotal role in natural language processing (NLP) tasks. Its proficiency lies in capturing intricate semantic relationships among words, thereby facilitating applications such as sentiment analysis, document classification, and machine translation to discern subtle nuances present in textual data. Bayesian networks introduce probabilistic modeling capabilities, predominantly in contexts marked by uncertainty. Their versatile applications encompass risk assessment, fault diagnosis, and recommendation systems. Gated recurrent units (GRU), a variant of recurrent neural networks, emerges as a formidable asset in modeling sequential data. Both training and testing are crucial to the success of an intrusion detection system (IDS). During the training phase, several models are created, each of which can recognize typical from anomalous patterns within a given dataset. To acquire passwords and credit card details, "phishing" usually entails impersonating a trusted company. Predictions of student performance on academic tasks are improved by hyper parameter optimization of the gradient boosting regression tree using the grid search approach.

Enhancing Image Captioning Using Deep Convolutional Generative Adversarial Networks

Introduction:: Introduction: Image caption generation has long been a fundamental challenge in the area of computer vision (CV) and natural language processing (NLP). In this research, we present an innovative approach that harnesses the power of Deep Convolutional Generative Adversarial Networks (DCGAN) and adversarial training to revolutionize the generation of natural and contextually relevant image captions. Method:: Our method significantly improves the fluency, coherence, and contextual relevance of generated captions and showcases the effectiveness of RL reward-based fine-tuning. Through a comprehensive evaluation of COCO datasets, our model demonstrates superior performance over baseline and state-of-the-art methods. On the COCO dataset, our model outperforms current state-of-the-art (SOTA) models across all metrics, achieving BLEU-4 (0.327), METEOR (0.249), Rough (0.525) and CIDEr (1.155) scores. Result:: The integration of DCGAN and adversarial training opens new possibilities in image captioning, with applications spanning from automated content generation to enhanced accessibility solutions. Conclusion:: This research paves the way for more intelligent and context-aware image understanding systems, promising exciting future exploration and innovation prospects.

The speech neuroprosthesis.

Loss of speech after paralysis is devastating, but circumventing motor-pathway injury by directly decoding speech from intact cortical activity has the potential to restore naturalcommunication and self-expression. Recent discoveries have defined how key features of speech production are facilitated by the coordinated activity of vocal-tract articulatory and motor-planning cortical representations. In this Review, we highlight such progress and how it has led to successful speech decoding, first in individuals implanted with intracranial electrodes for clinicalepilepsy monitoringand subsequently in individuals with paralysis as part ofearlyfeasibility clinical trials to restore speech. We discuss high-spatiotemporal-resolution neural interfaces and the adaptation of state-of-the-art speech computational algorithms that have driven rapidandsubstantial progress indecoding neural activity into text, audible speech, and facial movements. Although restoring natural speech is a long-term goal, speech neuroprostheses already have performance levels that surpass communication rates offered by current assistive-communication technology. Given this accelerated rate of progress in the field, we propose key evaluation metrics for speed and accuracy, among others, to help standardize across studies. We finish by highlighting several directions to more fully explore the multidimensional feature space of speech and language, which will continue to accelerate progress towards a clinically viable speech neuroprosthesis.

Advancing Accessibility through Automatic Speech Recognition and NLP Integration

Abstract: The integration of Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) technologies has the potential to revolutionize accessibility and inclusive communication. This article explores the fundamentals of ASR, including acoustic modeling, language modeling, and speech signal processing techniques, and discusses the challenges posed by language diversity in developing accurate ASR systems. The advent of ASR technology has opened up numerous possibilities for enhancing accessibility across various domains, such as assistive technology, education, healthcare, and media. The integration of ASR with NLP techniques enables the processing and analysis of spoken language data, leading to the development of voice-enabled virtual assistants, conversational AI systems, and cross-lingual communication tools. However, several challenges remain, including the need for robust and accurate ASR systems, privacy and security concerns, and ethical considerations in the development and deployment of these technologies. The article also presents future directions, such as the integration of ASR and NLP with emotion recognition and sentiment analysis, advances in deep learning techniques, and the application of these technologies in healthcare and accessibility. Overall, the integration of ASR and NLP holds immense promise for creating more natural, empathetic, and inclusive communication systems, but their development and deployment must be approached with care to ensure fairness, transparency, and user privacy.

Help Them Understand: Testing and Improving Voice User Interfaces

Voice-based virtual assistants are becoming increasingly popular. Such systems provide frameworks to developers for building custom apps. End-users can interact with such apps through a Voice User Interface (VUI), which allows the user to use natural language commands to perform actions. Testing such apps is not trivial: The same command can be expressed in different semantically equivalent ways. In this article, we introduce VUI-UPSET, an approach that adapts chatbot-testing approaches to VUI-testing. We conducted an empirical study to understand how VUI-UPSET compares to two state-of-the-art approaches (i.e., a chatbot testing technique and ChatGPT) in terms of (i) correctness of the generated paraphrases, and (ii) capability of revealing bugs. To this aim, we analyzed 14,898 generated paraphrases for 40 Alexa Skills. Our results show that VUI-UPSET generates more bug-revealing paraphrases than the two baselines with, however, ChatGPT being the approach generating the highest percentage of correct paraphrases. We also tried to use the generated paraphrases to improve the skills. We tried to include in the voice interaction models of the skills (i) only the bug-revealing paraphrases, (ii) all the valid paraphrases. We observed that including only bug-revealing paraphrases is sometimes not sufficient to make all the tests pass.

Replication in Requirements Engineering: The NLP for RE Case

Natural language processing (NLP) techniques have been widely applied in the requirements engineering (RE) field to support tasks such as classification and ambiguity detection. Despite its empirical vocation, RE research has given limited attention to replication of NLP for RE studies. Replication is hampered by several factors, including the context specificity of the studies, the heterogeneity of the tasks involving NLP, the tasks’ inherent hairiness , and, in turn, the heterogeneous reporting structure. To address these issues, we propose a new artifact, referred to as ID-Card , whose goal is to provide a structured summary of research papers emphasizing replication-relevant information. We construct the ID-Card through a structured, iterative process based on design science. In this article: (i) we report on hands-on experiences of replication; (ii) we review the state-of-the-art and extract replication-relevant information: (iii) we identify, through focus groups, challenges across two typical dimensions of replication: data annotation and tool reconstruction; and (iv) we present the concept and structure of the ID-Card to mitigate the identified challenges. This study aims to create awareness of replication in NLP for RE. We propose an ID-Card that is intended to foster study replication but can also be used in other contexts, e.g., for educational purposes.

DeepGD: A Multi-Objective Black-Box Test Selection Approach for Deep Neural Networks

Deep neural networks (DNNs) are widely used in various application domains such as image processing, speech recognition, and natural language processing. However, testing DNN models may be challenging due to the complexity and size of their input domain. In particular, testing DNN models often requires generating or exploring large unlabeled datasets. In practice, DNN test oracles, which identify the correct outputs for inputs, often require expensive manual effort to label test data, possibly involving multiple experts to ensure labeling correctness. In this article, we propose DeepGD , a black-box multi-objective test selection approach for DNN models. It reduces the cost of labeling by prioritizing the selection of test inputs with high fault-revealing power from large unlabeled datasets. DeepGD not only selects test inputs with high uncertainty scores to trigger as many mispredicted inputs as possible but also maximizes the probability of revealing distinct faults in the DNN model by selecting diverse mispredicted inputs. The experimental results conducted on four widely used datasets and five DNN models show that in terms of fault-revealing ability, (1) white-box, coverage-based approaches fare poorly, (2) DeepGD outperforms existing black-box test selection approaches in terms of fault detection, and (3) DeepGD also leads to better guidance for DNN model retraining when using selected inputs to augment the training set.

MizBERT: A Mizo BERT Model

This research investigates the utilization of pre-trained BERT transformers within the context of the Mizo language. BERT, an abbreviation for Bidirectional Encoder Representations from Transformers, symbolizes Google’s forefront neural network approach to Natural Language Processing (NLP), renowned for its remarkable performance across various NLP tasks. However, its efficacy in handling low-resource languages such as Mizo remains largely unexplored. In this study, we introduce MizBERT , a specialized Mizo language model. Through extensive pre-training on a corpus collected from diverse online platforms, MizBERT has been tailored to accommodate the nuances of the Mizo language. Evaluation of MizBERT’s capabilities is conducted using two primary metrics: masked language modeling and perplexity, yielding scores of 76.12% and 3.2565, respectively. Additionally, its performance in a text classification task is examined. Results indicate that MizBERT outperforms both the Multilingual BERT model and the Support Vector Machine algorithm, achieving an accuracy of 98.92%. This underscores MizBERT’s proficiency in understanding and processing the intricacies inherent in the Mizo language.

Measurement and optimisation of the perceptual equivalence of the Dutch consonant-vowel-consonant (CVC) word lists using synthetic speech and list pairs

Objectives (1) to determine whether the standard Dutch word lists for speech audiometry are equally intelligible in normal-hearing listeners (Experiment 1), (2) to investigate whether synthetic speech can be used to create word lists (Experiment 1) and (3) to determine whether the list effect found in Experiment 1 can be reduced by combining two lists into pairs (Experiment 2). Design Participants performed speech tests in quiet with the original (natural) and synthetic word lists (Experiment 1.). In Experiment 2, new participants performed speech tests with list pairs from the original lists constructed from the results of Experiment 1. Study samples Twenty-four and twenty-eight normal-hearing adults. Results There was a significant list effect in the natural speech lists; not in the synthetic speech lists. Variability in intelligibility was significantly higher in the former, with list differences up to 20% at fixed presentation levels. The 95% confidence interval of a list with a score of approximately 70% is around 10%-points wider than of a list pair. Conclusions The original Dutch word lists show large variations in intelligibility. List effects can be reduced by combining two lists per condition. Synthetic speech is a promising alternative to natural speech in speech audiometry in quiet.

Lightweight Deep Learning for Resource-Constrained Environments: A Survey

Over the past decade, the dominance of deep learning has prevailed across various domains of artificial intelligence, including natural language processing, computer vision, and biomedical signal processing. While there have been remarkable improvements in model accuracy, deploying these models on lightweight devices, such as mobile phones and microcontrollers, is constrained by limited resources. In this survey, we provide comprehensive design guidance tailored for these devices, detailing the meticulous design of lightweight models, compression methods, and hardware acceleration strategies. The principal goal of this work is to explore methods and concepts for getting around hardware constraints without compromising the model’s accuracy. Additionally, we explore two notable paths for lightweight deep learning in the future: deployment techniques for TinyML and Large Language Models. Although these paths undoubtedly have potential, they also present significant challenges, encouraging research into unexplored areas.

Machine Learning and Natural Language Processing Algorithms in the Remote Mobile Medical Diagnosis System of Internet Hospitals

In order to alleviate the contradiction between supply and demand of professional pharmacists, integrate medical resources, and ensure the safety of patients’ medication, the telemedicine diagnosis system has played a great role. Today, in this “Internet +” era, all walks of life have begun to integrate with Internet technology. The purpose of this article was to discuss the practical utility of machine learning and natural language processing algorithms in the remote mobile medical diagnosis system of Internet hospitals, for which this article conducted in-depth discussion. This article first introduced the basic concepts, development, and characteristics of machine learning and natural language processing algorithms in detail, and carefully studied and analyzed the development and culture of traditional offline medical diagnosis models. Based on machine learning and natural language processing algorithms, a remote mobile medical diagnosis is designed. By combining with the medical diagnosis system of traditional hospitals, a new type of remote mobile medical diagnosis system for Internet hospitals was designed and developed, and the combination of traditional medical industry and Internet technology was deeply studied. According to each functional requirement, the image module, heart rate measurement module, and user setting module are designed, respectively. Compared to traditional medical diagnosis systems, the accuracy of the remote mobile medical diagnosis system based on machine learning applied in Internet hospital diagnosis in this article reached 80% or even higher. At the same time, it was found through experiments that when the evolution number was 3, the maximum fit value and average fit value were the same, both of which were 0.6. This indicates that the system can accommodate more than 10,000 people at the same time, and patients can receive good treatment plans, with a very broad application prospect

Effectiveness Analysis of Entrepreneurial Legal Risk Prevention Based on Multi-Modal Deep Learning Model

With the emergence of the upsurge of entrepreneurship, entrepreneurs are increasingly concerned about legal risks. In the process of entrepreneurship, legal risk is the biggest hidden danger of entrepreneurial enterprises. The prevention and avoidance of legal risks is a long and arduous process, and not all risks can be identified and avoided in time. The deep learning method has brought great changes to the fields of speech recognition, image recognition and natural language processing. The tasks in these fields only involve single-mode input, but more recent applications need to involve multi-mode intelligence. Multimodal deep learning mainly includes three aspects: multimodal learning representation, multimodal signal fusion and multimodal application. Through the research on the legal risks in recent years, this paper believed that there are many legal problems in the development of entrepreneurial enterprises, including contract disputes, trade secret disputes, trademark infringement disputes, copyright infringement disputes, and so on. This paper aimed to study the problem of Legal Risks of Entrepreneurship (LRE) that entrepreneurs are concerned about, and proposed a new solution from the perspective of image recognition technology. 486 entrepreneurs were investigated by questionnaire. In the process of entrepreneurship, 87.04% of entrepreneurs encountered legal risks, and they would turn to lawyers for help, which is a better way. However, in most cases, entrepreneurs are exposed to LRE because of their insufficient understanding of economic law, so they solve it through lawyers. However, if a lawyer is hired, the cost would be very high, which would bring great economic pressure to the enterprise. Only 17.90% of entrepreneurs would safeguard their legitimate rights and interests through their own knowledge and legal weapons without resorting to lawyers. It can be seen that entrepreneurs have relatively low practical ability in the use of LRE, and their legal practical ability is obviously insufficient.

FETILDA: Evaluation Framework for Effective Representations of Long Financial Documents

In the financial sphere, there is a wealth of accumulated unstructured financial data, such as the textual disclosure documents that companies submit on a regular basis to regulatory agencies, such as the Securities and Exchange Commission. These documents are typically very long and tend to contain valuable soft information about a company’s performance that is not present in quantitative predictors. It is therefore of great interest to learn predictive models from these long textual documents, especially for forecasting numerical key performance indicators. In recent years, there has been great progress in natural language processing via pre-trained language models (LMs) learned from large corpora of textual data. This prompts the important question of whether they can be used effectively to produce representations for long documents, as well as how we can evaluate the effectiveness of representations produced by various LMs. Our work focuses on answering this critical question, namely, the evaluation of the efficacy of various LMs in extracting useful soft information from long textual documents for prediction tasks. In this article, we propose and implement a deep learning evaluation framework that utilizes a sequential chunking approach combined with an attention mechanism. We perform an extensive set of experiments on a collection of 10-K reports submitted annually by U.S. banks, and another dataset of reports submitted by U.S. companies, to investigate thoroughly the performance of different types of language models. Overall, our framework using LMs outperforms strong baseline methods for textual modeling as well as for numerical regression. Our work provides better insights into how utilizing pre-trained domain-specific and fine-tuned long-input LMs for representing long documents can improve the quality of representation of textual data and, therefore, help in improving predictive analyses.

Information in summaries of product characteristics about use in children is limited and needs standardisation: a systematic analysis in Switzerland

BackgroundA considerable proportion of drugs administered to children are not authorised for this purpose, and consequently off-label use is common in paediatric care. Our aims were to quantify systematically the...

Modeling and Verification of Natural Language Requirements based on States and Modes

The relationship between states (status of a system) and modes (capabilities of a system) used to describe system requirements is often poorly defined. The unclear relationship could make systems of interest out of control because of the out of boundaries of the systems caused by the newly added modes. Formally modeling and verifying requirements can clarify the relationship, making the system safer. To this end, an innovative approach to analyzing requirements is proposed. The MoSt language (a Domain Specific Language implemented on the Xtext framework) is firstly designed for requirements modeling and a model validator is realized to check requirements statically. A code generator is then provided to realize the automatic model transformation from the MoSt model to a NuSMV model, laying the foundation for the dynamic checks of requirements through symbolic model checking. Next, a NuSMV runner is designed to connect the NuSMV with the validator to automate the whole dynamic checks. The grammar, the model validator, the code generator, and the NuSMV runner are finally integrated into a publicly available Eclipse-based tool. Two case studies have been employed to illustrate the feasibility of our approach. For each case study, we injected 14 errors. The results show that the static and dynamic checks can successfully detect all the errors.

UniQRNet: Unifying Referring Expression Grounding and Segmentation with QRNet

Referring expression comprehension aims to align natural language queries with visual scenes, which requires establishing fine-grained correspondence between vision and language. This has important applications in multi-modal reasoning systems. Existing methods typically use text-agnostic visual backbones to extract features independently without considering the specific text input. However, we argue that the extracted visual features can be inconsistent with the referring expression, which hurts multi-modal understanding. To address this, we first propose Query-modulated Refinement Network (QRNet) that leverages language guidance to guide visual feature extraction. However, it only focuses on the grounding task that can only provide coarse-grained annotations in the form of bounding box coordinates. The guidance for the visual backbone is indirect, and the inconsistent issue still exists. To this end, we further propose UniQRNet, a multi-task framework over the QRNet to learn referring expression grounding and segmentation jointly. The framework introduces a multi-task head that leverages fine-grained pixel-level supervision from the segmentation task to directly guide the intermediate layers of QRNet to learn text-consistent visual features. Besides, UniQRNet also includes a loss balance strategy that allows two types of supervision signals to cooperate and optimize the model together. We conduct the most comprehensive comparison experiment covering four major datasets, ten evaluation set and three evaluation metrics used in previous work. UniQRNet outperforms previous state-of-the-art methods by a large margin on both referring comprehensive grounding (1.8%~5.09%) and segmentation tasks (0.57%~5.56%). Ablation and analysis reveal that UniQRNet can improve the consistency of visual features with text input and can bring significant performance improvement.

Towards Retrieval-Augmented Architectures for Image Captioning

The objective of image captioning models is to bridge the gap between the visual and linguistic modalities by generating natural language descriptions that accurately reflect the content of input images. In recent years, researchers have leveraged deep learning-based models and made advances in the extraction of visual features and the design of multimodal connections to tackle this task. This work presents a novel approach toward developing image captioning models that utilize an external k NN memory to improve the generation process. Specifically, we propose two model variants that incorporate a knowledge retriever component that is based on visual similarities, a differentiable encoder to represent input images, and a k NN-augmented language model to predict tokens based on contextual cues and text retrieved from the external memory. We experimentally validate our approach on COCO and nocaps datasets and demonstrate that incorporating an explicit external memory can significantly enhance the quality of captions, especially with a larger retrieval corpus. This work provides valuable insights into retrieval-augmented captioning models and opens up new avenues for improving image captioning at a larger scale.