Natural Language Understanding System Research Articles

Leveraging intent–entity relationships to enhance semantic accuracy in NLU models

Natural Language Understanding (NLU) components are used in Dialog Systems (DS) to perform intent detection and entity extraction. In this work, we introduce a technique that exploits the inherent relationships between intents and entities to enhance the performance of NLU systems. The proposed method involves the utilization of a carefully crafted set of rules that formally express these relationships. By utilizing these rules, we effectively address inconsistencies within the NLU output, leading to improved accuracy and reliability. We implemented the proposed method using the Rasa framework as an NLU component and used our own conversational dataset AWPS to evaluate the improvement. Then, we validated the results in other three commonly used datasets: ATIS, SNIPS, and NLU-Benchmark. The experimental results show that the proposed method has a positive impact on the semantic accuracy metric, reaching an improvement of 12.6% in AWPS when training with a small amount of data. Furthermore, the practical application of the proposed method can easily be extended to other Task-Oriented Dialog Systems (T-ODS) to boost their performance and enhance user satisfaction.

YouTube Scholar: Advancing Education with AI Assistance

In the contemporary landscape of online educa- tion, the proliferation of educational content on platforms like YouTube has opened new avenues for learners. However, the sheer volume of videos presents a challenge, requiring innovative solutions. This project takes a pioneering approach by developing a specialized AI assistant for educational YouTube lectures, lever- aging advanced language models such as Generative Pretrained Transformers (GPT). This project focuses on the development of an AI assistant tailored specifically for educational YouTube lectures and courses. The primary goal is to enhance the learning experience by leveraging the capabilities of GPT-based models to provide personalised and contextually relevant interactions for learners. By harnessing the power of natural language understanding, generation, and dialog systems, the AI assistant aims to assist learners in navigating, understanding, and engaging with educa- tional content more effectively. The ultimate goal is to bridge the gap between the abundance of online educational content and learners’ comprehension, thereby revolutionizing the educational landscape in the digital age. By providing a sophisticated AI-driven companion, this project aspires to empower learners, making the educational journey more accessible, engaging, and conducive to deeper learning experiences. Through the integration of cutting-edge technologies, it envisions a future where learners can unlock the full potential of online education with confidence and ease

Using Large Language Models in the Companion Cognitive Architecture: A Case Study and Future Prospects

The goal of the Companion cognitive architecture is to understand how to create human-like software social organisms. Thus natural language capabilities, both for reading and conversation, are essential. Recently we have begun experimenting with large language models as a component in the Companion architecture. This paper summarizes a case study indicating why we are currently using BERT with our symbolic natural language understanding system. It also describes some additional ways we are contemplating using large language models with Companions.

Annotation for computational argumentation analysis: Issues and perspectives

AbstractArgumentation has long been studied in a number of disciplines, including several branches of linguistics. In recent years, computational processing of argumentation has been added to the list, reflecting a general interest from the field of natural language processing (NLP) in building natural language understanding systems for increasingly intricate language phenomena. Computational argumentation analysis – referred to as argumentation mining in the NLP literature – requires large amounts of real‐world text with manually analyzed argumentation. This process is known as annotation in the NLP literature and such annotated datasets are used both as “gold standards” for assessing the quality of NLP applications and as training data for the machine learning algorithms underlying most state of the art approaches to NLP. Argumentation annotation turns out to be complex, both because argumentation can be complex in itself and because it does not come across as a unitary phenomenon in the literature. In this survey we review how argumentation has been studied in other fields, how it has been annotated in NLP and what has been achieved so far. We conclude with describing some important current and future issues to be resolved.

The DistilBERT Model: A Promising Approach to Improve Machine Reading Comprehension Models

Machine Reading Comprehension (MRC) is a challenging task in the field of Natural Language Processing (NLP), where a machine is required to read a given text passage and answer a set of questions based on it. This paper provides an overview of recent advances in MRC and highlights some of the key challenges and future directions of this research area. It also evaluates the performance of several baseline models on the dataset, evaluates the challenges that the dataset poses for existing MRC models, and introduces the DistilBERT model to improve the accuracy of the answer extraction process. The supervised paradigm for training machine reading and comprehension models represents a practical path forward for creating comprehensive natural language understanding systems. To enhance the DistilBERT basic model's functionality, we have experimented with a variety of question heads that differ in the number of layers, activation function, and general structure. DistilBERT is a model for question-resolution tasks that is successful and delivers state-of-the-art performance while requiring less computational resources than large models like BERT, according to the presented technique. We could enhance the model's functionality and obtain a better understanding of how the model functions by investigating other question head architectures. These findings could serve as a foundation for future study on how to make question-and-answer systems and other tasks connected to the processing of natural languages.

Few-shot re-identification of the speaker by social robots

Nowadays advanced machine learning, computer vision, audio analysis and natural language understanding systems can be widely used for improving the perceptive and reasoning capabilities of the social robots. In particular, artificial intelligence algorithms for speaker re-identification make the robot aware of its interlocutor and able to personalize the conversation according to the information gathered in real-time and in the past interactions with the speaker. Anyway, this kind of application requires to train neural networks having available only a few samples for each speaker. Within this context, in this paper we propose a social robot equipped with a microphone sensor and a smart deep learning algorithm for few-shot speaker re-identification, able to run in real time over an embedded platform mounted on board of the robot. The proposed system has been experimentally evaluated over the VoxCeleb1 dataset, demonstrating a remarkable re-identification accuracy by varying the number of samples per speaker, the number of known speakers and the duration of the samples, and over the SpReW dataset, showing its robustness in real noisy environments. Finally, a quantitative evaluation of the processing time over the embedded platform proves that the processing pipeline is almost immediate, resulting in a pleasant user experience.

Computational Linguistics with Deep-Learning-Based Intent Detection for Natural Language Understanding

Computational linguistics explores how human language is interpreted automatically and then processed. Research in this area takes the logical and mathematical features of natural language and advances methods and statistical procedures for automated language processing. Slot filling and intent detection are significant modules in task-based dialogue systems. Intent detection is a critical task in any natural language understanding (NLU) system and constitutes the base of a task-based dialogue system. In order to build high-quality, real-time conversational solutions for edge gadgets, there is a demand for deploying intent-detection methods on devices. This mandates an accurate, lightweight, and fast method that effectively operates in a resource-limited environment. Earlier works have explored the usage of several machine-learning (ML) techniques for detecting intent in user queries. In this article, we propose Computational Linguistics with Deep-Learning-Based Intent Detection and Classification (CL-DLBIDC) for natural language understanding. The presented CL-DLBIDC technique receives word embedding as input and learned meaningful features to determine the probable intention of the user query. In addition, the presented CL-DLBIDC technique uses the GloVe approach. In addition, the CL-DLBIDC technique makes use of the deep learning modified neural network (DLMNN) model for intent detection and classification. For the hyperparameter tuning process, the mayfly optimization (MFO) algorithm was used in this study. The experimental analysis of the CL-DLBIDC method took place under a set of simulations, and the results were scrutinized for distinct aspects. The simulation outcomes demonstrate the significant performance of the CL-DLBIDC algorithm over other DL models.

Semantic Vector Learning and Visualization with Semantic Cluster Using Transformers in Natural Language Understanding

Natural language understanding (NLU) is a fundamental technology for implementing natural interfaces. The embedding of sentences and correspondence between text and its extracted semantic knowledge, called semantic frame, has recently shown that a semantic vector representation is key in the implementation or support of robust NLU systems. Herein, we propose an extension of cluster-aware modeling with various types of pre-trained transformers for consideration of the many-to-1 relationships of text-to-semantic frames and semantic clusters. To attain this, we define the semantic cluster, and design the relationships between cluster members to learn semantically meaningful vector representations. In addition, we introduce novel ensemble methods to improve the semantic vector applications around NLU, i.e., similarity-based intent classification and a semantic search. Furthermore, novel semantic vector and corpus visualization techniques are presented. Using the proposed framework, we demonstrate that the proposed model can learn meaningful semantic vector representations in ATIS, SNIPS, SimM, and Weather datasets.

The Impact of Semi-Supervised Learning on the Performance of Intelligent Chatbot System

Artificial intelligent based dialog systems are getting attention from both business and academic communities. The key parts for such intelligent chatbot systems are domain classification, intent detection, and named entity recognition. Various supervised, unsupervised, and hybrid approaches are used to detect each field. Such intelligent systems, also called natural language understanding systems analyze user requests in sequential order: domain classification, intent, and entity recognition based on the semantic rules of the classified domain. This sequential approach propagates the downstream error; i.e., if the domain classification model fails to classify the domain, intent and entity recognition fail. Furthermore, training such intelligent system necessitates a large number of user-annotated datasets for each domain. This study proposes a single joint predictive deep neural network framework based on long short-term memory using only a small user-annotated dataset to address these issues. It investigates value added by incorporating unlabeled data from user chatting logs into multi-domain spoken language understanding systems. Systematic experimental analysis of the proposed joint frameworks, along with the semi-supervised multi-domain model, using open-source annotated and unannotated utterances shows robust improvement in the predictive performance of the proposed multi-domain intelligent chatbot over a base joint model and joint model based on adversarial learning.

Knowledge-driven Natural Language Understanding of English Text and its Applications

Understanding the meaning of a text is a fundamental challenge of natural language understanding (NLU) research. An ideal NLU system should process a language in a way that is not exclusive to a single task or a dataset. Keeping this in mind, we have introduced a novel knowledge driven semantic representation approach for English text. By leveraging the VerbNet lexicon, we are able to map syntax tree of the text to its commonsense meaning represented using basic knowledge primitives. The general purpose knowledge represented from our approach can be used to build any reasoning based NLU system that can also provide justification. We applied this approach to construct two NLU applications that we present here: SQuARE (Semantic-based Question Answering and Reasoning Engine) and StaCACK (Stateful Conversational Agent using Commonsense Knowledge). Both these systems work by ``truly understanding'' the natural language text they process and both provide natural language explanations for their responses while maintaining high accuracy.

Research on the Construction Path of Ideological and Political Education for Postgraduates in the Era of Artificial Intelligence

Postgraduate ideological and political education is an important part of ideological and political education in colleges and universities. It is the core of implementing the party’s educational policy, comprehensively improving the quality of education, and building a modern socialist education power. In the era of artificial intelligence, innovate classroom education of ideological and political education for graduate students through intelligent network systems, natural language understanding systems, and knowledge processing systems; use intelligent search systems, symbol processing systems, and combined planning systems to deepen the communication links of graduate ideological and political education; through enhancing digital intelligence tools such as genetic algorithm, deep learning algorithm and artificial neural network algorithm to reconstruct the evaluation criteria of graduate ideological and political education is an important engine to promote the intelligent, contemporary and diversified development of graduate ideological and political education in my country.

Research on Adopting Artificial Intelligence Technology to Improve Effectiveness of Vocational College English Learning

In the traditional English learning mode of vocational college, teachers and students spend a lot of time on the basic knowledge of English. Due to the singularity of teaching methods and the lack of effective student self-learning platform, many students consider that English learning is boring and the actual learning effect is poor[1-2]. As an emerging edge discipline, artificial intelligence covers a series of topics such as image recognition, pattern recognition, robotics, natural language understanding and expert systems. The simulation and application of human language intelligence is an essential progress of artificial intelligence technology[3]. This paper will analyze the connotation of artificial intelligence technology and its principles of natural language understanding, and sort out the application advantages of artificial intelligence in education field. Finally, it puts forward practical measures of adopting artificial intelligence tools in input and output stages of English language learning[4-5].

Multi-turn intent determination and slot filling with neural networks and regular expressions

Intent determination and slot filling are two prominent research areas related to natural language understanding (NLU). In a multi-turn NLU system, contextual information from dialogue history is exploited to mitigate the ambiguity of user utterance. State-of-the-art models employ memory networks to encode dialogue context, which is used by neural networks for determining user intent and associated slots. However, these methods rely on a large amount of labelled data, whereas we often have limited labelled data. To address this problem, we propose a multi-task learning model based on neural networks and regular expressions (REs), to jointly perform intent determination and slot filling tasks. The proposed model integrates neural networks with REs to encode domain knowledge and handle cases with a limited amount of labelled data in an end-to-end trainable manner. More specifically, the model employs a pre-trained BERT model to obtain contextual word representations of user utterances. These representations are utilized by a memory network to encode multi-turn information which is shared by the tasks. Furthermore, the convolutional neural network (CNN) and the recurrent neural network (RNN) are applied to contextual word representations and dialogue context for intent determination and slot filling tasks, respectively. These neural networks are then combined with REs which encode domain knowledge about a particular intent or slot value. Finally, the two neural networks are trained simultaneously by minimizing the joint loss. Extensive experiments on Key-Value Retrieval and Frames datasets show that the proposed model outperforms baseline methods in both tasks while requiring modest human effort.

PIQA: Reasoning about Physical Commonsense in Natural Language

To apply eyeshadow without a brush, should I use a cotton swab or a toothpick? Questions requiring this kind of physical commonsense pose a challenge to today's natural language understanding systems. While recent pretrained models (such as BERT) have made progress on question answering over more abstract domains – such as news articles and encyclopedia entries, where text is plentiful – in more physical domains, text is inherently limited due to reporting bias. Can AI systems learn to reliably answer physical commonsense questions without experiencing the physical world?In this paper, we introduce the task of physical commonsense reasoning and a corresponding benchmark dataset Physical Interaction: Question Answering or PIQA. Though humans find the dataset easy (95% accuracy), large pretrained models struggle (∼75%). We provide analysis about the dimensions of knowledge that existing models lack, which offers significant opportunities for future research.

GF + MMT = GLF – From Language to Semantics through LF

These days, vast amounts of knowledge are available online, most of it in written form. Search engines help us access this knowledge, but aggregating, relating and reasoning with it is still a predominantly human effort. One of the key challenges for automated reasoning based on natural-language texts is the need to extract meaning (semantics) from texts. Natural language understanding (NLU) systems describe the conversion from a set of natural language utterances to terms in a particular logic. Tools for the co-development of grammar and target logic are currently largely missing. We will describe the Grammatical Logical Framework (GLF), a combination of two existing frameworks, in which large parts of a symbolic, rule-based NLU system can be developed and implemented: the Grammatical Framework (GF) and MMT. GF is a tool for syntactic analysis, generation, and translation with complex natural language grammars and MMT can be used to specify logical systems and to represent knowledge in them. Combining these tools is possible, because they are based on compatible logical frameworks: Martin-L\"of type theory and LF. The flexibility of logical frameworks is needed, as NLU research has not settled on a particular target logic for meaning representation. Instead, new logics are developed all the time to handle various language phenomena. GLF allows users to develop the logic and the language parsing components in parallel, and to connect them for experimentation with the entire pipeline.

Enhancing Domain-Specific Supervised Natural Language Intent Classification with a Top-Down Selective Ensemble Model

Natural Language Understanding (NLU) systems are essential components in many industry conversational artificial intelligence applications. There are strong incentives to develop a good NLU capability in such systems, both to improve the user experience and in the case of regulated industries for compliance reasons. We report on a series of experiments comparing the effects of optimizing word embeddings versus implementing a multi-classifier ensemble approach and conclude that in our case, only the latter approach leads to significant improvements. The study provides a high-level primer for developing NLU systems in regulated domains, as well as providing a specific baseline accuracy for evaluating NLU systems for financial guidance.

Modeling Words for Qualitative Distance Based on Interval Type-2 Fuzzy Sets

Modeling qualitative distance words is important for natural language understanding, scene reconstruction and many decision support systems (DSSs) based on a geographic information system (GIS). However, it is difficult to establish the relationship between qualitative distance words and quantitative distance for special applications since the meanings of these words are influenced by both subjective and objective factors. Some existing methods are reviewed, and the Hao–Mendel approach (HMA) is improved to model qualitative distance words for four travel modes by using interval type-2 fuzzy sets (IT2 FSs), aiming at addressing the individual and interpersonal uncertainty among qualitative distance words. The area of the footprint of uncertainty (FOU), fuzziness (entropy), and variance are adopted to measure the uncertainties of qualitative distance words. The experimental results show that the improved HMA algorithm is better than the original HMA algorithm and can be used in spatial information retrieval and GIS-based DSSs.

Provably correct reactive control from natural language

This paper presents an integrated system for generating, troubleshooting, and executing correct-by-construction controllers for autonomous robots using natural language input, allowing non-expert users to command robots to perform high-level tasks. This system unites the power of formal methods with the accessibility of natural language, providing controllers for implementable high-level task specifications, easy-to-understand feedback on those that cannot be achieved, and natural language explanation of the reason for the robot's actions during execution. The natural language system uses domain-general components that can easily be adapted to cover the vocabulary of new applications. Generation of a linear temporal logic specification from the user's natural language input uses a novel data structure that allows for subsequent mapping of logical propositions back to natural language, enabling natural language feedback about problems with the specification that are only identifiable in the logical form. We demonstrate the robustness of the natural language understanding system through a user study where participants interacted with a simulated robot in a search and rescue scenario. Automated analysis and user feedback on unimplementable specifications is demonstrated using an example involving a robot assistant in a hospital.

Resolución de anáforas que requieren conocimiento cultural con la herramienta FunGramKB

While integrating linguistic knowledge of any kind is becoming an almost implicit practice in natural language understanding systems, the inclusion of cultural or world knowledge in these tools might have been neglected sometimes. However, a NLP system or knowledge base enriched with cultural information is a more robust, better cohesioned instrument for natural language understanding processes. The integration of this type of knowledge in NLP systems may be proven to contribute to solving some phenomena that occur in natural language, such as anaphor, metaphor and metonymy, ambiguity or co-reference, amongst others. The objective of this article is to describe the way FunGramKB (a knowledge base) integrates cultural knowledge in its conceptual modules and, in particular, how the information contained in the Onomasticon module of FunGramKB can contribute to maximising the informativeness and completeness of the whole system, thus resolving ambiguity problems in a determined linguistic phenomenon: anaphora.

Humanoid Robot Learning How to track and grip

Interaction with its environment is a key requisite for designing a humanoid robot especially to have the ability to recognize and manipulate unknown objects and it is crucial to successfully work in natural environments. However visual object recognition still remains a challenging problem. To get the robot capable of identifying the geometric shapes and colors of the objects, this paper proposes new approach using neuro Zernike moments. Furthermore, the paper proposes a natural language understanding system, where the robot will be able to effectively communicate with human through a dialogue developed in Arabic language. The developed dialogue and a dynamic object model are used for learning semantic categories, object descriptions, and new words acquisition for object learning. In this paper, a robot will be developed to interact with the users performing some specified actions. Moreover, integration between the proposed vision and natural language understanding systems has been presented. Finally, a hardware circuit is designed and Q-learning technique is presented assisting the robot to track and grip objects. Intensive experiments have been conducted indoor to address the validity of the complete system. Qualitative comparison among different techniques is accomplished. The achieved results show that the overall system performance of the proposed system outperforms in terms of accuracy and response time.