Form Of Natural Language Research Articles

Natural-Language-Driven Multimodal Representation Learning for Audio-Visual Scene-Aware Dialog System.

With the development of multimedia systems in wireless environments, the rising need for artificial intelligence is to design a system that can properly communicate with humans with a comprehensive understanding of various types of information in a human-like manner. Therefore, this paper addresses an audio-visual scene-aware dialog system that can communicate with users about audio-visual scenes. It is essential to understand not only visual and textual information but also audio information in a comprehensive way. Despite the substantial progress in multimodal representation learning with language and visual modalities, there are still two caveats: ineffective use of auditory information and the lack of interpretability of the deep learning systems' reasoning. To address these issues, we propose a novel audio-visual scene-aware dialog system that utilizes a set of explicit information from each modality as a form of natural language, which can be fused into a language model in a natural way. It leverages a transformer-based decoder to generate a coherent and correct response based on multimodal knowledge in a multitask learning setting. In addition, we also address the way of interpreting the model with a response-driven temporal moment localization method to verify how the system generates the response. The system itself provides the user with the evidence referred to in the system response process as a form of the timestamp of the scene. We show the superiority of the proposed model in all quantitative and qualitative measurements compared to the baseline. In particular, the proposed model achieved robust performance even in environments using all three modalities, including audio. We also conducted extensive experiments to investigate the proposed model. In addition, we obtained state-of-the-art performance in the system response reasoning task.

NLE-DM: Natural-Language Explanations for Decision Making of Autonomous Driving Based on Semantic Scene Understanding

In recent years, the advancement of deep-learning technologies has greatly promoted the research progress of autonomous driving. However, deep neural network is like a black box. Given a specific input, it is difficult to explain the output of the network. Without explainable results, it would be unsafe to deploy deep networks in unseen environments or environments with potential unexpected situations. Especially for decision-making networks, inappropriate outputs could lead to severe traffic accidents. To provide a solution to this problem, we propose a deep neural network that jointly predicts the decision-making actions and corresponding natural-language explanations based on semantic scene understanding. Two types of explanations, the reasons of driving actions and the surrounding environment descriptions of the ego-vehicle, are designed. Both the reasons and descriptions are in the form of natural language. The decision-making actions could be explained by the corresponding reasons or the environment descriptions. We also release a large-scale dataset with hand-labelled ground truth including driving actions and environment descriptions. The superiority of our network over other methods is demonstrated on both our dataset and a public dataset.

A Simple Framework for Scene Graph Reasoning with Semantic Understanding of Complex Sentence Structure

A rapidly expanding multimedia environment in recent years has led to an explosive increase in demand for multimodality that can communicate with humans in various ways. Even though the convergence of vision and language intelligence has shed light on the remarkable success over the last few years, there is still a caveat: it is unknown whether they truly understand the semantics of the image. More specifically, how they correctly capture relationships between objects represented within the image is still regarded as a black box. In order to testify whether such relationships are well understood, this work mainly focuses on the Graph-structured visual Question Answering (GQA) task which evaluates the understanding of an image by reasoning a scene graph describing the structural characteristics of an image in the form of natural language together with the image. Unlike the existing approaches that have been accompanied by an additional encoder for scene graphs, we propose a simple yet effective framework using pre-trained multimodal transformers for scene graph reasoning. Inspired by the fact that a scene graph can be regarded as a set of sentences describing two related objects with a relationship, we fuse them into the framework separately from the question. In addition, we propose a multi-task learning method that utilizes evaluating the grammatical validity of questions as an auxiliary task to better understand a question with complex structures. This utilizes the semantic role labels of the question to randomly shuffle the sentence structure of the question. We have conducted extensive experiments to evaluate the effectiveness in terms of task capabilities, ablation studies, and generalization.

Towards FAIR Principles in Biodiversity Research: Enabling computable taxonomic descriptions and ecological data with Phenoscript

Taxonomic descriptions hold immense phenotypic data, but their natural language (NL) format poses challenges for computer analysis. In this talk, we will present Phenoscript, a user-friendly computer language enabling computer-readable species descriptions and automated phenotype comparisons, in accordance with FAIR (Findable, Accessible, Interoperable, Reusable) principles. Phenoscript facilitates the creation of semantic species descriptions that represent a knowledge graph composed of terms from predefined biological ontologies. A Phenoscript description resembles a NL description, but follows a specific language grammar. We have developed the Phenospy package: a Python-based Phenoscript toolkit. Phenospy converts Phenoscript descriptions into both NL format, facilitating scientific publication, and the Web Ontology Language (OWL) format, enabling downstream analysis and computable phenotypic comparisons. OWL is a standard for sharing semantic data on the Web. While initially designed for phenotypes, Phenoscript can be extended to create semantic ecological data, encompassing environmental traits, functional traits, and species interactions. We will discuss the integration of species and ecological traits encoded in Phenoscript into downstream analysis, highlighting its potential for phenomic-level research in biology.

Implementation of Chatbot System on Tourism Objects in Banyumas Regency with AIML and Chatterbot

Information technology can be applied to all fields, including tourism. Tourism object information media can be implemented into the chatbot system to make the information search process more flexible. Currently, searching for tourist spot information is often done manually; this makes tourist information services limited in time, while the need for tourism information must always be available. This research aims to build a chatbot system using Artificial Intelligence Markup Language (AIML) and ChatterBot methods. Both methods are accessed from libraries in Python using input in the form of natural language that has been processed into certain patterns. The pattern determination process is carried out by classifying a collection of questions on the chatbot using the Support Vector Machines (SVM) method. Then the classification is divided into five attributes, namely address, ticket price, facilities, description, and access. The SVM model built obtained an accuracy rate of 88%. Based on the testing results on both models that have been tested, the approach with AIML results in an accuracy rate in answering questions correctly of 90%, while ChatterBot is 40%.

An Examination of Studies on the Use of Chatbot Technology in the Field of Education

Today, it is observed that technologies are rapidly advancing and being utilized in educational environments. Among these technologies, chatbots, which have gained significant popularity with ChatGPT, play a crucial role in enhancing learning experiences. Chatbots offer numerous opportunities to support learning performance, increase engagement, and provide extracurricular feedback and assistance to students. These chatbots are capable of interacting with users in a natural language format, enabling seamless communication. They are designed to follow predefined patterns, answer questions, and provide guidance within their defined scope. This article aims to examine the existing research on the utilization of chatbots in educational environments to shed light on the significance of their implementation. To achieve this objective, 19 articles and papers related to the field were analyzed using Google Scholar, incorporating keywords such as 'chatbot+education' and 'chatbot in education'. The articles and papers were subjected to a descriptive analysis, covering sections such as the subject, purpose, methodology, results, discussion, and recommendations. Based on the review, it was observed that many studies focused on the usability and product design of chatbots in education. The samples used in these studies predominantly comprised university-level participants, with limited applications targeting larger student populations. It is anticipated that the data and results presented in this study will serve as a guide to identify existing applications and research gaps in the utilization of chatbots in education.

Memory-Aware Attentive Control for Community Question Answering With Knowledge-Based Dual Refinement

The question answering system in open domain enables a machine to automatically select and generate the answer for questions posed by humans in a natural language form on the website. Previous approaches seek effective ways of extracting the semantic features between question and answer, but the contextual information effects in semantic matching are still limited by short-term memory. As an alternative, we propose an internal knowledge-based end-to-end model, enhanced by an attentive memory network for both answer selection and answer generation tasks by considering the full advantages of the semantics and multifacts (i.e., timescales, topics, and context). In detail, we design a long-term memory to learn the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> fine-grained similarity representations, where two memory-aware mechanisms aggregate the series of semantic word-level and sentence-level similarities to support the coarse contextual information. Furthermore, we propose a novel memory refinement mechanism with the two-dimensional of writing heads that offer an efficient approach to multiview selection of the salient word pairs. In the training stage, we adopt the transformer-based transfer learning skill to effectively pretrain the model. Experimentally, we compare the state-of-the-art approaches on four public datasets, the experimental results show that the proposed model achieves competitive performance.

University Student Dropout Prediction Using Pretrained Language Models

Predicting student dropout from universities is an imperative but challenging task. Numerous data-driven approaches that utilize both student demographic information (e.g., gender, nationality, and high school graduation year) and academic information (e.g., GPA, participation in activities, and course evaluations) have shown meaningful results. Recently, pretrained language models have achieved very successful results in understanding the tasks associated with structured data as well as textual data. In this paper, we propose a novel student dropout prediction framework based on demographic and academic information, using a pretrained language model to capture the relationship between different forms of information. To this end, we first formulate both types of information in natural language form. We then recast the student dropout prediction task as a natural language inference (NLI) task. Finally, we fine-tune the pretrained language models to predict student dropout. In particular, we further enhance the model using a continuous hypothesis. The experimental results demonstrate that the proposed model is effective for the freshmen dropout prediction task. The proposed method exhibits significant improvements of as much as 9.00% in terms of F1-score compared with state-of-the-art techniques.

Sentiment Analysis and Corpus: Cognitive Perspective and Overhead-Accuracy Tradeoff

Human logical thinking is in the form of natural language. With the development of computer science technique, it becomes easier and more convenient for natural language processing. Therefore, a variety number of natural language processing applications have emerged. Sentiment analysis is one of these novel applications, and has been applied in many areas. In Amazon.com, there are a large number of user comments and product discussions, which can help a person to decide whether to buy a product or not without asking the opinions from friends and family members. Therefore, sentiment analysis on user comments and product discussions, such as Amazon review becomes increasingly useful and important. In this paper, the effect of corpus on sentiment analysis of Amazon review dataset with the aid of support vector machine are studied. We generate eight different size datasets from Amazon review dataset filtered by different word frequency in Corpus of Contemporary American, and conduct some experiments on these eight datasets. According to the experimental result, we make some conclusion and give some suggestions to facilitate researchers to make a trade-off between accuracy and experimental cost.

A novel word-graph-based query rewriting method for question answering

PurposeQuestion answering (QA) answers the questions asked by people in the form of natural language. In the QA, due to the subjectivity of users, the questions they query have different expressions, which increases the difficulty of text retrieval. Therefore, the purpose of this paper is to explore new query rewriting method for QA that integrates multiple related questions (RQs) to form an optimal question. Moreover, it is important to generate a new dataset of the original query (OQ) with multiple RQs.Design/methodology/approachThis study collects a new dataset SQuAD_extend by crawling the QA community and uses word-graph to model the collected OQs. Next, Beam search finds the best path to get the best question. To deeply represent the features of the question, pretrained model BERT is used to model sentences.FindingsThe experimental results show three outstanding findings. (1) The quality of the answers is better after adding the RQs of the OQs. (2) The word-graph that is used to model the problem and choose the optimal path is conducive to finding the best question. (3) Finally, BERT can deeply characterize the semantics of the exact problem.Originality/valueThe proposed method can use word-graph to construct multiple questions and select the optimal path for rewriting the question, and the quality of answers is better than the baseline. In practice, the research results can help guide users to clarify their query intentions and finally achieve the best answer.

A knowledge-based decision support system for incident traffic congestion management

A Knowledge-based Decision Support System (KB-DSS) based on a multi-stage Fuzzy Logic Controller (MS-FLC) is developed for traffic congestion management on expressways. The MS-FLC receives real-time traffic and incident data to analyse and anticipate the traffic conditions, to recommend alternative control measures in the form of natural languages for the human operator to select control decisions, and to calculate control settings to manage traffic congestion. In a case study, the KB-DSS is evaluated on a simulated network in comparison to ALINEA\Q, a popular ramp control method, across various traffic and incident situations. The results showed that: (i) the KB-DSS provides a systematic procedure in deriving control actions and a good capability to deliver linguistic expressions; (ii) the KB-DSS outperforms ALINEA\Q with respect to global objectives across many scenarios, attains significant improvements of mainline travel conditions and substantial reductions of ramp queues. These advantages make the KB-DSS a robust tool for traffic control for incident congestion management on expressways.

NLP based AI Voice Assistant

All electronic devices will operate in the very near future using a remote helper, which is easily accessible but requires a certain level of weakness.This setup allowed users to access the system via voice commands. The user can ask the assistant to do anything that the framework is capable of doing, such as play music, open a certain application, open tabs, open websites, and so forth. Programming experts known as voice associates are able to understand human speech and respond with carefully crafted voices. Our voice assistant's primary goal is to educate individuals and provide quick, calculated results. The voice help receives voice input from our microphones (wired and Bluetooth) and turns it into computer- understandable English before providing the user with the necessary solutions and replies. This assistant makes a connection to the Internet to deliver answers to the user's queries. A natural language processing algorithm enables computer systems to communicate using many forms of natural human language. Keywords: NLP (Natural language processing), TTS (Text-to-Speech), AI (Artificial Intelligence), Voice Assistant, NLTK(Natural Language Toolkit),Python.

Integrating machine learning with linguistic features: A universal method for extraction and normalization of temporal expressions in Chinese texts

Background and ObjectiveWith the rapid development of information dissemination technology, the amount of events information contained in massive texts now far exceeds the intuitive cognition of humans, and it is hard to understand the progress of events in order of time. Temporal information runs through the whole process of beginning, proceeding, and ending of events, and plays an important role in many natural language processing applications, such as information extraction, question answering, and text summary. Accurately extracting temporal information from Chinese texts and automatically mapping the temporal expressions in natural language to the time axis are crucial to understanding the development of events and dynamic changes in them. MethodsThis study proposes a method integrating machine learning with linguistic features (IMLLF) for extraction and normalization of temporal expressions in Chinese texts to achieve the above objectives. Linguistic features are constructed by analyzing the expression rules of temporal information, and are combined with machine learning to map the natural language form of time onto a one-dimensional timeline. The web text dataset we build is divided into five parts for five-fold cross-validation, to compare the influence of different combinations of linguistic features and different methods. In the open medical dialog dataset, based on the training model obtained from the web text dataset, 200 disease descriptions are randomly selected each time for three rounds of experiments. ResultsThe F1 of multi-feature fusion is 95.2%, which is better than the single-feature and double-feature combination. The results of experiments showed that the proposed IMLLF method can improve the accuracy of recognition of temporal information in Chinese to a greater extent than classical methods, with an F1-score of over 95% on the web text dataset and medical conversation dataset. In terms of the normalization of time expressions, the accuracy of the IMLLF method is higher than 93%. ConclusionsIMLLF has better results in extracting and normalizing time expressions on the web text dataset and the medical conversation dataset, which verifies the universality of IMLLF to identify and quantify temporal information. IMLLF method can accurately map the time information to the time axis, which is convenient for doctors to intuitively see when and what happened to the patient, and helps to make better medical decisions.

Biologically Inspired Design Concept Generation Using Generative Pre-Trained Transformers

Abstract Biological systems in nature have evolved for millions of years to adapt and survive the environment. Many features they developed can be inspirational and beneficial for solving technical problems in modern industries. This leads to a specific form of design-by-analogy called bio-inspired design (BID). Although BID as a design method has been proven beneficial, the gap between biology and engineering continuously hinders designers from effectively applying the method. Therefore, we explore the recent advance of artificial intelligence (AI) for a data-driven approach to bridge the gap. This paper proposes a generative design approach based on the generative pre-trained language model (PLM) to automatically retrieve and map biological analogy and generate BID in the form of natural language. The latest generative pre-trained transformer, namely generative pre-trained transformer 3 (GPT-3), is used as the base PLM. Three types of design concept generators are identified and fine-tuned from the PLM according to the looseness of the problem space representation. Machine evaluators are also fine-tuned to assess the mapping relevancy between the domains within the generated BID concepts. The approach is evaluated and then employed in a real-world project of designing light-weighted flying cars during its conceptual design phase The results show our approach can generate BID concepts with good performance.

Design of a Novel Information System for Semi-automated Management of Cybersecurity in Industrial Control Systems

There is an urgent need in many critical infrastructure sectors, including the energy sector, for attaining detailed insights into cybersecurity features and compliance with cybersecurity requirements related to their Operational Technology (OT) deployments. Frequent feature changes of OT devices interfere with this need, posing a great risk to customers. One effective way to address this challenge is via a semi-automated cyber-physical security assurance approach, which enables verification and validation of the OT device cybersecurity claims against actual capabilities, both pre- and post-deployment. To realize this approach, this article presents new methodology and algorithms to automatically identify cybersecurity-related claims expressed in natural language form in ICS device documents. We developed an identification process that employs natural language processing (NLP) techniques with the goal of semi-automated vetting of detected claims against their device implementation. We also present our novel NLP components for verifying feature claims against relevant cybersecurity requirements. The verification pipeline includes components such as automated vendor identification, device document curation, feature claim identification utilizing sentiment analysis for conflict resolution, and reporting of features that are claimed to be supported or indicated as unsupported. Our novel matching engine represents the first automated information system available in the cybersecurity domain that directly aids the generation of ICS compliance reports.

Investigation of Hamming Network Procedures for Controlling Service Systems with Smprecisely Defined and Natural Language Data

Purpose. Models and methods, as well as software tools for the tasks of planning the flow of orders of service systems, or service systems (S&amp;S), are quite widespread. The task of developing processes for classification and management of S&amp;S based on the associative memory model of the Hamming neural network (HNN) with imprecisely defined data characteristics is relevant today, and has theoretical and practical significance. The main purpose of the work is to develop and study mathematical models of Hamming network procedures for S&amp;S with imprecisely defined and natural language data characteristics, comparative analysis of fuzzy set models and CF confidence coefficients. Methodology. The paper uses a modification of the Hamming neural network procedures and numerical experimental studies of the comparative possibilities of using fuzzy sets μX (X → [0; 1]) as models of primary data, as well as expert confidence indicators, confidence factors CF(A) from the set [–1; +1]. Findings. The formation and study of improved models of Hamming neural networks intended for classification procedures in S&amp;S with imprecisely defined and natural language data characteristics is carried out. Originality. For the first time, the comparative possibilities of using fuzzy values (NVs) and CFs as models for representing the properties of incomplete and imprecisely defined data, as well as data in natural language form, are investigated for the tasks of classification and management of S&amp;S. At the same time, the advantages of the CF confidence factor model are established and appropriate procedures for classifying and managing S&amp;S are formed. Practical value. The models and procedures for classifying the properties of multi-parameter S&amp;S objects based on modified Hamming neural networks developed in the article allow to effectively solve a wide range of tasks in the field of S&amp;S management under uncertainty and incompleteness of primary data.

Development of knowledge-based clinical decision support system for the management of cardiovascular diseases

Abstract Background Knowledge-based clinical decision support systems (CDSS) are technological tools that analyze patient information and present justified diagnostic and therapeutic suggestions to the user. They improve adherence to evidence-based medicine and reduce medical errors. Material and methods We have developed an advanced CDSS with the following components: 1. Knowledge Management System, which allows the introduction of interoperable clinical variables organized in taxonomies, and the generation, validation and direct maintenance of decision rules by experts doctors. 2. Smart Assistant for Medical Reports Generation, which facilitates user interaction for the introduction of clinical data (natural language and/or data entry forms) and decission making supported by real-time suggestions that are automatically generated by 3. Inference Engine. Rules and variables derived from the most relevant cardiovascular Clinical Practice Guidelines (CPGs) are introduced by cardiologists into the system. The rules undergo a process of peer review and internal and external validation with anonymized clinical cases. Results More than 2,000 clinical variables and more than 5,000 rules corresponding to more than 80 CPGs and consensuns papers have been created and validated. An internal and external pilot validation of the system has been successfully carried out for the main cardiovascular problems, such as heart failure, atrial fibrillation, valvulopathies, coronary syndromes, cardiomyopathies, hypertension, diabetes and dyslipidaemias, using more than 400 cases. Conclusion Our CDSS is capable of providing real-time diagnostic and therapeutic recommendations for complex scenarios in cardiovascular disease, combining thousands of variables and decision rules in a single system. Prospective evaluation of the system in real clinical environments, certification and integration in Electronic Medical Records for clinical use is required. Funding Acknowledgement Type of funding sources: Private company. Main funding source(s): DILEMMA solutions, SL

IvCDS: An End-to-End Driver Simulator for Personal In-Vehicle Conversational Assistant.

An advanced driver simulator methodology facilitates a well-connected interaction between the environment and drivers. Multiple traffic information environment language processing aims to help drivers accommodate travel demand: safety prewarning, destination navigation, hotel/restaurant reservation, and so on. Task-oriented dialogue systems generally aim to assist human users in achieving these specific goals by a conversation in the form of natural language. The development of current neural network based dialogue systems relies on relevant datasets, such as KVRET. These datasets are generally used for training and evaluating a dialogue agent (e.g., an in-vehicle assistant). Therefore, a simulator for the human user side is necessarily required for assessing an agent system if no real person is involved. We propose a new end-to-end simulator to operate as a human driver that is capable of understanding and responding to assistant utterances. This proposed driver simulator enables one to interact with an in-vehicle assistant like a real person, and the diversity of conversations can be simply controlled by changing the assigned driver profile. Results of our experiment demonstrate that this proposed simulator achieves the best performance on all tasks compared with other models.

GCP: Graph Encoder With Content-Planning for Sentence Generation From Knowledge Bases.

A knowledge base is a large repository of facts usually represented as triples, each consisting of a subject, a predicate, and an object. The triples together form a graph, i.e., a knowledge graph. The triple representation in a knowledge graph offers a simple interface for applications to access the facts. However, this representation is not in a natural language form, which is difficult for humans to understand. We address this problem by proposing a system to translate a set of triples (i.e., a graph) into natural sentences. We take an encoder-decoder based approach. Specifically, we propose a Graph encoder with Content-Planning capability (GCP) to encode an input graph. GCP not only works as an encoder but also serves as a content-planner by using an entity-order aware topological traversal to encode a graph. This way, GCP can capture the relationships between entities in a knowledge graph as well as providing information regarding the proper entity order for the decoder. Hence, the decoder can generate sentences with a proper entity mention ordering. Experimental results show that GCP achieves improvements over state-of-the-art models by up to 3.6%, 4.1%, and 3.8% in three common metrics BLEU, METEOR, and TER, respectively. The code is available at (https://github.com/ruizhang-ai/GCP/).

Development of knowledge-based clinical decision support system for the management of cardiovascular diseases

Abstract Background Knowledge-based clinical decision support systems (CDSS) are technological tools that analyze patient information and present justified diagnostic and therapeutic suggestions to the user. They improve adherence to evidence-based medicine and reduce medical errors. Material and methods We have developed an advanced CDSS with the following components: 1. Knowledge Management System, which allows the introduction of interoperable clinical variables organized in taxonomies, and the generation, validation and direct maintenance of decision rules by experts doctors. 2. Smart Assistant for Medical Reports Generation, which facilitates user interaction for the introduction of clinical data (natural language and/or data entry forms) and decission making supported by real-time suggestions that are automatically generated by 3. Inference Engine. Rules and variables derived from the most relevant cardiovascular Clinical Practice Guidelines (CPGs) are introduced by cardiologists into the system. The rules undergo a process of peer review and internal and external validation with anonymized clinical cases. Results More than 2,000 clinical variables and more than 5,000 rules corresponding to more than 80 CPGs and consensuns papers have been created and validated. An internal and external pilot validation of the system has been successfully carried out for the main cardiovascular problems, such as heart failure, atrial fibrillation, valvulopathies, coronary syndromes, cardiomyopathies, hypertension, diabetes and dyslipidaemias, using more than 400 cases. Conclusion Our CDSS is capable of providing real-time diagnostic and therapeutic recommendations for complex scenarios in cardiovascular disease, combining thousands of variables and decision rules in a single system. Prospective evaluation of the system in real clinical environments, certification and integration in Electronic Medical Records for clinical use is required. Funding Acknowledgement Type of funding sources: Private company. Main funding source(s): DILEMMA solutions, SL