Human-machine Dialogue Systems Research Articles

Ultra-broad sensing range, high sensitivity textile pressure sensors with heterogeneous fibre architecture and molecular interconnection strategy

Textile-based pressure sensors have garnered extensive attentions owing to their potentials in health monitoring, human–machine interactions, wearable human–machine interfaces (HMIs) and soft robotics. High sensitivity over a broad sensing range are highly desired yet challenging for textile pressure sensors due to the incompressibility of matrix materials and the stiffening of microstructures. Herein, we developed a high performance pressure sensor based on the silk/3-Aminopropyltriethoxy-silane/titanium carbide (Silk/APTES/MXene) film. The silk is designed with heterogeneous fiber architecture, which enables rich and multi-level contact pattern and could compensate for the effect of structural stiffening. Furthermore, APTES molecules are used to enhance the interface bonding between MXene and silk, promoting the mechanical stability of the sensor. Benefiting from these advantages, the Silk/APTES/MXene sensor device is achieved with high sensitivity of 17.1 KPa−1 over an ultra-broad sensing range up to 3.3 MPa (R2 = 0.997), ultra-low detection limit (0.25 Pa), and low fatigue toughness after 5000 cycles loading and unloading. With these merits, we have demonstrated its capability for a series of human motion detection (such as foot movement detection, arm/wrist/finger bending, etc) and an overall accuracy of 95 % is obtained with the help of CNN-based convolution neural architecture. More significantly, we have built an entire intelligent human–machine dialogue system and a patient-audience dialogue system, from lip/sign language recognition to real-time screen display and final voice output.

A Mood Semantic Awareness Model for Emotional Interactive Robots.

The rapid development of natural language processing technology and improvements in computer performance in recent years have resulted in the wide-scale development and adoption of human-machine dialogue systems. In this study, the Icc_dialogue model is proposed to enhance the semantic awareness of moods for emotional interactive robots. Equipped with a voice interaction module, emotion calculation is conducted based on model responses, and rules for calculating users' degree of interest are formulated. By evaluating the degree of interest, the system can determine whether it should transition to a new topic to maintain the user's interest. This model can also address issues such as overly purposeful responses and rigid emotional expressions in generated replies. Simultaneously, this study explores topic continuation after answering a question, the construction of dialogue rounds, keyword counting, and the creation of a target text similarity matrix for each text in the dialogue dataset. The matrix is normalized, weights are assigned, and the final text score is calculated. In the text with the highest score, the content of dialogue continuation is determined by calculating a subsequent sentence with the highest similarity. This resolves the issue in which the conversational bot fails to continue dialogue on a topic after answering a question, instead waiting for the user to voluntarily provide more information, resulting in topic interruption. As described in the experimental section, both automatic and manual evaluations were conducted to validate the significant improvement in the mood semantic awareness model's performance in terms of dialogue quality and user experience.

Beyond Words: An Intelligent Human-Machine Dialogue System with Multimodal Generation and Emotional Comprehension

Intelligent service robots have become an indispensable aspect of modern-day society, playing a crucial role in various domains ranging from healthcare to hospitality. Among these robotic systems, human-machine dialogue systems are particularly noteworthy as they deliver both auditory and visual services to users, effectively bridging the communication gap between humans and machines. Despite their utility, the majority of existing approaches to these systems primarily concentrate on augmenting the logical coherence of the system’s responses, inadvertently neglecting the significance of user emotions in shaping a comprehensive communication experience. To tackle this shortcoming, we propose the development of an innovative human-machine dialogue system that is both intelligent and emotionally sensitive, employing multimodal generation techniques. This system is architecturally comprised of three components: (1) data collection and processing, responsible for gathering and preparing relevant information, (2) a dialogue engine, which generates contextually appropriate responses, and (3) an interaction module, responsible for facilitating the communication interface between users and the system. To validate our proposed approach, we have constructed a prototype system and conducted an evaluation of the performance of the core dialogue engine by utilizing an open dataset. The results of our study indicate that our system demonstrates a remarkable level of multimodal generation response, ultimately offering a more human-like dialogue experience.

A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition

As an important development direction of natural language processing, emotion recognition in conversation (ERC) remains a challenge in sentiment analysis. Given the large-scale dialogue datasets and their wide application in the fields of recommendation systems and human–machine dialogue systems, researchers have begun to pay more attention to the issue of ERC. In recent research, the task of ERC has been largely based on the graph structure to model the speaker level. However, most existing studies simply splice multimodal features, and the heterogeneity of multimodal features tends to be overlooked. Hence, this paper proposes a multivariate messaging framework to embed heterogeneous information into multimodal relational graphs. In the process of aggregating graph node information, we take into account the homogeneity of nodes and assign different weights to different nodes so as to better aggregate semantic information. In order to improve the robustness of the model, we utilize the mechanism of sharing weights among neighbors to reduce the number of network parameters and improve the generalization ability of the model. In so doing, the node information is aggregated through the constructed graph network, and the final semantic vector representation is obtained. Experiments over two benchmark datasets for ERC show that our proposed model achieves improved performance in accuracy and F1 value.

Job sharing between human professionals and chatbots: How should ‘handovers’ happen?

Job sharing between humans and artificial intelligence (AI) systems is likely to become increasingly common in several domains of work. In this paper, we examine mechanisms for managing job sharing for one particular class of AI system: human–machine dialogue systems, or chatbots. This is a useful case to consider, as several mechanisms for managing job sharing are already emerging in these systems, and these mechanisms draw on those that human professionals already use to share work among themselves. A key concept in this domain is that of a ‘handover’, where a client is passed from one worker to another. We identify different types of handover, for human and AI workers, and discuss a range of issues that govern how these should take place for effective job sharing. We identify several questions that arise for engineers designing dialogue systems that support handover functionality, relating to timing and transparency. We conclude by arguing that handovers provide a useful way for structuring discussions of job sharing between humans and AI systems, in dialogue-based domains and perhaps beyond.

A Corpus-Based Study on Feedback in Daily Conversation: Forms, Position and Contexts.

Understanding how feedback is employed in various forms, positions, and contexts can provide valuable insights into improving communication and the design of human-machine dialogue systems. This paper aims to deepen the understanding of feedback in daily conversation and investigate how feedback is employed in various linguistic forms, position, preceding and following contexts, using a large corpus of telephone conversations. The study identifies three subclasses of feedback, including understandings, agreements, and answers, which account for almost one-third of the total utterances in the corpus. Acknowledge (backchannel) is the most frequently used subtype of feedback, accounting for almost 60% of the feedback, and is primarily used for conversational management and maintenance. Assessment/appreciation, on the other hand, is used less frequently, accounting for less than 10% of feedback, and is mainly realized by more creative, unpredictable, longer forms. The analysis also reveals that speakers are intentional in distinguishing the three subclasses of feedback based on various variables, such as position and the proximal discourse environment. Furthermore, the three subclasses of feedback are restricted by the function ofpreceding contexts, which shape the length of the remaining turn. The study suggests that future research should focus on exploring the individual differences and investigating the possible variations across different cultures and languages.

Research on Oral English Dialogue Understanding Based on Deep Learning

Oral English dialogue is a crucial part of a dialogue system that enables a computer to “understand” the input language as a human does, so the performance of a dialogue system is closely related to the performance of oral English dialogue understanding. In task-based human-machine dialogue systems, external knowledge bases can provide the machine with valid information beyond the training data, helping the model to better perform the oral English dialogue comprehension task. In this paper, we propose a deep recurrent neural network based on feature fusion, which directly stacks multiple nodes at a single time node to deepen the complexity of nonlinear transformations. The feature fusion network structure is applied to the ATIS dataset for oral English dialogue comprehension experiments, and the experimental results demonstrate that the feature fusion RNN network can further improve the effectiveness of the RNN network and the GRU network structure unit can obtain better results among different RNN node units.

Increasing naturalness of human–machine dialogue: The users’ choices inference of options in machine-raised questions

In many practical applications, the machine needs to actively ask humans to obtain their intents. The process that the machine raises questions and users return answers is called reverse QA, which is an important part of a human–machine dialogue. However, in many dialogue systems, the machine restricts users from answering questions by clicking on option items, which is unnatural and restricted. In addition, this method may lose important information expressed by users. Users should be allowed to answer questions in natural language in a more natural and intelligent dialogue system. To obtain users’ intents, users’ choices of questions’ options must be inferred from their answers. In this paper, we propose an advanced answer understanding network (UCINet) which infers users’ choices of options in machine-raised questions accurately and efficiently according to the users’ answer. Furthermore, metric learning is introduced for the model to learn better text representations. Based on the assumption that texts are determined by both semantics and styles, we propose a style-based answer generation network (SAGNet) which can generate various answers with different styles for a question. The generated answers are used to achieve data augmentation for UCINet’s training. Experimental results on two reverse QA data sets demonstrate that UCINet achieves impressive results compared to other strong competitors. Using SAGNet for answer generation, we obtain answers with various styles and good quality. Our work can be widely used in intelligent customer service, mobile phone assistants, and other human–machine dialogue systems.

A survey of multi-party dialogue research based on deep learning

In recent years, with the extensive application of deep learning technology, breakthroughs have been made in the study of human-computer dialogue. However, most of the current human-machine dialogue systems are designed under the assumption that both parties are involved, and the research and application of more challenging multi-party human-machine dialogues are not yet mature. Based on the field of natural language processing, this paper will review the research progress of multi-party dialogue based on deep learning in recent years. First, from the perspective of human-machine dialogue, we sort out the key problems and existing solutions of the multi-party dialogue system; then, we introduce other natural language processing tasks based on multi-party dialogue; afterwards, we summarize the existing multi-party dialogue research dataset and make a comparative analysis of limitations on the existing dataset; Finally, we look forward to the future development trend of multi-party dialogue research.

Research on Improved Intelligent Generative Dialogue Algorithm Based on Knowledge Graph

The rise of a new generation of artificial intelligence technology, represented by deep learning, has promoted the vigorous development of natural language processing technology. As a typical application of natural language processing technology, human-machine intelligent dialogue system, coupled with its commercial value in the fields of voice assistants and chat robots, has become a hot topic in the current academic and industrial circles. In the past, the responses generated by intelligent dialogue robots is single and universal, and even the content is inappropriate. Therefore, this study proposes to use the knowledge graph as the background knowledge when the dialogue model generates the responses[1]. In order to meet the needs of users more closely, it also proposes to introduce the user information participating in the dialogue and the dialogue scene into the model. The model is trained and evaluated on the DuRecDial public dataset, and the optimized model is compared with the original model. The experimental results show that the model with these two modules has better effect than the original model, especially in the generation index, the F1, BLEU2 and DIST-2 indexes have been improved by 0.91%, 0.5% and 0.9% respectively.

A Survey on Learning-Based Approaches for Modeling and Classification of Human-Machine Dialog Systems.

With the rapid development from traditional machine learning (ML) to deep learning (DL) and reinforcement learning (RL), dialog system equipped with learning mechanism has become the most effective solution to address human-machine interaction problems. The purpose of this article is to provide a comprehensive survey on learning-based human-machine dialog systems with a focus on the various dialog models. More specifically, we first introduce the fundamental process of establishing a dialog model. Second, we examine the features and classifications of the system dialog model, expound some representative models, and also compare the advantages and disadvantages of different dialog models. Third, we comb the commonly used database and evaluation metrics of the dialog model. Furthermore, the evaluation metrics of these dialog models are analyzed in detail. Finally, we briefly analyze the existing issues and point out the potential future direction on the human-machine dialog systems.

A Human–Machine Interaction Scheme Based on Background Knowledge in 6G-Enabled IoT Environment

6G-Enabled Internet of Things (IoT) is about to open a new era of Internet of Everything (IoE). It creates favorable conditions for new application services. The human-machine dialogue system, one of the most important forms of human–machine interaction, is expected to replace mobile applications in the future. This article proposes a dialogue generation scheme named background knowledge-aware dialogue generation model with pretrained encoders (BKADGPE). Dialogue generation, which takes the context as input and response as output, is a sequence-to-sequence (Seq2Seq) task. Instead of only generating the response based on the previous sequence of utterances, background knowledge-aware dialogue generation is also relying on background knowledge documents. This is because people often communicate based on their background knowledge. This article divides it into two tasks: 1) a knowledge selection task and 2) a response generation task. One of the latest language pretraining models, a lite bidirectional encoder representations from transformers (ALBERT), is applied as the encoder. In the knowledge selection task, ALBERT adds the linear layer and softmax layer to predict the content-related knowledge span. In the response generation task, the ALBERT after fine-tuning through the knowledge selection task adds the left-context-only transformer with a copy mechanism to incorporate background knowledge span into the generated response. Empirical studies on the HOLL-E dataset show that the result of BKADGPE is better than the related works.

Research and application of semantic understanding based on Attention-RNN

Abstract Spoken language understanding module is an important part of the human-machine dialogue system, which contains two important subtasks of intent recognition and slot filling. In the current human-machine dialogue system, intent recognition and slot filling are mostly in the form of independent tasks Exist, it is difficult to further mine the latent semantic information in the text. This paper attempts to introduce attention mechanism in the process of intent recognition, and share the latent information of intent recognition and slot filling to improve the result of intent recognition and slot filling. Experiments show that the introduction of attention mechanism and the sharing of hidden information are important for slot filling and intent recognition and have a certain improvement effect.

Human–machine dialogue modelling with the fusion of word- and sentence-level emotions

Emotion intelligence plays an important role in building a successful human–machine dialogue system. However, the extreme difficulty of capturing emotional information of social text, as well as the weakness of generative models for learning emotional expression, limits the performance of existing dialogue system. Combining dictionary matching and machine learning, this paper proposes a generative model, which fuses the word- and sentence-level emotions, to model the dialogue text and learn emotional expression. The model first obtains the emotional embedding of each word through dictionary matching, then concatenates the emotional word embedding with its traditional word embedding, and the finally formed vector is taken as the input of the encoder. In order to control the emotional feature of the generated response, our model employs a BernoulliNB-based classifier to extract the emotional feature of the post, which is used as the attribute of the original text, and subsequently adds it to the decoder. For further significantly improving the emotional expression of the response, the model leverages a discriminator to constrain the latent variable, which enables the latent variable to encode better the information of emotional feature of the post. With this model, we can generate the dialogue text that is consistent with the original emotion. Experimental results on our Twitter dataset demonstrate that our model outperforms several state-of-the-art methods in the emotion accuracy and quality of generated texts.

Research on acquisition and recognition of Naxi speaker's speech information

The human–machine dialogue system allows people and the machine to communicate basically. This technology is also attracting more and more attention. It is conducive to the realisation of resource sharing, inheritance, and protection of Dongba culture worldwide to analyse the Naxi speech recognition for the protection and inheritance needs of Naxi Dongba culture. The model was established for Naxi speaker recognition, and the method is studied with text-independent speaker speech voice recognition. The practical analysis method was used to compare the application effect of Gaussian mixture model (GMM) and hidden Markov model (HMM) in the processing of speaker recognition in Naxi speaker's pronunciation. It is very useful to provide a theoretical basis for the establishment of a Naxi language-based human–computer interaction system to find out the speaker's voice characteristics, personality differences, and different regions. This is a theoretically necessary analysis of ethnic minority languages and the protection of ethnic and folk cultures. It can also provide practical significance for carrying out ethnic cultural differences analysis.

Application of Human-Machine Dialogue in Foreign Language Teaching at Universities

With the development of internet technology, communication technology and artificial intelligence (AI), the versatile and convenient human-machine dialogue and interaction system has gradually penetrated into the education field and become a new type of teaching method. Focusing on the subject of the application of human-machine dialogue system in foreign language teaching at universities, this paper elaborated on how to apply the human-machine dialogue system in foreign language teaching, what effect can be achieved, and what application strategy can be adopted in human-machine dialogue interactive teaching. Combined with the problems existing in daily teaching, this paper presented suggestions for applying the human-machine dialogue system, which is expected to improve the effects of foreign language teaching.

Review of Intent Detection Methods in the Human-Machine Dialogue System

Spoken language understanding is an important part of the human-machine dialogue system, intent detection is a sub-task of spoken language understanding, and it is very important. The accuracy of intent detection is directly related to the performance of semantic slot filling, and it is helpful to the following research of the dialogue system. Considering the difficulty of intent detection in human-machine dialogue system, the traditional machine learning method cannot understand the deep semantic information of user’s discourse. This paper mainly analyzes, compares and summarizes the deep learning methods applied in the research of intent detection in recent years, and further considers how to apply deep learning model to multi-intent detection task, so as to promote the research of multi-intent detection methods based on deep neural network.

DREAM: Classification scheme for dialog acts in clinical research query mediation

Clinical data access involves complex but opaque communication between medical researchers and query analysts. Understanding such communication is indispensable for designing intelligent human–machine dialog systems that automate query formulation. This study investigates email communication and proposes a novel scheme for classifying dialog acts in clinical research query mediation. We analyzed 315 email messages exchanged in the communication for 20 data requests obtained from three institutions. The messages were segmented into 1333 utterance units. Through a rigorous process, we developed a classification scheme and applied it for dialog act annotation of the extracted utterances. Evaluation results with high inter-annotator agreement demonstrate the reliability of this scheme. This dataset is used to contribute preliminary understanding of dialog acts distribution and conversation flow in this dialog space.

Visual perception, language and gesture: A model for their understanding in multimodal dialogue systems

The way we see the objects around us determines speech and gestures we use to refer to them. The gestures we produce structure our visual perception. The words we use have an influence on the way we see. In this manner, visual perception, language and gesture present multiple interactions between each other. The problem is global and has to be tackled as a whole in order to understand the complexity of reference phenomena and to deduce a formal model. This model may be useful for any kind of human–machine dialogue system that focuses on deep comprehension. We show how a referring act takes place in a contextual subset of objects. This subset is called ‘reference domain’ and is implicit. It can be deduced from a lot of clues. Among these clues are those which come from the visual context and those which come from the multimodal utterance. We present the ‘multimodal reference domain’ model that takes these clues into account and that can be exploited in a multimodal dialogue system when interpreting.

Accent Patterning on Domain-Related Information in Swedish Travel Dialogues

Investigation of travel-domain dialogues reveals travel-agent (≈System) utterances with intonational contours characterized by late-timed focal accents on given information. These accents occur on content words in utterance-initial position. The accentuation can be assumed to be related to the interactive nature of the dialogue in which the travel agent links back to a domain-related concept introduced by the client (≈User) and comments on it in an engaged manner. A perception test using constructed human-machine dialogues in which the machine (synthesized) responses vary as to the type of accent pattern on the initial words was developed to test listeners' preference for accent type. Results indicate that i) focal accents on domain-related utterance-initial given concepts are indeed preferred to nonfocal accents and that ii) late-timed focal accents are preferred to early-timed focal accents. These results have implications for the design of the prosody-generating component of human-machine dialogue systems.