Conversational Interface Research Articles

InstructableCrowd: Creating IF-THEN Rules for Smartphones via Conversations with the Crowd

Natural language interfaces have become a common part of modern digital life. Chatbots utilize text-based conversations to communicate with users; personal assistants on smartphones such as Google Assistant take direct speech commands from their users; and speech-controlled devices such as Amazon Echo use voice as their only input mode. In this paper, we introduce InstructableCrowd, a crowd-powered system that allows users to program their devices via conversation. The user verbally expresses a problem to the system, in which a group of crowd workers collectively respond and program relevant multi-part IF-THEN rules to help the user. The IF-THEN rules generated by InstructableCrowd connect relevant sensor combinations (e.g., location, weather, device acceleration, etc.) to useful effectors (e.g., text messages, device alarms, etc.). Our study showed that non-programmers can use the conversational interface of InstructableCrowd to create IF-THEN rules that have similar quality compared with the rules created manually. InstructableCrowd generally illustrates how users may converse with their devices, not only to trigger simple voice commands, but also to personalize their increasingly powerful and complicated devices.

Usability Evaluation of Self-Diagnosis User Interfaces for Depression: A Comparative Study among Paper-Based, Text-Based, and Voice-Based Conversational Interfaces

Usability Evaluation of Self-Diagnosis User Interfaces for Depression: A Comparative Study among Paper-Based, Text-Based, and Voice-Based Conversational Interfaces

Reference Model and Architecture of Interactive Cognitive Health Advisor based on Evolutional Cyber-physical Systems

This study presents a reference model (RM) and the architecture of a cognitive health advisor (CHA) that integrates information with ambient intelligence. By controlling the information using the CHA platform, the reference model can provide various ambient intelligent solutions to a user. Herein, a novel approach to a CHA RM based on evolutional cyber-physical systems is proposed. The objective of the CHA RM is to improve personal health by managing data integration from many devices as well as conduct a new feedback cycle, which includes training and consulting to improve quality of life. The RM can provide an overview of the basis for implementing concrete software architectures. The proposed RM provides a standardized clarification for developers and service designers in the design and implementation process. The CHA RM provides a new approach to developing a digital healthcare model that includes integrated systems, subsystems, and components. New features for chatbots and feedback functions set the position of the conversational interface system to improve human health by integrating information, analytics, and decisions and feedback as an advisor on the CHA platform.

Towards Emotion-Sensitive Conversational User Interfaces in Healthcare Applications.

Perception of emotions and adequate responses are key factors of a successful conversational agent. However, determining emotions in a healthcare setting depends on multiple factors such as context and medical condition. Given the increase of interest in conversational agents integrated in mobile health applications, our objective in this work is to introduce a concept for analyzing emotions and sentiments expressed by a person in a mobile health application with a conversational user interface. The approach bases upon bot technology (Synthetic intelligence markup language) and deep learning for emotion analysis. More specifically, expressions referring to sentiments or emotions are classified along seven categories and three stages of strengths using treebank annotation and recursive neural networks. The classification result is used by the chatbot for selecting an appropriate response. In this way, the concerns of a user can be better addressed. We describe three use cases where the approach could be integrated to make the chatbot emotion-sensitive.

Communicability of traditional interfaces VS chatbots in healthcare and smart home domains

ABSTRACTThis paper presents a study about communicability of conversational interfaces (namely chatbots) under a semiotic perspective. A chatbot is a software system that allows you to simulate real conversations between devices and users by means of a conversational interface (CI). After introducing the chatbot concept, focusing on its advantages and issues, we will present two domains of use in which chatbot interfaces can be effective: healthcare and smart home. For carrying out simple tasks such as finding information or triggering operations, users need an easy-to-use and to an easy-to-learn system to communicate with. To face this, conversational interfaces represent the latest trend in the field of digital design. For studying the communicability aspects of a CI, we carried out a user test to compare traditional and chatbot interfaces. This paper aims at evaluating the benefits at the communicability level of a chatbot in comparison to traditional GUI for incrementing the effectiveness and efficacy of communication between users and the system specifically for users with poor attitude in using technologies. In details, we evaluated the communicability of two prototypes that can be used to solve simple tasks in order to favour user inclusion, including everyone with very little exposure to technologies.

Conception of a Conversational Interface to Provide a Guided Search of Study Related Data

Since the beginning of software development, solution approaches and technolo-gies have changed massively, including the requirements for a user interface. At the very beginning, it was the desktop application, with a classic Graphical User Interface (GUI), which fulfilled the needs of a user. Nowadays, many applica-tions moved to web respectively mobile and the user behavior changed. A very modern concept to handle the communication between a computer and a user is a chatbot. The range of functions of a chatbot can be very simple up to complex ar-tificial intelligence based solutions. This publication focuses on a chatbot solution for Graz University of Technology (TU Graz), which should support the student by finding study related information via a conversational interface.

Conversational User Interfaces and Interactions

The four papers in this special section explore applications, systems, methodologies, and technologies that relate broad aspects to realize successful conversational user interfaces and interactions. Speech recognition has become a part of our daily lives. Today, we can enjoy a wide array of conversational user interfaces and speech-based interactive services, from speech-based personal assistants in our smartphones and smart speakers, e.g.,Amazon Echo, Alexa, Cortana, and Google Home, to chatbots in traditional messaging interfaces. These interfaces are becoming pervasive as reflected by the exponential growth in their adoption. Hence, the expectations are that computer based conversational systems will soon reach the level of human–human conversation.

브랜드 서비스 강화를 위한 챗봇 CUI 체험 사례 연구

In this research, we look at concrete examples of utilization, mainly focusing on service fields on how to strengthen the chatbot marketing brand based on Bernd Schmitt's strategic experience module and how it is applied to service management, introducing chatbot through experience-based service operation case analysis provided by CUI(Conversational User Interface) of domestic chatbot, and aims to strengthen the effective service of brands and related organizations trying to utilize chatbot. By doing this, we categorize the types of experiences offered by CUI of domestic chatbot and want to analyze in what way each type of experience is applied to service management, so we display it with brand name chatbot that has high reputation we analyzed the elements of the five experiences of sense, feel, think, act, relate. Based on the qualitative research method of Robert K. Yin, we are conducting five stages of case analysis, designing an analytical model based on Bernd Schmitt's experience marketing theory, features according to chatbot example and usage method of CUI environment through grasping strategic module-specific experiences, tried to present empirical research on the chatbot CUI environment and to present an understanding and utilization scheme of the whole experience. As a result of research, in the field of finance, the experiences of think and act experiences were high, and sense and feel experiences appeared to be low overall, whereas in the field of distribution, all five elements of experience were highlighted. In the medical field, the remaining elements except the relate experience are displayed lower than in other fields, but this effort reflects the structural specificity of this medical field, so that the service completeness which means that it is required to be relatively high. In addition, in the case of legal fields with access to and analysis of legal information, special cases of conclusion derivation, the effect of think experience was high and showed a relatively low feel experience, and in the field of entertainment that focuses on user's sense and act to function, relate it is analyzed that the effect of all the elements of experience becomes higher. As described above, the experiences of each element are displayed differently depending on the specificity of the field and the completion degree of the CUI being operated, but it will further improve the effectiveness of communication with consumers through chatbot administration in order to fully utilize the five elements of experience marketing comprehensively, it is important to develop a service that can provide overall experience to consumers, and we hope that this case will provide suggestions and basics for preparing efficient schemes for strengthening brands service through expansion of various chatbot marketing.

Understanding and Measuring User Experience in Conversational Interfaces

AbstractAlthough various methods have been developed to evaluate conversational interfaces, there has been a lack of methods specifically focusing on evaluating user experience. This paper reviews the understandings of user experience (UX) in conversational interfaces literature and examines the six questionnaires commonly used for evaluating conversational systems in order to assess the potential suitability of these questionnaires to measure different UX dimensions in that context. The method to examine the questionnaires involved developing an assessment framework for main UX dimensions with relevant attributes and coding the items in the questionnaires according to the framework. The results show that (i) the understandings of UX notably differed in literature; (ii) four questionnaires included assessment items, in varying extents, to measure hedonic, aesthetic and pragmatic dimensions of UX; (iii) while the dimension of affect was covered by two questionnaires, playfulness, motivation, and frustration dimensions were covered by one questionnaire only. The largest coverage of UX dimensions has been provided by the Subjective Assessment of Speech System Interfaces (SASSI). We recommend using multiple questionnaires to obtain a more complete measurement of user experience or improve the assessment of a particular UX dimension.RESEARCH HIGHLIGHTSVarying understandings of UX in conversational interfaces literature. A UX assessment framework with UX dimensions and their relevant attributes. Descriptions of the six main questionnaires for evaluating conversational interfaces. A comparison of the six questionnaires based on their coverage of UX dimensions.

03:45 PM Abstract No. 38 The development of a machine learning smart speaker application for device sizing in interventional radiology

Smart speakers such as the Amazon Echo and Google Home offer a fundamentally new method for interacting with software. These tools offer unique value in interventional radiology (IR), as a conversational voice interface allows IR physicians to interact with software and retrieve information without breaking sterile scrub. We present a smart speaker application developed specifically for use during IR procedures, using natural language processing (NLP) and machine learning to rapidly provide information about device sizing and compatibility in IR. We have developed a device sizing application for the Google Home smart speaker device. This application processes a human voice query and provides sizing recommendations. It can, for example, specify the minimum sheath size needed for a particular Amplatzer plug, among many other sizing tasks. Size data was acquired in an extensive literature review, with data compiled for 475 IR devices (catheters, sheaths, stents, vascular plugs, etc.). Natural language processing was implemented using Dialogflow, which extracted information of interest from an input query. Logic operations and other data processing were performed using a Python script deployed to the cloud. The conversational application is found to offer an effective tool for device sizing in interventional radiology, achieving excellent performance for a wide range of tasks. Technically satisfactory outcomes were seen using a training data set of 10 samples for each information category and a classification threshold of 0.3. Smart speakers have the potential to offer immense value in interventional radiology, giving proceduralists the ability to access modern information technology while maintaining procedural sterility. This work represents an early step in developing a suite of smart speaker IR tools for intra-procedural support. Future applications may focus on sterile approaches to querying the inventory database, performing dictations during procedural downtime, and accessing key data from the electronic medical record. These tools offer significant value to interventional radiologists and allow us to benefit from the power of machine learning.

Miscommunication Detection and Recovery in Situated Human–Robot Dialogue

Even without speech recognition errors, robots may face difficulties interpreting natural-language instructions. We present a method for robustly handling miscommunication between people and robots in task-oriented spoken dialogue. This capability is implemented in TeamTalk, a conversational interface to robots that supports detection and recovery from the situated grounding problems of referential ambiguity and impossible actions. We introduce a representation that detects these problems and a nearest-neighbor learning algorithm that selects recovery strategies for a virtual robot. When the robot encounters a grounding problem, it looks back on its interaction history to consider how it resolved similar situations. The learning method is trained initially on crowdsourced data but is then supplemented by interactions from a longitudinal user study in which six participants performed navigation tasks with the robot. We compare results collected using a general model to user-specific models and find that user-specific models perform best on measures of dialogue efficiency, while the general model yields the highest agreement with human judges. Our overall contribution is a novel approach to detecting and recovering from miscommunication in dialogue by including situated context, namely, information from a robot’s path planner and surroundings.

Building Conversational Interfaces

Building Conversational Interfaces

Self-Anamnesis with a Conversational User Interface: Concept and Usability Study.

Self-anamnesis is a procedure in which a patient answers questions about the personal medical history without interacting directly with a doctor or medical assistant. If collected digitally, the anamnesis data can be shared among the health care team. In this article, we introduce a concept for digital anamnesis collection and assess the applicability of a conversational user interface (CUI) for realizing a mobile self-anamnesis application. We implemented our concept for self-anamnesis for the concrete field of music therapy. We collected requirements with respect to the application from music therapists and by reviewing the literature. A rule-based approach was chosen for realizing the anamnesis conversation between the system and the user. The Artificial Intelligence Markup Language was exploited for encapsulating the questions and responses of the system. For studying the quality of the system and analyzing performance, humanity, effect, and accessibility of the system, we performed a usability test with 22 persons. The current version of the self-anamnesis application is equipped with 63 questions on the music biography of a patient that are asked subsequently to the user by means of a chatbot conversation. The usability study showed that a CUI is a practical way for collecting anamnesis data. Users felt engaged of answering the questions and liked the human characteristics of the chatbot. They suggested to extend the conversation capabilities of the chatbot so that the system can react appropriately, in particular when the user is not feeling well. We could demonstrate the applicability of a CUI for collecting anamnesis data. In contrast to digital anamnesis questionnaires, the application of a CUI provides several benefits: the user can be encouraged to complete all queries and can ask clarifying questions in case something is unclear.

CADI — A Conversational Assistive Design Interface for Discovering Pong Variants

Mixed-initiative PCG systems provide a way to leverage the expressive power of algorithmic techniques for content generation in a manner that lowers the technical barrier for content creators. While these tools are a proof of concept of how PCG systems can aide aspiring designers reach their vision, there are issues pertaining capturing designer intent, and interface complexity. In this paper we introduce CADI (Conversational Assistive Design Interface) a mixed initiative PCG system for creating variations of the game Pong that utilizes natural language input through a natural language interface to explore the design space of Pong variations. We provide a motivation for the creation of CADI and discuss the implementation and design decisions taken to address the issues of designer intent and interface complexity in mixed-initiative PCG systems.

Enhancing Trust in Autonomous Vehicles through Intelligent User Interfaces That Mimic Human Behavior

Autonomous vehicles use sensors and artificial intelligence to drive themselves. Surveys indicate that people are fascinated by the idea of autonomous driving, but are hesitant to relinquish control of the vehicle. Lack of trust seems to be the core reason for these concerns. In order to address this, an intelligent agent approach was implemented, as it has been argued that human traits increase trust in interfaces. Where other approaches mainly use anthropomorphism to shape appearances, the current approach uses anthropomorphism to shape the interaction, applying Gricean maxims (i.e., guidelines for effective conversation). The contribution of this approach was tested in a simulator that employed both a graphical and a conversational user interface, which were rated on likability, perceived intelligence, trust, and anthropomorphism. Results show that the conversational interface was trusted, liked, and anthropomorphized more, and was perceived as more intelligent, than the graphical user interface. Additionally, an interface that was portrayed as more confident in making decisions scored higher on all four constructs than one that was portrayed as having low confidence. These results together indicate that equipping autonomous vehicles with interfaces that mimic human behavior may help increasing people’s trust in, and, consequently, their acceptance of them.

New Approaches to Collecting Family Health History – A Preliminary Study Investigating the Efficacy of Conversational Systems to Collect Family Health History

Knowing and compiling your family health information is an important, cost-effective and efficient way to help your doctor screen and monitor for risks of genomic diseases such as cancer. There are several ways to collect family health history, including the use of digital records. Digital records can be helpful for sharing and updating information among family members. However, minimal research has been conducted to compare the different data collection interfaces. This study focuses on evaluating the user’s performance and preference between a conversational interface that we developed and a traditional interface for compiling family health history. Using a within-subjects design, twenty participants were asked to perform several tasks using both platforms. Although the conversational interface required more clicks, participants reported lower workload, higher performance and greater ease-of-use with this platform, and preferred the guidance of the virtual assistant.

Using eMMA to Manage Medication

The electronic medication management assistant (eMMA) aims to empower patients to manage their medication and ask questions through a conversational user interface. In addition, eMMA supports a consistent flow of medication data among healthcare providers, reducing errors and increasing medication adherence.

차량용 음성대화 인터페이스의 Barge-in 기능이 주행 경험에 미치는 효과 연구 - 내비게이션 및 음악서비스 중심으로 -

차량용 음성대화 인터페이스의 Barge-in 기능이 주행 경험에 미치는 효과 연구 - 내비게이션 및 음악서비스 중심으로 -

Implementation of a Chatbot System using AI and NLP

For using software applications, user interfaces that can be used includes command line, graphical user interface (GUI), menu driven, form-based, natural language, etc. The mainstream user interfaces include GUI and web-based, but occasionally the need for an alternative user interface arises. A chatbot based conversational user interface fits into this space. The chatbot is a class of bots that have existed in the chat platforms. The user can interact with them via graphical interfaces or widgets, and the trend is in this direction. They generally provide a stateful service i.e. the application saves data of each session. On a college’s website, one often doesn’t know where to search for some kind of information. It becomes difficult to extract information for a person who is not a student or employee there. The solution to these comes up with a college inquiry chat bot, a fast, standard and informative widget to enhance college website’s user experience and provide effective information to the user. Chat bots are an intelligent system being developed using artificial intelligence (AI) and natural language processing (NLP) algorithms. It has an effective user interface and answers the queries related to examination cell, admission, academics, users’ attendance and grade point average, placement cell and other miscellaneous activities.

Sherlock: Experimental Evaluation of a Conversational Agent for Mobile Information Tasks

Controlled natural language (CNL) has great potential to support human–machine interaction (HMI) because it provides an information representation that is both human readable and machine processable. We investigated the effectiveness of a CNL-based conversational interface for HMI in a behavioral experiment called simple human experiment regarding locally observed collective knowledge ( Sherlock ). In Sherlock , individuals acted in groups to discover and report information to the machine using natural language (NL), which the machine then processed into CNL. The machine fused responses from different users to form a common operating picture, a dashboard showing the level of agreement for distinct information. To obtain information to add to this dashboard, users explored the real world in a simulated crowdsourced sensing scenario. This scenario represented a simplified controlled analog for tactical intelligence (i.e., direct intelligence of the environment), which is key for rapidly planning military, law enforcement, and emergency operations. Overall, despite close to zero training, 74% of the users inputted NL that was machine interpretable and addressed the assigned tasks. An experimental manipulation aimed to increase user–machine interaction, however, did not improve performance as hypothesized. Nevertheless, results indicate that the conversational interface may be effective in assisting humans with collection and fusion of information in a crowdsourcing context.