Text-based Conversational Agents Research Articles

Unavoidable social contagion of false memory from robots to humans.

Many of us interact with voice- or text-based conversational agents daily, but these conversational agents may unintentionally retrieve misinformation from human knowledge databases, confabulate responses on their own, or purposefully spread disinformation for political purposes. Does such misinformation or disinformation become part of our memory to further misguide our decisions? If so, can we prevent humans from suffering such social contagion of false memory? Using a social contagion of memory paradigm, here, we precisely controlled a social robot as an example of these emerging conversational agents. In a series of two experiments (ΣN = 120), the social robot occasionally misinformed participants prior to a recognition memory task. We found that the robot was as powerful as humans at influencing others. Despite the supplied misinformation being emotion- and value-neutral and hence not intrinsically contagious and memorable, 77% of the socially misinformed words became the participants' false memory. To mitigate such social contagion of false memory, the robot also forewarned the participants about its reservation toward the misinformation. However, one-time forewarnings failed to reduce false memory contagion. Even relatively frequent, item-specific forewarnings could not prevent warned items from becoming false memory, although such forewarnings helped increase the participants' overall cautiousness. Therefore, we recommend designing conversational agents to, at best, avoid providing uncertain information or, at least, provide frequent forewarnings about potentially false information. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Assessing the Feasibility of a Text-Based Conversational Agent for Asthma Support: Protocol for a Mixed Methods Observational Study.

Despite efforts, the UK death rate from asthma is the highest in Europe, and 65% of people with asthma in the United Kingdom do not receive the professional care they are entitled to. Experts have recommended the use of digital innovations to help address the issues of poor outcomes and lack of care access. An automated SMS text messaging-based conversational agent (ie, chatbot) created to provide access to asthma support in a familiar format via a mobile phone has the potential to help people with asthma across demographics and at scale. Such a chatbot could help improve the accuracy of self-assessed risk, improve asthma self-management, increase access to professional care, and ultimately reduce asthma attacks and emergencies. The aims of this study are to determine the feasibility and usability of a text-based conversational agent that processes a patient's text responses and short sample voice recordings to calculate an estimate of their risk for an asthma exacerbation and then offers follow-up information for lowering risk and improving asthma control; assess the levels of engagement for different groups of users, particularly those who do not access professional services and those with poor asthma control; and assess the extent to which users of the chatbot perceive it as helpful for improving their understanding and self-management of their condition. We will recruit 300 adults through four channels for broad reach: Facebook, YouGov, Asthma + Lung UK social media, and the website Healthily (a health self-management app). Participants will be screened, and those who meet inclusion criteria (adults diagnosed with asthma and who use WhatsApp) will be provided with a link to access the conversational agent through WhatsApp on their mobile phones. Participants will be sent scheduled and randomly timed messages to invite them to engage in dialogue about their asthma risk during the period of study. After a data collection period (28 days), participants will respond to questionnaire items related to the quality of the interaction. A pre- and postquestionnaire will measure asthma control before and after the intervention. This study was funded in March 2021 and started in January 2022. We developed a prototype conversational agent, which was iteratively improved with feedback from people with asthma, asthma nurses, and specialist doctors. Fortnightly reviews of iterations by the clinical team began in September 2022 and are ongoing. This feasibility study will start recruitment in January 2023. The anticipated completion of the study is July 2023. A future randomized controlled trial will depend on the outcomes of this study and funding. This feasibility study will inform a follow-up pilot and larger randomized controlled trial to assess the impact of a conversational agent on asthma outcomes, self-management, behavior change, and access to care. PRR1-10.2196/42965.

Ladderbot—A conversational agent for human-like online laddering interviews

In user research, laddering interviews are particularly helpful in eliciting goals and underlying values. However, laddering interviews do not scale due to being time and training intensive. In this study, we propose and evaluate Ladderbot, a text-based conversational agent (CA) capable of facilitating human-like online laddering interviews. Ladderbot uses techniques inspired by face-to-face laddering to engage in an interactive conversation with users. In a between-subject experimental study with 256 participants, we compare Ladderbot against established survey-based laddering approaches in exploring user values for smartphone use. We find that on average, participants participating in CA-based laddering interviews produce twice as many and significantly longer answers. Additionally, we identify the learnability of the CA-based interviews to be significantly higher compared to established survey-based laddering approaches. However, survey-based laddering more reliably produces ladders that end in values, while CA-based laddering trades clear attribute-consequence-value structures to explore negative gains. Therein, besides presenting a new CA-based laddering approach, our study has implications for how user researchers can utilize both survey- and CA-based laddering methods to paint a more complete and comprehensive picture.

Conversational Agents for Information Retrieval in the Education Domain

Text-based conversational agents (CAs) are widely deployed across a number of daily tasks, including information retrieval. However, most existing agents follow a default design that disregards user needs and preferences, ultimately leading to a lack of usage and an unsatisfying user experience. To better understand how CAs can be designed in order to lead to effective system use, we deduced relevant design requirements from both literature and 13 user interviews. We built and tested a question-answering, text-based CA for an information retrieval task in an education scenario. Results from our experimental test with 41 students indicate that following a user-centered design has a significant positive effect on enjoyment and trust in a CA as opposed to deploying a default CA. If not designed with the user in mind, CAs are not necessarily more beneficial than traditional question-answering systems. Beyond practical implications for effective CA design, this paper points towards key challenges and potential research avenues when deploying social cues for CAs.

Validity of Chatbot Use for Mental Health Assessment: Experimental Study

Mental disorders in adolescence and young adulthood are major public health concerns. Digital tools such as text-based conversational agents (ie, chatbots) are a promising technology for facilitating mental health assessment. However, the human-like interaction style of chatbots may induce potential biases, such as socially desirable responding (SDR), and may require further effort to complete assessments. This study aimed to investigate the convergent and discriminant validity of chatbots for mental health assessments, the effect of assessment mode on SDR, and the effort required by participants for assessments using chatbots compared with established modes. In a counterbalanced within-subject design, we assessed 2 different constructs-psychological distress (Kessler Psychological Distress Scale and Brief Symptom Inventory-18) and problematic alcohol use (Alcohol Use Disorders Identification Test-3)-in 3 modes (chatbot, paper-and-pencil, and web-based), and examined convergent and discriminant validity. In addition, we investigated the effect of mode on SDR, controlling for perceived sensitivity of items and individuals' tendency to respond in a socially desirable way, and we also assessed the perceived social presence of modes. Including a between-subject condition, we further investigated whether SDR is increased in chatbot assessments when applied in a self-report setting versus when human interaction may be expected. Finally, the effort (ie, complexity, difficulty, burden, and time) required to complete the assessments was investigated. A total of 146 young adults (mean age 24, SD 6.42 years; n=67, 45.9% female) were recruited from a research panel for laboratory experiments. The results revealed high positive correlations (all P<.001) of measures of the same construct across different modes, indicating the convergent validity of chatbot assessments. Furthermore, there were no correlations between the distinct constructs, indicating discriminant validity. Moreover, there were no differences in SDR between modes and whether human interaction was expected, although the perceived social presence of the chatbot mode was higher than that of the established modes (P<.001). Finally, greater effort (all P<.05) and more time were needed to complete chatbot assessments than for completing the established modes (P<.001). Our findings suggest that chatbots may yield valid results. Furthermore, an understanding of chatbot design trade-offs in terms of potential strengths (ie, increased social presence) and limitations (ie, increased effort) when assessing mental health were established.

Comparing Experts and Novices for AI Data Work: Insights on Allocating Human Intelligence to Design a Conversational Agent

Many AI system designers grapple with how best to collect human input for different types of training data. Online crowds provide a cheap on-demand source of intelligence, but they often lack the expertise required in many domains. Experts offer tacit knowledge and more nuanced input, but they are harder to recruit. To explore this trade off, we compared novices and experts in terms of performance and perceptions on human intelligence tasks in the context of designing a text-based conversational agent. We developed a preliminary chatbot that simulates conversations with someone seeking mental health advice to help educate volunteer listeners at 7cups.com. We then recruited experienced listeners (domain experts) and MTurk novice workers (crowd workers) to conduct tasks to improve the chatbot with different levels of complexity. Novice crowds perform comparably to experts on tasks that only require natural language understanding, such as correcting how the system classifies a user statement. For more generative tasks, like creating new lines of chatbot dialogue, the experts demonstrated higher quality, novelty, and emotion. We also uncovered a motivational gap: crowd workers enjoyed the interactive tasks, while experts found the work to be tedious and repetitive. We offer design considerations for allocating crowd workers and experts on input tasks for AI systems, and for better motivating experts to participate in low-level data work for AI.

“Welcome to your Daily Wellness Check”: The Proposed Evaluation of a SMS-based Conversational Agent for Managing Health and Wellbeing

University students tend to sacrifice healthy lifestyles in favor of academic and financial gain. Additionally, the COVID-19 pandemic has led to online schooling and reduced access to healthcare resources. To mitigate these pressures, artificial intelligence conversational agents (CA) have addressed mental wellness in the general population but are yet to address physical well-being. Therefore, a CA that can address barriers and improve physical and mental well-being by cultivating healthy behaviors is desired. This research is designed to evaluate the feasibility and user experience of a text-based CA on addressing student barriers for a healthy lifestyle using evidence-based theories including the Theory of Planned Behavior and Motivational Interviewing over a two-week timeframe. Data collection has been paused due to the pandemic; results are expected to show positive impacts on well-being outcomes compared to an information-only control group and support the implementation of CAs to those with less accessibility to healthcare resources.

Towards a decision support system for radiotherapy business continuity in a pandemic crisis

ABSTRACT The business continuity of radiotherapy practice is of utmost importance for a safe, effective and efficient fight against cancer. Patients rely on uninterrupted therapies even under adverse conditions. The COVID-19 pandemic has caught many radio- therapy institutions unprepared. Centers worldwide had to find ways to master the extraordinary challenges instantly. However, the lack of knowledge, specific guidelines and mitigation strategies exposed radiotherapy centres to unpredictable risks. Intelligent information systems can support the management of severe incidents. The aim of our research was to develop a decision support system that shall help to maintain safe, sustainable and efficient radiotherapy services in pandemic situa- tions. In this work, we present a text-based conversational agent that informs about mitigation measures for radiotherapy practice during a pandemic crisis.

I, Chatbot: Modeling the determinants of users’ satisfaction and continuance intention of AI-powered service agents

Chatbots are mainly text-based conversational agents that simulate conversations with users. This study aims to investigate drivers of users’ satisfaction and continuance intention toward chatbot-based customer service. We propose an analytical framework combining the expectation-confirmation model (ECM), information system success (ISS) model, TAM, and the need for interaction with a service employee (NFI-SE). Analysis of data collected from 370 actual chatbot users reveals that information quality (IQ) and service quality (SQ) positively influence consumers’ satisfaction, and that perceived enjoyment (PE), perceived usefulness (PU), and perceived ease of use (PEOU) are significant predictors of continuance intention (CI). The need for interaction with an employee moderates the effects of PEOU and PU on satisfaction. The findings also revealed that satisfaction with chatbot e-service is a strong determinant and predictor of users’ CI toward chatbots. Thus, chatbots should enhance their information and service quality to increase users’ satisfaction. The findings imply that digital technologies services, such as chatbots, could be combined with human service employees to satisfy digital users.

Do Multilingual Users Prefer Chat-bots that Code-mix? Let's Nudge and Find Out!

Despite their pervasiveness, current text-based conversational agents (chatbots) are predominantly monolingual, while users are often multilingual. It is well-known that multilingual users mix languages while interacting with others, as well as in their interactions with computer systems (such as query formulation in text-/voice-based search interfaces and digital assistants). Linguists refer to this phenomenon as code-mixing or code-switching. Do multilingual users also prefer chatbots that can respond in a code-mixed language over those which cannot? In order to inform the design of chatbots for multilingual users, we conduct a mixed-method user-study (N=91) where we examine how conversational agents, that code-mix and reciprocate the users' mixing choices over multiple conversation turns, are evaluated and perceived by bilingual users. We design a human-in-the-loop chatbot with two different code-mixing policies -- (a) always code-mix irrespective of user behavior, and (b) nudge with subtle code-mixed cues and reciprocate only if the user, in turn, code-mixes. These two are contrasted with a monolingual chatbot that never code-mixed. Users are asked to interact with the bots, and provide ratings on perceived naturalness and personal preference. They are also asked open-ended questions around what they (dis)liked about the bots. Analysis of the chat logs, users' ratings, and qualitative responses reveal that multilingual users strongly prefer chatbots that can code-mix. We find that self-reported language proficiency is the strongest predictor of user preferences. Compared to the Always code-mix policy, Nudging emerges as a low-risk low-gain policy which is equally acceptable to all users. Nudging as a policy is further supported by the observation that users who rate the code-mixing bot higher typically tend to reciprocate the language mixing pattern of the bot. These findings present a first step towards developing conversational systems that are more human-like and engaging by virtue of adapting to the users' linguistic style.

Human-Aided Bots

A chatbot is an example of a text-based conversational agent. While natural language understanding and machine learning techniques have advanced rapidly, current fully automated chatbots still struggle to serve their users well. Human intelligence, brought by crowd workers, freelancers, or even full-time employees can be embodied in the chatbot logic to fill the gaps caused by limitations of fully automated solutions. In this paper, we investigate human-aided bots, i.e., bots (including chatbots) using humans in the loop to operate. We survey industrial and academic examples of human-aided bots, discuss their differences and common patterns, and identify open research questions.