Chatbot Design Research Articles

Implications of artificial intelligence chatbot models in higher education

Artificial intelligence (AI) is becoming increasingly influential in the academic sector, which is why it is important to explore the ethical dilemmas and concerns surrounding AI chatbots’ design, development, and deployment in educational contexts. Conducted as a thematic literature review, this paper explores existing research on AI in education, AI chatbots, and their integration with higher education to gather evidence and insights that discuss ethical implications and challenges. The study has analyzed several articles on AI chatbots and their integration into academic fields. Significant gaps have been identified, such as the need for more practical implications and the recognition of AI chatbots as a collaborative tool for academic purposes. More AI chatbots should be explicitly trained on data relevant to the learners’ study to examine their usefulness properly. The paper discusses the ethical dilemmas and concerns about the design, development, and deployment of AI chatbots in higher education. It seeks to provide insights and recommendations to ensure the ethical use of AI chatbots in higher education by identifying significant gaps in the existing literature and providing scenarios to expect in the development of AI in education.

ChatGPT as a tool for self-learning English among EFL learners: A multi-methods study

This study investigates the factors influencing ChatGPT acceptance in self-directed English language learning and explores its potential to enhance the learning process. A multi-methods approach was employed, comprising a systematic review of 40 empirical articles, a quantitative survey of 344 English as a Foreign Language (EFL) learners, and 19 semi-structured interviews. Results indicated that interactivity, enjoyment, trust, and subjective norms significantly affect continued use through perceived usefulness, while human-likeness, self-efficacy, and technology anxiety influence it through perceived ease of use. EFL learners utilize ChatGPT for both primary and secondary English learning purposes, supporting various language dimensions including reading, writing, vocabulary, and grammar. The study also identified both passive and active language learning engagement facilitated by ChatGPT. This research contributes to the field by proposing an extended Technology Acceptance Model and a conceptual framework for chatbot-assisted language learning. For chatbot developers, the findings underscore the need to enhance interactivity, human-likeness, enjoyment, and trust while minimizing technology anxiety to increase sustained use. Educators can utilize these insights to better integrate ChatGPT into their curricula, supporting diverse language skills and engaging learners more effectively. Together, these findings provide concrete guidance for enhancing the design and implementation of AI chatbots in language learning contexts.

Chatbot Anthropomorphism, Social Presence, Uncanniness and Brand Attitude Effects

ABSTRACT As chatbots become the dominant technology in customer service, understanding how their design influences consumer perceptions and responses is crucial. This study investigates the impact of chatbot anthropomorphic design cues on perceived social presence and uncanniness, and their consequent effects on brand attitude. Through scenario-based experiments involving a fictitious retail brand’s chatbots, we demonstrate that anthropomorphism perceptions enhance brand attitudes mediated via social presence. However, we also find that sense of uncanniness moderates the effect of perceived anthropomorphism on social presence, with implications on brand attitude. These findings extend the CASA framework and offer novel insights into the psychological mechanisms underlying consumer–chatbot interactions. By highlighting the delicate balance between social presence and uncanniness, we provide insight that can inform chatbot design in ways that could optimize customer engagement while mitigating potential negative reactions. Our research advances theory and practice in leveraging AI-powered chatbots for enhanced brand experiences.

Towards Human-Centric Mental Health Chatbot Design: Leveraging on DASS-21

The existing landscape of mental health apps exhibits several limitations, particularly concerning their effectiveness in mental health prevention and their ability to provide adequate assistance for mental health practitioners. In this study, a human-centric design chatbot is proposed leveraging on the Depression Anxiety and Stress Scale 21 (DASS-21) for addressing these challenges. Based on the interviews carried out with five mental health professionals, a chatbot prototype was developed by integrating the DASS-21 assessment tool together with other features aimed at enhancing mental health prevention. The chatbot was then subjected to comprehensive testing involving users and two mental health practitioners. The usability testing with mental health practitioners revealed promising results, indicating that the chatbot effectively supports mental health prevention. Additionally, the feedback from mental health practitioners highlighted the usefulness of the chatbot as a tool in their practice. The usability testing involved 15 university students, with 40.0% female and 60.0% male participants. The majority of respondents were in the age range of 24-26 years old (53.3%), followed by 21-23 years old (40.0%), and a small percentage (6.7%) were 18-20 years old. In terms of their university year, 86.7% were Year 4 students, and 13.3% were Year 3 students. While most university students expressed agreement with the features in the mobile app, some (13.33%) rated certain aspects as "Average." These included the availability of sufficient information on specific pages and the user interface design, indicating that the chatbot may still be lacking in knowledge, data, and comprehensive functionalities. These findings demonstrate the potential of our human-centric design approach and the integration of DASS-21 within a chatbot framework to enhance mental health prevention. By addressing the limitations of existing mental health apps and garnering positive feedback from users and practitioners, our study contributes to the ongoing efforts in improving mental health support through technology.

Development and Evaluation of an AI-Based Chatbot for Preventing Social Media Addiction: A Waterfall Model Approach

The rapid growth of social media has transformed interaction and communication patterns, but it has also led to the rise of social media addiction, particularly among teenagers and young adults. This addiction, marked by compulsive usage and negative impacts on mental health and daily life, necessitates effective interventions. This research explores the development and evaluation of an AI-based chatbot designed to mitigate social media addiction by employing cognitive and behavioural strategies. The study utilizes the Waterfall model—a structured, sequential approach—in the chatbot’s development, encompassing stages from needs analysis to maintenance. The chatbot’s effectiveness was assessed through rigorous testing and user feedback. The methodology included problem analysis, system design, implementation, testing, and iterative improvements. A comprehensive needs analysis identified the psychological and behavioural factors contributing to social media addiction, leading to the design of a prototype chatbot integrated with AI for dynamic content adaptation and real-time feedback. The implementation phase focused on coding and system integration, followed by rigorous testing using Black Box Testing and the System Usability Scale (SUS) to ensure functionality and user-friendliness. Results indicate that the chatbot significantly reduced social media addiction scores, with a mean decrease from 55.21 to 50.17, supported by a highly significant p-value of <0.0001. User satisfaction was high, particularly regarding ease of use and information quality. However, user engagement declined over time, highlighting the need for ongoing content updates and feature enhancements. This study contributes to the field by providing insights into the application of the Waterfall model in AI chatbot development and offers a scalable solution for addressing social media addiction, with implications for future digital interventions in mental health.

Rule based chatbot design methods: A review

Rule based chatbot design methods: A review

Examining the interplay of chatbot interactivity and anthropomorphic design for enhancing brand relationships in service businesses

The current research examines how elements of chatbot design influence not only consumers’ perceptions and responses but also the processes by which they promote online-to-offline behavioral intention toward service brands. Focusing on the coffee service business, a study following a 2 × 2 between-subjects experimental design was conducted. Among the findings, the impact of interactivity on chatbot- and brand-related outcomes depended on the anthropomorphic conversational style employed; interactions with the warm, high-interactivity (vs. low-interactivity) chatbot enhanced the sense of social presence, whereas interactions with the competent, low-interactivity (vs. high-interactivity) chatbot promoted brand-related behavioral intention. Moreover, as the moderated serial mediation model demonstrated, although increased social presence shaped components of brand relationship quality (i.e. brand partner quality and brand attachment), only brand partner quality bolstered the intention to patronize the brand. The significant moderated indirect effect suggests that a warm, highly interactive chatbot design can strengthen social presence and thereby improve brand partner quality and raise the likelihood of offline patronage. Theoretical and managerial implications of the findings are discussed.

Empathy Toward Artificial Intelligence Versus Human Experiences and the Role of Transparency in Mental Health and Social Support Chatbot Design: Comparative Study.

Empathy is a driving force in our connection to others, our mental well-being, and resilience to challenges. With the rise of generative artificial intelligence (AI) systems, mental health chatbots, and AI social support companions, it is important to understand how empathy unfolds toward stories from human versus AI narrators and how transparency plays a role in user emotions. We aim to understand how empathy shifts across human-written versus AI-written stories, and how these findings inform ethical implications and human-centered design of using mental health chatbots as objects of empathy. We conducted crowd-sourced studies with 985 participants who each wrote a personal story and then rated empathy toward 2 retrieved stories, where one was written by a language model, and another was written by a human. Our studies varied disclosing whether a story was written by a human or an AI system to see how transparent author information affects empathy toward the narrator. We conducted mixed methods analyses: through statistical tests, we compared user's self-reported state empathy toward the stories across different conditions. In addition, we qualitatively coded open-ended feedback about reactions to the stories to understand how and why transparency affects empathy toward human versus AI storytellers. We found that participants significantly empathized with human-written over AI-written stories in almost all conditions, regardless of whether they are aware (t196=7.07, P<.001, Cohen d=0.60) or not aware (t298=3.46, P<.001, Cohen d=0.24) that an AI system wrote the story. We also found that participants reported greater willingness to empathize with AI-written stories when there was transparency about the story author (t494=-5.49, P<.001, Cohen d=0.36). Our work sheds light on how empathy toward AI or human narrators is tied to the way the text is presented, thus informing ethical considerations of empathetic artificial social support or mental health chatbots.

Bridging the health literacy gap through AI chatbot design: the impact of gender and doctor cues on chatbot trust and acceptance

PurposeNotwithstanding their potential benefits especially for individuals with low health literacy, users are still somewhat skeptical about the reliability of healthcare chatbots. The present study aims to address this challenge by investigating strategies to enhance users’ cognitive and emotional trust in healthcare chatbots. Particularly, this study aims to understand the effects of chatbot design cues in increasing trust and future chatbot use intention for low health literacy users.Design/methodology/approachWe conducted two experimental studies with a final sample of 327 (Study 1) and 241 (Study 2). Three different chatbots were developed (Chatbot design: Bot vs Male-doctor vs Female-doctor). Participants were asked to have a medical consultation with the chatbot. Participants self-reported their health literacy scores. The PROCESS model 7 was used to analyze the hypotheses.FindingsThe results showed that the female-doctor cues elicited greater cognitive and emotional trust, whereas the male-doctor cues only led to greater cognitive trust (vs bot-like cues). Importantly, this study found that users’ health literacy is a significant moderating factor in shaping cognitive and emotional trust. The results indicated that both the female and male-doctor cues’ positive effects on cognitive trust were significant for those with lower levels of health literacy. Furthermore, the positive effect of the female-doctor cues on emotional trust was also significant only for those whose health literacy level was low. The increased cognitive and emotional trust led to greater future intention to use the chatbot, confirming significant moderated mediation effects.Originality/valueDespite the strong economic and educational benefits of healthcare chatbots for low health literacy users, studies examining how healthcare chatbot design cues affect low health literate users surprisingly remained scarce. The results of this study suggest that healthcare chatbots can be a promising technological intervention to narrow the health literacy gap when aligned with appropriate design cues.

Identity Disclosure and Anthropomorphism in Voice Chatbot Design: A Field Experiment

Fueled by the widespread adoption of algorithms and artificial intelligence, the use of chatbots has become increasingly popular in various business contexts. In this paper, we study how to effectively and appropriately use voice chatbots, particularly by leveraging the two design features identity disclosure and anthropomorphism, and evaluate their impact on the firm operational performance. In collaboration with a large truck-sharing platform, we conducted a field experiment that randomly assigned 11,000 truck drivers to receive outbound calls from the voice chatbot dispatcher of our focal platform. Our empirical results suggest that disclosing the identity of the chatbot at the beginning of the conversation negatively affects operational performance, leading to around 11% reduction in the response probability. However, humanizing the voice chatbot by adding our proposed anthropomorphism features (i.e., interjections and filler words) significantly improves response probability, conversation length, and the probability of order acceptance intention by over 5.6%, 24.9%, and 10.1%, respectively. Moreover, even when the chatbot’s identity is disclosed along with humanizing features, the operational outcomes still improve. This finding suggests that enhancing anthropomorphism may potentially counteract the negative effects of chatbot identity disclosure. Finally, we propose one plausible explanation for the performance improvement—the enhanced trust between humans and algorithms—and provide empirical evidence that drivers are more likely to disclose information to chatbot dispatchers with anthropomorphism features. Our proposed anthropomorphism improvement solutions are currently being implemented and utilized by our collaborator platform. This paper has been This paper was accepted by Felipe Caro for the Special Issue on the Human-Algorithm Connection. Funding: This study is supported by the National Natural Science Foundation of China [Grants 72172169 and 91646125], Program for Innovation Research at the Central University of Finance and Economic, and Shanghai Pujiang Program. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2022.03833 .

Security and Privacy in Physical Activity Chatbots on Social Media: A Scoping Review.

Social media physical activity chatbots use both chatbots and social media platforms for physical activity promotion and, thus, could face privacy and security challenges inherent in both technologies. This study aims to provide an overview of physical activity chatbot interventions delivered via social media platforms, specifically focusing on security and privacy measures. We conducted a scoping review on this topic across 4 databases: PubMed, PsycINFO, ACM Digital Library, and IEEE Xplore. We extracted and summarised information on the author, population, country of study, social media platform, intervention, data processed (i.e. gathered or stored), and security/privacy measures. Out of 1299 identified articles, 12 were included in the analysis reporting about 9 different chatbots. Although all chatbots applied data processing methods, only a few considered anonymisation. One paper stated compliance with the General Data Protection Regulations. Other studies enforced some verification procedures before chatbot use. Current research fails to adequately report security considerations in social media physical activity chatbot design. However, integration of chatbots into social media platforms seems to be declining, possibly due to security concerns.

Adoption of Artificial Intelligence Chatbot Technology in Academic Activities of Jakarta Students: Ease of Use

Artificial Intelligence (AI) Chatbot technology has garnered significant attention recently as an innovative solution for various needs, from learning to counseling. However, its adoption among university students remains a crucial research area. This study aims to understand the usage patterns of university students in Jakarta regarding Chatbot technology, focusing on usage frequency, ease of use, and factors influencing adoption. Conducted from March to May 2024, the research involved 100 students from various Jakarta universities, selected through convenience sampling to capture diverse characteristics. Data collection methods included document studies and questionnaires. The analysis identified the level of Chatbot technology adoption among Jakarta students and the factors influencing it. Results indicate a high frequency of Chatbot use, with perceived usefulness, ease of use, and comfort being the primary drivers of adoption and positive attitudes toward this technology. This study provides significant insights for companies developing Chatbot technology, offering an in-depth understanding of students' preferences and behaviors. These insights can guide companies in creating more effective marketing strategies and enhancing Chatbot design and functionality to be more user-friendly and aligned with user expectations, thereby increasing adoption rates and user satisfaction.

Design of Intelligent Chatbot for Stress Management

This paper focuses on using natural language processing (NLP) in chatbots to manage stress in war-affected countries. A Java-based chatbot was designed to alleviate stress using two algorithms: TextRank and Stanford_CoreNLP. The problem was solved by integrating different languages using a plugin. The chatbot was tested with fifteen people and received positive feedback. Modifications were made based on user feedback, with journaling being a winner. However, the chatbot faced limitations like lack of Arabic language support and voice chat feature.

Designing chatbots in language classrooms: an empirical investigation from user learning experience

Amidst the technological evolution shaping the landscape of education, this research critically examines the imperative factors influencing the design of language-teaching chatbots in Thai language classrooms. Employing a comprehensive two-pronged methodology, our study delves into the intricacies of chatbot design by engaging with a diverse participant pool of pre-service teachers, in-service teachers, and educators. The study included the chatbot design opinion survey, a validated scale assessing attitudes toward chatbot design, and semi-structured interviews with teachers, educators, and experts to offer qualitative insights. Our findings reveal five key factors influencing chatbot design: learner autonomy and self-directed learning, content and interaction design for language skill development, implementation and usage considerations, alternative learning approaches and flexibility, and content presentation and format. Concurrently, thematic analysis of interviews results in five overarching themes: enhancing interactive language learning, motivational learning experience, inclusive language learning journey, blended learning companion, and communicative proficiency coaching. These findings inform the development of effective chatbots for Thai language classrooms within the evolving landscape of technology-driven education. These findings inform the development of effective chatbots for language classrooms within the evolving landscape of technology-driven education.

An overview of the applied use of chatbots for substance use disorder: Past, present and future

AbstractAimThe purpose of this paper was to critically evaluate the use of chatbots as an eHealth resource for substance use disorder. It considers their past, present and future use, including the progression of the new wave of generative chatbots.FindingsThis paper finds chatbots for substance use disorder to be an underutilised resource, noting that there are few solutions and that these have limitations. It identifies that, along with limitations with general chatbot design, there are also serious ethical concerns given the vulnerabilities of the target population.DiscussionHow a collaborative design can address some of the limitations is discussed, as well as the restrictions this could pose to leveraging the latest technological advances. The ethical concerns presented in the applied use of both existing and prospective chatbots are explored, and how there is a need to safeguard users in matters of mental health support.

Different dimensions of anthropomorphic design cues: How visual appearance and conversational style influence users’ information disclosure tendency towards chatbots

Text-based chatbots are widely used to deliver personalized services by leveraging user-provided information, and anthropomorphic design is crucial for their effectiveness. However, most earlier studies investigated the effects of anthropomorphic design of chatbots while manipulating only one dimension of anthropomorphic cues. The current research investigated how different dimensions of anthropomorphic design cues affect users’ information disclosure tendency towards chatbots. That is, the present study examined the effects of visual appearance (high anthropomorphism vs. low anthropomorphism), manipulating the visual cues dimension, and conversational style (human-like vs. mechanical), manipulating the verbal cues dimension, on users’ information disclosure tendency towards chatbots. Results showed positive effects of human-like conversational style on users’ information disclosure tendency. Of particular significance, an interaction effect between visual appearance and conversational style on users’ information disclosure tendency was found. Users reported a higher information disclosure tendency when the chatbot was designed with anthropomorphic cues consistent over dimensions. This finding suggested that an expectancy violation effect occurs when a chatbot exhibits inconsistent anthropomorphic design cues on two different dimensions. Besides, perceived security was identified as a positive mediating factor in the relationship between conversational style and users’ information disclosure tendency. This study advances research on users’ information disclosure tendency towards anthropomorphic chatbots and highlights the importance of different dimensions of anthropomorphic cues in chatbot design. Additionally, practical guidance for chatbot designers was also provided.

Human–AI communication in initial encounters: How AI agency affects trust, liking, and chat quality evaluation

Artificial intelligence (AI) agency plays an important role in shaping humans’ perceptions and evaluations of AI. This study seeks to conceptually differentiate AI agency from human agency and examine how AI’s agency manifested on source and language dimensions may be associated with humans’ perceptions of AI. A 2 (AI’s source autonomy: autonomous vs human-assisted) × 2 (AI’s language subjectivity: subjective vs objective) × 2 (topics: traveling vs reading) factorial design was adopted ( N = 376). The results showed autonomous AI was rated as more trustworthy, and AI using subjective language was rated as more trustworthy and likable. Autonomous AI using subjective language was rated as the most trustworthy, likable, and of the best quality. Participants’ AI literacy moderated the interaction effect of source autonomy and language subjectivity on human trust and chat quality evaluation. Results were discussed in terms of human–AI communication theories and the design and development of AI chatbots.

Exploring the influence of user characteristics on verbal aggression towards social chatbots

ABSTRACT Chatbots possess great potential benefits, yet concerns persist regarding users adopting inappropriate, offensive language. This research delved into the influence of user characteristics on verbally aggressive behaviours towards social chatbots. Employing a mixed-method study, we examined individual characteristics such as personal dispositions, offensive language patterns, academic majors, and prior experiences with conversational agents. Findings from a ten-day field experiment involving 33 participants using a real-world Telegram-based chatbot app unveiled that users' anthropomorphism, computer-related major, and gender significantly impact their moral emotions and evaluations of the chatbot's capabilities. Moreover, employing offensive language towards the chatbot detrimentally impacted users' perceptions of its abilities, helpfulness, and likability. The research findings advocate for ongoing monitoring and effective resolution of users' behaviours regarding the use of offensive language in their interactions with a chatbot. Additionally, the results underscore the importance of incorporating diverse perspectives into chatbot design to address biases and offensive utterances.

How Do Initial and Interactive Social Cues Increase Customers’ Continuance Usage Intention of Chatbots?

Enterprises widely utilize chatbots as “intelligent customer service” because of their efficiency and low cost. However, customer complaints and doubts occur occasionally, and improving users’ willingness to continue to use is a crucial issue that needs to be solved urgently. This study investigates whether and how initial social cues (avatar, name, and initial greeting) and interactive social cues (emoji) can improve users’ intention to continue using. Our findings from three scenario experiments reveal that both initial and interactive social cues can increase consumers’ warmth perception, while only interactive social cues elevate competence perception. The greater the warmth and competence perceived by users, the higher their trust in chatbots, leading to increased intention to continue using them. Furthermore, we identified a reinforcing effect when both types of social cues are combined, further augmenting users’ intention to persist with chatbots. This study contributes to the chatbot-human interaction literature by validating the effectiveness of these two social cues and offers practical insights for enhancing customer retention and optimizing chatbot design.

The way you assess matters: User interaction design of survey chatbots for mental health

The global pandemic has pushed human society into a mental health crisis, prompting the development of various chatbots to supplement the limited mental health workforce. Several organizations have employed mental health survey chatbots for public mental status assessments. These survey chatbots typically ask closed-ended questions (Closed-EQs) to assess specific psychological issues like anxiety, depression, and loneliness, followed by open-ended questions (Open-EQs) for deeper insights. While Open-EQs are naturally presented conversationally in a survey chatbot, Closed-EQs can be delivered as embedded forms or within conversations, with the length of the questionnaire varying according to the psychological assessment. This study investigates how the interaction style of Closed-EQs and the questionnaire length affect user perceptions regarding survey credibility, enjoyment, and self-awareness, as well as their responses to Open-EQs in terms of quality and self-disclosure in a survey chatbot. We conducted a 2 (interaction style: form-based vs. conversation-based) × 3 (questionnaire length: short vs. middle vs. long) between-subjects study (N=213) with a loneliness survey chatbot. The results indicate that the form-based interaction significantly enhances the perceived credibility of the assessment, thereby improving response quality and self-disclosure in subsequent Open-EQs and fostering self-awareness. We discuss our findings for the interaction design of psychological assessment in a survey chatbot for mental health.