Finding the sweet spot: Exploring the optimal communication delay for AI feedback tools

Abstract

AI writing assistants that are capable of offering rapid, automatic, and corrective feedback are becoming increasingly powerful, providing accurate and automatic feedback to knowledge contributors while they write. This helps people stay engaged and focused on their writing tasks. However, even with improved algorithms, AI still requires communication delays to perform sophisticated language processing and generate accurate responses. Prior research suggests that if the delay is too long, users may lose interest and become frustrated, perceiving the AI agent as ineffective. Conversely, if the AI-generated feedback is too quick, users may push back and perceive the AI as not fully comprehending their input, leading to low levels of trust. The impact of feedback timing on knowledge contribution behavior remains largely unknown. To address this gap, this research employs two sequential studies to explore the effects of feedback timing. The first study is a conjoint analysis, and the second study is a random-assignment-based experiment. Our results show that an optimal range of timing for communication delay is approximately one to three seconds, leading to improved performance. Presentation styles of the AI agent may affect these results but only slightly. Our study is the first to systematically investigate the optimality of communication delay on AI feedback mechanisms. Our findings will contribute to the literature on knowledge management, algorithm design, and anthropomorphism, and also inspire the software designers on the optimal timing of the communication feedback.