Abstract
Artificial intelligence (AI) agents are integral components of modern intelligent decision support systems (IDSS), providing their capability to assist in decision-making processes. Understanding the influence of AI involvement on human responses within these systems is crucial for fostering appropriate reliance on AI advice and ensuring a comfortable user experience. This study delves into the impact of AI agents on decision change, correct rate change, confidence rate change and pleasure rating among users. It explores the main and interactive effects of user proficiency and AI agent capability on user responses. A block-factorial experiment involving 45 participants was conducted. The results indicated that high-capability AI agents were associated with substantial decision change, correct rate change and confidence rate change. Participants with higher proficiency, both subjectively and objectively, exhibited reduced reliance on AI agents for task execution. Notably, the involvement of AI agents led to a slight decline in correct rates for participants of high competence (i.e., participants' independent correct rate exceeds AI agent capability), while still enhancing their confidence levels. Additionally, agents lacking human-like embodiment tended to enhance participants’ pleasure ratings. Individual differences in gender, collaboration experience with AI agent, and stereotype on AI agent capability also shaped the decision changes and pleasure ratings of participants. These findings underscore the presence of complementary competence interaction between user and AI agent within IDSS, and can be applied in designing capability of advisory AI agents with consideration of user proficiency and individual characteristics. Relevance to industryThe findings can be used as guidelines for designing AI agents and specifying their application strategies to integrate the assistance of AI agents effectively in intelligent decision support systems.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
