Effects of Anthropomorphic Phishing Detection Aids, Transparency Information, and Feedback on User Trust, Performance, and Aid Retention

Abstract

Phishing email attacks are a prevalent threat to internet users. Users often ignore or otherwise disregard automated aids, even when the aids’ reliability is high. The current study sought to fill a gap in the literature by examining the effects of anthropomorphism, feedback, and transparency information on user trust and performance within the domain of phishing email detection. Based upon previous studies in anthropomorphic automated systems, this study incorporated three levels of anthropomorphism (AI, human, text), two levels of aid gender (male, female), transparency information (present, absent), and feedback (present, absent). The 465 participants were recruited online through Amazon Mechanical Turk (MTurk) and performed the study on Qualtrics. Phishing was explained and instructions told the participants to judge whether the following emails are legitimate or phishing in three separate blocks of five emails. The first block was without any automated aid as a baseline of participants’ performance. The second block showed participants their respective aid and had them complete five more emails with the aid. The final block allowed participants to choose if they wanted to keep the aid or classify the emails alone. Afterwards, participants were asked how much they trusted the aid to help detect phishing threats using a trust in automation scale based on Jian, Bisantz, and Drury's (2000) study. Our results revealed improved performance on the phishing detection task for participants with an aid over participants without an aid. In addition, feedback was shown to be helpful for improving judgement accuracy as well as increase trust. Transparency also improved judgement accuracy for the human aid but was less helpful for the AI aid. This study compliments past research indicating improvements in performance with automated aids (Chen et al., 2018; Röttger, Bali, & Manzey, 2009; Wiegmann, Rich, & Zhang, 2001). Performance in blocks 2 and 3 was better than block 1. A significant positive correlation between trust and performance reinforces that high trust in a highly reliable aid begets good performance. Subsequently, if participants did not retain the aid for block 3, their performance was worse than those who retained the aid. Designers of automated aid systems should prioritize users interacting with and using the aid so that performance stays high. Feedback also helped improve judgement accuracy. By allowing participants to understand the reliability of the aid, they could learn to trust it more and rely on the suggestions of the aid. Feedback information should be offered to users if possible because it helps improve trust and performance, which is the goal of any automated aid system. Human aids with transparency information helped improve performance compared to human aids without transparency information. But this effect was not found for AI aids and nearly reversed. Transparency was expected to improve trust and performance (Hoff & Bashir, 2015), but it showed no differences in trust and only improved performance for human aids. This new finding demonstrates that there could be differences in the perception of human and AI aids, although further experiments would need to be conducted to further examine these findings.