It's not just about accuracy: An investigation of the human factors in users' reliance on anti-phishing tools

Abstract

Phishing attacks pose substantial threats to the security of individuals and organizations. Although current anti-phishing tools achieve high accuracy rates and present a potential solution to this problem, users are often reluctant to rely on the predictions of these competent tools. However, we continue to lack a means of resolving this reluctance—or even an explanation for it. To address this need and advance toward a solution, we investigate the factors that influence users' reliance on anti-phishing tools. Over the course of two studies, we test the effects of tool attributes (i.e., accuracy and frequency of phishing email predictions) and develop a model based on the notions of trust and distrust. Countering the common conjecture that tools are not accurate enough, we find that users' under-reliance is not an artifact of the insufficient accuracy of tools, as even in a 100% accuracy condition, users were under-reliant on tools. Rather, we find that while accuracy increases users' trust in tools, full reliance is inhibited by users' distrust, which is driven by a lack of transparency regarding tools' functionalities and the quantity of predictions provided. Thus, overall, our study shows the limits of accuracy in engendering reliance and explains the under-reliance phenomenon by showing that due to lack of knowledge or understanding, some users prefer to rely on their own inferior judgment instead of trusting and relying on the predictions provided by highly accurate tools.