SpearSim-V2: Synthetic Task Environment for Evaluating Attacker Behaviors

Abstract

Despite extensive research on phishing, a severe lack of work centered on attackers has resulted in a limited understanding of the adversarial behaviors conducive to attack success and failures. This work describes a novel method for conducting controlled laboratory studies of cognitive vulnerabilities that attackers experience during the design and execution phases of spear-phishing attacks. Based on the SpearSim platform, the new simulation environment integrates cognitive agents that model and predict end-user responses to spear-phishing attacks. This advancement to SpearSim allows the generation of real-time, automated, “human-like” responses to simulated spear-phishing attacks. This enables the execution of experiments focused on attackers and attacker behaviors. We describe the proposed simulation framework, provide details about the implemented simulation environment, and present results to evaluate the performance of the simulation environment. Compared to the earlier version of SpearSim involving human end-users, the new approach generates responses at a much faster rate (3 times faster than human end-users) and importantly with less variance in the time to respond. The cognitive agents used in the simulation predicted human responses to phishing and spear-phishing attackers with moderate accuracy (about 60%). Our proposed method intends to provide an effective and robust way to conduct laboratory experiments on spear-phishing attacks and further understand attackers' decision-making processes that could be exploited to thwart future attacks.