DESSRT: A Novel Framework for Empirical Red Teaming at Scale

Abstract

Background Red Teaming is widely used to discover vulnerabilities, test defensive measures, and anticipate emerging but novel threats. It has rarely been conducted both systematically and at scale, substantially limiting confidence in its results and the generalizability of its findings. Aim We introduce distributed, empirical, systematic, and scalable red teaming (DESSRT), a framework for translating tactical-level Red Teaming into a replicable research methodology. We apply DESSRT to address whether the information about and availability of computed tomography (CT) scanners influences adversary decision-making in aviation security. Method Using a convenience sample of 143 university students, participants role-played as adversaries in an eight-hour attack planning exercise. Via a custom instrument, participants were randomly assigned across three adversary profiles built on historical cases and then designed a simulated attack. Afterwards, one of three injects about CT scanners were randomly assigned, and participants were asked about potential changes in attack plans (including target changes). Differences among assigned profiles and CT scanner injects were evaluated using standard statistical tests of association. Results Although differences in explosive and weapon package selections were not statistically significant across profiles, security evasion methods were. Following injects, participants were equally as likely to change tactics across profiles, with the majority (53%) changing at least one tactical area. When asked, the majority (18) of those who changed targets (27/143) reported that the additional information on CT scanners did have some effect on their target change decision. Conclusion Overall, the DESSRT framework provides a novel mechanism for translating traditional Red Teaming exercises into a replicable and empirical research method. Although not a replacement for historical data, where available, DESSRT allows analysts and researchers to test theories about human decision-making, generate novel what-if insights to support planning efforts, and validate parameters within complex models.