Both eyes open: Vigilant Incentives help auditors improve AI safety

Abstract

Auditors can play a vital role in ensuring that tech companies develop and deploy AI systems safely, taking into account not just immediate, but also systemic harms that may arise from the use of future AI capabilities. However, to support auditors in evaluating the capabilities and consequences of cutting-edge AI systems, governments may need to encourage a range of potential auditors to invest in new auditing tools and approaches. We use evolutionary game theory to model scenarios where the government wishes to incentivise auditing but cannot discriminate between high and low-quality auditing. We warn that it is alarmingly easy to stumble on ‘Adversarial Incentives’, which prevent a sustainable market for auditing AI systems from forming. Adversarial Incentives mainly reward auditors for catching unsafe behaviour. If AI companies learn to tailor their behaviour to the quality of audits, the lack of opportunities to catch unsafe behaviour will discourage auditors from innovating. Instead, we recommend that governments always reward auditors, except when they find evidence that those auditors failed to detect unsafe behaviour they should have. These ‘Vigilant Incentives’ could encourage auditors to find innovative ways to evaluate cutting-edge AI systems. Overall, our analysis provides useful insights for the design and implementation of efficient incentive strategies for encouraging a robust auditing ecosystem.