A comparative evaluation of static analysis actionable alert identification techniques

Abstract

Automated static analysis (ASA) tools can identify potential source code anomalies that could lead to field failures. Developer inspection is required to determine if an ASA alert is important enough to fix, or an actionable alert. Supplementing current ASA tools with automated identification of actionable alerts could reduce developer inspection overhead, leading to an increase in industry adoption of ASA tools. The goal of this research is to inform the selection of an actionable alert identification technique for ranking the output of automated static analysis through a comparative evaluation of actionable alert identification techniques. We investigated six actionable alert identification techniques on three subject projects. Among these six techniques, the systematic actionable alert identification (SAAI) technique reported an average accuracy of 82.5% across the three subject projects when considering both ASA tools evaluated. Check 'n' Crash reported an average accuracy of 85.8% for the single ASA tool evaluated. The other actionable alert identification techniques had average accuracies ranging from 42.2%-78.2%.