Performance-Based Comparative Assessment of Open Source Web Vulnerability Scanners

Mansour Alsaleh,Noura Alomar,Abdulrahman Alarifi,Abdulmalik Al-Salman,Monirah Alshreef

doi:10.1155/2017/6158107

Abstract

The widespread adoption of web vulnerability scanners and the differences in the functionality provided by these tool-based vulnerability detection approaches increase the demand for testing their detection effectiveness. Despite the advantages of dynamic testing approaches, the literature lacks studies that systematically evaluate the performance of open source web vulnerability scanners. The main objectives of this study are to assess the performance of open source scanners from multiple perspectives and to examine their detection capability. This paper presents the results of a comparative evaluation of the security features as well as the performance of four web vulnerability detection tools. We followed this comparative assessment with a case study in which we evaluate the level of agreement between the results reported by two open source web vulnerability scanners. Given that the results of our comparative evaluation did not show significant performance differences among the scanners while the results of the conducted case study revealed high level of disagreement between the reports generated by different scanners, we conclude that the inconsistencies between the reports generated by different scanners might not necessarily correlate with their performance properties. We also present some recommendations for helping developers of web vulnerabilities scanners to improve their tools’ capabilities.

Highlights

The multilayered architectures of web-based systems and their sophisticated interactions with different types of subsystems increase the number of security flaws that can be exploited by attackers
In terms of crawler coverage, the obtained Web Input Vector Extractor Teaser (WIVET) scores for the four scanners show that 75% of them covered less than 50% of the crawled web-based systems
For SQL injection vulnerabilities, our observations showed that a false positive rate of 20% was observed for the two scanners; our case study showed that there was a notable difference between the numbers of SQL vulnerabilities detected by the two scanners

Summary

Introduction

The multilayered architectures of web-based systems and their sophisticated interactions with different types of subsystems increase the number of security flaws that can be exploited by attackers. According to Symantec [2], over 6500 web vulnerabilities were reported in 2014 and twenty percent of them were expected to have critical consequences. Dynamic security testing and static code analysis are ones of the main approaches that are used for detecting web vulnerabilities. While automated static code analysis techniques are more effective for detecting security defects (e.g., SQL vulnerabilities) [6, 7], the adoption of these techniques is limited compared to the widespread application of automated dynamic security testing tools. One of the reasons that contributed to limiting the adoption of static code analysis by software developers is that it cannot be performed without accessing the source code of the evaluated web-based application, which might not always be accessible [5]

Objectives

Methods

Results

Discussion

Conclusion