Prediction capabilities of vulnerability discovery models

Abstract

Quantitative approaches for software security are needed for effective testing, maintenance and risk assessment of software systems. Vulnerabilities that are present in an operating system after its release represent a great risk. Vulnerability discovery models (VDMs) have been proposed to model vulnerability discovery and have has been fined to vulnerability data against calendar time. The models have been shown to fit very well. In this paper, we investigate the prediction capabilities that these models offer by evaluating accuracy of predictions made with partial data. We examine both the recently proposed logistic model and a new linear model. In addition to VDMs, we consider static approaches to estimating some of the major attributes of the vulnerability discovery process, presenting a static approach to estimating the initial values of one of the VDM's parameters. We also suggest the use of constraints for parameter estimation during curve-fitting. Here we develop computational approaches for early applications of the models and examine the predictive capability of the models. We use data from Windows 98, Windows 2000 and Red Hat Linux 7.1. We examine the impact of using a specific constraint when the parameters of the logistic model are estimated plots for the prediction error are given. The results demonstrate that the prediction error is significantly less when a constraint based on past observations is added. It is observed that the linear model may yield acceptable projections for systems for which vulnerability discovery has not yet reached saturation. The results also suggest that it may be possible to improve the prediction capability by combining static and dynamic approaches, or by combing different models