Towards a Greater Understanding of Coordinated Vulnerability Disclosure Policy Documents

Abstract

Bug bounty programmes and vulnerability disclosure programmes, collectively referred to as Coordinated Vulnerability Disclosure (CVD) programmes, open up an organisation’s assets to the inquisitive gaze of (often eager) white-hat hackers. Motivated by the question What information do organisations convey to hackers through public CVD policy documents? , we aim to better understand the information available to hackers wishing to participate in the search for vulnerabilities. As such, in this article we consider three key issues. First, to address the differences in the legal language communicated to hackers, it is necessary to understand the formal constraints by which hackers must abide. Second, it is beneficial to understand the variation that exists in the informal constraints that are communicated to hackers through a variety of institutional elements. Third, for organisations wishing to better understand the commonplace elements that form current policy documents, we offer broad analysis of the components frequently included therein and identify gaps in programme policies. We report the results of a quantitative study, leveraging deep learning based natural language processing models, providing insights into the policy documents that accompany the CVD programmes of thousands of organisations, covering both stand-alone programmes and those hosted on 13 bug bounty programmes. We found that organisations often inadequately convey the formal constraints that are applicable to hackers, requiring hackers to have a deep understanding of the laws that underpin safe and legal security research. Furthermore, a lack of standardisation across similar policy components is prevalent, and may lead to a decreased understanding of the informal constraints placed upon hackers when searching for and disclosing vulnerabilities. Analysis of the institutional elements included in the policy documents of organisations reveals insufficient inclusion of many key components. Namely, legal information and information pertaining to restrictions on the backgrounds of hackers is found to be absent in a majority of policies analysed. Finally, to assist ongoing research, we provide novel annotated policy datasets that include human-labelled annotations at both the sentence and paragraph level, covering a broad range of CVD programme backgrounds.