Avaliação semântica da integração da gestão de riscos de segurança em documentos de software da administração pública

Abstract

Secure software has its security risks well managed regarding vulnerabilities. IT managers in the Brazilian Federal Public Sector (APF) are legally required to strive for software security. However, only a small number of software security professionals are employed in the technical support for development, maintenance and acquisition of such software in their public organizations. This problem, combined with recent advances in text mining, especially in natural language processing and multi-label classification, motivate this work on research for a computational solution that can understand the semantics of sentences in documents written in Portuguese and connect them to previously defined software security risks, such as OWASP Top Ten. This solution (A2E) can improve the software factories bidding process of the APF by providing the authors and reviewers of technical specifications with a computational tool which can automatically evaluate the integration between security risks management and software processes described in these documents. After applying A2E to more than 120 thousand sentences extracted from OWASP ASVS and past APF specifications, a survey was conducted to compare its performance with the opinion of software engineers and security specialists through objective metrics like Precision, Recall, Hamming Loss and Negative Predictive Values (NPV). A2E’s final version, after a series of improvements in the development process, obtained a significantly better Hamming Loss measure when compared to the specialists’ assessments. Additionally, experiments showed that its NPV is statistically as good as the NPV from the surveyed experts. These results bring interesting new perspectives to future of software security in APF biddings.