Athena: A framework to automatically generate security test oracle via extracting policies from source code and intended software behaviour

Abstract

Context: Software security testing aims to check the security behaviour of a program. To determine whether the program behaves securely on a particular execution, we need an oracle who knows the expected security behaviour. Security test oracle decides whether test cases violate the intended security policies of the program. Thus, it is necessary for the oracle to model the detailed security policies. Unfortunately, these policies are usually poorly documented. Even worse, in some cases, the source code is the only available document of the program.Objective: We propose a method to automatically extract the intended security policies of the program under test from the source code and expected execution traces. We introduce a security test oracle, Athena, which utilises these policies to differentiate between the secure and potentially insecure behaviour of the program.Method: We use a hybrid analysis approach to obtain the intended security policies. We investigate the program statements (gates) in which the software communicates with the environment. We analyse the transmitted messages in the gates and the control and data flow of the program to extract some security properties. Moreover, we specify the intended navigation paths of the program. These properties and paths form the expected security policies. Athena utilises these policies to detect potential security breaches.Results: Investigating common types of software vulnerabilities illustrates the flexibility of Athena in modelling various kinds of security policies. Moreover, we show the usefulness of the method by applying it to the real web applications and evaluating its capability to detect actual attacks.Conclusions: Our proposed approach takes a step towards solving the test oracle automation problem in the domain of security testing.