Testing for software vulnerability using environment perturbation

Abstract

AbstractWe describe a methodology for testing a software system for possible security flaws. Based on the observation that most security flaws are caused by the program's inappropriate interactions with the environment, and are triggered by a user's malicious perturbation on the environment (which we call an environment fault), we view the security testing problem as the problem of testing for the fault‐tolerance properties of a software system. We consider each environment perturbation as a fault and the resulting security compromise a failure in the toleration of such faults. Our approach is based on the well‐known technique of fault injection. Environment faults are injected into the system under test and system behavior observed. The failure to tolerate faults is an indicator of a potential security flaw in the system. An Environment‐Application Interaction (EAI) fault model is proposed which guides us to decide what faults to inject. Based on EAI, we have developed a security testing methodology, and applied it to several applications. We have successfully identified a number of vulnerabilities including vulnerabilities in the Windows NT operating system. Copyright © 2002 John Wiley & Sons, Ltd.