Dynamic Overflow Vulnerability Detection Method Based on Finite CSP

Abstract

Overflow vulnerability is one of the most dangerous vulnerability types.Attackers can make use of this vulnerability to execute arbitrary code and control the vulnerable system.Thus,overflow vulnerability detection has become a hot topic in software analysis field.Vulnerability detection methods can usually be divided into two classes: white-box testing and black-box testing.The former one analyzes instructions of software statically,but it has the problem of inefficiency and impreciseness.The latter one cannot guarantee the coverage of programs.The generated input data is usually not effectiveness,either.This paper presents a dynamic method to detect vulnerabilities based on finite CSP(Constraint Satisfaction Problem).By launching target programs in virtual surroundings,we choose suitable instructions to generate CSP expressions dynamically with the help of taint analysis and loop analysis.If the CSP expression is satisfiable,the vulnerability exists and the solution can trigger the vulnerability.In this way,it can analyze binary code directly without any source.Our method can solve the problems such as alias pointers and indirect jumps,which is not easy to solve in static analysis.We developed a proof-of-concept system and made several experiments on it.The results show that our method improves the efficiency to detect overflow vulnerabilities.