Analysis to Heap Overflow Exploit in Linux with Symbolic Execution

Abstract

Heap overflow is a common error of buffer overflow in Linux. The control flow of a program may be hijacked when the program satisfies several specific conditions. The existing automatic exploit generation technologies for buffer overflow find vulnerability trigger point and generate exploit by checking the control flow state. However, the heap overflow data rarely lead to a control flow hijacking as well as protection mechanisms limit the trigger condition. It is difficult to analyze the exploitability of heap overflow automatically through the existing analysis technology. For the heap overflow errors in Linux, we summarize the features of exploit on the basis of analyzing the instances, building the detection model of the exploitability of heap overflow, and proposing a method for analyzing the exploitability of heap overflow based on the model. The proposed method monitors the input data and insecurity functions of the program by using taint analysis; builds the path constraints and data constraints which satisfy the conditions of heap overflow exploit through selective symbolic execution; solves the abovementioned constraints and generates the test case automatically. All the steps of our method can be finished automatically by using the symbolic execution tool S2E. The experiments show that this method can automatically analyze and detect the exploitability of heap overflow errors.