Memory Allocation Vulnerability Analysis and Analysis Optimization for C Programs Based on Formal Methods

Abstract

The information security problems caused by the software vulnerabilities have became more and more complex. Among these vulnerabilities, the ones existing in memory allocations appear to be difficult to diagnose due to the absence of an appropriate method. In order to solve this problem, we introduce a methodology including four novel frameworks in this paper. The formalization for a program called algebraic transition system is proposed first. It aims to transform the data exchange process and its security attribute of a program into algebraic systems which are able to be considered as objection functions and constraint conditions, respectively. Based on the systems, the behavior and structure of formalization are optimized with bisimulation to reduce the computing cost in the subsequent processes. The determination of bisimulation is implemented by numerical and symbolic computation. Finally, the specific detection of the memory allocation vulnerability in the C program can be changed into a constraints solving problem called Max function which is able to be resolved with the filled function method. The experiment results represent that our approach is feasible.