An approach to dynamic malware analysis based on system and application code split

Abstract

This paper discusses the development of tools for dynamic malware analysis. The main idea is to provide total control on a suspicious sample execution on the test computer. The approach we propose is to separate the application code from the system code by using memory pages access control. Thus, we are able to detect all system API calls and non-standard ways to transfer the control flow. Our tools (codename ToolChain) intentionally consist of a Control module, a Scheduling module, and a Cloaking module. In this paper, we focus mainly on the Control module. We monitor internal target process events by using invasive methods such as a system call hook or an executable file patch. This research describes the key creation stages of the prototype, with the basic functionality and technical ideas on handling several issues, such as analysis of multi-threaded applications, cloaking of the presence of analytical tools, and mitigation of the performance degradation of the operation system.