Selective Instrumentation Mechanism and its Application in a Virtual Machine

Abstract

Among existing approaches to software analysis one stands out: dynamic binary analysis, implemented with dynamic binary instrumentation (DBI). Instrumentation allows to perform analysis by inserting user-defined instructions into examined code flow. DBI frameworks allows to perform analysis in the absence of original source code, as well as providing functionality to change and supplement analysis conditions on-the-go. These capabilities provide performing analysis of any complexity and for any software. However, analysis quality and ease of use of dynamic binary instrumentation directly depends on implemented functionality in a chosen framework. One of the key features, allowing convenient analysis process is a possibility to specify and to narrow instrumentation target from operating system to smaller and more precise entities in system, like: process, thread, memory range. This ability is called selective instrumentation. Having this feature analyst may switch freely between whole system instrumentation and selective instrumentation both ways, which allows to benefit from both approaches while using the same framework. Whole system instrumentation affords the most comprehensive overview of all running applications in the system and the system itself. However the downside is a noticeable slowdown of the analyzed system, which can lead to malfunctioning of the system, and excessive amount of data that needs to be processed and analyzed. Selective instrumentation allows one to specify the area of interest for analysis routines. This can be performed at the right time and for specific entities, which provides a more accurate result depending on the goals. In this paper we are going to look through existing approaches for selective instrumentation and define their flaws. Then we will propose an approach for instrumentation of processes, threads, fibers and memory, and will describe test implementation for ARM and x86 architectures. In the last part of the paper we will describe application examples of developed selective instrumentation approaches.