Playing Inside the Black Box Using Dynamic Instrumentation to Create Security Holes

Abstract

Programs running on insecure or malicious hosts have often been cited as ripe targets for security attacks. The enabling technology for these attacks is the ability to easily analyze and control the running program. Dynamic instrumentation provides the necessary technology for this analysis and control. As embodied in the DynInst API library, dynamic instrumentation allows easy construction of tools that can: (1) inspect a running process, obtaining structural information about the program; (2) control the execution of the program, (3) cause new libraries to be dynamically loaded into the process’ address space; (4) splice new code sequences into the running program and remove them; and (5) replace individual call instructions or entire functions. With this technology, we have provided two demonstrations of its use: exposing vulnerabilities in a distributed scheduling system (Condor), and bypassing access to a license server by a word processor (Framemaker). The first demonstration shows the danger of remote execution of a job on a system of unknown pedigree, and the second demonstration shows the vulnerabilities of software license protection schemes. While these types of vulnerabilities have long been speculated, we show how, with the right tool (the DynInst API), they can be easily accomplished. Along with this discussion of vulnerabilities, we also discuss strategies for compensating for them. Programs in execution have long been considered to be immutable objects. Object code and libraries are emitted by the compiler, linked and then executed; any changes to the program require revisiting the compile or link steps. In contrast, we consider a running program to be an object that can be examined, instrumented, and re-arranged on the fly. The DynInst API provides a portable library for tool builders to construct tools that operate on a running program. Where previous tools might have required a special compiler, linker, or run-time library, tools based on DynInst can operate directly on unmodified binary programs during execution. In this papers, we show how this technology can be used to subvert system security. The discussions will be based on two example cases: exposing vulnerabilities in a distributed scheduling system (Condor), and bypassing access to a license server by a word processor (Framemaker). For the Condor study, we create “lurker” processes that can be left latent on a host in the Condor pool. These lurker processes lie in wait for subsequent Condor jobs to arrive on the infected host. The lurker will then use Dyninst to attach to the newly-arrived victim job and take control. Once in control, the lurker can cause the victim job to make requests back to its home host, causing it execute a wide variety of system calls. For the license-server study, we constructed a collection of Dyninst-based tools that allowed us to understand the control flow within the application (Framemaker) program. As a result, we were able to detect and remove Framemaker’s contact with the license server. In addition, there are frequent checks within Framemaker to see if it has cached a valid license credential. Using our Dyninst-based tools, we were able to locate and neutralize these checks. For each of these studies, we provide some suggestions as to how to make them less vulnerable to attack.