Binary-centric defense of production operating systems against kernel queue injection attacks

Abstract

Kernel callback queues (KQs) are the established mechanism for event handling in modern kernels. Unfortunately, real-world malware has abused KQs to run malicious logic, through an attack called kernel queue injection (KQI). Current kernel-level defense mechanisms have difficulties with KQI attacks, since they work without necessarily changing legitimate kernel code or data. In this paper, we present the design, implementation, and evaluation of KQguard, an efficient and effective protection mechanism of KQs. KQguard employs static and dynamic analysis of kernel and device drivers to learn specifications of legitimate event handlers. At runtime, KQguard rejects all the unknown KQ requests that cannot be validated. We implement KQguard on the Windows Research Kernel (WRK), Windows XP, and Linux, using source code instrumentation or binary patching. Our extensive experimental evaluation shows that KQguard is effective (i.e., it can have zero false positives against representative benign workloads after enough training and very low false negatives against 125 real-world malware), and it incurs a small overhead (up to ~5%). We also present the result of an automated analysis of 1,528 real-world kernel-level malware samples aiming to detect their KQ Injection behaviors. KQguard protects KQs in both Windows and Linux kernels, can accommodate new device drivers, and can support closed source device drivers through dynamic analysis of their binary code.