Growing Grapes in Your Computer to Defend Against Malware

Abstract

Behavior-based detection is promising to resolve the pressing security problem of malware. However, the great challenge lies in how to detect malware in a both accurate and light-weight manner. In this paper, we propose a novel behavior-based detection method, named growing grapes, aiming to enable accurate online detection. It consists of a clustering engine and detection engine. The clustering engine groups the objects, e.g., processes and files, of a suspicious program together into a cluster, just like growing grapes. The detection engine recognizes the cluster as malicious if the behaviors of the cluster match a predefined behavior template formed by a set of discrete behaviors. The approach is accurate since it identifies a malware based on multiple behaviors and the source of the processes requesting the behaviors. The approach is also light-weight as it uses OS-level information flows instead of data flows that generally impose significant performance impact on the system. To further improve the performance, a novel method of organizing the behavior template and template database is proposed, which not only makes the template matching process very quick, but also makes the storage space small and fixed. Furthermore, the detection accuracy and performance are optimized to the best degree using a combinatorial optimization algorithm, which properly selects and combines multiple behaviors to form a template for malware detection. Finally, the approach novelly identifies malicious OS objects in a cluster fashion rather than one by one as done in traditional methods, which help users to thoroughly eliminate the changes of a malware without malware family knowledge. Compared with commercial antimalware tools, extensive experiments show that our approach can detect new malware samples with higher detection rate and lower false positive rate while imposing low overhead on the system.