An efficient block-discriminant identification of packed malware

Abstract

Advanced persistent attacks, incorporated by sophisticated malware, are on the rise against hosts, user applications and utility software. Modern malware hide their malicious payload by applying packing mechanism. Packing tools instigate code encryption to protect the original malicious payload. Packing is employed in tandem with code obfuscation/encryption/compression to create malware variants. Despite being just a variant of known malware, the packed malware invalidates the traditional signature based malware detection as packing tools create an envelope of packer code around the original base malware. Therefore, unpacking becomes a mandatory phase prior to anti-virus scanning for identifying the known malware hidden behind packing layers. Existing techniques of unpacking solutions increase execution overhead of AV scanners in terms of time. This paper illustrates an easy to use approach which works in two phases to reduce this overhead. The first phase (ESCAPE) discriminates the packed code from the native code (non-packed) by using random block entropy. The second phase (PEAL) validates inferences of ESCAPE by employing bi-classification (packed vs native) model using relevant hex byte features extracted blockwise. The proposed approach is able to shrink the overall execution time of AV scanners by filtering out native samples and avoiding excessive unpacking overhead. Our method has been evaluated against a set consisting of real packed instances of malware and benign programs.