TKRD: Trusted kernel rootkit detection for cybersecurity of VMs based on machine learning and memory forensic analysis.

Xiao Wang,Jianbiao Zhang,Ai Zhang,Jinchang Ren

doi:10.3934/mbe.2019132

Abstract

The promotion of cloud computing makes the virtual machine (VM) increasingly a target of malware attacks in cybersecurity such as those by kernel rootkits. Memory forensic, which observes the malicious tracks from the memory aspect, is a useful way for malware detection. In this paper, we propose a novel TKRD method to automatically detect kernel rootkits in VMs from private cloud, by combining VM memory forensic analysis with bio-inspired machine learning technology. Malicious features are extracted from the memory dumps of the VM through memory forensic analysis method. Based on these features, various machine learning classifiers are trained including Decision tree, Rule based classifiers, Bayesian and Support vector machines (SVM). The experiment results show that the Random Forest classifier has the best performance which can effectively detect unknown kernel rootkits with an Accuracy of 0.986 and an AUC value (the area under the receiver operating characteristic curve) of 0.998.

Highlights

With the popularization of cloud computing [1], cybersecurity has become a major concern [2,3]
Aiming at solving the cybersecurity problem of virtual machine in private cloud computing environment, a TKRD method is proposed to detect known and unknown kernel rootkits based on machine learning method
We leveraged memory forensic technology to obtain the malicious tracks in the virtual machine (VM) memory

Summary

Introduction

With the popularization of cloud computing [1], cybersecurity has become a major concern [2,3]. According to the model of service, cloud computing can be divided into public, private and hybrid cloud. Kernel rootkits modify the critical kernel data structures, making it much more difficult for detection than any other kinds of malwares. Traditional anti-virus solutions are based on the signatures of the already known malwares whilst installed in the same system with the malware. They can only detect known malwares and can be spotted and evaded by the smart malwares

Methods

Results

Conclusion