Behavior based authentication mechanism to prevent malicious code attacks in windows

Abstract

Most modern kernel of the operating system fails to ensure the authenticity of a suspicious process while servicing its system call. As a result, preventing kernel level malicious code attacks that target system table hooking becomes a challenging and serious security issue. The traditional process authentication techniques such as the process name, process identifier and execution path exercised by the kernel are not reliable. Therefore, in this paper, we proposed a kernel level authentication prototype to verify the originality of each suspicious process during runtime. The verification and authentication tasks are performed well in advance before each suspicious process getting the kernel service. We designed, implemented, and assessed the prototype in Windows. The evaluation results confirm that the prototype successfully blocked all malicious code attacks that target invoking system services directly in the kernel mode with minimal overhead.