Rootkit Attacks Research Articles

Machine Learning and Deep Learning Based Model for the Detection of Rootkits Using Memory Analysis

Rootkits are malicious programs designed to conceal their activities on compromised systems, making them challenging to detect using conventional methods. As the threat landscape continually evolves, rootkits pose a serious threat by stealthily concealing malicious activities, making their early detection crucial to prevent data breaches and system compromise. A promising strategy for monitoring system activities involves analyzing volatile memory. This study proposes a rootkit detection model that combines memory analysis with Machine Learning (ML) and Deep Learning (DL) techniques. The model aims to identify suspicious patterns and behaviors associated with rootkits by analyzing the contents of a system’s volatile memory. To train the model, a diverse dataset of known rootkit samples is employed, and ML and deep learning algorithms are utilized. Through extensive experimentation and evaluation using SVM, RF, DT, k-NN, and LSTM algorithms, it is determined that SVM achieves the highest accuracy rate of 96.2%, whereas Execution Time (ET) shows that k-NN depicts the best performance, and LSTM (a DL model) shows the worst performance among the tested algorithms. This research contributes to the development of advanced defense mechanisms and enhances system security against the constantly evolving threat of rootkit attacks.

Root-Of-Trust for Continuous Integration and Continuous Deployment Pipeline in Cloud Computing

Cloud computing has gained significant use over the last decade due to its several benefits, including cost savings associated with setup, deployments, delivery, physical resource sharing across virtual machines, and availability of on-demand cloud services. However, in addition to usual threats in almost every computing environment, cloud computing has also introduced a set of new threats as consumers share physical resources due to the physical co-location paradigm. Furthermore, since there are a growing number of attacks directed at cloud environments (including dictionary attacks, replay code attacks, denial of service attacks, rootkit attacks, code injection attacks, etc.), customers require additional assurances before adopting cloud services. Moreover, the continuous integration and continuous deployment of the code fragments have made cloud services more prone to security breaches. In this study, the model based on the root of trust for continuous integration and continuous deployment is proposed, instead of only relying on a single sign-on authentication method that typically uses only id and password. The underlying study opted hardware security module by utilizing the Trusted Platform Module (TPM), which is commonly available as a cryptoprocessor on the motherboards of the personal computers and data center servers. The preliminary proof of concept demonstrated that the TPM features can be utilized through RESTful services to establish the root of trust for continuous integration and continuous deployment pipeline and can additionally be integrated as a secure microservice feature in the cloud computing environment.

Virtual Wall: Filtering Rootkit Attacks To Protect Linux Kernel Functions

Linux servers are being used in almost all clouds, datacenters and supercomputers today. Linux Kernel functions are facing a kind of malware attacks, known as rootkits with root-access capability. The rootkits appear as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">loadable kernel modules</i> (LKM) in today's Linux servers. These modules hide from other kernel objects, and can redirect the kernel control flow by tampering with the metadata needed in kernel service functions. The kernel rootkits are invisible to users after loading, which may bypass most security shields. Both spatial and temporal appearance of rootkits are randomly distributed, which makes it difficult to detect or removal. To deal with rootkit threats, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Virtual Wall</i> (VTW) approach to filtering out the rootkit-embedded LKMs by tracing the incurred kernel activities. This VTW is essentially a lightweight hypervisor built with rootkit detection and event tracing capabilities. Normally, the Linux runs in a guest mode. When a LKM execution violates the security policy set by the VTW, the OS control will switch to a host mode. The VTW at host mode enables the detection and tracing of rootkit events timely. In other words, potential rootkit attacks are detected, traced and classified to make meaningful filtering decisions. The whole detection and tracing process is based on memory access control and event injection mechanisms. Experimental results show that the VTW defense system is effective to detect and defend against kernel rootkits timely. The CPU overhead for executing VTW is less than 2 percent. Compared with other defense schemes (such as DIKernel, etc.), our vs is easier to implement with low performance degradation on Linux servers. We will demonstrate the advantages of VTW through its simplicity in implementation and potential performance gains. We will also compare our system with seven other rootkit defense systems.

An Investigation on Feasibility and Security for Cyberattacks on Generator Synchronization Process

Electric power supply is crucial for the effective operation of many sectors such as manufacturing, health care, etc. Further, electrification of the transportation network including metro-rail systems and personal electric vehicles necessitates capacity expansion of power distribution system (PDS). Capacity expansion in PDSs is seeing a clear paradigm shift from component upgrades to the incorporation of distributed energy resources. Though renewable energy sources are attractive from the sustainability aspect, conventional generators are also essential for many applications such as maintaining the grid stability. In this article, for the first time, we report a new type of attack on the generators; attacks on the synchronization process, targeting the availability of generators. The threat model considered in this article is similar to the popular real-world attacks, such as “Stuxnet worm” or programmable logic controller root-kit attacks such as “Harvey,” however, the reported attack goal is novel. We propose a defense mechanism that can be decoupled from the network data and is named as physical layer security (PLS). We present theoretical analysis, simulation results, and experimental validation for the reported attack. We also propose an alternative implementation named as Level 0–1 security for cases where PLS may not be practical. The proposed defense is generic and can be extended to many other applications/attack vectors.

HPCgnature: a hardware‐based application‐level intrusion detection system

In the past decade, commodity software applications have been deployed more than ever in almost every domain. Having the ability to differentiate the original trusted application at run-time from its compromised, mimic or trojanised versions would mitigate a broad range of intrusion threats to these applications. This has been addressed by application-level intrusion detection systems, however, such schemes mostly depend on the system software for either monitoring or modelling the application. This is while system software can itself get compromised by kernel-level rootkit attacks. In this study, the authors have proposed a new hardware-based app-IDS, which works independent of the system software of the target system. The proposed method, referred to as HPCgnature, includes a new abstraction corresponding to the repetitious functionalities of programs. Such functionalities generate a distinguishing sequence of periods, referred to in this study as the Operational Periodicity. The method uses monitoring scheme based on external access to the hardware performance counters of CPUs. Implementing a prototype, they have shown how HPCgnature can detect intrusions in 12 complex interactive desktop applications. Evaluation results indicate this model could differentiate applications with 98% accuracy, and can detect even small run-time code injection attacks by an accuracy of >75%

Domain Isolated Kernel: A lightweight sandbox for untrusted kernel extensions

Monolithic kernel is one of the prevalent configurations out of various kernel design models. While monolithic kernel excels in performance and management, they are unequipped for runtime system update; and this brings the need for kernel extension. Although kernel extensions are a convenient measure for system management, it is well established that they make the system prone to rootkit attacks and kernel exploitation as they share the single memory space with the rest of the kernel. To address this problem, various forms of isolation (e.g., making into a process), are so far proposed, yet their performance overhead is often too high or incompatible for a general purpose kernel. In this paper, we propose Domain Isolated Kernel (DIKernel), a new kernel architecture which securely isolates the untrusted kernel extensions with minimal performance overhead. DIKernel leverages hardware-based memory domain feature in ARM architecture; and prevents system manipulation attacks originated from kernel extensions, such as rootkits and exploits caused by buggy kernel extensions. We implemented DIKernel on top of Linux 4.13 kernel with 1500 LOC. Performance evaluation indicates that DIKernel imposes negligible overhead which is observed by cycle level microbenchmark.

A game theoretic-based distributed detection method for VM-to-hypervisor attacks in cloud environment

Cloud computing is a pool of scalable virtual resources serving a large number of users who pay fees depending on the extent of utilized service. From payment perspective, cloud is like electricity and water as people who use more of this shared pool should pay larger fees. Cloud computing involves a diverse set of technologies including networking, virtualization and transaction scheduling. Thus, it is vulnerable to a wide range of security threats. Some of the most important security issues threatening the cloud computing systems originate from virtualization technology, as it constitutes the main body and basis of these systems. The most important virtualization-based security threats include VM side channel, VM escape and rootkit attacks. The previous works on the subject of virtualization security rely on hardware approaches such as the use of firewalls, which are expensive, the use of schedulers to control the side channels along with noise injection, which impose high overhead, or the use of agents to collect information and send them back to a central intrusion detection system, which itself can become the target of attacker. In the method presented in this paper, a group of mobile agents act as the sensors of invalid actions in the cloud environment. They start a noncooperative game with the suspected attacker and then calculate the Nash equilibrium value and utility so as to differentiate an attack from legitimate requests and determine the severity of attack and its point of origin. The simulation results show that this method can detect the attacks with 86% accuracy. The use of mobile agents and their trainability feature has led to reduced system overhead and accelerated detection process.

Detecting and Preventing Kernel Rootkit Attacks with Bus Snooping

To protect the integrity of operating system kernels, we present Vigilare system , a kernel integrity monitor that is architected to snoop the bus traffic of the host system from a separate independent hardware. This snoop-based monitoring enabled by the Vigilare system, overcomes the limitations of the snapshot-based monitoring employed in previous kernel integrity monitoring solutions. Being based on inspecting snapshots collected over a certain interval, the previous hardware-based monitoring solutions cannot detect transient attacks that can occur in between snapshots, and cannot protect the kernel against permanent damage. We implemented three prototypes of the Vigilare system by adding Snooper hardware connections module to the host system for bus snooping, and a snapshot-based monitor to be comared with, in order to evaluate the benefit of snoop-based monitoring. The prototypes of Vigilare system detected all the transient attacks and the second one protected the kernel with negligible performance degradation while the snapshot-based monitor could not detect all the attacks and induced considerable performance degradation as much as 10 percent in our tuned STREAM benchmark test.

Combating Kernel Rootkits on Linux Version 2.6 (Analysis of Rootkit Prevention, Detection and Correction)

Rootkits are a major security concern for smartphones today. They have always been around, though largely operational on desktops and PCs. On the mobile platform, their presence was not very popular until the advent of smartphones and advanced mobile devices. The rapid developments and trends recorded on smartphones today make them highly vulnerable to rootkit attacks. Smartphone operating systems now come highly sophisticated and packaged with advanced functionality to keep record of users’ diary, sensitive personal and security details, among others. These features make them a prime choice for attacks from rootkit authors, who explore all available avenues to exploit and extract this information for malicious purposes. Cases of rootkit attacks have continued to increase with more of such attacks targeted at popular smartphone operating systems like Android. In this research, we discuss rootkits, illustrating their operational architecture and operation with a design of a kernel rootkit for the Linux kernel 2.6. We explore possible measures to combat rootkits on the mobile platform, using Android as a case study

Efficient Kernel Integrity Monitor Design for Commodity Mobile Application Processors

In recent years, there are increasing threats of rootkits that undermine the integrity of a system by manipulating OS kernel. To cope with the rootkits, in Vigilare, the snoop-based monitoring which snoops the memory traffics of the host system was proposed. Although the previous work shows its detection capability and negligible performance loss, the problem is that the proposed design is not acceptable in recent commodity mobile application processors (APs) which have become de facto the standard computing platforms of smart devices. To mend this problem and adopt the idea of snoop-based monitoring in commercial products, in this paper, we propose a snoop-based monitor design called S-Mon, which is designed for the AP platforms. In designing S-Mon, we especially consider two design constraints in the APs which were not addressed in Vigilare; the unified memory model and the crossbar switch interconnect. Taking into account those, we derive a more realistic architecture for the snoop-based monitoring and a new hardware module, called the region controller, is also proposed. In our experiments on a simulation framework modeling a production-quality device, it is shown that our S-Mon can detect the rootkit attacks while the runtime overhead is also negligible.

Virtual ghost

Applications that process sensitive data can be carefully designed and validated to be difficult to attack, but they are usually run on monolithic, commodity operating systems, which may be less secure. An OS compromise gives the attacker complete access to all of an application's data, regardless of how well the application is built. We propose a new system, Virtual Ghost, that protects applications from a compromised or even hostile OS. Virtual Ghost is the first system to do so by combining compiler instrumentation and run-time checks on operating system code, which it uses to create ghost memory that the operating system cannot read or write. Virtual Ghost interposes a thin hardware abstraction layer between the kernel and the hardware that provides a set of operations that the kernel must use to manipulate hardware, and provides a few trusted services for secure applications such as ghost memory management, encryption and signing services, and key management. Unlike previous solutions, Virtual Ghost does not use a higher privilege level than the kernel. Virtual Ghost performs well compared to previous approaches; it outperforms InkTag on five out of seven of the LMBench microbenchmarks with improvements between 1.3x and 14.3x. For network downloads, Virtual Ghost experiences a 45% reduction in bandwidth at most for small files and nearly no reduction in bandwidth for large files and web traffic. An application we modified to use ghost memory shows a maximum additional overhead of 5% due to the Virtual Ghost protections. We also demonstrate Virtual Ghost's efficacy by showing how it defeats sophisticated rootkit attacks.

Virtual ghost

Applications that process sensitive data can be carefully designed and validated to be difficult to attack, but they are usually run on monolithic, commodity operating systems, which may be less secure. An OS compromise gives the attacker complete access to all of an application's data, regardless of how well the application is built. We propose a new system, Virtual Ghost, that protects applications from a compromised or even hostile OS. Virtual Ghost is the first system to do so by combining compiler instrumentation and run-time checks on operating system code, which it uses to create ghost memory that the operating system cannot read or write. Virtual Ghost interposes a thin hardware abstraction layer between the kernel and the hardware that provides a set of operations that the kernel must use to manipulate hardware, and provides a few trusted services for secure applications such as ghost memory management, encryption and signing services, and key management. Unlike previous solutions, Virtual Ghost does not use a higher privilege level than the kernel. Virtual Ghost performs well compared to previous approaches; it outperforms InkTag on five out of seven of the LMBench microbenchmarks with improvements between 1.3x and 14.3x. For network downloads, Virtual Ghost experiences a 45% reduction in bandwidth at most for small files and nearly no reduction in bandwidth for large files and web traffic. An application we modified to use ghost memory shows a maximum additional overhead of 5% due to the Virtual Ghost protections. We also demonstrate Virtual Ghost's efficacy by showing how it defeats sophisticated rootkit attacks.

Design on Framework for Dynamic Monitoring of Virtual Machines

In order to ensure that the cloud platform client runtime kernel virtual machine security, this paper proposes a new framework for dynamic monitoring of virtual machines, it is for the kernel rootkit attacks, study the cloud client virtual machine operating system kernel safety, presented Hyperchk virtual machine dynamic monitoring framework. This framework mainly for kernel rootkit attacks, ensure that customers running virtual machine kernel security.

A framework for prototyping and testing data-only rootkit attacks

Kernel rootkits—attacks which modify a running operating system kernel in order to hide an attacker's presence—are significant threats. Recent advances in rootkit defense technology will force rootkit threats to rely on only modifying kernel data structures without injecting and executing any new code; however these data-only kernel rootkit attacks are still both realistic and powerful. In this work we present DORF, a framework for prototyping and testing data-only rootkit attacks. DORF is an object-oriented framework that allows researchers to construct attacks that can be easily ported between various Linux distributions and versions. The current implementation of DORF contains a group of existing and new data-only attacks, and the portability of DORF is demonstrated by porting it to 6 different Linux distributions. The goal of DORF is to allow researchers to construct repeatable experiments with little effort, which will in turn advance research into data-only attacks and defenses.

Comprehensive and Efficient Protection of Kernel Control Data

Protecting kernel control data (e.g., function pointers and return addresses) has been a serious issue plaguing rootkit defenders. In particular, rootkit authors only need to compromise one piece of control data to launch their attacks, while defenders need to protect thousands of such values widely scattered across kernel memory space. Worse, some of this data (e.g., return addresses) is volatile and can be dynamically generated at run time. Existing solutions, however, offer either incomplete protection or excessive performance overhead. To overcome these limitations, we present indexed hooks, a scheme that greatly facilitates kernel control-flow enforcement by thoroughly transforming and restricting kernel control data to take only legal jump targets (allowed by the kernel's control-flow graph). By doing so, we can severely limit the attackers' possibility of exploiting them as an infection vector to launch rootkit attacks. To validate our approach, we have developed a compiler-based prototype that implements this technique in the FreeBSD 8.0 kernel, transforming 49 025 control transfer instructions (~7.25% of the code base) to use indexed hooks instead of direct pointers. Our evaluation results indicate that our approach is generic, effective, and can be implemented on commodity hardware with a low performance overhead (<;5% based on benchmarks).

Rootkit attacks and protection

A rootkit is a type of malware designed to gain privileged access to a computer while hiding itself from the user and the operating system by, for example, compromising the communication channels w...

Automated containment of rootkits attacks

Rootkit attacks are a serious threat to computer systems. Packaged with other malwares such as worms, viruses and spyware, rootkits pose a more potent threat than ever before by allowing malware to evade detection. In the absence of appropriate tools to counter such attacks, compromised machines stay undetected for extended periods of time. Leveraging virtual machine technology, we propose a solution for real-time automated detection and containment of rootkit attacks. We have developed a prototype using VMware Workstation to illustrate the solution. Our analysis and experimental results indicate that this approach can very successfully detect and contain the effects of a large percentage of rootkits found for Linux today. We also demonstrate with an example, how this approach is particularly effective against malwares that use rootkits to hide.

Root Kits

Root Kits are tool boxes containing a collection of highly skilled tools for attacking computer systems. Their algorithms and databases contain professional knowledge about methods and mechanisms for completely automated attacks both over a network as well as from within a system. Root kits attack by maneuvering a system into executing a script with supervisor privileges. Once having gained full control, such scripts begin to install several software packages, including backdoors for easy future access, deception packages and modified versions of administration utilities that conceal system modifications and refuse to counterattack any future infiltration.The security threat imposed by root kits is quite serious. A root kit attack is swift, fully automatic, and has long-lasting effects. An attack has a high success probability, and it requires only a very small amount of knowledge. Last not least, root kits axe easily available in the Internet.This paper is a survey of the works of root kits from an operating systems point of view. Keywords: error exploitation, error proliferation, privilege proliferation, kernel abstractions, trusted computing base, reference monitor, security domains, mandatory and discretionary access control, secure booting, secure program execution

Recognizing and recovering from Rootkit attacks

Recognizing and recovering from Rootkit attacks