Evaluation of isolation in virtual machine environments encounter in effective attacks against memory

Abstract

AbstractOne of the main features of the hypervisor is the isolation among virtual machine (VM) environments. By isolation between VMs, malicious activity in one VM could not affect all other VMs, so it is necessary to apply security mechanisms in order to improve isolation between VMs. Before applying security policies to a virtualization system, it is necessary to quantitatively measure the hypervisor from isolation point of view aiming at increasing security of isolation among VMs; then considering the circumstances of the VM execution environments and the results of measurements, we can find areas in the virtualization system with the most effective on enhancing isolation. This paper, proposed a semi‐Markov model for evaluation of isolation, by studying the Xen virtualization architecture. We considered certain type of vulnerability that successfully exploiting it can lead to the attacker's malicious codes execution in part of memory address space. We included all three layers in virtualization for the evaluation purpose, because we wanted to consider strength and weakness areas in virtualization system and not just specific layer such as hypervisor, so it can be figured out that improving security in which layer of virtualization is most effective in improving security of isolation, in respect to the increasing or decreasing attacker's (defender's) ability to be successful. The sensitivity analysis results show that MTTSF is more sensitive to the increasing ability of defensive mechanisms to be successful at the application layer and decreasing the attacker's ability to successfully exploit vulnerabilities at the guest operating system layer, model parameters. Copyright © 2015 John Wiley & Sons, Ltd.