Guest Memory Research Articles

Improving live migration efficiency in QEMU: An eBPF-based paravirtualized approach

Virtual Machines are the key technology in cloud computing. In order to upgrade, repair or service the physical machine where a Virtual Machine is hosted, a common practice is to live-migrate the Virtual Machine to a different server. This involves copying all the guest memory over the network, which may take a non-negligible amount of time. In this work, we propose a technique to speed up the migration time by reducing the amount of guest memory to be transferred with the help of the guest OS. In particular, during live-migration, an eBPF program is injected in the guest kernel to obtain, and send to the Virtual Machine Monitor, the list of guest page frames that are currently unused. The VMM can then safely skip these pages during the copy. We have integrated this technique in the live-migration implementation of QEMU (Bellard, 2005), and we show the effects of our work in some experiments comparing the results against the QEMU default implementation.

Trinity: High-Performance and Reliable Mobile Emulation through Graphics Projection

Mobile emulation, which creates full-fledged software mobile devices on a physical PC/server, is pivotal to the mobile ecosystem. Unfortunately, existing mobile emulators perform poorly on graphics-intensive apps in terms of efficiency and compatibility. To address this, we introduce graphics projection , a novel graphics virtualization mechanism that adds a small-size projection space inside the guest memory, which processes graphics operations involving control contexts and resource handles without host interactions. While enhancing performance, the decoupled and asynchronous guest/host control flows introduced by graphics projection can significantly complicate emulators’ reliability issue diagnosis when faced with a variety of uncommon or non-standard app behaviors in the wild, hindering practical deployment in production. To overcome this drawback, we develop an automatic reliability issue analysis pipeline that distills the critical code paths across the guest and host control flows by runtime quarantine and state introspection. The resulting new Android emulator, dubbed Trinity, exhibits an average of 97% native hardware performance and 99.3% reliable app support, in some cases outperforming other emulators by more than an order of magnitude. It has been deployed in Huawei DevEco Studio, a major Android IDE with millions of developers.

GuaNary: Efficient Buffer Overflow Detection In Virtualized Clouds Using Intel EPT-based Sub-Page Write Protection Support

Write buffer overflow is a widespread and prevalent memory safety violation in C/C++, reported as the top vulnerability in 2022 and 2023. Secure memory allocators are generally used to protect systems against attacks that may exploit buffer overflows. Existing allocators mainly rely on two types of countermeasures to prevent or detect write overflows: canaries and guard pages, each with pros and cons in terms of detection latency and memory footprint. For virtualized cloud applications, this paper follows the Out of Hypervisor (OoH) trend and introduces GuaNary, a safety guard against write overflows, allowing synchronous detection at a low memory footprint cost. OoH is a new virtualization research axis introduced in 2022 advocating the exposure of hardware features for virtualization to the guest OS so that its processes can take advantage of them. Based on the OoH principle, GuaNary leverages Intel Sub-Page write Permission (SPP), a recent hardware virtualization feature that allows to write-protect guest memory at the granularity of 128B (namely, sub-page) instead of 4KB. We implement a software stack, LeanGuard, which promotes the utilization of SPP from inside virtual machines by new secure allocators that use GuaNary. Our evaluation shows that for the same number of protected buffers, LeanGuard consumes 8.3× less memory than SlimGuard, a recent state-of-art secure allocator. Further, for the same memory consumption, LeanGuard allows protecting 25× more buffers than SlimGuard.

Optimizing data permutations in structured loads/stores translation and SIMD register mapping for a cross-ISA dynamic binary translator

More and more modern processors have been supporting non-contiguous SIMD data accesses. However, translating such instructions has been overlooked in the Dynamic Binary Translation (DBT) area. For example, in the popular QEMU dynamic binary translator, guest memory instructions with strides are emulated by a sequence of scalar instructions, leaving a significant room for performance improvement when the SIMD instructions are available on the host machines. Structured loads/stores, such as VLDn/VSTn in ARM NEON, are one type of strided SIMD data access instructions. They are widely used in signal processing, multimedia, mathematical, and 2D matrix transposition applications. Efficient translation of such structured loads/stores is a critical issue when migrating ARM executables to other ISAs. However, it is quite challenging since not only the translation of structured loads/stores is not trivial, but also the mapping of SIMD registers between guest and host is complicated. This paper presents the design of translating structured loads/stores in DBT, including target code generation, efficient SIMD register mapping, and optimizations for reducing data permutations. Our proposed register mapping mechanisms and optimizations are not limited to handle structured loads/stores, they can be extended to deal with normal SIMD instructions. This paper evaluates how different factors affect the translation performance and code size. These factors include guest SIMD register length, strides, and use cases for structured loads. On a set of OpenCV benchmarks, our QEMU-based system has achieved a maximum speedup of 5.03x, with an average improvement of 2.87x. On a set of BLAS benchmarks, our system has obtained a maximum speedup of 2.22x and an average improvement of 1.78x.

High-Performance Memory Snapshotting for Real-Time, Consistent, Hypervisor-Based Monitors

This paper presents a concurrent-computing approach—high-performance memory snapshotting—to improving security-introspection of virtual machine guest memory. Efficient introspection improves security monitoring in existing hypervisor systems with real-time, consistent memory introspection capabilities. Efficient introspection has three requirements that each must be met to provide protection against evasive threats: native memory introspection performance, accpetable guest performance, and consistent introspection view of guest memory. Existing introspection systems have provided one or two of these properties but not all three at once. High-performance memory snapshots are evaluated as a solution for meeting all three efficient introspection requirements. In this work we describe how existing system performance can be improved with high-performance snapshotting, present an efficient introspection prototype that has been released as an element of the open-source LibVMI introspection library1, evaluate the efficient introspection prototype on both applications and microbenchmarks, provide demonstrations of introspection application modules enabled by efficient introspection, and provide performance guidance for developing introspection applications utilizing efficient introspection.

Dynamic translation of structured Loads/Stores and register mapping for architectures with SIMD extensions

More and more modern processors have been supporting non-contiguous SIMD data accesses. However, translating such instructions has been overlooked in the Dynamic Binary Translation (DBT) area. For example, in the popular QEMU dynamic binary translator, guest memory instructions with strides are emulated by a sequence of scalar instructions, leaving a significant room for performance improvement when the host machines have SIMD instructions available. Structured loads/stores, such as VLDn/VSTn in ARM NEON, are one type of strided SIMD data access instructions. They are widely used in signal processing, multimedia, mathematical and 2D matrix transposition applications. Efficient translation of such structured loads/stores is a critical issue when migrating ARM executables to other ISAs. However, it is quite challenging since not only the translation of structured loads/stores is not trivial, but also the difference between guest and host register configurations must be taken into consideration. In this work, we present the design and implementation of translating structured loads/stores in DBT, including target code generation as well as efficient SIMD register mapping. Our proposed register mapping mechanisms are not limited to handling structured loads/stores, they can be extended to deal with normal SIMD instructions. On a set of OpenCV benchmarks, our QEMU-based system has achieved a maximum speedup of 5.41x, with an average improvement of 2.93x. On a set of BLAS benchmarks, our system has also obtained a maximum speedup of 2.19x and an average improvement of 1.63x.

Hardware-Accelerated Cross-Architecture Full-System Virtualization

Hardware virtualization solutions provide users with benefits ranging from application isolation through server consolidation to improved disaster recovery and faster server provisioning. While hardware assistance for virtualization is supported by all major processor architectures, including Intel, ARM, PowerPC, and MIPS, these extensions are targeted at virtualization of the same architecture, for example, an x86 guest on an x86 host system. Existing techniques for cross-architecture virtualization, for example, an ARM guest on an x86 host, still incur a substantial overhead for CPU, memory, and I/O virtualization due to the necessity for software emulation of these mismatched system components. In this article, we present a new hardware-accelerated hypervisor called C aptive , employing a range of novel techniques that exploit existing hardware virtualization extensions for improving the performance of full-system cross-platform virtualization. We illustrate how (1) guest memory management unit (MMU) events and operations can be mapped onto host memory virtualization extensions, eliminating the need for costly software MMU emulation, (2) a block-based dynamic binary translation engine inside the virtual machine can improve CPU virtualization performance, (3) memory-mapped guest I/O can be efficiently translated to fast I/O specific calls to emulated devices, and (4) the cost for asynchronous guest interrupts can be reduced. For an ARM-based Linux guest system running on an x86 host with Intel VT support, we demonstrate application performance levels, based on SPEC CPU2006 benchmarks, of up to 5.88× over state-of-the-art Q emu and 2.5× on average, achieving a guest dynamic instruction throughput of up to 1280 MIPS (million instructions per second) and 915.52 MIPS, on average.

Efficient Checkpointing of Live Virtual Machines

The ability to save the state of a running virtual machine (VM) for later restoration is an important tool for home, server, and virtual desktop cloud (VDC) environments in order to achieve optimal and balanced hardware utilization. With guest memory sizes of four to eight gigabytes being the norm the time- and space-overhead of storing VM checkpoints still prevents an effective use of the technique. This work presents a method for fast and space-efficient checkpointing of VMs. Based on the observation that operating systems cache disk blocks in memory, the proposed technique transparently intercepts I/O operations and maintains an up-to-date mapping of memory pages and disk blocks containing identical data. At a checkpoint, those memory pages are excluded from the checkpoint image leading to a significant reduction of both the time and space required to take a checkpoint of a running VM. The broad applicability and good performance of the proposed method is demonstrated by an extensive set of experiments. We have implemented the technique for para-virtualized (PV), PVHVM, and fully-virtualized (HVM) guests in the Xen hypervisor. In comparison with an unmodified Xen hypervisor, experiments with Linux and Windows guests, on average, achieve a 86, 76, 53, and 47 percent reduction in the stored data and a 73, 62, 47, and 38 percent shorter time required to take a checkpoint for PV, PVHVM, HVM Linux, and HVM Windows guests, respectively.

ProbeBuilder: Uncovering Opaque Kernel Data Structures for Automatic Probe Construction

VM-based inspection tools generally implement probes in the hypervisor to monitor events and the state of kernel of the guest system. The most important function of a probe is to carve information of interest out of the memory of the guest when it is triggered. Implementing probes for a closed-source OS demands manually reverse-engineering the undocumented code/data structures in the kernel binary image. Furthermore, the reverse-engineering result is often non-reusable between OS versions or even kernel updates due to the rapid change of these structures. In this paper, we propose ProbeBuilder, a system automating the process to inference kernel data structures. Based on dynamic execution, ProbeBuilder searches for data structures matching the “pointer-offset-pointer” pattern in guest memory. The sequences of these offsets, which are referred to as dereferences , are then verified by ProbeBuilder with instruction evidence that traverse them. The experiment on Windows kernel shows that ProbeBuilder efficiently narrows thousands of choices for kernel-level probes down to dozens. The finding allows analysts to quickly implement probes, facilitating rapid development/update of inspection tools for different OSes. With these features, ProbeBuilder is the first system capable of automatically generating practical probes that extracts information through dereferences to opaque kernel data structures.

Dynamic Memory Balancing for Virtualization

Allocating memory dynamically for virtual machines (VMs) according to their demands provides significant benefits as well as great challenges. Efficient memory resource management requires knowledge of the memory demands of applications or systems at runtime. A widely proposed approach is to construct a miss ratio curve (MRC) for a VM, which not only summarizes the current working set size (WSS) of the VM but also models the relationship between its performance and the target memory allocation size. Unfortunately, the cost of monitoring and maintaining the MRC structures is nontrivial. This article first introduces a low-cost WSS tracking system with effective optimizations on data structures, as well as an efficient mechanism to decrease the frequency of monitoring. We also propose a Memory Balancer (MEB), which dynamically reallocates guest memory based on the predicted WSS. Our experimental results show that our prediction schemes yield a high accuracy of 95.2% and low overhead of 2%. Furthermore, the overall system throughput can be significantly improved with MEB, which brings a speedup up to 7.4 for two to four VMs and 4.54 for an overcommitted system with 16 VMs.

Paravirtual Remote I/O

The traditional "trap and emulate" I/O paravirtualization model conveniently allows for I/O interposition, yet it inherently incurs costly guest-host context switches. The newer "sidecore" model eliminates this overhead by dedicating host (side)cores to poll the relevant guest memory regions and react accordingly without context switching. But the dedication of sidecores on each host might be wasteful when I/O activity is low, or it might not provide enough computational power when I/O activity is high. We propose to alleviate this problem at rack scale by consolidating the dedicated sidecores spread across several hosts onto one server. The hypervisor is then effectively split into two parts: the local hypervisor that hosts the VMs, and the remote hypervisor that processes their paravirtual I/O. We call this model vRIO---paraVirtual Remote I/O. We find that by increasing the latency somewhat, it provides comparable throughput with fewer sidecores and superior throughput with the same number of sidecores as compared to the state of the art. vRIO additionally constitutes a new, cost-effective way to consolidate I/O devices (on the remote hypervisor) while supporting efficient programmable I/O interposition.

Paravirtual Remote I/O

The traditional "trap and emulate" I/O paravirtualization model conveniently allows for I/O interposition, yet it inherently incurs costly guest-host context switches. The newer "sidecore" model eliminates this overhead by dedicating host (side)cores to poll the relevant guest memory regions and react accordingly without context switching. But the dedication of sidecores on each host might be wasteful when I/O activity is low, or it might not provide enough computational power when I/O activity is high. We propose to alleviate this problem at rack scale by consolidating the dedicated sidecores spread across several hosts onto one server. The hypervisor is then effectively split into two parts: the local hypervisor that hosts the VMs, and the remote hypervisor that processes their paravirtual I/O. We call this model vRIO---paraVirtual Remote I/O. We find that by increasing the latency somewhat, it provides comparable throughput with fewer sidecores and superior throughput with the same number of sidecores as compared to the state of the art. vRIO additionally constitutes a new, cost-effective way to consolidate I/O devices (on the remote hypervisor) while supporting efficient programmable I/O interposition.

Leveraging Transactional Execution for Memory Consistency Model Emulation

System emulation is widely used in today’s computer systems. This technology opens new opportunities for resource sharing as well as enhancing system security and reliability. System emulation across different instruction set architectures (ISA) can enable further opportunities. For example, cross-ISA emulation can enable workload consolidation over a wide range of microprocessors and potentially facilitate the seamless deployment of new processor architectures. As multicore and manycore processors become pervasive, it is important to address the challenges toward supporting system emulation on these platforms. A key challenge in cross-ISA emulation on multicore systems is ensuring the correctness of emulation when the guest and the host memory consistency models differ. Many existing cross-ISA system emulators are sequential, thus they are able to avoid this problem at the cost of significant performance degradation. Recently proposed parallel emulators are able to address the performance limitation; however, they provide limited support for memory consistency model emulation. When the host system has a weaker memory consistency model compared to the guest system, the emulator can insert memory fences at appropriate locations in the translated code to enforce the guest memory ordering constraints. These memory fences can significantly degrade the performance of the translated code. Transactional execution support available on certain recent microprocessors provides an alternative approach. Transactional execution of the translated code enforces sequential consistency (SC) at the coarse-grained transaction level, which in turn ensures that all memory accesses made on the host machine conform to SC. Enforcing SC on the host machine guarantees that the emulated execution will be correct for any guest memory model. In this article, we compare and evaluate the overheads associated with using transactions and fences for memory consistency model emulation on the Intel Haswell processor. Our experience of implementing these two approaches on a state-of-the-art parallel emulator, COREMU, demonstrates that memory consistency model emulation using transactions performs better when the transaction sizes are large enough to amortize the transaction overhead and the transaction conflict rate is low, whereas inserting memory fences is better for applications in which the transaction overhead is high. A hybrid implementation that dynamically determines which approach to invoke can outperform both approaches. Our results, based on the SPLASH-2 and the PARSEC benchmark suites, demonstrate that the proposed hybrid approach is able to outperform the fence insertion mechanism by 4.9% and the transactional execution approach by 24.9% for two-thread applications, and outperform them by 4.5% and 44.7%, respectively, for four-threaded execution.

Proactively Breaking Large Pages to Improve Memory Overcommitment Performance in VMware ESXi

VMware ESXi leverages hardware support for MMU virtualization available in modern Intel/AMD CPUs. To optimize address translation performance when running on such CPUs, ESXi preferably uses host large pages (2MB in x86-64 systems) to back VM's guest memory. While using host large pages provides best performance when host has sufficient free memory, it increases host memory pressure and effectively defeats page sharing. Hence, the host is more likely to hit the point where ESXi has to reclaim VM memory through much more expensive techniques such as ballooning or host swapping. As a result, using host large pages may significantly hurt consolidation ratio. To deal with this problem, we propose a new host large page management policy that allows to: a) identify 'cold' large pages and break them even when host has plenty of free memory; b) break all large pages proactively when host free memory becomes scarce, but before the host starts ballooning or swapping; c) reclaim the small pages within the broken large pages through page sharing. With the new policy, the shareable small pages can be shared much earlier and the amount of memory that needs to be ballooned or swapped can be largely reduced when host memory pressure is high. We also propose an algorithm to dynamically adjust the page sharing rate when proactively breaking large pages using a VM large page shareability estimator for higher efficiency. Experimental results show that the proposed large page management policy can improve the performance of various workloads up to 2.1x by significantly reducing the amount of ballooned or swapped memory when host memory pressure is high. Applications still fully benefit from host large pages when memory pressure is low.

Introspection-based memory de-duplication and migration

Memory virtualization abstracts a physical machine's memory resource and presents to the virtual machines running on it a piece of physical memory that could be shared, compressed and moved. To optimize the memory resource utilization by fully leveraging the flexibility afforded by memory virtualization, it is essential that the hypervisor have some sense of how the guest VMs use their allocated physical memory. One way to do this is virtual machine introspection (VMI), which interprets byte values in a guest memory space into semantically meaningful data structures. However, identifying a guest VM's memory usage information such as free memory pool is non-trivial. This paper describes a bootstrapping VM introspection technique that could accurately extract free memory pool information from multiple versions of Windows and Linux without kernel version-specific hard-coding, how to apply this technique to improve the efficiency of memory de-duplication and memory state migration, and the resulting improvement in memory de-duplication speed, gain in additional memory pages de-duplicated, and reduction in traffic loads associated with memory state migration.

Fast and space-efficient virtual machine checkpointing

Checkpointing, i.e., recording the volatile state of a virtual machine (VM) running as a guest in a virtual machine monitor (VMM) for later restoration, includes storing the memory available to the VM. Typically, a full image of the VM's memory along with processor and device states are recorded. With guest memory sizes of up to several gigabytes, the size of the checkpoint images becomes more and more of a concern. In this work we present a technique for fast and space-efficient checkpointing of virtual machines. In contrast to existing methods, our technique eliminates redundant data and stores only a subset of the VM's memory pages. Our technique transparently tracks I/O operations of the guest to external storage and maintains a list of memory pages whose contents are duplicated on non-volatile storage. At a checkpoint, these pages are excluded from the checkpoint image. We have implemented the proposed technique for paravirtualized as well as fully-virtualized guests in the Xen VMM. Our experiments with a paravirtualized guest (Linux) and two fullyvirtualized guests (Linux, Windows) show a significant reduction in the size of the checkpoint image as well as the time required to complete the checkpoint. Compared to the current Xen implementation, we achieve, on average, an 81% reduction in the stored data and a 74% reduction in the time required to take a checkpoint for the paravirtualized Linux guest. In a fully-virtualized environment runningWindows and Linux guests, we achieve a 64% reduction of the image size along with a 62% reduction in checkpointing time.

Glossing gloss and its undertow

Click to increase image sizeClick to decrease image size Notes This article continues the discussion of Guest's poetry in my two previous articles: ‘“Couplings of Such Sonority”: Reading a Poem by Barbara Guest’, Textual Practice 23.3 (2009), pp. 481–502, and ‘Lost and Found in The Türler Losses’, Chicago Review 53.4 & 54.1/2 (2008), pp. 98–106. Thanks are once again due to The National Humanities Center, North Carolina, where I was Carl and Lily Pforzheimer Fellow from 2007 to 2008. Additionally, I wish to thank Nicola Masciandaro and Ryan Dobran for an invitation to the conference Glossing is a Glorious Thing: The Past, Present, and Future of Commentary at The Graduate Center, City University of New York, April 9–10, 2009, where I presented an earlier and abbreviated version of this article. ‘The Barbara Guest Memory Bank’ is at http://www.asu.edu/pipercwcenter/how2journal/bg_memorybank/bg_memory.html ‘Letter from Barbara Guest to Susan Gervitz, excerpt’, HOW(ever), vol. 4, no. 4, April 1988, http://www.asu.edu/pipercwcenter/how2journal/archive/print_archive/0488post.html; Erika Duncan, ‘ENCOUNTERS: Hearing a Poet, but Understanding Little’, New York Times, September 25, 1994. For a discussion, see Susan Hayward, ‘Cohesive Relations and Texture in Bresson's Film L'Argent’, Sub-Stance 51 (1987), pp. 52–68, esp. pp. 60–64. C/f Theodore W. Adorno, ‘Presuppositions’ in Notes to Literature (New York, NY: Columbia University Press, 1992), vol. 2, esp. p. 97 and footnote. Also my essay ‘Following the Poem' in John Wilkinson, The Lyric Touch. Essays on the Poetry of Excess (Cambridge: Salt Publishing, 2007), esp. p. 196. For Marjorie Welish's poems, see, for instance, The Annotated ‘Here' and Selected Poems (Minneapolis, MN: Coffee House Press, 2000). See my account of Guest's poem ‘Saving Tallow’ in ‘Couplings of Such Sonority’. Barbara Guest, Rocks on a Platter (Hanover, NH: Wesleyan University Press, 1999). Reprinted in Barbara Guest, in Hadley Haden Guest (ed.), The Collected Poems of Barbara Guest (Middletown, CT: Wesleyan University Press, 2008), pp. 425–448 [henceforward CP]. References in the text are to the first publication. Rocks on a Platter, p. 40. Rocks on a Platter, p. 35; Theodore W. Adorno, Aesthetic Theory, tr. Robert Hullot-Kentor (Minneapolis, MN: University of Minnesota Press, 1997), p. 6, summarising ‘Hegel's argument against Kant'. Rocks on a Platter, p. 44. Rocks on a Platter, p. 43. Barbara Guest, If So, Tell Me (London: Reality Street Editions, 1999), p. 25. CP p. 380. Robert Kaufman, ‘A Future for Modernism: Barbara Guest's Recent Poetry’, American Poetry Review 29.4 (2000), p. 14. Kaufman also notes the allusion to Adorno in the book's subtitle. Barbara Guest, The Blue Stairs (New York, NY: Corinth Books, 1968), pp. 16–17. The poem is reprinted unaltered in Barbara Guest, Selected Poems (Los Angeles, CA: Sun & Moon Press, 1995), pp. 38–39. CP pp. 71–72. Susan Stewart, Poetry and the Fate of the Senses (Chicago: University of Chicago Press, 2002), p. 111. Rocks on a Platter, p. 30: ‘dissident morning! ‖ with no ulterior purpose'. At ‘The Barbara Guest Memory Bank’, http://www.asu.edu/pipercwcenter/how2journal/bg_memorybank/bg_memory.html Barbara Guest, Biography (Providence, RI: Burning Deck, 1980), np. The line occurs in part 6, CP p. 186. Guest confirms ‘that poem did come out of those trips' in ‘Barbara Guest and Kathleen Fraser, in conversation with Elisabeth Frost and Cynthia Hogue’, Jacket 25, February 2004, http://jacketmagazine.com/25/guest-iv.html Barbara Guest, Herself Defined: The Poet H.D. and Her World (Garden City, NY: Doubleday, 1984). Herself Defined, p. ix–x. Herself Defined, p. 228. Both contributing to the ‘Barbara Guest Feature’ at Jacket 10, January 2000, http://jacketmagazine.com/10/index.shtml Rocks on a Platter, pp. 8, 45. Sara Lundquist, ‘Two voices, one joking’: the Metapoetic Comedy of Barbara Guest's Poetry, Jacket 25, February 2004, http://jacketmagazine.com/25/guest-lund.html Barbara Guest, Forces of Imagination, Writing on Writing (Berkeley, CA: Kelsey St Press, 2003). It is true that Guest's writing on visual art is far more fluent, but it seems reasonable to assume that subediting had a hand. See Barbara Guest, Dürer in the Window, Reflections on Art (New York, NY: Roof Books, 2003). Barbara Guest, Seeking Air (Los Angeles, CA: Sun & Moon Press, 1997), p. 84. Seeking Air, p. 168. If So, Tell Me, p. 27. CP pp. 381–382. Veronica Forrest-Thomson, Poetic Artifice: A Theory of Twentieth-Century Poetry (Manchester: Manchester University Press, 1978), ch. 3, pp. 65–80. Ian Patterson, The Glass Bell (London: Barque, 2009), np. The line is from the first sonnet in the sequence ‘Glossolalia’.