Virtual Machine Images Research Articles

Secure the Cloud

In response to the revival of virtualized technology by Rosenblum and Garfinkel [2005], NIST defined cloud computing, a new paradigm in service computing infrastructures. In cloud environments, the basic security mechanism is ingrained in virtualization—that is, the execution of instructions at different privilege levels. Despite its obvious benefits, the caveat is that a crashed virtual machine (VM) is much harder to recover than a crashed workstation. When crashed, a VM is nothing but a giant corrupt binary file and quite unrecoverable by standard disk-based forensics. Therefore, VM crashes should be avoided at all costs. Security is one of the major contributors to such VM crashes. This includes compromising the hypervisor, cloud storage, images of VMs used infrequently, and remote cloud client used by the customer as well as threat from malicious insiders. Although using secure infrastructures such as private clouds alleviate several of these security problems, most cloud users end up using cheaper options such as third-party infrastructures (i.e., private clouds), thus a thorough discussion of all known security issues is pertinent. Hence, in this article, we discuss ongoing research in cloud security in order of the attack scenarios exploited most often in the cloud environment. We explore attack scenarios that call for securing the hypervisor, exploiting co-residency of VMs, VM image management, mitigating insider threats, securing storage in clouds, abusing lightweight software-as-a-service clients, and protecting data propagation in clouds. Wearing a practitioner's glasses, we explore the relevance of each attack scenario to a service company like Infosys. At the same time, we draw parallels between cloud security research and implementation of security solutions in the form of enterprise security suites for the cloud. We discuss the state of practice in the form of enterprise security suites that include cryptographic solutions, access control policies in the cloud, new techniques for attack detection, and security quality assurance in clouds.

I-sieve: An inline high performance deduplication system used in cloud storage

Data deduplication is an emerging and widely employed method for current storage systems. As this technology is gradually applied in inline scenarios such as with virtual machines and cloud storage systems, this study proposes a novel deduplication architecture called I-sieve. The goal of I-sieve is to realize a high performance data sieve system based on iSCSI in the cloud storage system. We also design the corresponding index and mapping tables and present a multi-level cache using a solid state drive to reduce RAM consumption and to optimize lookup performance. A prototype of I-sieve is implemented based on the open source iSCSI target, and many experiments have been conducted driven by virtual machine images and testing tools. The evaluation results show excellent deduplication and foreground performance. More importantly, I-sieve can co-exist with the existing deduplication systems as long as they support the iSCSI protocol.

계산과학 시뮬레이션을 위한 실시간 가상 클러스터 생성 및 I/O 성능 향상 기법

Computational science simulations have been used to enable discovery in a broad spectrum of application areas, these simulations show irregular demanding characteristics of computing resources from time to time. The adoption of virtualized high performance cloud, rather than CPU-centric computing platform (such as supercomputers), is gaining interest of interests mainly due to its ease-of-use, multi-tenancy and flexibility. Basically, provisioning a virtual cluster, which consists of a lot of virtual machines, in a real-time has a critical impact on the successful deployment of the virtualized HPC clouds for computational science simulations. However, the cost of concurrently creating many virtual machines in constructing a virtual cluster can be as much as two orders of magnitude worse than expected. One of the main factors in this bottleneck is the time spent to create the virtual images for the virtual machines. In this paper, we propose a novel technique to minimize the creation time of virtual machine images and improve I/O performance of the provisioned virtual clusters. We also confirm that our proposed technique outperforms the conventional ones using various sets of experiments.

BioImg.org: A Catalog of Virtual Machine Images for the Life Sciences.

Virtualization is becoming increasingly important in bioscience, enabling assembly and provisioning of complete computer setups, including operating system, data, software, and services packaged as virtual machine images (VMIs). We present an open catalog of VMIs for the life sciences, where scientists can share information about images and optionally upload them to a server equipped with a large file system and fast Internet connection. Other scientists can then search for and download images that can be run on the local computer or in a cloud computing environment, providing easy access to bioinformatics environments. We also describe applications where VMIs aid life science research, including distributing tools and data, supporting reproducible analysis, and facilitating education. BioImg.org is freely available at: https://bioimg.org.

Management of virtual machine images in heterogeneous clouds

This paper presents the image server of the VISOR cloud agnostic virtual machine images management service. An evaluation approach is also described and the results are discussed. VISOR is not intended to fit in a specific cloud framework but rather to overreach sharing and interoperability limitations among different frameworks. Such feature is achieved by relying on a flexible image metadata schema and in a storage abstraction layer, where the latter seamlessly abstracts the heterogeneity of distinct cloud storage systems into which images can be saved. VISOR is an open source highly distributed system composed of several independent web services. In its development we have focused in modularity, extensibility and performance. Conducted evaluation tests have shown encouraging results regarding performance and scalability.

Tunable Version Control System for Virtual Machines in an Open-Source Cloud

Open-source cloud platforms provide a feasible alternative of deploying cloud computing in low-cost commodity hardware and operating systems. To enhance the reliability of an open-source cloud, we design and implement CloudVS, a practical add-on system that enables version control for virtual machines (VMs). CloudVS targets a commodity cloud platform that has limited available resources. It exploits content similarities across different VM versions using redundancy elimination (RE), such that only non-redundant data chunks of a VM version are transmitted over the network and kept in persistent storage. Using RE as a building block, we propose a suite of performance adaptation mechanisms that make CloudVS amenable to different commodity settings. Specifically, we propose a tunable mechanism to balance the storage and disk seek overheads, as well as various I/O optimization techniques to minimize the interferences to other co-resident processes. We further exploit a higher degree of content similarity by applying RE to multiple VM images simultaneously, and support the copy-on-write image format. Using real-world VM snapshots, we experiment CloudVS in an open-source cloud testbed built on Eucalyptus. We demonstrate how CloudVS can be parameterized to balance the performance trade-offs between version control and normal VM operations.

Allele Workbench: transcriptome pipeline and interactive graphics for allele-specific expression.

Sequencing the transcriptome can answer various questions such as determining the transcripts expressed in a given species for a specific tissue or condition, evaluating differential expression, discovering variants, and evaluating allele-specific expression. Differential expression evaluates the expression differences between different strains, tissues, and conditions. Allele-specific expression evaluates expression differences between parental alleles. Both differential expression and allele-specific expression have been studied for heterosis (hybrid vigor), where the hybrid has improved performance over the parents for one or more traits. The Allele Workbench software was developed for a heterosis study that evaluated allele-specific expression for a mouse F1 hybrid using libraries from multiple tissues with biological replicates. This software has been made into a distributable package, which includes a pipeline, a Java interface to build the database, and a Java interface for query and display of the results. The required input is a reference genome, annotation file, and one or more RNA-Seq libraries with optional replicates. It evaluates allelic imbalance at the SNP and transcript level and flags transcripts with significant opposite directional allele-specific expression. The Java interface allows the user to view data from libraries, replicates, genes, transcripts, exons, and variants, including queries on allele imbalance for selected libraries. To determine the impact of allele-specific SNPs on protein folding, variants are annotated with their effect (e.g., missense), and the parental protein sequences may be exported for protein folding analysis. The Allele Workbench processing results in transcript files and read counts that can be used as input to the previously published Transcriptome Computational Workbench, which has a new algorithm for determining a trimmed set of gene ontology terms. The software with demo files is available from https://code.google.com/p/allele-workbench. Additionally, all software is ready for immediate use from an Atmosphere Virtual Machine Image available from the iPlant Collaborative (www.iplantcollaborative.org).

VMThunder: Fast Provisioning of Large-Scale Virtual Machine Clusters

Infrastructure as a service (IaaS) allows users to rent resources from the Cloud to meet their various computing requirements. The pay-as-you-use model, however, poses a nontrivial technical challenge to the IaaS cloud service providers: how to fast provision a large number of virtual machines (VMs) to meet users' dynamic computing requests? We address this challenge with VMThunder, a new VM provisioning tool, which downloads data blockson demandduring the VM booting process and speeds up VM image streaming by strategically integrating peer-to-peer (P2P) streaming techniques with enhanced optimization schemes such as transfer on demand, cache on read, snapshot on local, and relay on cache. In particular, VMThunder stores the original images in a share storage and in the meantime it adopts a tree-based P2P streaming scheme so that common image blocks are cached and reused across the nodes in the cluster. We implement VMThunder in CentOS Linux and thoroughly test its performance. Comprehensive experimental results show that VMThunder outperforms the state-of-the-art VM provisioning methods, with respect to scalability, latency, and VM runtime I/O performance.

Rethink the storage of virtual machine images in clouds

As one of the most prevalent cloud services, Infrastructure-as-a-Service provides virtual machines (VMs) with high flexibility. How to effectively manage a huge amount of VM images becomes a big challenge. On one hand, images affect VMs’ disk IO performance significantly, which is essential to the quality of services, especially for those having intensive disk IO workloads. On the other hand, they consume many storage resources and cause much management cost, which is cared by cloud managers. Current ways to optimize images usually focus on either improving performance or decreasing image size, which unfortunately cannot satisfy the requirements of high IO performance, low storage consumption, and low management cost simultaneously. Typically, high IO performance requires images storing close to VMs, but this increases redundant data and consumes extra storage at the same time. Besides, a closer image means more data stored in local disks rather than a normal shared storage, which increases management cost as well. In this paper, we analyze these requirements and potential tradeoffs among them, and propose Zone-based model to well balance the requirements. We partition computing nodes into many zones, and construct a shared storage in each zone to cache hot data for high IO performance and low storage consumption. In addition, we improve the normal Copy-on-Write and cache mechanisms, providing new image types and cache functions to enhance the eventual effectiveness. The evaluations show that, our solution improves IO performance by more than 100% in general and even 10 times while adopting a friendly VM placement strategy, and gets close or less storage consumption and management cost than the other models at the same time.

Private Cloud using OpenStack for Knowledge Sharing System

is an open source cloud computing platform which is a joint venture by Rackspace and NASA. OpenStack consists of many projects such as Nova (Cpute), Swift (Storage), Neutron (Network), Glance (VM Images) etc. Swift is used as Object storage. Amazon Web Services provide way of managing cloud instances; the similar purpose can be achieved by OpenStack at virtually no cost. System aims at developing a private cloud using OpenStack and building a website for the purpose of knowledge sharing. This system consists of firstly, building a private cloud using OpenStack for object based storage as well as compute node and secondly a website to implement knowledge sharing through the medium of web. Motivation of this system is to fill the gap between the growing technology and its use in academia. Key features for this system include abstraction of complexity, simplicity and fault tolerance of object storage and managing of cloud instances. KeywordsInstances, OpenStack, REST, Swift

Using Puppet to contextualize computing resources for ATLAS analysis on Google Compute Engine

With the advent of commercial as well as institutional and national clouds, new opportunities for on-demand computing resources for the HEP community become available. The new cloud technologies also come with new challenges, and one such is the contextualization of computing resources with regard to requirements of the user and his experiment. In particular on Google's new cloud platform Google Compute Engine (GCE) upload of user's virtual machine images is not possible. This precludes application of ready to use technologies like CernVM and forces users to build and contextualize their own VM images from scratch. We investigate the use of Puppet to facilitate contextualization of cloud resources on GCE, with particular regard to ease of configuration and dynamic resource scaling.

Implementation of Grid Tier 2 and Tier 3 facilities on a Distributed OpenStack Cloud

The Australian Government is making a $AUD 100 million investment in Compute and Storage for the academic community. The Compute facilities are provided in the form of 30,000 CPU cores located at 8 nodes around Australia in a distributed virtualized Infrastructure as a Service facility based on OpenStack. The storage will eventually consist of over 100 petabytes located at 6 nodes. All will be linked via a 100 Gb/s network. This proceeding describes the development of a fully connected WLCG Tier-2 grid site as well as a general purpose Tier-3 computing cluster based on this architecture. The facility employs an extension to Torque to enable dynamic allocations of virtual machine instances. A base Scientific Linux virtual machine (VM) image is deployed in the OpenStack cloud and automatically configured as required using Puppet. Custom scripts are used to launch multiple VMs, integrate them into the dynamic Torque cluster and to mount remote file systems. We report on our experience in developing this nation-wide ATLAS and Belle II Tier 2 and Tier 3 computing infrastructure using the national Research Cloud and storage facilities.

A Validation Framework for the Long Term Preservation of High Energy Physics Data

The study group on data preservation in high energy physics, DPHEP, is moving to a new collaboration structure, which will focus on the implementation of preservation projects, such as those described in the group's large scale report published in 2012. One such project is the development of a validation framework, which checks the compatibility of evolving computing environments and technologies with the experiments software for as long as possible, with the aim of substantially extending the lifetime of the analysis software, and hence of the usability of the data. The framework is designed to automatically test and validate the software and data of an experiment against changes and upgrades to the computing environment, as well as changes to the experiment software itself. Technically, this is realised using a framework capable of hosting a number of virtual machine images, built with different configurations of operating systems and the relevant software, including any necessary external dependencies.

Micro-CernVM: slashing the cost of building and deploying virtual machines

The traditional virtual machine (VM) building and and deployment process is centered around the virtual machine hard disk image. The packages comprising the VM operating system are carefully selected, hard disk images are built for a variety of different hypervisors, and images have to be distributed and decompressed in order to instantiate a virtual machine. Within the HEP community, the CernVM File System (CernVM-FS) has been established in order to decouple the distribution from the experiment software from the building and distribution of the VM hard disk images. We show how to get rid of such pre-built hard disk images altogether. Due to the high requirements on POSIX compliance imposed by HEP application software, CernVM-FS can also be used to host and boot a Linux operating system. This allows the use of a tiny bootable CD image that comprises only a Linux kernel while the rest of the operating system is provided on demand by CernVM-FS. This approach speeds up the initial instantiation time and reduces virtual machine image sizes by an order of magnitude. Furthermore, security updates can be distributed instantaneously through CernVM-FS. By leveraging the fact that CernVM-FS is a versioning file system, a historic analysis environment can be easily re-spawned by selecting the corresponding CernVM-FS file system snapshot.

Liquid: A Scalable Deduplication File System for Virtual Machine Images

A virtual machine (VM) has been serving as a crucial component in cloud computing with its rich set of convenient features. The high overhead of a VM has been well addressed by hardware support such as Intel virtualization technology (VT), and by improvement in recent hypervisor implementation such as Xen, KVM, etc. However, the high demand on VM image storage remains a challenging problem. Existing systems have made efforts to reduce VM image storage consumption by means of deduplication within a storage area network (SAN) cluster. Nevertheless, an SAN cannot satisfy the increasing demand of large-scale VM hosting for cloud computing because of its cost limitation. In this paper, we propose Liquid, a scalable deduplication file system that has been particularly designed for large-scale VM deployment. Its design provides fast VM deployment with peer-to-peer (P2P) data transfer and low storage consumption by means of deduplication on VM images. It also provides a comprehensive set of storage features including instant cloning for VM images, on-demand fetching through a network, and caching with local disks by copy-on-read techniques. Experiments show that Liquid's features perform well and introduce minor performance overhead.

Dynamic Management of Virtual Infrastructures

Cloud infrastructures are becoming an appropriate solution to address the computational needs of scientific applications. However, the use of public or on-premises Infrastructure as a Service (IaaS) clouds requires users to have non-trivial system administration skills. Resource provisioning systems provide facilities to choose the most suitable Virtual Machine Images (VMI) and basic configuration of multiple instances and subnetworks. Other tasks such as the configuration of cluster services, computational frameworks or specific applications are not trivial on the cloud, and normally users have to manually select the VMI that best fits, including undesired additional services and software packages. This paper presents a set of components that ease the access and the usability of IaaS clouds by automating the VMI selection, deployment, configuration, software installation, monitoring and update of Virtual Appliances. It supports APIs from a large number of virtual platforms, making user applications cloud-agnostic. In addition it integrates a contextualization system to enable the installation and configuration of all the user required applications providing the user with a fully functional infrastructure. Therefore, golden VMIs and configuration recipes can be easily reused across different deployments. Moreover, the contextualization agent included in the framework supports horizontal (increase/decrease the number of resources) and vertical (increase/decrease resources within a running Virtual Machine) by properly reconfiguring the software installed, considering the configuration of the multiple resources running. This paves the way for automatic virtual infrastructure deployment, customization and elastic modification at runtime for IaaS clouds.

VIRTUAL APPLIANCE CREATION AND OPTIMIZATION IN CLOUD

The large scale IaaS systems could store virtual appliances in several repositories. The deployment time could heavily vary depending on the connection properties of the repository storing the appliance. A virtual appliance is a virtual machine image designed to run on a virtualization platform i.e. Virtual Box, Xen, VMware Workstation. Virtual appliance delivery requires the modification of the underlying IaaS systems. IaaS is the virtual delivery of computing resources in the form of hardware, networking, and storage services This concept will reduce the variance in deployment time by introducing the concept of online active repositories and appliance optimization. To provide efficient delivery time in IaaS and to increase the efficiency of IaaS (Infra structure as a service) system To calculate the delivery time when deployed it in the virtualized platform Combining the both online and manual repositories for calculating the delivery time To construct the appliance in online using various online repositories The constructed appliance is deployed in the virtualized platform (using virtual box). And the appliance is optimized for increasing the efficiency and decreasing the delivery time. The delivery time of the online appliance is compared with the appliance which has been created manually.

CodeCloud: A platform to enable execution of programming models on the Clouds

This paper presents a platform that supports the execution of scientific applications covering different programming models (such as Master/Slave, Parallel/MPI, MapReduce and Workflows) on Cloud infrastructures. The platform includes (i) a high-level declarative language to express the requirements of the applications featuring software customization at runtime, (ii) an approach based on virtual containers to encapsulate the logic of the different programming models, (iii) an infrastructure manager to interact with different IaaS backends, (iv) a configuration software to dynamically configure the provisioned resources and (v) a catalog and repository of virtual machine images. By using this platform, an application developer can adapt, deploy and execute parallel applications agnostic to the Cloud backend.

A Platform for Interactive Software Experiments in Massive Open Online Courses

Massive open online courses enjoy a surge of popularity: Numerous platforms such as Coursera, Udacity, edX and many more offer a variety of high-quality courses to learners worldwide. Hasso Plattner Institute is operating its own platform openHPI where selected courses are available. These platforms are well-suited for the presentation of video and reading material and offer opportunities for interaction in web forums. Assignments, however, are usually limited to simple quizzes or basic question & answer tasks, although, a very important component of teaching are practical assignments that allow students to gather hands-on experiences. With InstantLab, we have created a self-service web platform for interactive software experiments that is used in our curriculum: Experiments are provided as VM images which are executed on our in-house private cloud and accessed over a remote-desktop connection embedded in the web browser – requiring no additional software on the users computers. In this paper, we discuss how interactive software experiments can be added to state-of-the-art MOOCs using the example of openHPI and InstantLab. We propose XCloud – a hybrid cloud architecture that takes into consideration the unique challenges of MOOC scenarios of scaling issues and resource management for anonymous users. It provides the following benefits to MOOC infrastructures: hosting in a private cloud with support for dynamic scale out to public cloud resources in peak load situations; support for live hands-on experiments with single machines and networks of machines; support to access a limited set of machines with special hardware resources (e.g. GPUs, hardware accelerators); and a trust-based access control scheme that can be used to achieve self-managing resource assignment. We report on our practical experiences with InstantLab and outline its transition to the XCloud architecture. This effort bridges the two fields of computer science and education research: Collected data and practical experience from the from our platform can be used to improve the quality of distance learning and its didactic methods.

Simulation as a service (SMaaS): a cloud-based framework to support the educational use of scientific software

Large manufacturers increasingly leverage modelling and simulation to improve quality and reduce cost. Small manufacturers have not adopted these techniques due to sizable upfront costs for expertise, software and hardware. The software as a service (SaaS) model provides access to applications hosted in a cloud environment, allowing users to try services at low cost and scale as needed. We have extended SaaS to include high-performance computing-hosted applications, thus creating simulation as a service (SMaaS). Polymer portal is a first-generation SMaaS platform designed to integrate access to multiple modelling, simulation and training services. Polymer portal provides a number of features including an e-commerce front end, common AAA service, and support for both cloud-hosted virtual machine (VM) images and high-performance computing (HPC) jobs. It has been deployed for six months and has been used successfully for a number of training and simulation activities. This paper describes the requirements, challenges, design and implementation of the polymer portal.