Isolated Execution Environment Research Articles

HClave: An isolated execution environment design for hypervisor runtime security

Virtualization is the cornerstone of cloud computing, but the hypervisor, the crucial software component that enables virtualization, is known to suffer from various attacks. It is challenging to secure the hypervisor due to at least two reasons. On one hand, commercial hypervisors are usually integrated into a privileged Operating System (OS), which brings in a larger attack surface. On the other hand, multiple Virtual Machines (VM) share a single hypervisor, thus a malicious VM could leverage the hypervisor as a bridge to launch “cross-VM” attacks. In this work, we propose HClave, an isolated execution environment (IEE) design for hypervisor runtime. We decouple the virtualization layer into a tiny trusted computing base (TCB), a large non-secure OS, and multiple HClave IEEs through a bidirectional isolation approach. HClave extends the nested kernel approach to deprive the traditional OS from accessing the tiny TCB’s memory and creates an IEE for hypervisor runtime. We implemented HClave based on KVM and evaluated its effectiveness and efficiency through case studies. Experimental results show that HClave can significantly improve the security of the hypervisor with reasonable runtime overhead.

CacheIEE: Cache-Assisted Isolated Execution Environment on ARM Multi-Core Platforms

ARM TrustZone technology has been widely used to create Trusted Execution Environments (TEEs) for enhancing the security of applications. However, the increasing number of installed security-sensitive applications in the secure world will inevitably enlarge the trusted computing base (TCB) of TEE systems. To minimize the TCB of the secure world and increase application portability, Isolated Execution Environments (IEEs) are proposed to protect applications in enclaves created in the normal world. However, existing IEE systems cannot provide the same level of security as the TEE systems, particularly, on resolving the multi-vector attacks that include both physical memory disclosure attacks and software attacks. In this paper, we develop a new cache-assisted IEE system called CacheIEE that creates enclaves in the L1 data cache of the normal world to protect sensitive data against multi-vector attacks. First, by always storing the sensitive data in the L1 data cache, CacheIEE can effectively prevent physical memory disclosure attacks. Second, we protect the L1 data cache against untrusted rich OS running in other cores. To support more applications, CacheIEE can process large-size sensitive data in the L1 data cache with constrained capacity. We implement a system prototype of CacheIEE and verify its security and practicability.

Supporting AI-powered real-time cyber-physical systems on heterogeneous platforms via hypervisor technology

The heavy use of machine learning algorithms in safety-critical systems poses serious questions related to safety, security, and predictability issues, requiring novel architectural approaches to guarantee such properties. This paper presents an architecture solution that leverages heterogeneous platforms and virtualization technologies to support AI-powered applications consisting of modules with mixed criticalities and safety requirements. The hypervisor exploits the security features of the Xilinx ZCU104 MPSoCs to create two isolated execution environments: a high performance domain running deep learning algorithms under the Linux operating system and a safety-critical domain running control and monitoring functions under the freeRTOS real-time operating system. The proposed approach is validated by a use case consisting of an unmanned aerial vehicle capable of tracking moving targets using a deep neural network accelerated on the FGPA available on the platform.

ASM: An Adaptive Secure Multicore for Co-located Mutually Distrusting Processes

With the ever-increasing virtualization of software and hardware, the privacy of user-sensitive data is a fundamental concern in computation outsourcing. Secure processors enable a trusted execution environment to guarantee security properties based on the principles of isolation, sealing, and integrity. However, the shared hardware resources within the microarchitecture are increasingly being used by co-located adversarial software to create timing-based side-channel attacks. State-of-the-art secure processors implement the strong isolation primitive to enable non-interference for shared hardware, but suffer from frequent state purging and resource utilization overheads, leading to degraded performance. This paper proposes ASM , an adaptive secure multicore architecture that enables a reconfigurable, yet strongly isolated execution environment. For outsourced security-critical processes, the proposed security kernel and hardware extensions allow either a given process to execute using all available cores, or co-execute multiple processes on strongly isolated clusters of cores. This spatio-temporal execution environment is configured based on resource demands of processes, such that the secure processor mitigates state purging overheads and maximizes hardware resource utilization.

MA-TEECM: Mutual Anonymous Authentication-Based Credential Migration Technology for Mobile Trusted Execution Environments

ARM TrustZone is the most widely used mobile trusted execution environment (TEE) technology today. Its hardware-enabled isolated execution environment provides reliable assurance of secure storage of credentials in mobile devices. However, the research on managing credentials stored in the TEE throughout the lifecycle of mobile devices has received little attention in recent years, and the credentials in TEE generally face usability problems caused by the mobile device lifecycle events. Aiming at the risk of information disclosure caused by the third-party service providers in the traditional credential migration scheme, this paper presents a mutual anonymous authentication-based credential migration framework for mobile trusted execution environments. First, we propose a peer-to-peer credential migration model between mobile terminals based on TrustZone and SGX, which solves the single point of failure caused by attacks on trusted third parties that act as credential transfer stations and managers in traditional solutions; Second, we propose an identity authentication protocol between TEEs based on mutual anonymous authentication, and a detailed authentication process is designed based on the universal mobile TEE model; Third, we build a formal verification model using High-Level Protocol Specification Language (HLPSL). Finally, the formal and informal security analysis indicate that the improved scheme meets the expected security requirements and is secure against several known attacks.

ISotEE: A Hypervisor Middleware for IoT-Enabled Resource-Constrained Reliable Systems

Embedded systems for critical applications are often based on resource-constrained devices to meet the requirements like performance predictability and energy consumption. To deal with the increased software complexity, many of these systems have adopted reliable RTOSes (Real-Time Operating Systems) with advanced protection functionalities. Meanwhile, the concept of IoT (Internet of Things) is gaining momentum. Many IoT OSes, specialized to provide the large software stack required by IoT applications, have been released. Nevertheless, neither reliable RTOS nor IoT OS can satisfy all the requirements of IoT-enabled reliable systems. Dual-OS configuration (i.e. the coexistence of reliable RTOS and IoT OS) is a promising approach to achieve high reliability and productivity simultaneously. Existing dual-OS solutions, however, depend on additional hardware features (e.g. virtualization extensions, ARM TrustZone), which are unavailable in most resource-constrained devices. This paper presents iSotEE (iSolated Execution Environment), a middleware allowing IoT OS to run inside an isolated environment on top of a reliable RTOS without special hardware. Open-source implementations of iSotEE for Renesas RX (with TOPPERS/HRP3 as reliable RTOS, Amazon FreeRTOS as IoT OS) and ARMv7-M (with two configurations of Zephyr as reliable RTOS and IoT OS) architectures are provided and evaluated. The results show that iSotEE can create reliable systems with a small footprint for resource-constrained devices, high real-time performance for critical applications, and high productivity and throughput for IoT applications.

SOTPM: Software One-Time Programmable Memory to Protect Shared Memory on ARM Trustzone

In ARM TrustZone-based architecture, shared memory is one of the most useful schemes to enable isolated execution environments supported by TrustZone to communicate between environments. However, it is already known that shared memory is vulnerable to man-in-the-middle attacks since mechanisms to check integrity or authenticate callers for the shared memory payload are not supported in TrustZone. While an encryption-based method that resolves this limitation does exist, there are some architectural limitations. Indeed, even with key protection countermeasures applied, there is a risk that encryption keys may be leaked, as they are placed in insecure user memory during communication. Moreover, countermeasures for key leakage cause system performance overhead. In this paper, we propose a lightweight and secure scheme for shared memory, called Software One-Time Programmable Memory (SOTPM). SOTPM is a software-implemented, one-time programmable shared memory. It is based on the idea that payload encryption in the shared memory layer is unnecessary because sensitive data is already encrypted in the application layer before being written to the shared memory. SOTPM is set to read-only after data is written into SOTPM due to the one-time programmable characteristic. Therefore, attackers are unable to manipulate content in SOTPM during communication. Since it is not necessary for SOTPM to encrypt the payload in order to prevent malicious payload manipulation, it is possible to remove the risk of key leakage posed in previous studies. Additionally, in contrast with the existing method, our method can dramatically reduce system performance overhead. We implemented our prototype on an open-source hardware board with an Armv8-A processor and performed a security analysis and performance evaluation. The results show that SOTPM provides a sufficient level of security and less than 1% performance overhead, implying that SOTPM is a reasonable solution for current commercial products.

Boolean Searchable Symmetric Encryption with Filters on Trusted Hardware

The prevalence and availability of cloud infrastructures has made them the de facto solution for storing and archiving data, both for organizations and individual users. Nonetheless, the cloud's wide spread adoption is still hindered by dependability and security concerns, particularly in applications with large data collections where efficient search and retrieval services are also major requirements. This leads to an increased tension between security, efficiency, and search expressiveness. In this paper we tackle this tension by proposing BISEN, a new provably-secure boolean searchable symmetric encryption scheme that improves these three complementary dimensions by exploring the design space of isolation guarantees offered by novel commodity hardware such as Intel SGX, abstracted as Isolated Execution Environments (IEEs). BISEN is the first scheme to support multiple users and enable highly expressive and arbitrarily complex boolean queries, with minimal information leakage regarding performed queries and accessed data, and verifiability regarding fully malicious adversaries. Furthermore, BISEN extends the traditional SSE model to support filter functions on search results based on generic metadata created by the users. Experimental validation and comparison with the state of art shows that BISEN provides better performance with enriched search semantics and security properties.

Hardware-Assisted Transparent Tracing and Debugging on ARM

The existing malware analysis platforms leave detectable fingerprints such as uncommon string properties in QEMU, signatures in Android Java virtual machine, and artifacts in Linux kernel profiles. Since these fingerprints provide the malware a chance to split its behavior depending on whether the analysis system is present or not, the existing analysis systems are not sufficient to analyze the sophisticated malware. In this paper, we propose NINJA, a transparent malware analysis framework on the ARM platform with low artifacts. NINJA leverages a hardware-assisted isolated execution environment TrustZone to transparently trace and debug a target application with the help of performance monitor unit and embedded trace macrocell. These hardware features help NINJA to achieve transparency while avoiding heavy performance overhead. NINJA does not modify system software and is OS-agnostic on the ARM platform. We implement a prototype of NINJA (i.e., tracing and debugging subsystems), and the experimental results show that NINJA is efficient and transparent for malware analysis. An improved fast system restoration mechanism is also designed to facilitate the continuous malware analysis.

Safe and Efficient Implementation of a Security System on ARM using Intra-level Privilege Separation

Security monitoring has long been considered as a fundamental mechanism to mitigate the damage of a security attack. Recently, intra-level security systems have been proposed that can efficiently and securely monitor system software without any involvement of more privileged entity. Unfortunately, there exists no full intra-level security system that can universally operate at any privilege level on ARM. However, as malware and attacks increase against virtually every level of privileged software including an OS, a hypervisor, and even the highest privileged software armored by TrustZone, we have been motivated to develop an intra-level security system, named Hilps . Hilps realizes true intra-level scheme in all these levels of privileged software on ARM by elaborately exploiting a new hardware feature of ARM’s latest 64-bit architecture, called TxSZ, that enables elastic adjustment of the accessible virtual address range. Furthermore, Hilps newly supports the sandbox mechanism that provides security tools with individually isolated execution environments, thereby minimizing security threats from untrusted security tools. We have implemented a prototype of Hilps on a real machine. The experimental results demonstrate that Hilps is quite promising for practical use in real deployments.

Research on ARM TrustZone

ARM TrustZone [1] is a hardware-based security feature that can provide software with a high-privilege and isolated execution environment. Such isolation is ensured by hardware, which is usually considered as more trustworthy than software. Thus the execution environment is also known as trusted execution environment (TEE). TrustZone technology was proposed in 2002, but did not get widely used until 2009, when Apple released iPhone 5s. In iPhone 5s, Apple leveraged TrustZone to protect its Touch ID, which ensures that even if the iOS is fully compromised, the user's fingerprint data can still be safe. In 2017, Google made TEE a mandatory requirement on any Android devices with a fingerprint scanner. Nowadays, almost all mobile phones and tablets have TEE deployed. Meanwhile, ARM integrates TrustZone in ARM64 and ARMv8-M to support a broader range of platforms including servers and IoT devices.

SPACE: Semi-Partitioned CachE for Energy Efficient, Hard Real-Time Systems

Multi-core processors are increasingly popular because they yield higher performance, but they also present new challenges for hard real-time systems in that they make it much more difficult to estimate a task's worst-case execution time (WCET). Partitioned cache architecture is being used to ease the problem by providing an isolated execution environment for each thread. Although simple to implement and use, this method may be sub-optimal with respect to both energy consumption and performance since it prevents taking advantage of information shared across threads for both instructions and data. This work presents a new cache architecture termed SPACE (Semi-Partitioned CachE) that makes it possible to leverage information sharing, yielding in turn a tighter WCET. The SPACE architecture together with our new WCET algorithm can be used to maintain the predictability of the execution time of the parallel threads while reducing the overall energy consumption of the system. The new proposed cache architecture was implemented using Verilog and deployed on a Xilinx MicroBlaze multi-core design for testing, validation and measurements. The application level experiments were conducted using the Chronos tool for estimation and the Wattch/SimpleScalar simulator for execution. Using three real-time programs–a radar tracker, a DES encryption algorithm, and an FM radio–we showed that SPACE together with the enhanced WCET algorithm reduce the average system WCET of these applications by 31 percent and reduce the actual energy consumption by 18 percent in comparison with other cache architectures.

MIPE: a practical memory integrity protection method in a trusted execution environment

With the rapid development of Internet of Things technology and the promotion of embedded devices’ computation performance, smart devices are probably open to security threats and attacks while connecting with rich and novel Internet. Attracting lots of attention in embedded system security community recently, Trusted Execution Environment (TEE), allows for the execution of arbitrary code within environments completely isolated from the rest of a system. However, existing memory protection methods in a TEE are inadequate. In general, the software-based formal methods are not practical and the hardware-based implementation approaches lack of theoretical proof. To address the memory isolation and protection problems in TEE, in this paper, we propose a practical memory integrity protection method on an ARM-based platform, called MIPE, to defend against security threats including kernel data attacks and direct memory access attacks. MIPE utilizes TrustZone technique to create a isolated execution environment, which can protect the sensitive code and data against attacks. To present the integrity protection strategies, we provide the design of MIPE using B method, which is a practical formal method. We also implement MIPE on the Xilinx Zynq ZC702 evaluation board. The evaluation results show that the automatic proof rate of machines using B method is about 78.32%, and the proposed method is effective and feasible in terms of both load time and overhead.

A Trusted IaaS Environment with Hardware Security Module

With the proliferation of cloud computing, security concerns about confidentiality violations of user data by the privileged domain and system administrators have been growing. This paper proposes secure cloud architecture with a hardware security module, which isolates cloud user data from potentially malicious privileged domains or cloud administrators. Within a securely isolated execution environment, the hardware security module provides essential security functionality with only restricted interfaces exposed to vulnerable management systems or cloud administrators. Such restriction prevents cloud administrators from affecting the security of guest VMs. The proposed architecture not only defends against wide attack vectors but also achieves a small TCB. This paper discusses our hardware and software implementation of the proposed cloud architecture, analyzes its security, and presents its performance results.

The Study and Evaluation of ARM-Based Mobile Virtualization

In common sense, virtualization technology is adopted to offer several isolated execution environments and make better use of computational resources in server environment. However, in embedded systems, the significance of virtualization does not come into the picture. The extensive utilization of mobile smart devices has led to a series of issues such as security, wasting of resources, and power consumption. In this paper, we discuss how mobile virtualization addresses these challenges and then present a detail analysis of four mainstream mobile virtualization solutions: containers, paravirtualization, hardware-assisted full virtualization, and microkernel. At last, we carry out a series of performance comparisons among these solutions and make some suggestions for further research.

Accuracy, Security, and Architecture Impacts and Challenges of Mobile and Web Technologies: Geolocation Field Data Collection in Washington State Water Resources

The purpose of this paper is to examine accuracy, security, and architecture impacts and challenges of mobile and web technologies through the case study of collecting geolocation field data in Washington (WA) State water resources. Effective management of water as a public resource relies on the capture, storage, and retrieval of accurate geographic position data. This is also true of a broad range of business domains beyond water resources, such as earth sciences, city planning, and navigation to name a few. Application developers must choose how to capture this information (with enough accuracy to be useful for its intended application) and get that data to a place where it can be processed and used. A traditional monotonic application using a Global Positioning System (GPS) and a mobile app using a smartphone are available today. The advent of HTML 5 now allows the development of a mobile web app, which is not dependent on any particular smartphone platform. These multiple technical options lead to two research questions: How does an HTML5 mobile web app solution work in terms of accuracy, security, and architecture, compared with a GPS-based solution and a mobile native app solution for collecting geolocation field data? And second, as HTML5 mobile web apps are a relatively new technology, what best practices can we uncover to assist in the process of choosing between an HTML5 mobile web app and a mobile native app, and also what are the best practices for building a mobile web app that can operate offline? To answer these questions, we build an HTML5 mobile web app called „LocationSharpener‟ for collecting GPS locations leveraging three of the new HTML5 features: IndexedDB, AppCache, and GeoLocation. We use the mobile web app to collect locations of water resources facilities like wells and diversions and analyze how accurately the app collects the geolocation data. We apply threat risk modeling to the mobile web app to analyze its security and privacy compared to that of the native mobile app approach. In addition, by documenting the architecture of the mobile web app with multiple views, we analyze how HTML5 affects the architecture and present best practices for building a mobile web app that can operate online and offline. The analysis of location accuracy shows the HTML5 mobile web app approach provides acceptable location accuracy even when offline. Threat modeling demonstrates that, in contrast to native mobile apps, mobile web apps offer a clear advantage to users and developers: an isolated execution environment makes it more difficult for a malicious mobile web app to read private data; As an outcome of the architecture documentation we also propose new best practices of developing a mobile web app using HTML5: the developer must consider two subsystems for online and offline use cases and a sequence of connected, disconnected, and connected deployment modes. Also, standards-based web apps are portable across virtually any operating system. This also eases the burden on developers, since they can write mobile web app once and deploy anywhere.

Modeling and Reasoning about States in Late Launch Based on Horn Clauses

Late Launch, which is a kind of dynamic measurement technology proposed by both Intel and AMD, offers isolated execution environment for codes needed to be protected. However, since the specifications and documents of Late Launch have hundreds of pages, they are too long and complicated to be fully covered and analyzed. A model based on Horn clauses is presented to solve the problem that there is a lack of realistic models and of automated tools for the verification of security protocols based on Late Launch. A running example is taken to show the execution details of Late Launch. Based on the example, secrecy properties of Late Launch are verified. Whats more, the automatic theorem proving tool ProVerif is used to make the verification more fast and accurate.

Batch scheduling of consolidated virtual machines based on their workload interference model

The use of virtualization technology (VT) has become widespread in modern datacenters and Clouds in recent years. In spite of their many advantages, such as provisioning of isolated execution environments and migration, current implementations of VT do not provide effective performance isolation between virtual machines (VMs) running on a physical machine (PM) due to workload interference of VMs. Generally, this interference is due to contention on physical resources that impacts performance in different workload configurations. To investigate the impacts of this interference, we formalize the concept of interference for a consolidated multi-tenant virtual environment. This formulation, represented as a mathematical model, can be used by schedulers to estimate the interference of a consolidated virtual environment in terms of the processing and networking workloads of running VMs, and the number of consolidated VMs. Based on the proposed model, we present a novel batch scheduler that reduces the interference of running tenant VMs by pausing VMs that have a higher impact on proliferation of the interference. The scheduler achieves this by selecting a set of VMs that produce the least interference using a 0–1 knapsack problem solver. The selected VMs are allowed to run and other VMs are paused. Users are not troubled by the pausing and resumption of VMs for a short time because the scheduler has been designed for the execution of batch type applications such as scientific applications. Evaluation results on the makespan of VMs executed under the control of our scheduler have shown nearly 33% improvement in the best case and 7% improvement in the worst case compared to the case in which all VMs are running concurrently. In addition, the results show that our scheduling algorithm outperforms serial and random scheduling of VMs as well.

Security-Preserving Live Migration of Virtual Machines in the Cloud

Hypervisor-based process protection is a novel approach that provides isolated execution environments for applications running on untrusted commodity operating systems. It is based on off-the-shelf hardware and trusted hypervisors while it meets the requirement of security and trust for many cloud computing models, especially third-party data centers and a multi-tenant public cloud, in which sensitive data are out of the control of the users. However, as the hypervisor extends semantic protection to the process granularity, such a mechanism also breaks the platform independency of virtual machines and thus prohibits live migration of virtual machines, which is another highly desirable feature in the cloud. In this paper, we extend hypervisor-based process protection systems with live migration capabilities by migrating the protection-related metadata maintained in the hypervisor together with virtual machines and protecting sensitive user contents using encryption and hashing. We also propose a security-preserving live migration protocol that addresses several security threats during live migration procedures including timing-related attacks, replay attacks and resumption order attacks. We implement a prototype system base on Xen and Linux. Evaluation results show that performance degradation in terms of both total migration time and downtime are reasonably low compared to the unmodified Xen live migration system.

A Trusted Virtual Machine in an Untrusted Management Environment

Virtualization is a rapidly evolving technology that can be used to provide a range of benefits to computing systems, including improved resource utilization, software portability, and reliability. Virtualization also has the potential to enhance security by providing isolated execution environments for different applications that require different levels of security. For security-critical applications, it is highly desirable to have a small trusted computing base (TCB), since it minimizes the surface of attacks that could jeopardize the security of the entire system. In traditional virtualization architectures, the TCB for an application includes not only the hardware and the virtual machine monitor (VMM), but also the whole management operating system (OS) that contains the device drivers and virtual machine (VM) management functionality. For many applications, it is not acceptable to trust this management OS, due to its large code base and abundance of vulnerabilities. For example, consider the computing-as-a-service” scenario where remote users execute a guest OS and applications inside a VM on a remote computing platform. It would be preferable for many users to utilize such a computing service without being forced to trust the management OS on the remote platform. In this paper, we address the problem of providing a secure execution environment on a virtualized computing platform under the assumption of an untrusted management OS. We propose a secure virtualization architecture that provides a secure runtime environment, network interface, and secondary storage for a guest VM. The proposed architecture significantly reduces the TCB of security-critical guest VMs, leading to improved security in an untrusted management environment. We have implemented a prototype of the proposed approach using the Xen virtualization system, and demonstrated how it can be used to facilitate secure remote computing services. We evaluate the performance penalties incurred by the proposed architecture, and demonstrate that the penalties are minimal.