Fine-grained Fault Tolerance Research Articles

Aggressive Fault Tolerance for Memristor Crossbar-Based Neural Network Accelerators by Operational Unit Level Weight Mapping

The memristor crossbar has the characteristic of high parallelism in implementing the matrix vector multiplication which can speed up the computation of the neural network. However, faulty memristors resulted from the hardware defects significantly degrade the classification accuracy of neural networks deployed onto the crossbar. Weight mapping is a type of low cost fault tolerant scheme. Unfortunately, the existing schemes usually conduct fault aware weight mapping in the entire row granularity which constrains the optimization space for fault tolerance. To overcome above mentioned problem, in this paper, we propose the operational unit (OU) level weight mapping which further adjusts mapping of the weight blocks inside each OU after the row granularity weight mapping. Such strategy can implement fine-grained fault tolerance. Moreover, we unify the similar inputs for the different OUs in order to constrain the increase in the number of input vectors resulted from the OU level mapping adjustment. The simulation results demonstrate that with the defect rates of 5%, 10%, 15% and 20%, on average classification accuracies of the networks optimized by the proposed scheme are improved by 1.165%, 4.38%, 10.73% and 12.58% compared to the ones of the row level weight mapping. Moreover, on average numbers of the input vectors are reduced by 61.8%, 73.2%, 75.3% and 64.9%, compared with the OU level weight mapping without considering unification of the similar inputs.

A Fine-Grained Software-Implemented DMA Fault Tolerance for SoC Against Soft Error

In system-on-chips (SoCs), DMA, as a peripheral module, plays an important role in data transmission. However, the structure shrinking of SoC leads to its proneness to radiation-induced soft errors, especially for DMA. This paper presents a fine-grained software-implemented fault tolerance for SoC, named DCRH, to enhance the reliability of DMA against soft errors. DCRH achieves fine-grained selective fault tolerance, protecting DMA without interfering other modules of SoC. Furthermore, it is transparent to the user application because it performs on driver layer. In this paper, we present our fault source analysis for DMA based on Xilinx Zynq-7010 SoC and the detailed design of DCRH. The method is then applied to bare-metal MicroZed so that a DCRH-enhanced DMA driver is developed. Finally, SSIFFI is engaged in the simulated DMA fault injection experiments to validate DCRH. The experimental results prove that DCRH can achieve high fault coverage for DMA, above 97%, with stable performance.

Versionized process based on non-volatile random-access memory for fine-grained fault tolerance

Non-volatile random-access memory (NVRAM) technology is maturing rapidly and its byte-persistence feature allows the design of new and efficient fault tolerance mechanisms. In this paper we propose the versionized process (VerP), a new process model based on NVRAM that is natively non-volatile and fault tolerant. We introduce an intermediate software layer that allows us to run a process directly on NVRAM and to put all the process states into NVRAM, and then propose a mechanism to versionize all the process data. Each piece of the process data is given a special version number, which increases with the modification of that piece of data. The version number can effectively help us trace the modification of any data and recover it to a consistent state after a system crash. Compared with traditional checkpoint methods, our work can achieve fine-grained fault tolerance at very little cost.

JetScope

Interactive, reliable, and rich data analytics at cloud scale is a key capability to support low latency data exploration and experimentation over terabytes of data for a wide range of business scenarios. Besides the challenges in massive scalability and low latency distributed query processing, it is imperative to achieve all these requirements with effective fault tolerance and efficient recovery, as failures and fluctuations are the norm in such a distributed environment. We present a cloud scale interactive query processing system, called JetScope, developed at Microsoft. The system has a SQL-like declarative scripting language and delivers massive scalability and high performance through advanced optimizations. In order to achieve low latency, the system leverages various access methods, optimizes delivering first rows, and maximizes network and scheduling efficiency. The system also provides a fine-grained fault tolerance mechanism which is able to efficiently detect and mitigate failures without significantly impacting the query latency and user experience. JetScope has been deployed to hundreds of servers in production at Microsoft, serving a few million queries every day.

Prospects, challenges and latest developments in designing a scalable big data stream computing system

In the big data era, big data stream computing, as a computing paradigm, is gaining traction for real-time and online data computing applications, and is specially designed to solve the dilemma of real-time data stream computing by processing data online within real-time constraints. It is used to compute large amounts of data in the form of continuous data streams. Each computation is represented by a data stream graph, usually a directed graph. In this paper, the computing paradigm of big data stream computation in data stream graph is given, and some application scenarios and big data stream characteristics are presented. A series of challenges in designing a scalable big data stream computing system in big data environments are summarised by referring to some research results of big data stream computing system, which include stateless system architecture, elastically adaptive scheduling strategy and fine-grained fault tolerance strategy. All these challenges will greatly help us to understand big data steam computing, and provide a high-level guidance in designing an excellent and useful big data stream computing system.

Multi-query optimization in MapReduce framework

MapReduce has recently emerged as a new paradigm for large-scale data analysis due to its high scalability, fine-grained fault tolerance and easy programming model. Since different jobs often share similar work (e.g., several jobs scan the same input file or produce the same map output), there are many opportunities to optimize the performance for a batch of jobs. In this paper, we propose two new techniques for multi-job optimization in the MapReduce framework. The first is a generalized grouping technique (which generalizes the recently proposed MRShare technique) that merges multiple jobs into a single job thereby enabling the merged jobs to share both the scan of the input file as well as the communication of the common map output. The second is a materialization technique that enables multiple jobs to share both the scan of the input file as well as the communication of the common map output via partial materialization of the map output of some jobs (in the map and/or reduce phase). Our second contribution is the proposal of a new optimization algorithm that given an input batch of jobs, produces an optimal plan by a judicious partitioning of the jobs into groups and an optimal assignment of the processing technique to each group. Our experimental results on Hadoop demonstrate that our new approach significantly outperforms the state-of-the-art technique, MRShare, by up to 107%.

Tiled-MapReduce

The prevalence of chip multiprocessors opens opportunities of running data-parallel applications originally in clusters on a single machine with many cores. MapReduce, a simple and elegant programming model to program large-scale clusters, has recently been shown a promising alternative to harness the multicore platform. The differences such as memory hierarchy and communication patterns between clusters and multicore platforms raise new challenges to design and implement an efficient MapReduce system on multicore. This article argues that it is more efficient for MapReduce to iteratively process small chunks of data in turn than processing a large chunk of data at a time on shared memory multicore platforms. Based on the argument, we extend the general MapReduce programming model with a “tiling strategy”, called Tiled - MapReduce (TMR). TMR partitions a large MapReduce job into a number of small subjobs and iteratively processes one subjob at a time with efficient use of resources; TMR finally merges the results of all subjobs for output. Based on Tiled-MapReduce, we design and implement several optimizing techniques targeting multicore, including the reuse of the input buffer among subjobs, a NUCA/NUMA-aware scheduler, and pipelining a subjob’s reduce phase with the successive subjob’s map phase, to optimize the memory, cache, and CPU resources accordingly. Further, we demonstrate that Tiled-MapReduce supports fine-grained fault tolerance and enables several usage scenarios such as online and incremental computing on multicore machines. Performance evaluation with our prototype system called Ostrich on a 48-core machine shows that Ostrich saves up to 87.6% memory, causes less cache misses, and makes more efficient use of CPU cores, resulting in a speedup ranging from 1.86x to 3.07x over Phoenix. Ostrich also efficiently supports fine-grained fault tolerance, online, and incremental computing with small performance penalty.

Fine-grained fault tolerance using device checkpoints

Recovering faults in drivers is difficult compared to other code because their state is spread across both memory and a device. Existing driver fault-tolerance mechanisms either restart the driver and discard its state, which can break applications, or require an extensive logging mechanism to replay requests and recreate driver state. Even logging may be insufficient, though, if the semantics of requests are ambiguous. In addition, these systems either require large subsystems that must be kept up-to-date as the kernel changes, or require substantial rewriting of drivers. We present a new driver fault-tolerance mechanism that provides fine-grained control over the code protected. Fine-Grained Fault Tolerance (FGFT) isolates driver code at the granularity of a single entry point. It executes driver code as a transaction, allowing roll back if the driver fails. We develop a novel checkpointing mechanism to save and restore device state using existing power management code. Unlike past systems, FGFT can be incrementally deployed in a single driver without the need for a large kernel subsystem, but at the cost of small modifications to the driver. In the evaluation, we show that FGFT can have almost zero runtime cost in many cases, and that checkpoint-based recovery can reduce the duration of a failure by 79% compared to restarting the driver. Finally, we show that applying FGFT to a driver requires little effort, and the majority of drivers in common classes already contain the power-management code needed for checkpoint/restore.

Fine-grained fault tolerance using device checkpoints

Recovering faults in drivers is difficult compared to other code because their state is spread across both memory and a device. Existing driver fault-tolerance mechanisms either restart the driver and discard its state, which can break applications, or require an extensive logging mechanism to replay requests and recreate driver state. Even logging may be insufficient, though, if the semantics of requests are ambiguous. In addition, these systems either require large subsystems that must be kept up-to-date as the kernel changes, or require substantial rewriting of drivers. We present a new driver fault-tolerance mechanism that provides fine-grained control over the code protected. Fine-Grained Fault Tolerance (FGFT) isolates driver code at the granularity of a single entry point. It executes driver code as a transaction, allowing roll back if the driver fails. We develop a novel checkpointing mechanism to save and restore device state using existing power management code. Unlike past systems, FGFT can be incrementally deployed in a single driver without the need for a large kernel subsystem, but at the cost of small modifications to the driver. In the evaluation, we show that FGFT can have almost zero runtime cost in many cases, and that checkpoint-based recovery can reduce the duration of a failure by 79% compared to restarting the driver. Finally, we show that applying FGFT to a driver requires little effort, and the majority of drivers in common classes already contain the power-management code needed for checkpoint/restore.

The performance of MapReduce

MapReduce has been widely used for large-scale data analysis in the Cloud. The system is well recognized for its elastic scalability and fine-grained fault tolerance although its performance has been noted to be suboptimal in the database context. According to a recent study [19], Hadoop, an open source implementation of MapReduce, is slower than two state-of-the-art parallel database systems in performing a variety of analytical tasks by a factor of 3.1 to 6.5. MapReduce can achieve better performance with the allocation of more compute nodes from the cloud to speed up computation; however, this approach of "renting more nodes" is not cost effective in a pay-as-you-go environment. Users desire an economical elastically scalable data processing system, and therefore, are interested in whether MapReduce can offer both elastic scalability and efficiency. In this paper, we conduct a performance study of MapReduce (Hadoop) on a 100-node cluster of Amazon EC2 with various levels of parallelism. We identify five design factors that affect the performance of Hadoop, and investigate alternative but known methods for each factor. We show that by carefully tuning these factors, the overall performance of Hadoop can be improved by a factor of 2.5 to 3.5 for the same benchmark used in [19], and is thus more comparable to that of parallel database systems. Our results show that it is therefore possible to build a cloud data processing system that is both elastically scalable and efficient.

Self-Adapting Event Configuration in Ubiquitous Wireless Sensor Networks

Wireless Sensor Networks are the key-enabler for low cost ubiquitous applications in the area of homeland security, health-care, and environmental monitoring. A necessary prerequisite is reliable and efficient event detection in spite of sudden failures and environmental changes. Due to the fact that the sensors need to be low cost, they have only scarce resources leading to a certain level of failures of sensor nodes or sensing devices attached to the nodes. Available fault tolerant solutions are mainly customized approaches that revealed several shortcomings, particularly in adaptability and energy efficiency. The authors present a complete event detection concept including all necessary steps from formal event definition to autonomous device configuration. It features an event definition language that allows defining complex events as well as enhance the reliability by tailor-made voting schemes and application constraints. Based on that, this paper introduces a novel approach for self-adapting on-node and in-network processing, called Event Decision Tree (EDT). EDT autonomously adapts to available resources and environmental conditions, even though it requires to (re-)organize collaboration between neighboring nodes for evaluation. The authors’ approach achieves fine-grained event-related fault tolerance with configurable adaptation rate while enhancing maintainability and energy efficiency.