Error Recovery Mechanisms Research Articles

FPGA-Based Reliable Fault Secure Design for Protection against Single and Multiple Soft Errors

Field programmable gate arrays (FPGAs) are increasingly used in industry (e.g., biomedical, space, and automotive industries). FPGAs are subjected to single, as well as multiple event upsets (SEUs and MEUs), due to the continuous shrinking of transistor dimensions. These upsets inevitably decrease system lifetime. Fault-tolerant techniques are often used to mitigate these problems. In this research, penta and hexa modular redundancy, as well as dynamic partial reconfiguration (DPR), are used to increase system reliability. We show, depending on the relative rates of the SEUs and MEUs, that penta modular redundancy has a higher reliability than hexa modular redundancy, which is a counter-intuitive result in some cases since increasing redundancy is expected to increase reliability. Focusing on penta modular redundancy, an error detection and recovery mechanism (voter) is designed. This mechanism uses the internal configuration access port (ICAP) and its associated controller, as well as DPR to mitigate SEUs and MEUs. Then, it is implemented on Xilinx Vivado tools targeting the Kintex7 7k410tfbg676 device. Finally, we show how to render this design fault secure in the event that SEUs or MEUs affect the voter itself. This fault secure voter either produces the correct output or gives an indication that the output is incorrect.

FT-PBLAS: PBLAS-Based Fault-Tolerant Linear Algebra Computation on High-performance Computing Systems

As high-performance computing (HPC) systems have scaled up, resilience has become a great challenge. To guarantee resilience, various kinds of hardware and software techniques have been proposed. However, among popular software fault-tolerant techniques, both the checkpoint-restart approach and the replication technique face challenges of scalability in the era of peta- and exa-scale systems due to their numerous processes. In this situation, algorithm-based approaches, or algorithm-based fault tolerance (ABFT) mechanisms, have become attractive because they are efficient and lightweight. Although the ABFT technique is algorithm-dependent, it is possible to implement it at a low level (e.g., in libraries for basic numerical algorithms) and make it application-independent. However, previous ABFT approaches have mainly aimed at achieving fault tolerance in integrated circuits (ICs) or at the architecture level and are therefore not suitable for HPC systems; e.g., they use checksums of rows and columns of matrices rather than checksums of blocks to detect errors. Furthermore, they cannot deal with errors caused by node failure, which are common in current HPC systems. To solve these problems, this paper proposes FT-PBLAS, a PBLAS-based library for fault-tolerant parallel linear algebra computations that can be regarded as a fault-tolerant version of the parallel basic linear algebra subprograms (PBLAS), because it provides a series of fault-tolerant versions of interfaces in PBLAS. To support the underlying error detection and recovery mechanisms in the library, we propose a block-checksum approach for non-fatal errors and a scheme for addressing node failure, respectively. We evaluate two fault-tolerant mechanisms and FT-PBLAS on HPC systems, and the experimental results demonstrate the performance of our library.

Automatic syntax error reporting and recovery in parsing expression grammars

Error recovery is an essential feature for a parser that should be plugged in Integrated Development Environments (IDEs), which must build Abstract Syntax Trees (ASTs) even for syntactically invalid programs in order to offer features such as automated refactoring and code completion. Parsing Expressions Grammars (PEGs) are a formalism that naturally describes recursive top-down parsers using a restricted form of backtracking. Labeled failures are a conservative extension of PEGs that adds an error reporting mechanism for PEG parsers, and these labels can also be associated with recovery expressions to provide an error recovery mechanism. These expressions can use the full expressivity of PEGs to recover from syntactic errors. Manually annotating a large grammar with labels and recovery expressions can be difficult. In this work, we present two approaches, Standard and Unique , to automatically annotate a PEG with labels, and to build their corresponding recovery expressions. The Standard approach annotates a grammar in a way similar to manual annotation, but it may insert labels incorrectly, while the Unique approach is more conservative to annotate a grammar and does not insert labels incorrectly. We evaluate both approaches by using them to generate error recovering parsers for four programming languages: Titan, C, Pascal and Java. In our evaluation, the parsers produced using the Standard approach, after a manual intervention to remove the labels incorrectly added, gave an acceptable recovery for at least 70% of the files in each language. By it turn, the acceptable recovery rate of the parsers produced via the Unique approach, without the need of manual intervention, ranged from 41% to 76%. • We discuss two approaches, Standard and Unique, to build PEG-based error recovering parsers in a more automatic way. • We build error recovering parsers for Titan, C, Pascal and Java. • Algorithm Standard, with the help of manual intervention, gives an acceptable recovery for at least 70% of the syntactically invalid files of each language. • Algorithm Unique, without manual intervention, gives an acceptable recovery rate that ranges from 41% to 76%.

Towards comprehensive dependability-driven resource use and message log-analysis for HPC systems diagnosis

Failure analysis plays an important role in the reliability of data centers and high-performance computing (HPC) systems. Recent work have shown that both resource use data and failure logs can, separately and together, be used to detect system failure-inducing errors and diagnose system failures; the result of error propagation and (unsuccessful) execution of error recovery mechanisms. For more accurate and detailed failure diagnosis, knowledge of error propagation patterns and unsuccessful error recovery is important. To improve system reliability, knowledge of recovery protocols deployment is important. This paper describes and demonstrates application of a new diagnostics framework (CORRMEXT). CORRMEXT analyzes and reports error propagation patterns and degrees of success and failure of error recovery protocols. The steps in the framework are correlations of resource use metrics and error messages, and identification of the earliest times of change of system behaviour. The framework is illustrated with analyses of resource use data and message logs for three HPC systems operated by the Texas Advanced Computing Center (TACC). The illustrations are focused on groups of resource use counters and groups of errors; they reveal many interesting insights into patterns of: (i) network data and software errors, (ii) Lustre file-system and Linux operating system process errors, and (iii) memory and storage errors. We also confirm that: (i) correlations of resource use and errors can only be identified by applying different correlation algorithms, and (ii) the earliest times of change in system behaviour can only be identified by analyzing both the correlated resource use counters and correlated errors. We believe CORRMEXT is the first tool that have diagnosed error propagation paths and error recovery attempts on three different HPC systems. CORRMEXT will be put on the public domain to support systems administrators in diagnosing HPC system failures, on August 2018.

Block-Based Carry Speculative Approximate Adder for Energy-Efficient Applications

In this brief, a low energy consumption block-based carry speculative approximate adder is proposed. Its structure is based on partitioning the adder into some non-overlapped summation blocks whose structures may be selected from both the carry propagate and parallel-prefix adders. Here, the carry output of each block is speculated based on the input operands of the block itself and those of the next block. In this adder, the length of the carry chain is reduced to two blocks (worst case), where in most cases only one block is employed to calculate the carry output leading to a lower average delay. In addition, to increase the accuracy and reduce the output error rate, an error detection and recovery mechanism is proposed. The effectiveness of the proposed approximate adder is compared with state-of-the-art approximate adders using a cost function based on the energy, delay, area, and output quality. The results indicate an average of 50% reduction in terms of the cost function compared to other approximate adders.

An effective system for video transmission and error recovery mechanisms in multimedia networks

An effective system for video transmission and error recovery mechanisms in multimedia networks

HTD: A Light-Weight Holosymmetrical Transition Detector for Wide-Voltage-Range Variation Resilient ICs

To mitigate the excess timing margin in conventional digital integrated circuit design, resilient circuits were proposed. The current techniques however either have a large error-detecting register or have difficulty in wide-voltage-range (down to near-threshold) operation. In this paper, we propose a light-weight holosymmetrical transition detector (HTD) that can work reliably at near-threshold to normal VDD with fast response time. Combined with a conventional latch, the HTD-latch is used to replace the end point flip-flop for timing error prediction with an area overhead of only 9.5% over a conventional flip-flop. Besides, we optimize the speculation window and design an adaptive duty-cycle control mechanism to reduce the short-path padding overhead in a resilient circuit. In addition, with voltage scaling, we keep the system working at the point before the first failure by using local detection and global clock gating, which needs neither architectural circuit change nor complicated error recovery mechanism. We apply the above-mentioned three techniques on an eighth-order filter test chip in a 40-nm CMOS process. The measurement results demonstrate that the proposed HTD can work robustly at 0.474–1.1 V. With 4.37% area overhead in total, the proposed resilient technique improves performance by 179.31%/28.2% and energy efficiency by 45.6%/28.1% in a near- $V_{\mathrm {TH}}$ (0.474 V)/super- $V_{\mathrm {TH}}$ (1.1 V) operation, as compared with the baseline design. Therefore, our proposed HTD-based resilient technique can improve the energy efficiency of circuits by almost eliminating all the timing margins.

Error recovery in parsing expression grammars through labeled failures and its implementation based on a parsing machine

Parsing Expression Grammars (PEGs) are a formalism used to describe top-down parsers with backtracking. As PEGs do not provide a good error recovery mechanism, PEG-based parsers usually do not recover from syntax errors in the input, or recover from syntax errors using ad-hoc, implementation-specific features. The lack of proper error recovery makes PEG parsers unsuitable for use with Integrated Development Environments (IDEs), which need to build syntactic trees even for incomplete, syntactically invalid programs.We discuss a conservative extension, based on PEGs with labeled failures, that adds a syntax error recovery mechanism for PEGs. This extension associates recovery expressionsto labels, where a label now not only reports a syntax error but also uses this recovery expression to reach a synchronization point in the input and resume parsing. We give an operational semantics of PEGs with this recovery mechanism, as well as an operational semantics for a parsing machinethat we can translate labeled PEGs with error recovery to, and prove the correctness of this translation. We use an implementation of labeled PEGs with error recovery via a parsing machine to build robust parsers, which use different recovery strategies, for the Lua language. We evaluate the effectiveness of these parsers, alone and in comparison with a Lua parser with automatic error recovery generated by ANTLR, a popular parser generator .

Fault tolerance on-chip: a reliable computing paradigm using self-test, self-diagnosis, and self-repair (3S) approach

If your computer crashes, you can revive it by a reboot, an empirical solution that usually turns out to be effective. The rationale behind this solution is that transient faults, either in hardware or software, can be fixed by refreshing the machine state. Such a “silver bullet", however, could be futile in the future because the faults, especially those existing in the hardware such as Integrated Circuit (IC) chips, cannot be eliminated by refreshing. What we need is a more sophisticated mechanism to steer the system back to the right track. The “magic cure" is the Fault Tolerance On-Chip (FTOC) mechanism, which relies on a suite of built-in design-for-reliability logic, including fault detection, fault diagnosis, and error recovery, working in a self-supportive manner. We have exploited the FTOC to build a holistic solution ranging from on-chip fault detection to error recovery mechanisms to address faults caused by chips progressively aging. Besides fault detection, the FTOC paradigm provides attractive benefits, such as facilitating graceful performance degradation, mitigating the impact of verification blind spots, and improving the chip yield.

Bandwidth Estimation Algorithm of WestwoodNR for Wireless Network

Two widely known parameters of Transmission Control Protocol (TCP) used to control the flow of packets are Congestion Window (cwnd) & Slow Start Threshold (ssthresh). After congestion, slow start phase or fast-retransmit phase come in action wherein TCP has an important role in the reduction of these parameters. This is in response to packet loss identified by TCP. This in turn will cause unnecessary reduction of data flow & degradation of TCP throughput. Researchers have developed some algorithms to come out of this problem, WestwoodNR is one of them. WestwoodNR is using Bandwidth Estimation algorithm to estimate available bandwidth, to make effective use of available network capacity even after the congestion episode. It allows higher values of ssthresh & cwnd when it enters the fast-retransmit phase and slow start phase. In turn this algorithm claims better performance in terms of bandwidth utilization. The focus of this paper is on error recovery mechanisms suitable for WestwoodNR operating over the wireless sub path. These mechanisms have to address the increased bit error probability and temporary disruptions of wireless links. The efficiency of WestwoodNR within wireless scenarios is investigated and possible modifications that lead to higher performance are pointed out.

Modelling reversible execution of robotic assembly

SUMMARYProgramming robotic assembly for industrial small-batch production is challenging; hence, it is vital to increase robustness and reduce development effort in order to achieve flexible robotic automation. A human who has made an assembly error will often simply undo the process until the error is undone and then restart the assembly. Conceptually, robots could do the same. This paper introduces a programming model that enables robot assembly programs to be executed in reverse. We investigate the challenges in running robot programs backwards and present a classification of reversibility characteristics. We demonstrate how temporarily switching the direction of program execution can be an efficient error recovery mechanism. Moreover, we demonstrate additional benefits arising from supporting reversibility in an assembly language, such as increased code reuse and automatically derived disassembly sequences. As a default approach to reversibility, we use program inversion and statement-level inversion of commands, but with a novel override option providing alternative sequences for asymmetric reverse actions. To efficiently program for this model, this paper introduces a new domain-specific language, SCP-RASQ (Simple C++ Reversible Assembly SeQuences). In initial experiments, where 200 consecutive assemblies of two industrial cases were performed, 18 of 22 errors were corrected automatically using only the trial-and-error capabilities that come from reverse execution.

Error Recovery in the Time-Triggered Paradigm with FTT-CAN.

Data networks are naturally prone to interferences that can corrupt messages, leading to performance degradation or even to critical failure of the corresponding distributed system. To improve resilience of critical systems, time-triggered networks are frequently used, based on communication schedules defined at design-time. These networks offer prompt error detection, but slow error recovery that can only be compensated with bandwidth overprovisioning. On the contrary, the Flexible Time-Triggered (FTT) paradigm uses online traffic scheduling, which enables a compromise between error detection and recovery that can achieve timely recovery with a fraction of the needed bandwidth. This article presents a new method to recover transmission errors in a time-triggered Controller Area Network (CAN) network, based on the Flexible Time-Triggered paradigm, namely FTT-CAN. The method is based on using a server (traffic shaper) to regulate the retransmission of corrupted or omitted messages. We show how to design the server to simultaneously: (1) meet a predefined reliability goal, when considering worst case error recovery scenarios bounded probabilistically by a Poisson process that models the fault arrival rate; and, (2) limit the direct and indirect interference in the message set, preserving overall system schedulability. Extensive simulations with multiple scenarios, based on practical and randomly generated systems, show a reduction of two orders of magnitude in the average bandwidth taken by the proposed error recovery mechanism, when compared with traditional approaches available in the literature based on adding extra pre-defined transmission slots.

An Effective System for Video Transmission and Error Recovery Mechanisms in Multimedia Networks

In this paper, an effective system has been proposed for video transmission. This system solves the problems arised due to occurrence of errors in transmitted video and delivers video with required quality of services. The reconstructed video maintains the quality of services at the decoder side. Video dynamics-based error concealment algorithm is applied for recovering errors occurred during transmission. The performance of the proposed system is measured by means of simulations using JM reference software.

Edge-TM

Scaling of semiconductor devices has enabled higher levels of integration and performance improvements at the price of making devices more susceptible to the effects of static and dynamic variability. Adding safety margins (guardbands) on the operating frequency or supply voltage prevents timing errors, but has a negative impact on performance and energy consumption. We propose Edge-TM , an adaptive hardware/software error management policy that ( i ) optimistically scales the voltage beyond the edge of safe operation for better energy savings and ( ii ) works in combination with a Hardware Transactional Memory (HTM)-based error recovery mechanism. The policy applies dynamic voltage scaling (DVS) (while keeping frequency fixed) based on the feedback provided by HTM, which makes it simple and generally applicable. Experiments on an embedded platform show our technique capable of 57% energy improvement compared to using voltage guardbands and an extra 21-24% improvement over existing state-of-the-art error tolerance solutions, at a nominal area and time overhead.

A performance study for the multicast collision prevention mechanism for IEEE 802.11

In IEEE 802.11 networks a data packet is delivered simultaneously to multiple receivers through the multicast paradigm. The standard defines a simple mechanism that does not implement any error-recovery mechanism, thus, the reliability of the service provided to the multicast users is penalized. This issue is more important as the number of collisions increases due to a large number of active stations and/or a high load network. In this paper we carry out a detailed optimization study of the multicast collision prevention (MCP) mechanism, a highly-efficient multicast collision avoidance mechanism for IEEE 802.11 previously introduced by the authors. Besides a more in deep explanation of MCP, this study includes a comparative performance evaluation of the optimized MCP with the IEEE 802.11 standard. Results shown that, through this optimization, the number of collisions in MCP can be made negligible for any network load.

Relay Assisted Effective Loss Recovery Technique for Wireless Sensor Network

In wireless sensor network (WSN), the existing error recovery mechanisms do not concentrate on physical aspects such as error correction and coding. Also a complete balance needs to be maintained among the performance and energy tradeoff in order to prevent communication overhead. In this paper, we propose a relay assisted effective loss recovery technique for WSN. In this technique, the relay nodes among the source and destination node is selected based on the parameters such as channel state information and combined score using a decentralized partially observable Markov decision process. The combined score for each sensor is defined involving the parameters such as queue length, link bandwidth, MAC contention and residual energy. Then the low density parity check (LDPC) codes are used to encode and decode for performing error recovery. By simulation results, we show that the proposed technique reduces the number of errors and also reduces the computational complexity while selecting the node for retransmission of packets at the destination without fail. Also the decision making can be done locally with low overhead.

Modeling of Compensation in Long-Running Transactions

nowadays, the most controversial issue is transaction in database systems or web services. Specifically, in the area of service-oriented computing, where business transactions always need long periods of time to finish. In the case of a failure rollback, which is the traditional method, it will not be enough and not suitable for handling errors during long running transactions. As a substitute, the most appropriate approach is compensation which is used as an error recovery mechanism. Therefore, transactions that need a long time to complete are programmed as a composition of a set of compensable transactions. This study attempts to design several compensation policies in the long running web transaction especially when the transaction has parallel threads. Meabwhile, one thread in sequence steps of the transaction may fail. This paper also describes and models many different ways to compensate to the thread. Moreover, this study proposes a system to implement creating long running transactions as well as simulating failures by using compensation policies.

A task scheduling strategy based on weighted round robin for distributed crawler

SummaryWith the rapid development of the network, stand‐alone crawlers are finding hard to find and gather information. Distributed crawlers are gradually accepted to solve this problem. This paper proposes a task scheduling strategy based on weighted round robin for small‐scale distributed crawler with formula weights for the current node based on crawling efficiency, implements a distributed crawler system with multithreading support and deduplication which takes the algorithm as core, and discusses some possible extensions and details. The design of the error recovery mechanism and the node table allows crawling nodes have flexible scalability and fault tolerance. Finally, we conducted some experiments to prove the good load balancing performance of the system. Concurrency and Computation: Practice and Experience, 2015.© 2015 Wiley Periodicals, Inc. Copyright © 2015 John Wiley & Sons, Ltd.

Timely Error Detection for Effective Recovery in Light-Lockstep Automotive Systems

Safety-relevant systems in the automotive domain often implement features such as lockstep execution for error detection, and reset and re-execution for error correction. Light-lockstep has already been adopted in some such systems due to its relatively low-implementation cost given that it does not require deep changes into nonlockstep hardware. Instead, as only off-core activities (i.e., data/addresses sent) need to be compared across different cores, light-lockstep designs are lowly intrusive. This approach has been proven sufficient to guarantee functional correctness of the system in the presence of errors in the cores, in particular in relation with certification against safety standards such as ISO26262 in the automotive domain. However, error detection in light-lockstep systems may occur long after the error actually occurs, thus jeopardizing timing guarantees, which are as critical as functional ones in hard real-time systems. In this paper, we analyze the timing behavior of errors due to transient and permanent faults in light-lockstep systems. Our results show that the time elapsed until an error is detected can be inordinately large, especially for permanent faults. Based on this observation and building upon the specific characteristics of light-lockstep systems, we propose lightly verbose (LiVe), a new mechanism to enforce the early detection of errors, due to both transient and permanent faults, thus enabling the computation of tight error detection timing bounds. We also analyze how existing mechanisms for error recovery in multicore systems increase their effectiveness when light-lockstep operates in LiVe mode in the context of mixed-criticality workloads.

The Error Reporting in the ATLAS TDAQ System

The ATLAS Error Reporting provides a service that allows experts and shift crew to track and address errors relating to the data taking components and applications. This service, called the Error Reporting Service (ERS), gives to software applications the opportunity to collect and send comprehensive data about run-time errors, to a place where it can be intercepted in real-time by any other system component. Other ATLAS online control and monitoring tools use the ERS as one of their main inputs to address system problems in a timely manner and to improve the quality of acquired data.The actual destination of the error messages depends solely on the run-time environment, in which the online applications are operating. When an application sends information to ERS, depending on the configuration, it may end up in a local file, a database, distributed middleware which can transport it to an expert system or display it to users. Thanks to the open framework design of ERS, new information destinations can be added at any moment without touching the reporting and receiving applications.The ERS Application Program Interface (API) is provided in three programming languages used in the ATLAS online environment: C++, Java and Python. All APIs use exceptions for error reporting but each of them exploits advanced features of a given language to simplify the end-user program writing. For example, as C++ lacks language support for exceptions, a number of macros have been designed to generate hierarchies of C++ exception classes at compile time. Using this approach a software developer can write a single line of code to generate a boilerplate code for a fully qualified C++ exception class declaration with arbitrary number of parameters and multiple constructors, which encapsulates all relevant static information about the given type of issues. When a corresponding error occurs at run time, the program just need to create an instance of that class passing relevant values to one of the available class constructors and send this instance to ERS.This paper presents the original design solutions exploited for the ERS implementation and describes how it was used during the first ATLAS run period. The cross-system error reporting standardization introduced by ERS was one of the key points for the successful implementation of automated mechanisms for online error recovery.