Post-silicon Validation Research Articles

An Agile Post-Silicon Validation Methodology for the Address Translation Mechanisms of Modern Microprocessors

Detection of bugs in the complex address translation mechanisms (ATMs) of modern microprocessors is much harder than in other microprocessor structures because their output (the physical address) is not directly observable to program or architecture visible locations during post-silicon validation. Errata reports, by major microprocessor vendors about ATM-related bugs found after chips massive production, plea for more efficient validation solutions. In this paper, we formulate the problem of post-silicon validation of microprocessors’ address translation hardware, by defining a comprehensive set of bug models. We then propose an ISA-independent post-silicon validation methodology for the ATMs of high-end microprocessors. The method detects ATM bugs right after their manifestation, reveals mismatches due to them and facilitates diagnosis and debug. Our experiment evaluation on an enhanced version of the Gem5 simulator (including MMU caches) demonstrates the effectiveness of the method in detecting known ATM bugs, five orders of magnitude faster than traditional approaches.

Postsilicon Trace Signal Selection Using Machine Learning Techniques

A key problem in postsilicon validation is to identify a small set of traceable signals that are effective for debug during silicon execution. Structural analysis used by traditional signal selection techniques leads to a poor restoration quality. In contrast, simulation-based selection techniques provide superior restorability but incur significant computation overhead. In this paper, we propose an efficient signal selection technique using machine learning to take advantage of simulation-based signal selection while significantly reducing the simulation overhead. The basic idea is to train a machine learning framework with a few simulation runs and utilize its effective prediction capability (instead of expensive simulation) to identify beneficial trace signals. Specifically, our approach uses: 1) bounded mock simulations to generate training vectors for the machine learning technique and 2) a compound search-space exploration approach to identify the most profitable signals. Experimental results indicate that our approach can improve restorability by up to 143.1% (29.2% on average) while maintaining or improving runtime compared with the state-of-the-art signal selection techniques.

Feature-based Signal Selection for Post-silicon Debug using Machine Learning

A key challenge of post-silicon validation methodology is to select a limited number of trace signals that are effective during post-silicon debug. Structural analysis used by traditional signal selection techniques are fast but lead to poor restoration quality. In contrast, simulation-based selection techniques provide superior restorability but incur significant computation overhead. While early work on machine learning based signal selection is promising [1] , it is still not applicable on large industrial designs since it needs thousands of simulations of large and complex designs. In this paper, we propose a signal selection technique that addresses the scalability issue of simulation-based techniques while maintaining a high restoration performance. The basic idea is to train a machine learning framework using a small set of circuits, and apply the trained model to the bigger circuit under test, without any need for simulating the large industry-scale designs. This paper makes two fundamental contributions: i) this is the first attempt to show that learning from small related circuits can be useful for signal selection, and ii) this is the first automated signal selection approach that is applicable on industrial designs without sacrificing restoration quality. Experimental results indicate that our approach can improve restorability by up to 135.4 percent (8.8 percent on average) while significantly reduce (up to 37X, 16.6X on average) the runtime compared to existing signal selection approaches.

Symbolic Quick Error Detection for Pre-Silicon and Post-Silicon Validation: Frequently Asked Questions

Reducing the error detection latency is critical for improving the design visibility while searching for design errors. This article uses a FAQ format to discuss the key points of the symbolic QED method that can be applied during both pre-silicon and post-silicon validation.

PHAETON: A SAT-Based Framework for Timing-Aware Path Sensitization

Knowledge about sensitizable paths through combinational logic is essential for numerous design tasks. We present the framework PHAETON which identifies sensitizable paths and generates test pairs to exercise these paths using Boolean satisfiability (SAT). PHAETON supports a large number of models and sensitization conditions and provides a generic interface that can be used by applications. It incorporates a novel application-specific unary representation of integer numbers to integrate timing information with logical conditions within the same monolithic SAT formula. Due to a number of further elaborate speed-up techniques, PHAETON scales to industrial circuits. Experimental results show the performance of PHAETON in classical K longest path generation tasks and in new post-silicon validation and characterization scenarios.

On-Chip Cube-Based Constrained-Random Stimuli Generation for Post-Silicon Validation

Post-silicon validation is critical for exposing subtle design errors that have escaped to the silicon prototypes. Its effectiveness is conditioned by in-system application of a large volume of functionally-compliant stimuli. In this paper, we present a methodology to design constrained-random stimuli generators, which are placed on-chip and are configurable at design-time to generate in-system functionally-compliant stimuli subject to user-programmable constraints provided at validation time. Central to our method is a cube-based representation of constraints. These cubes are used as masks that force pseudo-random sequences to map onto functionally-compliant stimuli. To reduce the on-chip storage requirements, masks are compressed at design-time and expanded on-the-fly at validation time using decompression circuitry. Experimental results evaluate the impact of our method on the requirements for on-chip logic and memory resources.

Area-Aware Cache Update Trackers for Postsilicon Validation

The internal state of the complex modern processors often needs to be dumped out frequently during postsilicon validation. Since the caches hold most of the state, the volume of data dumped and the transfer time are dominated by the large caches present in the architecture. The limited bandwidth to transfer data present in these large caches off-chip results in stalling the processor for long durations when dumping the cache contents off-chip. To alleviate this, we propose to transfer only those cache lines that were updated since the previous dump. Since maintaining a bit-vector with a separate bit to track the status of individual cache lines is expensive, we propose two methods: 1) where a bit tracks multiple cache lines and 2) an Interval Table which stores only the starting and ending addresses of continuous runs of updated cache lines. Both methods require significantly lesser space compared with a bit-vector, and allow the designer to choose the amount of space to allocate for this design-for-debug feature. The impact of reducing storage space is that some nonupdated cache lines are dumped too. We attempt to minimize such overheads. We propose a scheme to share such cache update tracking hardware (or Update Trackers) across multiple caches in case of physically distributed caches so that they are replicated fewer times, thereby limiting the area overhead. We show that the proposed Update Trackers occupy less than 1% of cache area for both the shared and distributed caches.

Efficient Selection of Trace and Scan Signals for Post-Silicon Debug

Post-silicon validation is a critical part of integrated circuit design methodology. The primary objective is to detect and eliminate the bugs that have escaped pre-silicon validation phase. One of the key challenges in post-silicon validation is the limited observability of internal signals in manufactured chips. A promising direction to improve observability is to combine trace and scan signals—a small set of trace signals are stored in every cycle, whereas a large set of scan signals are dumped across multiple cycles. Existing techniques are not very effective, since they explore a coarse-grained combination of trace and scan signals. In this paper, we propose a fine-grained architecture that addresses this issue using various scan chains with different dumping periods. We also propose efficient algorithms to select beneficial signals based on this architecture. Our experimental results demonstrate that our approach can improve restoration ratio up to 127% (36% on average) compared with existing trace-only techniques. Our approach also shows up to 125% improvement (61.7% on average) compared with techniques that allow a combination of trace and scan signals with minor (<1%) area and power overhead.

Automated Selection of Assertions for Bit-Flip Detection During Post-Silicon Validation

Post-silicon validation deals with detection and diagnosis of errors that, due to existing limitations in pre-silicon verification, escape to the silicon prototypes and need to be fixed before committing to high-volume manufacturing. Electrical errors, such as those caused by cross-talk or power droops, are particularly difficult to catch during the pre-silicon phase because of the insufficient accuracy of device models, which is often traded-off against simulation time. This challenge is further aggravated by the rising number of voltage domains, especially if subtle errors are excited in unique electrical states. In fact these electrically-induced subtle errors most commonly manifest in the logic domain as bit-flips and, to the best of our knowledge, there are no systematic methods for designing embedded hardware monitors for generic logic blocks that can detect bit-flips with low detection latency. Moreover, unlike pre-silicon verification and manufacturing test that benefit from well-defined and universally accepted coverage metrics, there is no generic metric from which confidence can be implied at the end of post-silicon validation. Toward these goals, we present a method that relies on design invariants (assertions) that are ranked based on their potential to detect bit-flips. We also introduce two metrics bit-flip coverage estimate and flip-flop coverage estimate that can be used to assess the quality of the selected assertions, and, in general, the effectiveness of the post-silicon validation process.

Unified Coverage Methodology for SoC Post-Silicon Validation

The System-on-Chip’s increased complexity and shortened design cycle calls for innovation in design and validation. A high quality System-on-Chip creates distinction and position in the market, and validation is the key to a quality product. Validation consumes >60% of the product cycle. Therefore, validation should be carried out efficiently. Validation must be quantified to aid in determining its quality. Pre-silicon uses various coverage metrics for quantifying the validation. The available on-chip coverage logic limits the use of pre-silicon-like coverage metrics in post-silicon. Although on-chip coverage logic increases observability, it does not contribute to the functional logic; hence, they are controlled and limited. Discounting the need for the on-chip coverage logic, the question to be answered is whether or not these pre-silic-on coverage metrics applicable to post-silicon. We discuss the reasons for limited applicability of pre-silicon coverage metrics in post-silicon. This paper presents a unified SoC post-silicon coverage methodology centered on functional coverage metrics.

Generating Cyclic-Random Sequences in a Constrained Space for In-System Validation

The constrained-random methodology is widely used during the pre-silicon verification of very-large scale integrated circuits. Recently, research efforts have been made to support the application of constrained-random patterns during the post-silicon validation stage. In this paper, we present a new method, including both software algorithms and on-chip hardware structures, for in-system constrained-random generation of stimuli sequences that are uniformly distributed. More specifically, we facilitate in-system application of constrained-random sequences that are cyclic-random, i.e., all the valid values from the user-constrained space are generated only once before the entire sample space is exhausted. While software simulation environments commonly support this feature, e.g., randc in SystemVerilog, to the best of our knowledge this is the first time it is shown how such feature can be ported to hardware environments.

A Survey of Virtual Prototyping Techniques for System Development and Validation

Recently, different kinds of computer systems like smart phones, embedded systems and cloud servers, are more and more widely used and the system development and validation is under great pressure. Hardware device, firmware and device driver development account for a significant portion of system development and validation effort. In traditional device, firmware and driver development largely has to wait until a stable version of the device becomes available. This dependency often leaves not enough time for software validation. Recently, virtual prototyping techniques have been widely explored and utilized by both industry engineers and academic researchers. White box nature of virtual prototyping brings better observability, traceability, debugging support and adaptability. First, virtual prototyping has found their way into enabling early firmware and driver development and validation. Second, there has been some research utilizing virtual prototyping for post-silicon functional validation. Third, the industry has built hybrid emulation and hybrid FPGA systems for system validation using virtual prototyping. In this paper, we demonstrate how recent work and products utilize virtual prototyping techniques for system development and validation in the above three domains.

Post-Silicon Validation of Multiprocessor Memory Consistency

Shared-memory chip-multiprocessor (CMP) architectures define memory consistency models that establish the ordering rules for memory operations from multiple threads. Validating the correctness of a CMP's implementation of its memory consistency model requires extensive monitoring and analysis of memory accesses while multiple threads are executing on the CMP. In this paper, we present a low overhead solution for observing, recording and analyzing shared-memory interactions for use in an emulation and/or post-silicon validation environment. Our approach leverages portions of the CMP's own data caches, augmented only by a small amount of hardware logic, to log information relevant to memory accesses. After transferring this information to a central memory location, we deploy our own analysis algorithm to detect any possible memory consistency violations. We build on the property that a violation corresponds to a cycle in an appropriately defined graph representing memory interactions. The solution we propose allows a designer to choose where to run the analysis algorithm: 1) on the CMP itself; 2) on a separate processor residing on the validation platform; or 3) off-line on a separate host machine. Our experimental results show an 83% bug detection rate, in our testbed CMP, over three distinct memory consistency models, namely: relaxed-memory order, total-store order, and sequential consistency. Finally, note that our solution can be disabled in the final product, leading to zero performance overhead and a per-core area overhead that is smaller than the size of a physical integer register file in a modern processor.

Fine-Grained Access Management in Reconfigurable Scan Networks

Modern very large scale integration designs incorporate a high amount of instrumentation that supports post-silicon validation and debug, volume test and diagnosis, as well as in-field system monitoring and maintenance. Reconfigurable scan architectures, as allowed by the novel IEEE Std 1149.1-2013 (JTAG) and IEEE Std 1687-2014 [Internal JTAG (IJTAG)], emerge as a scalable mechanism for access to such on-chip instruments. While the on-chip instrumentation is crucial for meeting quality, dependability, and time-to-market goals, it is prone to abuse and threatens system safety and security. A secure access management method is mandatory to assure that critical instruments be accessible to authorized entities only. This paper presents a novel protection method for fine-grained access management in complex reconfigurable scan networks based on a challenge-response authentication protocol. The target scan network is extended with an authorization instrument and secure segment insertion bits that together control the accessibility of individual instruments. To the best of the authors’ knowledge, this is the first fine-grained access management scheme that scales well with the number of protected instruments and offers a high level of security. Compared with recent state-of-the-art techniques, this scheme is more favorable with respect to implementation cost, performance overhead, and provided security level.

Reconfigurable Scan Networks

Efficient access to on-chip instrumentation is a key requirement for post-silicon validation, test, debug, bringup, and diagnosis. Reconfigurable scan networks, as proposed by, for example, IEEE Std 1687-2014 and IEEE Std 1149.1-2013, emerge as an effective and affordable means to cope with the increasing complexity of on-chip infrastructure. Reconfigurable scan networks are often hierarchical and may have complex structural and functional dependencies. Common approaches for scan verification based on static structural analysis and functional simulation are not sufficient to ensure correct operation of these types of architectures. To access an instrument in a reconfigurable scan network, a scan-in bit sequence must be generated according to the current state and structure of the network. Due to sequential and combinational dependencies, the access pattern generation process ( pattern retargeting ) poses a complex decision and optimization problem. This article presents the first generalized formal model that considers structural and functional dependencies of reconfigurable scan networks and is directly applicable to 1687-2014-based and 1149.1-2013-based scan architectures. This model enables efficient formal verification of complex scan networks, as well as automatic generation of access patterns. The proposed pattern generation method supports concurrent access to multiple target scan registers ( access merging ) and generates short scan-in sequences.

Debugging post-silicon fails in the IBM POWER8 bring-up lab

Debugging post-silicon fails continues to be a difficult problem that is becoming even more challenging as chips integrate more functionality and implement increasingly complicated functions. Additionally, the complexity of hardware systems, coupled with the difficulty in observing the state of the system that led to the failure, make the debugging effort a unique challenge. In this paper, we review the techniques and mechanisms used to facilitate effective debugging in the POWER8™ processor post-silicon validation phase. We further describe several functional bugs and describe the debugging process that drove the identification of their root cause.

Pattern Generation for Understanding Timing Sensitivity to Power Supply Noise

Timing prediction has become more and more difficult with shrinking technology nodes. Combining the pre-silicon delay model with post-silicon timing measurements has the potential to improve the accuracy of timing analysis. In this work, we address the problem of automatic test pattern generation for understanding circuit timing sensitivity to power supply noise (PSN) during post-silicon validation. Long paths are selected from a pseudo functional test set to span the power delivery network. To determine the sensitivity of timing to on-chip noise, the patterns are intelligently filled to achieve the desired PSN level. Our previous PSN control scheme is enhanced to consider both spatial and temporal information for better correlation with functional PSN. These patterns can be used to understand timing sensitivity in post-silicon validation by repeatedly applying the path delay test while sweeping the PSN experienced by the path from low to high.

Machine Learning Adaptation in Post Silicon Server Validation

As the complexity of modern server processors increases so the validation challenges. Current design validation methods cover less, resulting in bug escape and more regress postsilicon validation. The biggest problem is manual debugging of several failures by large number of test cases. By using machine learning in server validation, validation efforts and resource requirement will reduce. Validation of future generation server will be done through the learning set generated from the previous generation device, which is a set of test cases being passed. General Terms Algorithm, Machine learning, Validation.

Effective Post-Silicon Validation of System-on-Chips Using Quick Error Detection

This paper presents the Quick Error Detection (QED) technique for systematically creating families of post-silicon validation tests that quickly detect bugs inside processor cores and uncore components (cache controllers, memory controllers, and on-chip interconnection networks) of multicore system on chips (SoCs). Such quick detection is essential because long error detection latency, the time elapsed between the occurrence of an error due to a bug and its manifestation as an observable failure, severely limits the effectiveness of traditional post-silicon validation approaches. QED can be implemented completely in software, without any hardware modification. Hence, it is readily applicable to existing designs. Results using multiple hardware platforms, including the Intel® Core™ i7 SoC, and a state-of-the-art commercial multicore SoC, along with simulation results using an OpenSPARC T2-like multicore SoC with bug scenarios from commercial multicore SoCs demonstrate: 1) error detection latencies of post-silicon validation tests can be very long, up to billions of clock cycles, especially for bugs inside uncore components; 2) QED shortens error detection latencies by up to nine orders of magnitude to only a few hundred cycles for most bug scenarios; and 3) QED enables up to a fourfold increase in bug coverage.

Post-silicon platform for the functional diagnosis and debug of networks-on-chip

The increasing number of units in today's systems-on-chip and multicore processors has led to complex intra-chip communication solutions. Specifically, Networks-on-Chip (NoCs) have emerged as a favorable fabric to provide high bandwidth and low latency in connecting many units in a same chip. To achieve these goals, the NoC often includes complex components and advanced features, leading to the development of large and highly complex interconnect subsystems. One of the biggest challenges in these designs is to ensure the correct functionality of this communication infrastructure. To support this goal, an increasing fraction of the validation effort has shifted to post-silicon validation, because it permits exercising network activities that are too complex to be validated in pre-silicon. However, post-silicon validation is hindered by the lack of observability of the network's internal operations and thus, diagnosing functional errors during this phase is very difficult. In this work, we propose a post-silicon validation platform that improves observability of network operations by taking periodic snapshots of the traffic traversing the network. Each node's local cache is configured to temporarily store the snapshot logs in a designated area reserved for post-silicon validation and relinquished after product release. Each snapshot log is analyzed locally by a software algorithm running on its corresponding core, in order to detect functional errors. Upon error detection, all snapshot logs are aggregated at a central location to extract additional debug data, including an overview of network traffic surrounding the error event, as well as a partial reconstruction of the routes followed by packets in flight at the time. In our experiments, we found that this approach allows us to detect several types of functional errors, as well as observe, on average, over 50% of the network's traffic and reconstruct at least half of each of their routes through the network.