Probabilistic Timing Analysis Research Articles

Worst Case Execution Time and Power Estimation of Multicore and GPU Software: A Pedestrian Detection Use Case

Worst Case Execution Time estimation of software running on parallel platforms is a challenging task, due to resource interference of other tasks and the complexity of the underlying CPU and GPU hardware architectures. Similarly, the increased complexity of the hardware, challenges the estimation of worst case power consumption. In this paper, we employ Measurement Based Probabilistic Timing Analysis (MBPTA), which is capable of managing complex architectures such as multicores. We enable its use by software randomisation, which we show for the first time that is also possible on GPUs. We demonstrate our method on a pedestrian detection use case on an embedded multicore and GPU platform for the automotive domain, the NVIDIA Xavier. Moreover, we extend our measurement based probabilistic method in order to predict the worst case power consumption of the software on the same platform.

Hybrid probabilistic timing analysis with Extreme Value Theory and Copulas

The primary challenge of time-critical systems is to guarantee that a task completes its execution before its deadline. In order to ensure compliance with timing requirements, it is necessary to analyze the timing behavior of the overall software. Worst-Case Execution Time (WCET) represents the maximum amount of time an individual software unit takes to execute and is used for scheduling analysis in safety-critical systems. Recent studies focus on statistical approaches, which augments measurement-based timing analysis with probabilistic confidence level by applying stochastic methods. Common approaches either utilize Extreme Value Theory (EVT) for end-to-end measurements or convolution techniques for a group of program units to derive probabilistic upper bounds for the program. The former method does not ensure path coverage while the latter suffers from ignoring possible extreme cases. Furthermore, current state-of-the-art convolution methods employed in a commercial WCET analysis tool overestimates the results because of using the assumption of worst-case dependence between basic blocks. In this paper, we propose a hybrid probabilistic timing analysis framework and modeling the program units with EVT to capture extreme cases and use Copulas to model the dependency between the units to derive tighter distributional bounds in order to mitigate the effects of co-monotonic assumptions.

Dealing with Uncertainty in pWCET Estimations

The problem of estimating a tight and safe Worst-Case Execution Time (WCET), needed for certification in safety-critical environment, is a challenging problem for modern embedded systems. A possible solution proposed in past years is to exploit statistical tools to obtain a probability distribution of the WCET. These probabilistic real-time analyses for WCET are, however, subject to errors, even when all the applicability hypotheses are satisfied and verified. This is caused by the uncertainties of the probabilistic-WCET distribution estimator. This article aims at improving the measurement-based probabilistic timing analysis approach providing some techniques to analyze and deal with such uncertainties. The so-called region of acceptance model based on state-of-the-art statistical test procedures is defined over the distribution space parameters. From this model, a set of strategies is derived and discussed to provide the methodology to deal with the trade-off safety/tightness of the WCET estimation. These techniques are then tested over real datasets, including industrial safety-critical applications, to show the increased value of using the proposed approach in probabilistic WCET analyses.

The Misconception of Exponential Tail Upper-Bounding in Probabilistic Real Time

Measurement-based probabilistic timing analysis, a probabilistic real-time computing method, is based on the extreme value theory (EVT), a statistical theory applied to worstcase execution time (WCET) analysis on real-time embedded systems. The output of the EVT theory is a statistical distribution, in the form of generalized extreme value distribution or generalized Pareto distribution. Their cumulative distribution function can asymptotically assume one of the three possible forms: 1) light; 2) exponential; or 3) heavy tail. Recently, several works proposed to upper-bound the light-tail distributions with their exponential version. In this letter, we show that this assumption is valid only under certain conditions and that it is often misinterpreted. This leads to unsafe estimations of the WCET, which cannot be accepted in applications targeting safety critical embedded systems.

A Survey of Probabilistic Timing Analysis Techniques for Real-Time Systems

This survey covers probabilistic timing analysis techniques for real-time systems. It reviews and critiques the key results in the field from its origins in 2000 to the latest research published up to the end of August 2018. The survey provides a taxonomy of the different methods used, and a classification of existing research. A detailed review is provided covering the main subject areas: static probabilistic timing analysis, measurement-based probabilistic timing analysis, and hybrid methods. In addition, research on supporting mechanisms and techniques, case studies, and evaluations is also reviewed. The survey concludes by identifying open issues, key challenges and possible directions for future research.

Probabilistic Worst-Case Timing Analysis

The unabated increase in the complexity of the hardware and software components of modern embedded real-time systems has given momentum to a host of research in the use of probabilistic and statistical techniques for timing analysis. In the last few years, that front of investigation has yielded a body of scientific literature vast enough to warrant some comprehensive taxonomy of motivations, strategies of application, and directions of research. This survey addresses this very need, singling out the principal techniques in the state of the art of timing analysis that employ probabilistic reasoning at some level, building a taxonomy of them, discussing their relative merit and limitations, and the relations among them. In addition to offering a comprehensive foundation to savvy probabilistic timing analysis, this article also identifies the key challenges to be addressed to consolidate the scientific soundness and industrial viability of this emerging field.

Time-Randomized Wormhole NoCs for Critical Applications

Wormhole-based NoCs (wNoCs) are widely accepted in high-performance domains as the most appropriate solution to interconnect an increasing number of cores in the chip. However, wNoCs suitability in the context of critical real-time applications has not been demonstrated yet. In this article, in the context of probabilistic timing analysis (PTA), we propose a PTA-compatible wNoC design that provides tight time-composable contention bounds. The proposed wNoC design builds on PTA ability to reason in probabilistic terms about hardware events impacting execution time (e.g., wNoC contention), discarding those sequences of events occurring with a negligible low probability. This allows our wNoC design to deliver improved guaranteed performance w.r.t. conventional time-deterministic setups. Our results show that performance guarantees of applications running on top of probabilistic wNoC designs improve by 40% and 93% on average for 4 × 4 and 6 × 6 wNoC setups, respectively.

Probabilistic timing analysis of time‐randomised caches with fault detection mechanisms

In the real-time systems domain, time-randomised caches have been proposed as a way to simplify software timing analysis, i.e. the process of estimating the probabilistic worst case execution time (pWCET) of an application. However, the technology scaling of the cache memory manufacturing process is rendering transient and permanent faults more and more likely. These faults, in turn, affect a system's timing behaviour and the complexity of its analysis. In this study, the authors propose a static probabilistic timing analysis approach for time-randomised caches that is able to account for the presence of faults – and their detection mechanisms – using a state-space modelling technique. Their experiments show that the proposed methodology is capable of providing tight pWCET estimates. In their analysis, the effects on the estimation of safe pWCET bounds of two online mechanisms for the detection and classification of faults, i.e. a rule-based system and dynamic hidden Markov models (D-HMMs), are compared. The experimental results show that different mechanisms can greatly affect safe pWCET margins and that, by using D-HMMs, the pWCET of the system can be improved with respect to rule-based detection.

Locality-aware cache random replacement policies

Abstract Measurement-Based Probabilistic Timing Analysis (MBPTA) facilitates the analysis of complex software running on hardware comprising high-performance features. MBPTA also aims at preventing additional analysis costs for timing analysis techniques and preserving the confidence on derived WCET estimates. Cache behavior has a deep influence on WCET estimates and hence on “the amount of software” that can be consolidated onto a single hardware platform. Deterministic replacement policies such as LRU (Least Recently Used) and NMRU (Non-Most Recently Used) have systematic pathological cases that may lead to high execution times and WCET estimates. Instead, random replacement (RR) decreases pathological cases probability, at the cost of temporal locality. We present two new MBPTA-amenable replacement policies that completely remove the presented pathological cases. The first policy, Random Permutations (RP) preserves higher temporal locality than RR; while the second, NMRU Random Permutations (NMRURP), also protects the Most Recently Used line from eviction. Both proposed policies build upon restricted random replacement choices. Our simulation evaluation (validated against a real prototype) using the Malardalen benchmarks and a case study shows that RP and NMRURP deliver both high average performance (within 1% of LRUs and NRMU performance) and tight WCET estimates 11% and 24% lower than those of RR.

Fitting Software Execution-Time Exceedance into a Residual Random Fault in ISO-26262

Car manufacturers relentlessly replace or augment the functionality of mechanical subsystems with electronic components. Most such subsystems (e.g., steer-by-wire) are safety related, hence, subject to regulation. ISO-26262, the dominant standard for road vehicles, regards software faults as systematic , while differentiating hardware faults between systematic and random . The analysis of systematic faults entails rigorous processes and qualitative considerations. The increasing complexity of modern on-board computers, however, questions the very notion of treating the violation of execution-time envelopes for software programs as a systematic fault. Modern hardware in fact reduces the user's ability to delve deep enough into the fabric of hardware–software interaction to gage its extent of contribution to the worst-case execution time (WCET). Changing the nature of the WCET-analysis problem may help address that challenge effectively. To this end, we propose a solution that should allow ISO-26262 to quantify the likelihood of execution-time exceedance events, relating it to target failure metrics employed in support of certification arguments, similarly to random faults in hardware. To this end, we inject randomization in the timing behavior of the computer hardware to relieve the user from the need to control hard-to-reach low-level parts, and use measurement-based probabilistic timing analysis to quantify, constructively, the failure rates resulting from the likelihood of execution-time exceedance events.

On the Reliability and Tightness of GP and Exponential Models for Probabilistic WCET Estimation

As computer architectures evolve, guaranteeing that Real-Time Systems’ (RTSs’) timing requirements are met through Worst Case Execution Time (WCET) upper bounds becomes increasingly difficult. Techniques such as Measurement-Based Probabilistic Timing Analysis (MBPTA) have emerged that estimate WCET bounds exceeded only with arbitrarily low probabilities (i.e., pWCETs) through Extreme Value Theory (EVT). The Peaks Over Threshold (POT) approach for applying EVT involves adjusting a tail-shaped distribution, e.g., Generalized Pareto (GP) or Exponential, to the values that exceed a carefully selected high threshold. Several works suggest that GP should be used within POT for best representing different tail shapes, while others consider the Exponential model more adequate for providing upper bounds with increased reliability. This work presents empirical reliability and tightness evaluations of the pWCET estimates yielded by the GP and Exponential models while applying MBPTA through the POT approach. It mainly provides counter-evidence to the GP model reliability and evidence of the Exponential model adequacy in this context.

On the analysis of random replacement caches using static probabilistic timing methods for multi-path programs

Probabilistic hard real-time systems, based on hardware architectures that use a random replacement cache, provide a potential means of reducing the hardware over-provision required to accommodate pathological scenarios and the associated extremely rare, but excessively long, worst-case execution times that can occur in deterministic systems. Timing analysis for probabilistic hard real-time systems requires the provision of probabilistic worst-case execution time (pWCET) estimates. The pWCET distribution can be described as an exceedance function which gives an upper bound on the probability that the execution time of a task will exceed any given execution time budget on any particular run. This paper introduces a more effective static probabilistic timing analysis (SPTA) for multi-path programs. The analysis estimates the temporal contribution of an evict-on-miss, random replacement cache to the pWCET distribution of multi-path programs. The analysis uses a conservative join function that provides a proper over-approximation of the possible cache contents and the pWCET distribution on path convergence, irrespective of the actual path followed during execution. Simple program transformations are introduced that reduce the impact of path indeterminism while ensuring sound pWCET estimates. Evaluation shows that the proposed method is efficient at capturing locality in the cache, and substantially outperforms the only prior approach to SPTA for multi-path programs based on path merging. The evaluation results show incomparability with analysis for an equivalent deterministic system using an LRU cache. For some benchmarks the performance of LRU is better, while for others, the new analysis techniques show that random replacement has provably better performance.

An Adaptive Markov Model for the Timing Analysis of Probabilistic Caches

Accurate timing prediction for real-time embedded software execution is becoming a problem due to the increasing complexity of computer architecture, and the presence of mixed-criticality workloads. Probabilistic caches were proposed to set bounds to Worst Case Execution Time (WCET) estimates and help designers improve real-time embedded system resource use. Static Probabilistic Timing Analysis (SPTA) for probabilistic caches is nevertheless difficult to perform, because cache accesses depend on execution history, and the computational complexity of SPTA makes it intractable for calculation as the number of accesses increases. In this paper, we explore and improve SPTA for caches with evict-on-miss random replacement policy using a state space modeling technique. A nonhomogeneous Markov model is employed for single-path programs in discrete-time finite state space representation. To make this Markov model tractable, we limit the number of states and use an adaptive method for state modification. Experiments show that compared to the state-of-the-art methodology, the proposed adaptive Markov chain approach provides better results at the occurrence probability of 10 −15 : in terms of accuracy, the state-of-the-art SPTA results are more conservative, by 11% more on average. In terms of computation time, our approach is not significantly different from the state-of-the-art SPTA.

Adapting TDMA arbitration for measurement-based probabilistic timing analysis

Critical Real-Time Embedded Systems require functional and timing validation to prove that they will perform their functionalities correctly and in time. For timing validation, a bound to the Worst-Case Execution Time (WCET) for each task is derived and passed as an input to the scheduling algorithm to ensure that tasks execute timely. Bounds to WCET can be derived with deterministic timing analysis (DTA) and probabilistic timing analysis (PTA), each of which relies upon certain predictability properties coming from the hardware/software platform beneath. In particular, specific hardware designs are needed for both DTA and PTA, which challenges their adoption by hardware vendors.This paper makes a step towards reconciling the hardware needs of DTA and PTA timing analyses to increase the likelihood of those hardware designs to be adopted by hardware vendors. In particular, we show how Time Division Multiple Access (TDMA), which has been regarded as one of the main DTA-compliant arbitration policies, can be used in the context of PTA and, in particular, of the industrially-friendly Measurement-Based PTA (MBPTA). We show how the execution time measurements taken as input for MBPTA need to be padded to obtain reliable and tight WCET estimates on top of TDMA-arbitrated hardware resources with no further hardware support. Our results show that TDMA delivers tighter WCET estimates than MBPTA-friendly arbitration policies, whereas MBPTA-friendly policies provide higher average performance. Thus, the best policy to choose depends on the particular needs of the end user.

Measurement-Based Worst-Case Execution Time Estimation Using the Coefficient of Variation

Extreme Value Theory (EVT) has been historically used in domains such as finance and hydrology to model worst-case events (e.g., major stock market incidences). EVT takes as input a sample of the distribution of the variable to model and fits the tail of that sample to either the Generalised Extreme Value (GEV) or the Generalised Pareto Distribution (GPD). Recently, EVT has become popular in real-time systems to derive worst-case execution time (WCET) estimates of programs. However, the application of EVT is not straightforward and requires a detailed analysis of, and customisation for, the particular problem at hand. In this article, we tailor the application of EVT to timing analysis. To that end, (1) we analyse the response time of different hardware resources (e.g., cache memories) and identify those that may lead to radically different types of execution time distributions. (2) We show that one of these distributions, known as mixture distribution, causes problems in the use of EVT. In particular, mixture distributions challenge not only properly selecting GEV/GPD parameters (i.e., location, scale and shape) but also determining the size of the sample to ensure that enough tail values are passed to EVT and that only tail values are used by EVT to fit GEV/GPD. Failing to select these parameters has a negative impact on the quality of the derived WCET estimates. We tackle these problems, by (3) proposing Measurement-Based Probabilistic Timing Analysis using the Coefficient of Variation (MBPTA-CV), a new mixture-distribution aware, WCET-suited MBPTA method that builds on recent EVT developments in other fields (e.g., finance) to automatically select the distribution parameters that best fit the maxima of the observed execution times. Our results on a simulation environment and a real board show that MBPTA-CV produces high-quality WCET estimates.

On the assessment of probabilistic WCET estimates reliability for arbitrary programs

Measurement-Based Probabilistic Timing Analysis (MBPTA) has been shown to be an industrially viable method to estimate the Worst-Case Execution Time (WCET) of real-time programs running on processors including several high-performance features. MBPTA requires hardware/software support so that program’s execution time, and so its WCET, has a probabilistic behaviour and can be modelled with probabilistic and statistic methods. MBPTA also requires that those events with high impact on execution time are properly captured in the (R) runs made at analysis time. Thus, a representativeness argument is needed to provide evidence that those events have been captured.This paper addresses the MBPTA representativeness problems caused by set-associative caches and presents a novel representativeness validation method (ReVS) for cache placement. Building on cache simulation, ReVS explores the probability and impact (miss count) of those cache placements that can occur during operation. ReVS determines the number of runs R′, which can be higher than R, such that those cache placements with the highest impact are effectively observed in the analysis runs, and hence, MBPTA can be reliably applied to estimate the WCET.

Aging Assessment and Design Enhancement of Randomized Cache Memories

Critical real-time systems require the estimation of the worst-case execution time (WCET) for scheduling purposes and resource budgeting. Measurement-based probabilistic timing analysis (MBPTA) has been shown recently as a powerful approach for WCET estimation. MBPTA builds upon time-randomized cache memories. While aging has been deeply analyzed for conventional time-deterministic caches (i.e., implementing modulo placement), little work has been done to assess the reliability of time-randomized caches. In this line, only the WCET analysis of faulty time-randomized caches has been introduced recently. However, the intrinsic robustness of randomized hardware designs has not been assessed yet. In this paper we perform, for the first time, an assessment of the agingrobustness of random placement cache designs: random modulo and hash-based random placement. We propose a new random modulo implementation preserving its key benefits in terms of low critical path impact, low miss rates and MBPTA compliance; while reducing hot-carrier injection aging by achieving a better, yet random, activity distribution across cache sets. On the other hand, we show that gains in terms of bias temperature instability aging are limited for random placement designs on their own.

Fitting processor architectures for measurement-based probabilistic timing analysis

The pressing market demand for competitive performance/cost ratios compels Critical Real-Time Embedded Systems industry to employ feature-rich hardware. The ensuing rise in hardware complexity however makes worst-case execution time (WCET) analysis of software programs – which is often required, especially for programs at the highest levels of integrity – an even harder challenge. State-of-the-art WCET analysis techniques are hampered by the soaring cost and complexity of obtaining accurate knowledge of the internal operation of advanced processors and the difficulty of relating data obtained from measurement observations with reliable worst-case behaviour. This frustrating conundrum calls for novel solutions, with low intrusiveness on development practice. Measurement-Based Probabilistic Timing Analysis (MBPTA) techniques offer the opportunity to simultaneously reduce the cost of acquiring the knowledge needed for computing reliable WCET bounds and gain increased confidence in the representativeness of measurement observations. This paper describes the changes required in the design of several high-performance features – massively used in modern processors – to meet MBPTA requirements.

Correlation-Aware Probabilistic Timing Analysis for the Dynamic Segment of FlexRay

We propose an analytical framework for probabilistic timing analysis of the event-triggered Dynamic segment of the FlexRay communication protocol. Specifically, our framework computes the Deadline Miss Ratio of each message. The core problem is formulated as a Mixed Integer Linear Program (MILP). Given the intractability of the problem, we also propose several techniques that help to mitigate the running times of our tool. This includes the re-engineering of the problem to run it on GPUs as well as reformulating the MILP itself. Most importantly, we also show how our framework can handle correlations between the queuing events of messages. This is challenging because one cannot apply the convolution operator in the same way as in the case of independent queuing events.

The probabilistic time and cost risk analysis of a challenging part of an urban tunneling project

A predominant characteristic of geotechnical works and more particularly tunneling projects, is the uncertainty that surrounds the geologic medium in which the construction is to be performed. This paper is dealing with the risk analysis of both the construction time and cost of a challenging part of Tehran Metro Line 7, an urban tunneling project which is currently being under development. The Decision Aids for Tunneling (DAT) has been utilized for this assessment, concerning the risk analysis by taking into account different scenarios relating to the measures implemented to prevent surface subsidence problems. The results have shown that selection of different sorts of methods will help control the risk of escalating the project’s total construction cost and time to completion. This research delivers useful information to concurrently help the owner, the contractor as well as the designer of the project in making the most appropriate decisions under such a highly uncertain and challenging geological environment.