Measurement-based Timing Analysis Research Articles

Hybrid probabilistic timing analysis with Extreme Value Theory and Copulas

The primary challenge of time-critical systems is to guarantee that a task completes its execution before its deadline. In order to ensure compliance with timing requirements, it is necessary to analyze the timing behavior of the overall software. Worst-Case Execution Time (WCET) represents the maximum amount of time an individual software unit takes to execute and is used for scheduling analysis in safety-critical systems. Recent studies focus on statistical approaches, which augments measurement-based timing analysis with probabilistic confidence level by applying stochastic methods. Common approaches either utilize Extreme Value Theory (EVT) for end-to-end measurements or convolution techniques for a group of program units to derive probabilistic upper bounds for the program. The former method does not ensure path coverage while the latter suffers from ignoring possible extreme cases. Furthermore, current state-of-the-art convolution methods employed in a commercial WCET analysis tool overestimates the results because of using the assumption of worst-case dependence between basic blocks. In this paper, we propose a hybrid probabilistic timing analysis framework and modeling the program units with EVT to capture extreme cases and use Copulas to model the dependency between the units to derive tighter distributional bounds in order to mitigate the effects of co-monotonic assumptions.

HRM: Merging Hardware Event Monitors for Improved Timing Analysis of Complex MPSoCs

The performance monitoring unit (PMU) in multiprocessor system-on-chips (MPSoCs) is at the heart of the latest measurement-based timing analysis techniques in critical embedded systems. In particular, hardware event monitors (HEMs) in the PMU are used as building blocks in the process of budgeting and verifying software timing by tracking and controlling access counts to shared resources. While the number of HEMs in current MPSoCs reaches hundreds, they are read via performance monitoring counters whose number is usually limited to 4-8, thus requiring multiple runs of each experiment in order to collect all desired HEMs. Despite the effort of engineers in controlling the execution conditions of each experiment, the complexity of current MPSoCs makes it arguably impossible to completely remove the noise affecting each run. As a result, HEMs read in different runs are subject to different variability, and hence, those HEMs captured in different runs cannot be “blindly” merged. In this work, we focus on the NXP T2080 platform where we observed up to 59% variability across different runs of the same experiment for some relevant HEMs (e.g., processor cycles). We develop a HEM reading and merging (HRM) approach to join reliably HEMs across different runs as a fundamental element of any measurement-based timing budgeting and verification technique. Our method builds on order statistics and the selection of an anchor HEM read in all runs to derive the most plausible combination of HEM readings that keep the distribution of each HEM and their relationship with the anchor HEM intact.

Increasing the Reliability of Software Timing Analysis for Cache-Based Processors

Real-time systems are witnessing a significant increase in critical software's size, complexity, and performance needs, which can only be satisfied with high-performance hardware features. Cache memories, pervasively used to improve average performance, complicate Worst-Case Execution Time analysis: cache placement (i.e., how software objects are mapped to cache) during the testing phase does not only critically affect the observed performance, but also proves to be arduous to control and preserve up to operation. The probabilistic variant of Measurement-Based Timing Analysis (MBPTA) responds to this challenge by deploying time-randomized caches that naturally explore a different random cache placement in each run, relieving the user from producing tests that intercept relevant Cache Conflict Placements (CCP). Yet, to meet an adequate probabilistic CCP coverage, the user is required to collect a minimum number of measurements. We present two mechanisms, CCP-RM and CCP-HRP, to identify CCP with relevant probability of occurrence and large impact on execution-time, for the random modulo (RM) and hash-based random placement (HRP) policies. CCP-RM and CCP-HRP enable a reliable application of MBPTA by computing the number of runs $R^{\prime }$R' necessary to meet the desired CCP coverage. We exhaustively evaluate CCP-RM and CCP-HRP, showing their effectiveness on well-known benchmarks and a railway case study, on top of an accurate simulator and a concrete RTL implementation.

Reconciling Time Predictability and Performance in Future Computing Systems

This article presents hardware design approaches to achieve predictability with measurement-based timing analysis.

Reproducibility and representativity

The increased number of systems consisting of multiple interacting components imposes the evolution of timing analyses towards methods able to estimate the timing behavior of an entire system by aggregating timings bounds of its components. In this paper we propose the first discussion on the properties required by measurement-based timing analyses to ensure such compositionality. We identify the properties of reproducibility and representativity as necessary conditions to ensure the convergence of any measurement protocol allowing a compositional measurement-based timing analysis.

Calculating WCET estimates from timed traces

Real-time systems engineers face a daunting duty: they must ensure that each task in their system can always meet its deadline. To analyse schedulability they must know the worst-case execution time (WCET) of each task. However, determining exact WCETs is practically infeasible in cost-constrained industrial settings involving real-life code and COTS hardware. Static analysis tools that could yield sufficiently tight WCET bounds are often unavailable. As a result, interest in portable analysis approaches like measurement-based timing analysis is growing. We present an approach based on integer linear programming (ILP) for calculating a WCET estimate from a given database of timed execution traces. Unlike previous work, our method specifically aims at reducing overestimation, by means of an automatic classification of code executions into scenarios with differing worst-case behaviour. To ease the integration into existing analysis tool chains, our method is based on the implicit path enumeration technique. It can thus reuse flow facts from other analysis tools and produces ILP problems that can be solved by off-the-shelf solvers.

On the evaluation of the impact of shared resources in multithreaded COTS processors in time-critical environments

Commercial Off-The-Shelf (COTS) processors are now commonly used in real-time embedded systems. The characteristics of these processors fulfill system requirements in terms of time-to-market, low cost, and high performance-per-watt ratio. However, multithreaded (MT) processors are still not widely used in real-time systems because the timing analysis is too complex. In MT processors, simultaneously-running tasks share and compete for processor resources, so the timing analysis has to estimate the possible impact that the inter-task interferences have on the execution time of the applications. In this paper, we propose a method that quantifies the slowdown that simultaneously-running tasks may experience due to collision in shared processor resources. To that end, we designed benchmarks that stress specific processor resources and we used them to (1) estimate the upper limit of a slowdown that simultaneously-running tasks may experience because of collision in different shared processor resources, and (2) quantify the sensitivity of time-critical applications to collision in these resources. We used the presented method to determine if a given MT processor is a good candidate for systems with timing requirements. We also present a case study in which the method is used to analyze three multithreaded architectures exhibiting different configurations of resource sharing. Finally, we show that measuring the slowdown that real applications experience when simultaneously-running with resource-stressing benchmarks is an important step in measurement-based timing analysis. This information is a base for incremental verification of MT COTS architectures.