Performance Monitoring Unit Research Articles

SLOPE: Safety LOg PEripherals implementation and software drivers for a safe RISC-V microcontroller unit

The focus of this manuscript is related to the main safety issues regarding a mixed criticality system running multiple concurrent tasks. Our concerns are related to the guarantee of Freedom of Interference between concurrent partitions, and to the respect of the Worst Case Execution Time for tasks. Moreover, we are interested in the evaluation of resources budgeting and the study of system behavior in case of occurring random hardware failures. In this paper we present a set of Safety LOg PEripherals (SLOPE): Performance Monitoring Unit (PMU), Execution Tracing Unit (ETU), Error Management Unit (EMU), Time Management Unit (TMU) and Data Log Unit (DLU); then, an implementation of SLOPE on a single core RISC-V architecture is proposed. Such peripherals are able to collect software and hardware information about execution, and eventually trigger recovery actions to mitigate a possible dangerous misbehavior. We show results of the hardware implementation and software testing of the units with a dedicated software library. For the PMU we standardized the software layer according to embedded Performance Application Programming Interface (ePAPI), and compared its functionality with a bare-metal use of the library. To test the ETU we compared the hardware simulation results with software ones, to understand if overflow may occur in internal hardware buffers during tracing. In conclusion, designed devices introduce new instruments for system investigation for RISC-V technologies and can generate an execution profile for safety related tasks.

Investigating Black-Box Function Recognition Using Hardware Performance Counters

This paper presents new methods and results for recognising black-box program functions using hardware performance counters (HPC), where an investigator can invoke and measure function calls. Important use cases include analysing compiled libraries, e.g. static and dynamic link libraries, and trusted execution environment (TEE) applications. We develop a generic approach to classify a comprehensive set of hardware events, e.g. branch mis-predictions and instruction retirements, to recognise standard benchmarking and cryptographic library functions. This includes various signing, verification and hash functions, and ciphers in numerous modes of operation. Three architectures are evaluated using off-the-shelf Intel/X86-64, ARM, and RISC-V CPUs. Next, we show that several known CVE-numbered OpenSSL vulnerabilities can be detected using HPC differences between patched and unpatched library versions. Further, we demonstrate that standardised cryptographic functions within ARM TrustZone TEE applications can be recognised using non-secure world HPC measurements, applying to platforms that insecurely perturb the performance monitoring unit (PMU) during TEE execution. High accuracy was achieved in all cases (86.22–99.83%) depending on the application, architectural, and compilation assumptions. Lastly, we discuss mitigations, outstanding challenges, and directions for future research.

Cross-World Covert Channel on ARM Trustzone through PMU.

The TrustZone technology is incorporated in a majority of recent ARM Cortex A and Cortex M processors widely deployed in the IoT world. Security critical code execution inside a so-called secure world is isolated from the rest of the application execution within a normal world. It provides hardware-isolated area called a trusted execution environment (TEE) in the processor for sensitive data and code. This paper demonstrates a vulnerability in the secure world in the form of a cross-world, secure world to normal world, covert channel. Performance counters or Performance Monitoring Unit (PMU) events are used to convey the information from the secure world to the normal world. An encoding program generates appropriate PMU event footprint given a secret S. A corresponding decoding program reads the PMU footprint and infers S using machine learning (ML). The machine learning model can be trained entirely from the data collected from the PMU in user space. Lack of synchronization between PMU start and PMU read adds noise to the encoding/decoding ML models. In order to account for this noise, this study proposes three different synchronization capabilities between the client and trusted applications in the covert channel. These are synchronous, semi-synchronous, and asynchronous. Previously proposed PMU based covert channels deploy L1 and LLC cache PMU events. The latency of these events tends to be 100–1000 cycles limiting the bandwidth of these covert channels. We propose to use microarchitecture level events with latency of 10–100 cycles captured through PMU for covert channel encoding leading to a potential 100× higher bandwidth. This study conducts a series of experiments to evaluate the proposed covert channels under various synchronization models on a TrustZone supported Cortex-A processor using OP-TEE framework. As stated earlier, switch from signaling based on PMU cache events to PMU microarchitectural events leads to approximately 15× higher covert channel bandwidth. This proposed finer-grained microarchitecture event encoding covert channel can achieve throughput of the order of 11 Kbits/s as opposed to previous work’s throughput of the order of 760 bits/s.

Robust and Accurate Fine-Grain Power Models for Embedded Systems With No On-Chip PMU

This letter a novel approach to event-based power modeling for embedded platforms that do not have a performance monitoring unit (PMU). The method involves complementing the target hardware platform, where the physical power data is measured, with another platform on which the CPU performance data, that is needed for model generation, can be collected. The methodology is used to generate accurate fine-grain power models for the Gaisler GR712RC dual-core LEON3 fault-tolerant SPARC processor with onboard power sensors and no PMU. A Kintex UltraScale field-programmable gate array (FPGA) is used as the support platform to obtain the required CPU performance data, by running a soft-core representation of the dual-core LEON3 as on the GR712RC but with a PMU implementation. Both platforms execute the same benchmark set and data collection is synchronized using per-sample timestamps so that the power sensor data from the GR712RC board can be matched to the PMU data from the FPGA. The synchronized samples are then processed by the Robust Energy and Power Predictor Selection (REPPS) software in order to generate power models. The models achieve less than 2% power estimation error when validated on an industrial use case and can follow program phases, which makes them suitable for runtime power profiling during development.

PMU-Events-Driven DVFS Techniques for Improving Energy Efficiency of Modern Processors

This paper describes the results of our measurement-based study, conducted on an Intel Core i7 processor running the SPEC CPU2017 benchmark suites, that evaluates the impact of dynamic voltage frequency scaling (DVFS) on performance (P), energy efficiency (EE) , and their product (PxEE) . The results indicate that the default DVFS-based power management techniques heavily favor performance, resulting in poor energy efficiency. To remedy this problem, we introduce, implement, and evaluate four DVFS-based power management techniques driven by the following metrics derived from the processor's performance monitoring unit: (i) the total pipeline slot stall ratio (FS-PS), (ii) the total cycle stall ratio (FS-TS), (iii) the total memory-related cycle stall ratio (FS-MS), and (iv) the number of last level cache misses per kilo instructions (FS-LLCM). The proposed techniques linearly map these metrics onto the available processor clock frequencies. The experimental evaluation results show that the proposed techniques significantly improve EE and PxEE metrics compared to the existing approaches. Specifically, EE improves from 44% to 92%, and PxEE improves from 31% to 48% when all the benchmarks are considered together. Furthermore, we find that the proposed techniques are particularly effective for a class of memory-intensive benchmarks – they improve EE from 121% to 183% and PxEE from 100% to 141%. Finally, we elucidate the advantages and disadvantages of each of the proposed techniques and offer recommendations on using them.

Microarchitectural Characterization on a Mobile Workload

Geekbench is one of the most referenced cross-platform benchmarks in the mobile world. Most of its workloads are synthetic but some of them aim to simulate real-world behavior. In the mobile world, its microarchitectural behavior has been reported rarely since the hardware profiling features are limited to the public. As a popular mobile performance workload, it is hard to find Geekbench’s microarchitecture characteristics in mobile devices. In this paper, a thorough experimental study of Geekbench performance characterization is reported with detailed performance metrics. This study also identifies mobile system on chip (SoC) microarchitecture impacts, such as the cache subsystem, instruction-level parallelism, and branch performance. After the study, we could understand the bottleneck of workloads, especially in the cache sub-system. This means that the change of data set size directly impacts performance score significantly in some systems and will ruin the fairness of the CPU benchmark. In the experiment, Samsung’s Exynos9820-based platform was used as the tested device with Android Native Development Kit (NDK) built binaries. The Exynos9820 is a superscalar processor capable of dual issuing some instructions. To help performance analysis, we enable the capability to collect performance events with performance monitoring unit (PMU) registers. The PMU is a set of hardware performance counters which are built into microprocessors to store the counts of hardware-related activities. Throughout the experiment, functional and microarchitectural performance profiles were fully studied. This paper describes the details of the mobile performance studies above. In our experiment, the ARM DS5 tool was used for collecting runtime PMU profiles including OS-level performance data. After the comparative study is completed, users will understand more about the mobile architecture behavior, and this will help to evaluate which benchmark is preferable for fair performance comparison.

HRM: Merging Hardware Event Monitors for Improved Timing Analysis of Complex MPSoCs

The performance monitoring unit (PMU) in multiprocessor system-on-chips (MPSoCs) is at the heart of the latest measurement-based timing analysis techniques in critical embedded systems. In particular, hardware event monitors (HEMs) in the PMU are used as building blocks in the process of budgeting and verifying software timing by tracking and controlling access counts to shared resources. While the number of HEMs in current MPSoCs reaches hundreds, they are read via performance monitoring counters whose number is usually limited to 4-8, thus requiring multiple runs of each experiment in order to collect all desired HEMs. Despite the effort of engineers in controlling the execution conditions of each experiment, the complexity of current MPSoCs makes it arguably impossible to completely remove the noise affecting each run. As a result, HEMs read in different runs are subject to different variability, and hence, those HEMs captured in different runs cannot be “blindly” merged. In this work, we focus on the NXP T2080 platform where we observed up to 59% variability across different runs of the same experiment for some relevant HEMs (e.g., processor cycles). We develop a HEM reading and merging (HRM) approach to join reliably HEMs across different runs as a fundamental element of any measurement-based timing budgeting and verification technique. Our method builds on order statistics and the selection of an anchor HEM read in all runs to derive the most plausible combination of HEM readings that keep the distribution of each HEM and their relationship with the anchor HEM intact.

Enabling Hardware Performance Counters for Microkernel-Based Virtualization on Embedded Systems

Virtualization techniques continue to evolve at rapid speed and have now come to find its application in embedded and mobile computing devices. Virtualization improves the utilization of system resources effectively and also enhances security by providing isolated environments to run untrusted applications. There are various approaches to virtualization of embedded systems, from among them, we have chosen microkernel-based virtualization for our analysis due to its low memory requirements and advantages in terms of security. The microkernel selected for our work is the L4/Fiasco microkernel. Most of the modern CPUs consist of Performance Monitoring Unit (PMUs), which have a set of hardware counters that can be configured to monitor events. These hardware performance counters in the PMU block is not accessible from a microkernel-based virtualization environment. Access to PMU from a virtual environment would facilitate profiling with better accuracy and reduced overheads. To get this realized we propose a method to access the hardware performance counters from a microkernel-based virtualization environment. We have used this implementation to analyze the performance of applications in a microkernel-based virtual environment and to compare its performance in a non-virtual environment.

Intra- and inter-core power modelling for single-ISA heterogeneous processors

This research presents a systematic methodology for producing accurate power models for single instruction set architecture (ISA) heterogeneous processors. We use the hardware event counters from the processor performance monitoring unit (PMU) to accurately capture the CPU states and ordinary least squares (OLS), assisted by automated event selection algorithms, to compute the power models. Several estimators for single-thread and multi-thread benchmarks are proposed capable of performing power predictions across different frequency levels for one processor as well as between the heterogeneous processors with less than 3% error. The models are compared to related work showing significant improvement in accuracy and good computational efficiency which makes them suitable for run-time deployment.

ProRace

This paper presents ProRace, a dynamic data race detector practical for production runs. It is lightweight, but still offers high race detection capability. To track memory accesses, ProRace leverages instruction sampling using the performance monitoring unit (PMU) in commodity processors. Our PMU driver enables ProRace to sample more memory accesses at a lower cost compared to the state-of-the-art Linux driver. Moreover, ProRace uses PMU-provided execution contexts including register states and program path, and reconstructs unsampled memory accesses offline. This technique allows \ProRace to overcome inherent limitations of sampling and improve the detection coverage by performing data race detection on the trace with not only sampled but also reconstructed memory accesses. Experiments using racy production software including apache and mysql shows that, with a reasonable offline cost, ProRace incurs only 2.6% overhead at runtime with 27.5% detection probability with a sampling period of 10,000.

ProRace

This paper presents ProRace, a dynamic data race detector practical for production runs. It is lightweight, but still offers high race detection capability. To track memory accesses, ProRace leverages instruction sampling using the performance monitoring unit (PMU) in commodity processors. Our PMU driver enables ProRace to sample more memory accesses at a lower cost compared to the state-of-the-art Linux driver. Moreover, ProRace uses PMU-provided execution contexts including register states and program path, and reconstructs unsampled memory accesses offline. This technique allows \ProRace to overcome inherent limitations of sampling and improve the detection coverage by performing data race detection on the trace with not only sampled but also reconstructed memory accesses. Experiments using racy production software including apache and mysql shows that, with a reasonable offline cost, ProRace incurs only 2.6% overhead at runtime with 27.5% detection probability with a sampling period of 10,000.

ProRace

This paper presents ProRace, a dynamic data race detector practical for production runs. It is lightweight, but still offers high race detection capability. To track memory accesses, ProRace leverages instruction sampling using the performance monitoring unit (PMU) in commodity processors. Our PMU driver enables ProRace to sample more memory accesses at a lower cost compared to the state-of-the-art Linux driver. Moreover, ProRace uses PMU-provided execution contexts including register states and program path, and reconstructs unsampled memory accesses offline. This technique allows \ProRace to overcome inherent limitations of sampling and improve the detection coverage by performing data race detection on the trace with not only sampled but also reconstructed memory accesses. Experiments using racy production software including apache and mysql shows that, with a reasonable offline cost, ProRace incurs only 2.6% overhead at runtime with 27.5% detection probability with a sampling period of 10,000.

Inside 6th-Generation Intel Core: New Microarchitecture Code-Named Skylake

Skylake's core, processor graphics, and system on chip were designed to meet a demanding set of requirements for a wide range of power-performance points. Its coherent fabric was designed to provide high-memory bandwidth from multiple memory sources. Skylake's power management, which includes Intel Speed Shift technology, was designed to provide the largest dynamic power range among prior Intel processors. The Intel Architecture core delivers higher power efficiency, higher frequency, and a wider dynamic power range, supporting smaller form factors. Skylake's Gen9 graphics provides new features designed to maximize energy efficiency and bring the best visual experience for gaming and media. Skylake offers a rich performance monitoring unit that enhances software developers' ability to optimize their applications.

The Effectiveness of Performance Contracting in Service Delivery in Public Universities in Kenya

The concept of performance contracting was mooted by the government of Kenya for effective and efficient achievement of set targets in the public service. All public institutions are expected to set performance targets and indicators for their achievement. The introduction of performance contracting in public universities is an effort to institute a framework of accountability and enhance service delivery in Kenya. Though difficult and challenging to implement, performance contracting leads to higher levels of performance through periodic monitoring and evaluation of performance targets. The main objective of carrying out this study was to determine the effectiveness of performance contracting and ways of improving its implementation and measurement in Kenyan public universities. The study was carried out in public universities and targeted staff in various cadres as respondents. The researchers employed descriptive survey design where questionnaires were administered. Stratified random sampling was used in a sample size of 80 respondents. The findings indicate that performance contracting has not been cascaded to all staff members, only a quarter of the staff had signed performance contracts. However, public universities have made moderate achievements in the entire performance contract indicators. The major challenge that affects the performance contracting process is the low level of coordination. The study recommends that the performance monitoring and evaluation unit be strengthened in order to carry out its mandate effectively and departmental performance contract coordinators be appointed. Also, a clear reward and sanctions scheme is put in place to inculcate a competitive culture in public universities. The study further recommends sensitization and training of university staff to demystify performance contracting. The process should be more inclusive by involving all stakeholders.

Optimizing ATLAS code with different profilers

After the current maintenance period, the LHC will provide higher energy collisions with increased luminosity. In order to keep up with these higher rates, ATLAS software needs to speed up substantially. However, ATLAS code is composed of approximately 6M lines, written by many different programmers with different backgrounds, which makes code optimisation a challenge. To help with this effort different profiling tools and techniques are being used. These include well known tools, such as the Valgrind suite and Intel Amplifier; less common tools like Pin, PAPI, and GOoDA; as well as techniques such as library interposing. In this paper we will mainly focus on Pin tools and GOoDA. Pin is a dynamic binary instrumentation tool which can obtain statistics such as call counts, instruction counts and interrogate functions' arguments. It has been used to obtain CLHEP Matrix profiles, operations and vector sizes for linear algebra calculations which has provided the insight necessary to achieve significant performance improvements. Complimenting this, GOoDA, an in-house performance tool built in collaboration with Google, which is based on hardware performance monitoring unit events, is used to identify hot-spots in the code for different types of hardware limitations, such as CPU resources, caches, or memory bandwidth. GOoDA has been used in improvement of the performance of new magnetic field code and identification of potential vectorization targets in several places, such as Runge-Kutta propagation code.

BPM/BPM+

The main memory system is a shared resource in modern multicore machines that can result in serious interference leading to reduced throughput and unfairness. Many new memory scheduling mechanisms have been proposed to address the interference problem. However, these mechanisms usually employ relative complex scheduling logic and need modifications to Memory Controllers (MCs), which incur expensive hardware design and manufacturing overheads. This article presents a practical software approach to effectively eliminate the interference without any hardware modifications. The key idea is to modify the OS memory management system and adopt a page-coloring-based Bank-level Partitioning Mechanism (BPM) that allocates dedicated DRAM banks to each core (or thread). By using BPM, memory requests from distinct programs are segregated across multiple memory banks to promote locality/fairness and reduce interference. We further extend BPM to BPM+ by incorporating channel-level partitioning, on which we demonstrate additional gain over BPM in many cases. To achieve benefits in the presence of diverse application memory needs and avoid performance degradation due to resource underutilization, we propose a dynamic mechanism upon BPM/BPM+ that assigns appropriate bank/channel resources based on application memory/bandwidth demands monitored through PMU (performance-monitoring unit) and a low-overhead OS page table scanning process. We implement BPM/BPM+ in Linux 2.6.32.15 kernel and evaluate the technique on four-core and eight-core real machines by running a large amount of randomly generated multiprogrammed and multithreaded workloads. Experimental results show that BPM/BPM+ can improve the overall system throughput by 4.7%/5.9%, on average, (up to 8.6%/9.5%) and reduce the unfairness by an average of 4.2%/6.1% (up to 15.8%/13.9%).

Taming Hardware Event Samples for Precise and Versatile Feedback Directed Optimizations

Feedback-directed optimization (FDO) is effective in improving application runtime performance, but has not been widely adopted due to the tedious dual-compilation model, the difficulties in generating representative training data sets, and the high runtime overhead of profile collection. The use of hardware-event sampling overcomes these drawbacks by providing a lightweight approach to collect execution profiles in the production environment, which naturally consumes representative input. Yet, hardware event samples are typically not precise at the instruction or basic-block granularity. These inaccuracies lead to missed performance when compared to instrumentation-based FDO. In this paper, we use Performance Monitoring Unit (PMU)-based sampling to collect the instruction frequency profiles. By collecting profiles using multiple events, and applying heuristics to predict the accuracy, we improve the accuracy of the profile. We also show how emerging techniques can be used to further improve the accuracy of the sample-based profile. Additionally, these emerging techniques are used to collect value profiles, as well as to assist a lightweight interprocedural optimizer. All these profiles are represented in a portable form, thus they can be used across different platforms. We demonstrate that sampling-based FDO can achieve an average of 92 percent of the performance gains obtained using instrumentation-based exact profiles for both SPEC CINT2000 and CINT2006 benchmarks. The overhead of collection is only 0.93 percent on average, while compiler-based instrumentation incurs 2.0-351.5 percent overhead (and 10x overhead on an industrial web search application).

A flexible and dynamic page migration infrastructure based on hardware counters

Performance counters, also known as hardware counters, are a powerful monitoring mechanism included in the Performance Monitoring Unit (PMU) of most of the modern microprocessors. Their use is gaining popularity as an analysis and validation tool for profiling, since their impact is virtually imperceptible and their precision has noticeably increased thanks to the new Precise Event-Based Sampling (PEBS) features. In this paper, we present and evaluate a novel user-level tool, based on hardware counters, for monitoring and migrating pages dynamically. This tool supports different migration strategies, being able to attach and monitor a target application without need to modify it whatsoever. The page migration process is performed timely and its overhead is overcome by the benefit of the data locality achieved. As a case study, an access-based migration algorithm was implemented and integrated into our tool. Performance results on a NUMA system show a noticeable reduction of remote accesses and execution time, achieving speedups of up to ~21 % in a multiprogrammed environment.

System-Level Online Power Estimation Using an On-Chip Bus Performance Monitoring Unit

Quality power estimation is a basis of efficient power management of electronic systems. Indirect power measurement, such as power estimation using a CPU performance monitoring unit (PMU), is widely used for its low cost and area overheads. However, the existing CPU PMUs only monitor the core and cache activities, which result in a significant accuracy limitation in the system-wide power estimation including off-chip memory devices. In this paper, we propose an on-chip bus (OCB) PMU that directly captures on-chip and off-chip component activities by snooping the OCB. The OCB PMU stores the activity information in separate counters, and online software converts counter values into actual power values with simple first-order linear power models. We also introduce an optimization algorithm that minimizes the energy model to reduce the number of counters in the OCB PMU. We compare the accuracy of the power estimation using the proposed OCB PMU with real hardware measurement and cycle-accurate system-level power estimation, and demonstrate high estimation accuracy compared with CPU PMU-based estimation method.

PMU Guided Structure Data-Layout Optimization

Existing methods of obtaining runtime feedback for structure data-layout optimization have several drawbacks, such as large overhead and difficulty composing training sets. As a result, structure data-layout optimization is not widely used. To overcome these drawbacks, a performance monitoring unit (PMU) sampling method was developed with much less overhead and better portability and usability. An algorithm was developed to correct incomplete and inaccurate PMU sampling. With the corrected PMU feedback, a structure data-layout optimizer achieved a 45.1% performance improvement compared to a design without data-layout optimization, which is 97.6% of the performance improvement achieved with instrumented feedback. Calculation of the PMU feedback increased the execution time by 12.3%, compared to the overhead for the instrumented feedback of 341.5%. Tests show that the PMU feedback is efficient and effective for structure data-layout optimization.