L2 Cache Research Articles

Design of RISC-V CPU for 100 Gbps Network Application

<p indent=0mm>As a new open-source reduced instruction set architecture, RISC-V has advantages of low power consumption, small area and high performance. Therefore, technology and products based on RISC-V are developing rapidly. However, currently there are few medium and high end 64 bit CPU design instances based on the RISC-V architecture, and it is also hard to find corresponding commercial IP, especially for high-speed network applications. In this paper, we firstly improved the open source 64 bit U500 RISC-V SoC by extending the Bus width and adding L2 Cache, and etc. Secondly we implemented a complete 100 Gbps Ethernet function, which includes MAC, PCS, and SerDes, and the TX buffer and RX buffer used for the function. Finally we proved correctness and effectiveness of entire 64 bit RISC-V CPU design and 100 Gbps Ethernet function by simulation, FPGA verification and boot of Linux operation system. The designed RISC-V CPU and 100 Gbps Ethernet function can be applied to data center application such as smart network interface cards.

Code integrity verification using cache memory monitoring

ABSTRACT This paper addresses the challenges of building a secure software system to prevent Man-at-the-End attacks. Ensuring the security of systems is challenging due to unfavorable constraints faced by the end-point host system. Constraints such as hostile environments leave the host system at the peril of would-be attackers. In this paper, we verify program integrity through L3 cache by monitoring the security-sensitive code points and verify them in memory. This paper uses a cache-based monitoring program to verify code integrity. In particular, we show that side-channel information can be used to encode the invariant of the program execution state. These invariants can be periodically and externally monitored as a proxy for application integrity. This monitoring system uses a sliding window scheme that can detect the violation of these invariant with high reliability. The proposed solution is transparent to the attacker and utilizes a side-channel technique (Flush + Reload) along with a sliding window scheme to monitor security-sensitive code and detect MATE attacks to prevent malicious manipulation of software.

Electromagnetic fault injection against a complex CPU, toward new micro-architectural fault models

The last years have seen the emergence of fault attacks targeting modern central processing units (CPUs). These attacks are analyzed at a very high abstraction level and, due to the modern CPUs complexity, the underlying fault effect is usually unknown. Recently, a few articles have focused on characterizing faults on modern CPUs. In this article, we focus on the electromagnetic fault injection (EMFI) characterization on a bare-metal implementation. With this approach, we discover and understand new effects on micro-architectural subsystems. We target the BCM2837 where we successfully demonstrate persistent faults on L1 instruction cache, L1 data cache and L2 cache. We also show that faults can corrupt the memory management unit (MMU). To validate our fault model, we realize a persistent fault analysis to retrieve an AES key.

NVIDIA A100 Tensor Core GPU: Performance and Innovation

NVIDIA A100 Tensor Core GPU is NVIDIA's latest flagship GPU. It has been designed with many new innovative features to provide performance and capabilities for HPC, AI, and data analytics workloads. Feature enhancements include a Third-Generation Tensor Core, new asynchronous data movement and programming model, enhanced L2 cache, HBM2 DRAM, and third-generation NVIDIA NVLink I/O.

Agglomerative Memory and Thread Scheduling for High-Performance Ray-Tracing on GPUs

Ray-tracing rendering has long been considered as a promising technology to enable a higher level of visual experience. The democratization of the ray-tracing rendering to consumer platforms, however, poses significant challenges to rendering hardware and software due to its highly irregular computing patterns. In fact, modern ray-tracing techniques typically depend on a tree-based acceleration structure to reduce the computing complexity of intersection testing of rays and graphics primitives. The traversal by a massive number of rays on a graphics processing unit (GPU) incurs a significant amount of irregular memory traffic, which turns out to be a major stumbling block for real-time performance. In this work, a scheduling mechanism, so-called Agglomerative Memory and Thread Scheduling, is proposed to unleash the inherence parallelism in the ray-tracing process on GPUs. It is associated with a tile-based ray-tracing framework in which the acceleration structure (i.e., KD-tree in this work) is partitioned into subtrees that can be completely loaded into the on-chip L1 cache inside a streaming multiprocessor. An effective scheduling mechanism collects threads with regard to the subtrees hit by their respective rays and regroup threads into warps for dispatching. In addition, subtrees are dynamically preloaded into the L1 cache of multiprocessors in an on-demand fashion. The proposed scheduler can be integrated on today&#x2019;s high-end GPUs with only minor overhead. Microarchitecture simulation results prove that the proposed framework significantly improves memory efficiency and outperforms a traditional GPU microarchitecture by 47.4&#x0025; for average.

Investigating memory prefetcher performance over parallel applications: From real to simulated

AbstractMemory prefetcher algorithms are widely used in processors to mitigate the performance gap between the processors and the memory subsystem. The complexities behind the architectures and prefetcher algorithms, however, not only hinder the development of accurate architecture simulators, but also hinder understanding the prefetcher's contribution to performance, on both a real hardware and in a simulated environment. In this paper, we contribute to shed light on the memory prefetcher's role in the performance of parallel High‐Performance Computing applications, considering the prefetcher algorithms offered by both the real hardware and the simulators. We performed a careful experimental investigation, executing the NAS parallel benchmark (NPB) on a real Skylake machine, and as well in a simulated environment with the ZSim and Sniper simulators, taking into account the prefetcher algorithms offered by both Skylake and the simulators. Our experimental results show that: (i) prefetching from the L3 to L2 cache presents better performance gains, (ii) the memory contention in the parallel execution constrains the prefetcher's effect, (iii) Skylake's parallel memory contention is poorly simulated by ZSim and Sniper, and (iv) Skylake's noninclusive L3 cache hinders the accurate simulation of NPB with the Sniper's prefetchers.

Binned k-d Tree Construction for Sparse Volume Data on Multi-Core and GPU Systems.

While k-d trees are known to be effective for spatial indexing of sparse 3-d volume data, full reconstruction, e.g. due to changes to the alpha transfer function during rendering, is usually a costly operation with this hierarchical data structure. In a recent publication we showed how to port a clever state of the art k-d tree construction algorithm to a multi-core CPU architecture and by means of thorough optimization we were able to obtain interactive reconstruction rates for moderately sized to large data sets. The construction scheme is based on maintaining partial summed-volume tables that fit in the L1 cache of the multi-core CPU and that allow for fast occupancy queries. In this work we propose a GPU implementation of the parallel k-d tree construction algorithm and compare it with the original multi-core CPU implementation. We conduct a thorough comparative study that outlines performance and scalability of our implementation.

Monolithic 3D-Based SRAM/MRAM Hybrid Memory for an Energy-Efficient Unified L2 TLB-Cache Architecture

Monolithic 3D (M3D) integration has been emerged as a promising technology for fine-grained 3D stacking. As the M3D integration offers extremely small dimension of via in a nanometer-scale, it is beneficial for small microarchitectural blocks such as caches, register files, translation look-aside buffers (TLBs), etc. However, since the M3D integration requires low-temperature process for stacked layers, it causes lower performance for stacked transistors compared to the conventional 2D process. In contrast, non-volatile memory (NVM) such as magnetic RAM (MRAM) is originally fabricated at a low temperature, which enables the M3D integration without performance degradation. In this paper, we propose an energy-efficient unified L2 TLB-cache architecture exploiting M3D-based SRAM/MRAM hybrid memory. Since the M3D-based SRAM/MRAM hybrid memory consumes much smaller energy than the conventional 2D SRAM-only memory and 2D SRAM/MRAM hybrid memory, while providing comparable performance, our proposed architecture improves energy efficiency significantly. Especially, as our proposed architecture changes the memory partitioning of the unified L2 TLB-cache depending on the L2 cache miss rate, it maximizes the energy efficiency for parallel workloads suffering extremely high L2 cache miss rate. According to our analysis using PARSEC benchmark applications, our proposed architecture reduces the energy consumption of L2 TLB + L2 cache by up to 97.7% (53.6% on average), compared to the baseline with the 2D SRAM-only memory, with negligible impact on performance. Furthermore, our proposed technique reduces the memory access energy consumption by up to 32.8% (10.9% on average), by reducing memory accesses due to TLB misses.

Analytical Modeling the Multi-Core Shared Cache Behavior With Considerations of Data-Sharing and Coherence

To mitigate the ever worsening “Power wall” and “Memory wall” problems, multi-core architectures with multi-level cache hierarchies have been widely accepted in modern processors. However, the complexity of the architectures makes modeling of shared caches extremely complex. In this article, we propose a data-sharing aware analytical model for estimating the miss rates of the downstream shared cache under multi-core scenarios. To avoid time-consuming full simulations of the cache architecture required by conventional approaches, the proposed model can also be integrated with our refined upstream cache analytical model, which also evaluates coherence misses with similar accuracies of state-of-the-art approach with only one tenth time overhead. We validate our analytical model against gem5 simulation results under 13 applications from PARSEC 2.1 benchmark suites. Compared to the results from gem5 simulations under 8 hardware configurations including dual-core and quad-core architectures, the average absolute error of the predicted shared L2 cache miss rates is less than 2% for all configurations. After integrated with the refined upstream model with coherence misses, the overall average absolute error in 4 hardware configurations is degraded to 4.82% due to the error accumulations. As an application case of the integrated model, we also evaluate the miss rates of 57 different multi-core and multi-level cache configurations.

COPE

Prefetching helps in reducing the memory access latency in multi-banked NUCA architecture, where the Last Level Cache (LLC) is shared. In such systems, an application running on core generates significant traffic on the shared resources, the underlying network and LLC. While prefetching helps to increase application performance, but an inaccurate prefetcher can cause harm by generating unwanted traffic that additionally increases network and LLC contention. Increased network contention results in untimely prefetching of cache blocks, thereby reducing the effectiveness of a prefetcher. Prefetch accuracy is extensively used to reduce unwanted prefetches that can mitigate the prefetcher caused contention. However, the conventional prefetch accuracy parameter has major limitations in NUCA architectures. The article exposes that prefetch accuracy can create two major false-positive cases of prefetching, Under-estimation and Over-estimation problems, and false feedback loop that can mislead a prefetcher in generating more unwanted traffic. We propose a novel technique, Coordinated Prefetching for Efficient (COPE), which addresses these issues by redefining prefetch accuracy for such architectures and identifies additional parameters that can avoid generating unwanted prefetch requests. Experiment conducted using PARSEC benchmark on a 64-core system shows that COPE achieve 3% reduction in L1 cache miss rate, 12.64% improvement in IPC, 23.2% reduction in average packet latency and 18.56% reduction in dynamic power consumption of the underlying network.

Distributed Shared Mamory for NoC-based MPSoCs

As the number of cores and functionalities integrated in embedded devices increases, the amount of memory used on these devices also increases, justifying the development of memory architectures that present scalability, low energy consumption and low latency. Two factors impact the scalability of MPSoC systems: cache access latency and energy consumption. The distance of the processors to the cache banks and the cache coherence protocol influences both factors. This work proposes a physically distributed data L2 cache as the cache architecture for a NoC-based MPSoC, because it allows the concept of clustering, and the implementation of data migration algorithms to reducing cache access latency. Results show that the number of cycles required to execute a given application might reduce 23% with the appropriate number of L2 cache banks. Also, a directory-based cache coherence protocol was implemented, exploiting the NoC physical services to improve performance. Results show a reduction of 17% in the number of clock cycles and a reduction up to 86% (average reduction: 39%) in energy consumption for some cache transactions.

DTaPO: Dynamic Thermal-Aware Performance Optimization for Dark Silicon Many-Core Systems

Future many-core systems need to handle high power density and chip temperature effectively. Some cores in many-core systems need to be turned off or ‘dark’ to manage chip power and thermal density. This phenomenon is also known as the dark silicon problem. This problem prevents many-core systems from utilizing and gaining improved performance from a large number of processing cores. This paper presents a dynamic thermal-aware performance optimization of dark silicon many-core systems (DTaPO) technique for optimizing dark silicon a many-core system performance under temperature constraint. The proposed technique utilizes both task migration and dynamic voltage frequency scaling (DVFS) for optimizing the performance of a many-core system while keeping system temperature in a safe operating limit. Task migration puts hot cores in low-power states and moves tasks to cooler dark cores to aggressively reduce chip temperature while maintaining high overall system performance. To reduce task migration overhead due to cold start, the source core (i.e., active core) keeps its L2 cache content during the initial migration phase. The destination core (i.e., dark core) can access it to reduce the impact of cold start misses. Moreover, the proposed technique limits tasks migration among cores that share the last level cache (LLC). In the case of major thermal violation and no cooler cores being available, DVFS is used to reduce the hot cores temperature gradually by reducing their frequency. Experimental results for different threshold temperatures show that DTaPO can keep the average system temperature below the thermal limit. Affirmatively, the execution time penalty is reduced by up to 18% compared with using only DVFS for all thermal thresholds. Moreover, the average peak temperature is reduced by up to 10.8°C. In addition, the experimental results show that DTaPO improves the system’s performance by up to 80% compared to optimal sprinting patterns (OSP) and reduces the temperature by up to 13.6°C.

Approximate Cache in GPGPUs

There is a growing number of application domains ranging from multimedia to machine learning where a certain level of inexactness can be tolerated. For these applications, approximate computing is an effective technique that trades off some loss in output data integrity for energy and/or performance gains. In this article, we present the approximate cache, which approximates similar values and saves energy in the L2 cache of general-purpose graphics processing units (GPGPUs). The L2 cache is a critical component in memory hierarchy of GPGPUs, as it accommodates data of thousands of simultaneously executing threads. Simply increasing the size of the L2 cache is not a viable solution to keep up with the growing size of data in many-core applications. This work is motivated by the observation that threads within a warp write values into memory that are arithmetically similar. We exploit this property and propose a low-cost and implementation-efficient hardware to trade off accuracy for energy. The approximate cache identifies similar values during the runtime and allows only one thread writes into the cache in the event of similarity. Since the approximate cache is able to pack more data in a smaller space, it enables downsizing of the data array with negligible impact on cache misses and lower-level memory. The approximate cache reduces both dynamic and static energy. By storing data of a thread into a cache block, each memory instruction requires accessing fewer cache cells, thus reducing dynamic energy. In addition, the approximate cache increases frequency of bank idleness. By power gating idle banks, static energy is reduced. Our evaluations reveal that the approximate cache reduces energy by 52% with minimal quality degradation while maintaining performance of a diverse set of GPGPU applications.

IBM z15: Physical design improvements to significantly increase content in the same technology

The IBM Z processor continues to improve over previous System Z processors, but for the first time it does so without a technology improvement as the baseline enabler. The IBM z15 was designed in the same 14-nm High-Performance GLOBALFOUNDRIES technology as the IBM z14 and yet still added 20% more cores, doubled the L3 cache, and increased the L2 cache by a third while also adding a third peripheral component interconnect express (PCIe) port to the chip and an elliptic curve cryptography engine into each core. This article discusses the design, tool, and methodology enhancements required to increase the design content so significantly while maintaining the chip size and power limits from the previous z14 design. This article also discusses other design and methodology improvements that were made possible via the deeper understanding of the technology and how to more fully leverage it in a second generation.

An analysis of concurrency control protocols for in-memory databases with CCBench

This paper presents yet another concurrency control analysis platform, CCBench. CCBench supports seven protocols (Silo, TicToc, MOCC, Cicada, SI, SI with latch-free SSN, 2PL) and seven versatile optimization methods and enables the configuration of seven workload parameters. We analyzed the protocols and optimization methods using various workload parameters and a thread count of 224. Previous studies focused on thread scalability and did not explore the space analyzed here. We classified the optimization methods on the basis of three performance factors: CPU cache, delay on conflict, and version lifetime. Analyses using CCBench and 224 threads, produced six insights. The performance of optimistic concurrency control protocol for a read-only workload rapidly degrades as cardinality increases even without L3 cache misses. (I2) Silo can outperform TicToc for some write-intensive workloads by using invisible reads optimization. (I3) The effectiveness of two approaches to coping with conflict (wait and no-wait) depends on the situation. (I4) OCC reads the same record two or more times if a concurrent transaction interruption occurs, which can improve performance. (I5) Mixing different implementations is inappropriate for deep analysis. (I6) Even a state-of-the-art garbage collection method cannot improve the performance of multi-version protocols if there is a single long transaction mixed into the workload. On the basis of I4, we defined the read phase extension optimization in which an artificial delay is added to the read phase. On the basis of I6, we defined the aggressive garbage collection optimization in which even visible versions are collected. The code for CCBench and all the data in this paper are available online at GitHub.

Mach-RT: A Many Chip Architecture for High Performance Ray Tracing.

Data movement, particularly access to the main memory, has been the bottleneck of most computing problems. Ray tracing is no exception. We propose an unconventional solution that combines a ray ordering scheme that minimizes access to the scene data with a large on-chip buffer acting as near-compute storage that is spread over multiple chips. We demonstrate the effectiveness of our approach by introducing Mach-RT (Many chip - Ray Tracing), a new hardware architecture for accelerating ray tracing. Extending the concept of dual streaming, we optimize the main memory accesses to a level that allows the same memory system to service multiple processor chips at the same time. While a multiple chip solution might seem to imply increased energy consumption as well, because of the reduced memory traffic we are able to demonstrate, performance increases while maintaining reasonable energy usage compared to academic and commercial architectures. This article extends our previous work E. Vasiou, K. Shkurko, E. Brunvand, and C. Yuksel, "Mach-RT: A many chip architecture for high-performance ray tracing," in Proc. High-Perform. Graph. Conf., 2019 with design space exploration of the L3 cache size, more detailed evaluation of energy and memory performance, a discussion of energy delay product, and a brief exploration of boards with 16 chips. We also introduce new treelet enqueueing logic for the predictive scheduler.

Energy-Efficient GPU L2 Cache Design Using Instruction-Level Data Locality Similarity

This article presents a novel energy-efficient cache design for massively parallel, throughput-oriented architectures like GPUs. Unlike L1 data cache on modern GPUs, L2 cache shared by all of the streaming multiprocessors is not the primary performance bottleneck, but it does consume a large amount of chip energy. We observe that L2 cache is significantly underutilized by spending 95.6% of the time storing useless data. If such “dead time” on L2 is identified and reduced, L2’s energy efficiency can be drastically improved. Fortunately, we discover that the SIMT programming model of GPUs provides a unique feature among threads: instruction-level data locality similarity, which can be used to accurately predict the data re-reference counts at L2 cache block level. We propose a simple design that leverages this Lo cality S imilarity to build an energy-efficient GPU L2 Cache , named LoSCache . Specifically, LoSCache uses the data locality information from a small group of cooperative thread arrays to dynamically predict the L2-level data re-reference counts of the remaining cooperative thread arrays. After that, specific L2 cache lines can be powered off if they are predicted to be “dead” after certain accesses. Experimental results on a wide range of applications demonstrate that our proposed design can significantly reduce the L2 cache energy by an average of 64% with only 0.5% performance loss. In addition, LoSCache is cost effective, independent of the scheduling policies, and compatible with the state-of-the-art L1 cache designs for additional energy savings.

Comparative Analysis of Processor-FPGA Communication Performance in Low-Cost FPSoCs

Field-programmable system-on-chip (FPSoC) devices, combining high-performance processors and FPGA fabric in the same chip, are currently a leading technology in the design of complex digital systems. Since design times are longer than those of systems based on graphic processing units or standalone processors, many efforts are being devoted to develope efficient compilers from high-level languages. Even though, efficient processor-FPGA communication is still an important open issue. To contribute to this area, this article presents an extensive characterization of the processor-FPGA communication delays in Zynq-7000 devices. Although partial analyses of communication performance in these devices have been reported, this is the first work to address very important issues such as the use of DMA for data transfers or the effect of L2 cache controller settings and external RAM controller settings. As a result, data transfer rates are analyzed considering all parameters that influence them. The performance of Zynq-7000 devices is also compared to that of Cyclone V devices, hence covering the two most important current families that dominate the FPSoC market. This information is of utmost importance for designers to optimize processor-FPGA communication and, in turn, the performance of their FPSoC-based systems.

ECC-United Cache: Maximizing Efficiency of Error Detection/Correction Codes in Associative Cache Memories

Error Detection/Correction Codes (EDCs/ECCs) are the most conventional approaches to protect on-chip caches against radiation-induced soft errors. The overhead of EDCs/ECCs is a major concern and is of decisive importance when a higher protection capability is required to tolerate multiple adjacent bit errors (burst errors). This article proposes the ECC-United Cache (EUC) architecture to improve the efficiency of EDCs/ECCs in set-associative L1 caches. EUC architecture extends the data protection granularity from a single word to multiple words by exploiting the parallel cache lines access, which is inherently available in the cache. As compared with the conventional architecture, EUC can be configured to provide: 1) the same protection capability with a significantly lower overhead, 2) a significantly higher protection capability with the same number of check bits, or 3) a trade-off between the former two features. Simulation results show that, when configured to minimize the overhead, EUC reduces the number of check bits by 69 and 75 percent in data-cache and instruction-cache, respectively. When configured to maximize the protection capability, EUC provides fourfold higher burst error detection/correction capability. Moreover, EUC is orthogonal to previous protection schemes and they can be redesigned based on the EUC architecture to further improve their efficiency.

Execution Model to Reduce the Interference of Shared Memory in ARINC 653 Compliant Multicore RTOS

Multicore architecture is applied to contemporary avionics systems to deal with complex tasks. However, multicore architectures can cause interference by contention because the cores share hardware resources. This interference reduces the predictable execution time of safety-critical systems, such as avionics systems. To reduce this interference, methods of separating hardware resources or limiting capacity by core have been proposed. Existing studies have modified kernels to control hardware resources. Additionally, an execution model has been proposed that can reduce interference by adjusting the execution order of tasks without software modification. Avionics systems require several rigorous software verification procedures. Therefore, modifying existing software can be costly and time-consuming. In this work, we propose a method to apply execution models proposed in existing studies without modifying commercial real-time operating systems. We implemented the time-division multiple access (TDMA) and acquisition execution restitution (AER) execution models with pseudo-partition and message queuing on VxWorks 653. Moreover, we propose a multi-TDMA model considering the characteristics of the target hardware. For the interference analysis, we measured the L1 and L2 cache misses and the number of main memory requests. We demonstrated that the interference caused by memory sharing was reduced by at least 60% in the execution model. In particular, multi-TDMA doubled utilization compared to TDMA and also reduced the execution time by 20% compared to the AER model.