Cache Access Research Articles

Partially shared cache and adaptive replacement algorithm for NoC-based many-core systems

Abstract The Network-on-Chip(NoC) is a promising alternative to traditional bus-based architectures that has been widely applied to interconnect multi/many-core systems due to its scalable and modular design. Undoubtedly, the memory wall problem is one of the most important challenges; however, this problem can now be somewhat be alleviated by cache subsystems. In this paper, to overcome the high resource consumption and low data-sharing rate problems of the private cache scheme, we propose a partially shared cache structure and a corresponding replacement algorithm based on a mesh NoC. In this scheme, the L2 cache is shared by each group of four cores that connected as a cluster to a given node by the local bus. To maximize the performance of this partially shared cache structure, we propose a core-aware re-reference interval prediction (CA-RRIP) replacement algorithm. The algorithm performs dynamic virtual partitioning on the partially shared cache; the core that initiated the cache access request will be given top priority when a cache area needs to be replaced or inserted. This approach guarantees cache exclusivity and can mitigate interactions among cores using different access patterns. We implement the traditional private, the proposed partially shared and the row-shared cache subsystems in our experiments. The comparisons indicate that the overall system resource occupation can be reduced by 20% with the same number of cores, and the instructions per cycle(IPC) of the system could increase by up to 49.2%. Moreover, the system throughput(STP) increased by an average of 5.89%. Our experimental results showed that the proposed CA-RRIP algorithm also reduces the average cache miss rate of the system under various cache access patterns.

Periodic learning-based region selection for energy-efficient MLC STT-RAM cache

The emerging multi-level cell (MLC) spin-transfer torque RAM (STT-RAM) is becoming one of the most promising candidates to replace SRAM as on-chip last-level caches. Compared with single-level cell (SLC) STT-RAM design, MLC cache outperforms SLC cache in terms of storage capacity. However, due to the cell design constrains, MLC STT-RAM suffers from considerably long write latency and high write energy. To explore the potential benefits of MLC STT-RAM cache, this paper proposes a scheme named periodic learning-based region selection (PLRS). We first formulate the region selection problem with greedy algorithm and then profile and collect the cache access behavior through periodic learning. Finally, PLRS will determine region selection based on the behavior information. The experimental results show that PLRS reduces dynamic energy consumption by 22.7% and reduces execution time by 16.2% on average compared to conventional MLC STT-RAM, with negligible overhead.

Design and Implementation of Cache Memory with Dual Unit Tile/Line Accessibility

In recent years, the increasing disparity between the data access speed of cache and processing speeds of processors has caused a major bottleneck in achieving high-performance 2-dimensional (2D) data processing, such as that in scientific computing and image processing. To solve this problem, this paper proposes new dual unit tile/line access cache memory based on a hierarchical hybrid Z-ordering data layout and multibank cache organization supporting skewed storage schemes. The proposed layout improves 2D data locality and reduces L1 cache misses and Translation Lookaside Buffer (TLB) misses efficiently and it is transformed from conventional raster layout by a simple hardware-based address translation unit. In addition, we proposed an aligned tile set replacement algorithm (ATSRA) for reduction of the hardware overhead in the tag memory of the proposed cache. Simulation results using Matrix Multiplication (MM) illustrated that the proposed cache with parallel unit tile/line accessibility can reduce both the L1 cache and TLB misses considerably as compared with conventional raster layout and Z-Morton order layout. The number of parallel load instructions for parallel unit tile/line access was reduced to only about one-fourth of the conventional load instruction. The execution time for parallel load instruction was reduced to about one-third of that required for conventional load instruction. By using 40 nm Complementary Metal-Oxide-Semiconductor (CMOS) technology, we combined the proposed cache with a SIMD-based data path and designed a 5 × 5 mm2 Large-Scale Integration (LSI) chip. The entire hardware overhead of the proposed ATSRA-cache was reduced to only 105% of that required for a conventional cache by using the ATSRA method.

A Novel High Performance and Energy Efficient NUCA Architecture for STT-MRAM LLCs With Thermal Consideration

As the speed gap of the modern processor and the off-chip main memory enlarges, on-chip cache capacity increases to sustain the performance scaling. As a result, the cache power occupies a large portion of the total power budget. Spin transfer torque magnetic memory (STT-MRAM) is proposed as a promising solution for the low power cache design due to its high integration density and ultralow leakage power. Nevertheless, the high write power and latency of STT-MRAM become new barriers for the commercialization of this emerging technology. In this paper, we investigate the thermal effect on the access performance of STT-MRAM, and observe that the temperature can affect the write delay and energy significantly. Then, we explore the nonuniform cache access (NUCA) design of the chip-multiprocessors with STT-MRAM-based last level cache (LLC). A thermal aware data migration policy, called “Thermosiphon,” which takes advantage of the thermal property of STT-MRAM, is proposed to reduce the LLC write energy. This policy splits the LLC into different regions dynamically based on the thermal distribution monitored by thermal sensors available on-chip, and adaptively migrates write intensive data among different thermal regions considering the thermal gradient. Compared to the conventional NUCA design, our proposed design can save 41.2% write energy at most and 13.01% on average with negligible hardware overhead.

Motion estimation using maximum sub-image and sub-pixel phase correlation on a DSP platform

Motion estimation is a key step for many video process systems, but it usually suffers from low precision in complex imaging scenarios, as well as the problem of real-time processing. This paper proposes an accurate and fast motion estimation algorithm using maximum sub-image and sub-pixel phase correlation implemented on a DSP platform. Firstly, a maximum sub-image is extracted to contain the image content as much as possible, which both satisfies the requirement of Fourier precondition and the wide coverage of complete background. And then, the extracted sub-image is down-sampled with median filter to increase the signal-noise-ratio and decrease the computation load. Secondly, a sub-pixel motion estimation is used to compensate the losing precision due to down-sampling, and keep the range of motion estimation. Finally, the proposed motion estimation algorithm is implemented on a single core TMS320C6678 DSP platform, and it is accelerated by applying multistage data cache and advanced data access. Experiments demonstrate the accuracy of the proposed motion estimation algorithm in complex scenarios. Meanwhile, it can achieve 7.4 ms/frame for sub-image with 512 × 512 pixels size and 29.6 ms/frame for sub-image with 1024 × 1024 pixels size, respectively.

A Cache Replacement Policy with Considering Global Fluctuations of Priority Values

In the high-associativity caches, the hardware overheads of the cache replacement policy become a problem. To avoid this problem, the Adaptive Demotion Policy (ADP) is proposed. The ADP focuses on the priority value demotion at a cache miss, and it can achieve a higher performance compared with conventional cache replacement policies. The ADP can be implemented with small hardware resources, and the priority value update logic can be implemented with a small hardware cost. The ADP can suit for various applications by the appropriate selection of its insertion, promotion and selection policies. If the dynamic selection of the suitable policies for the running application is possible, the performance of the cache replacement policy will be increased. In order to achieve the dynamic selection of the suitable policies, this paper focuses on the global fluctuations of the priority values. At first, the cache is partitioned into several partitions. At every cache access, the total of priority values in each partition is calculated. At every set interval, the fluctuations of total priority values in all the partitions are checked, and the information is used to detect the behavior of the application. This paper adapts this mechanism to the ADP, and the adapted cache replacement policy is called the ADP-G. The performance evaluation shows that the ADP-G achieves the MPKI reductions and the IPC improvements, compared to the LRU policy, the RRIP policy and the ADP.

Hybrid Self-Backhaul and Cache Assisted Millimeter Wave Two-Tier Heterogeneous Networks With MIMO Equipped Backhaul Access Points

This paper focuses on effective content delivering over the co-channel and full millimeter-wave (mm-Wave) heterogeneous networks (HetNets) by using hybrid self-backhaul and cache-assisted (HSBC-assisted) content delivering. Especially, the small base stations (SBSs) are equipped with a single antenna, a portion of which has the capacity-limited cache and the remaining of which has not to cache. The macro base stations (MBSs) are equipped with multiple antennas or even massive MIMO, which provide the wireless self-backhaul for the SBSs without cache. Instead of the oversimplified flat-top mm-Wave antenna pattern, a more actual and accurate one for the uniform linear array at MBSs is used. By modeling the locations of line-of-sight (LoS) and non-LoS base stations as independent Poisson point processes, this paper first presents the downlink content delivering model as well as the distributions of interferences. Then, to explore the impact of the specific MIMO self-backhaul on content delivering, a comprehensive investigation is conducted in terms of the coverage probability, achievable average area rate (AAR), average delay (AD), and energy effective (EE). The analytical and numerical results show that the utilization of multi-antenna MBSs not only improves the AAR and AD of self-backhaul transmission from MBS to SBS but also does the ones of cache and non-cache access links due to narrow beam and direction transmission decreasing interference. The results also show that in the proposed HSBC-assisted HetNets, under the scenario where the low-resolution antenna array is used and the cache capacity is large, the effect of hardware power consumption on the aggregate EE is negligible. This result shows a trade-off between AD and EE. With a projected EE, the joint utilization of cache and low-resolution antenna array allows more antennas equipped at MBSs so that the near-field propagation loss is compensated by the higher gain of directional antennas. Meanwhile, the results show that the achieved performance gain by the HSBC-assisted system over the traditional self-backhaul (TSB)-assisted greatly depends on the system parameters. With this observation, an adaptive TSB-HSBC-assisted system model is proposed. When the ratio factor of cache-assisted SBSs is less than the threshold, the sole TSB-assisted model is selected; otherwise, the HSBC-assisted model is used.

Fast and Efficient In-Memory Big Data Processing

With the passage of time, the data is growing exponentially and the mostly endured areas are social media networks, media hosting applications, and servers. They have thousands of Tera-bytes of data and the efficient systems, however, they are as yet confronting issue to oversee such volume of information and its size is growing each day. Data systems retrieve information with less time of In-memory. Instead of each factor data systems are required to define good usage of cache and fast memory access with help of optimization. The proposed technique to solve this problem can be the optimal indexing technique with better and efficient utilization of Cache and having less overhead of DRAM with the goal that energy can also be saved for the high-end servers.

MemWander: Memory Dynamic Remapping via Hypervisor Against Cache-Based Side-Channel Attacks

Current countermeasures against last level cache (LLC) based non-memory-sharing side-channel attacks (LNSA), which is a powerful and practical cache attack in the cloud, fail in practical due to their shortage of generality or efficiency. Motivated by random cache access, we propose a novel defense method called dynamic remapping which dynamically changes mapping relationship from virtual memory to cache at hypervisor layer. It aims to confuse the attacker about the relationship between observed cache activities and values of demanded secret. In order to guarantee scalability and deployability, we formalize the problem of remapping, and design a sequential classified selection algorithm to decide how to remap all those protected memory. Furthermore, we implement MemWander, a prototype system integrated in Xen and OpenStack which are popular cloud settings. And its security improvement and performance overhead are evaluated on various applications and an apache server as a simulated cloud service. The experimental results show that MemWander not only provides enough security guarantees for general services in cloud, but also induces low performance overhead as no more than 7%, which is tolerable in most scenarios.

A smart protocol-level task mapping for energy efficient traffic on network-on-chip

Previous task mapping assumes that applications directly fetch data on remote nodes and build up their energy efficient mapping based on pattern of the NoC traffic. However, the data movement is actually managed by a cache coherence mechanism that executes a much more complicated protocol under application layer to guarantee data correction. Thus, we propose an energy efficient task mapping referring the protocol layer, rather than application layer, to precisely model protocol activities and energy dissipation. By a probabilistic description of cache accessing, we find a efficient method to convert application activities to energy consumption, generating an energy evaluation as a global optimization goal. We also propose a task mapping algorithm to minimize the energy consumption by referring activity intensity of the protocol. The experimental results show that the proposed energy model achieves a precision with less than 2% error and provides a credible quantitative criterion for energy optimization of cache coherence protocols. Comparing to application-layer optimization, our task mapping can obtain 20% energy saving and 15% latency reduction on average.

Covert Timing Channels Exploiting Cache Coherence Hardware: Characterization and Defense

Information leakage of sensitive data has become one of the fast growing concerns among computer users. With adversaries turning to hardware for exploits, caches are frequently a target for timing channels since they present different timing profiles for cache miss and hit latencies. Such timing channels operate by having an adversary covertly communicate secrets to a spy simply through modulating resource timing without leaving any physical evidence. In this article, we demonstrate a new vulnerability exposed by cache coherence protocols where adversaries could manipulate the coherence states on certain cache blocks to alter cache access timing and communicate secrets illegitimately. Our threat model assumes the trojan and spy can either exploit explicitly shared read-only physical pages (e.g., shared library code), or use memory deduplication feature to implicitly force create shared physical pages. We demonstrate a template that adversaries may use to construct covert timing channels through manipulating combinations of coherence states and data placement in different caches. We investigate several classes of cache coherence protocols, and observe that both directory-based and snoopy protocols can be subject to covert timing channel attacks. We identify that the root cause of the vulnerability to be the existence of access latency difference for cache lines in read-only cache coherence states: Exlusive and Shared. For defense, we propose a slightly modified cache coherence scheme that will enable the last level cache to directly respond to read data requests in these read-only coherence states, and avoid any latency difference that could enable timing channels.

PrODACT: Prefetch-Obfuscator to Defend Against Cache Timing Channels

Cache timing channels operate stealthily through modulating the cache access latencies, and exfiltrate sensitive information to malicious adversaries. Among several forms of such timing channels, covert channels are especially dangerous since they involve two colluding processes (namely, the trojan and spy), and are often difficult to stop or prevent. In this article, we propose and demonstrate PrODACT, a low-cost mitigation mechanism using hardware prefetchers to defend against cache-based timing channels. Our detection mechanism first identifies the target cache sets that are being exploited by the adversaries, and then the counterattack mechanism fetches cache blocks to obliterate the pattern of cache accesses (misses and hits) created to construct timing channel between the trojan and the spy. We evaluate PrODACT on different classes of cache timing channel protocols that use different numbers of cache block groups for covert communication in a round-robin or parallel fashion. We observe that the cache timing channels suffer an average 50% bit error rate (with a minimum of at least 30%) which makes it very difficult or impossible for spy to decipher any useful information.

New massively parallel scheme for Incompressible Smoothed Particle Hydrodynamics (ISPH) for highly nonlinear and distorted flow

A new massively parallel scheme is developed to simulate free-surface flows with the meshless method incompressible smoothed particle hydrodynamics (ISPH) for simulations involving more than 100 million particles. As a pressure-projection method, ISPH requires the solution of a sparse matrix for the pressure Poisson equation (PPE) which is non trivial for large problems where the particles are moving with continuously evolving sparsity. The new scheme uses a Hilbert space filling curve with a cell-linked list to map the entire domain so that domain decomposition and load balancing can be achieved easily to take advantage of geometric locality in order to reduce latency in memory cache access. The computational domain can be subdivided into more than 12,000 partitions using the message passing interface (MPI) for communication between partitions. Load balancing is achieved using the open-source Zoltan library using a new particle weighting system. To solve the PPE for large problems using tens of thousands of partitions, the open-source PETSc library is used which requires the HYPRE BoomerAMG preconditioner to ensure rapid convergence for ISPH. The performance of the code is benchmarked on the U.K. National Supercomputer ARCHER. The results show that domain decomposition with a space filling curve can efficiently treat irregularly distributed particles creating a well-balanced scheme demonstrating that the approach is well matched to the highly irregular subdomains and non-uniform distribution of ISPH free-surface simulations. The benchmark results show that massively parallel ISPH code can achieve over 90% efficiency for the solution of the PPE, but the efficiency of computing matrix coefficients decreases when using more than 12000 partitions giving overall efficiencies in excess of 43% up to 6144 MPI partitions, highlighting future improvements required. The work demonstrates that the Zoltan and PETSc libraries can be effectively combined with ISPH to offer the capability of developing a massively parallel ISPH toolkit.

Coverage Performance of NOMA in Wireless Caching Networks

In order to keep a balance between the transmission delay of backhaul and the spectrum efficiency of access links, the coverage performance of wireless networks with non-orthogonal multiple access (NOMA) and content caching is studied. First, an explicit expression for the coverage probability of a typical user is presented by using stochastic geometry and order statistics. This expression can provide useful insights in order to improve the coverage performance of NOMA communications. Second, a closed-form expression for the average coverage probability is derived. Finally, simulation results are provided to validate the accuracy of the analytical framework and demonstrate the performance gain, due to NOMA.

IBM POWER9 opens up a new era of acceleration enablement: OpenCAPI

Open Coherent Accelerator Processor Interface (OpenCAPI) is a new industry-standard device interface that enables the development of host-agnostic devices that can coherently connect to any host platform that supports the OpenCAPI standard. This in turn allows such devices to coherently cache host memory to facilitate accelerator execution, perform direct memory access and atomics to host memory, send messages and interrupts to the host, and act as a host memory home agent. OpenCAPI utilizes high-frequency differential signaling technology while providing the high bandwidth and low latency needed by advanced accelerators. OpenCAPI encapsulates the serializing cache access and address translation constructs in high-speed host silicon technology to minimize overhead and design complexity in attached silicon such as field-programmable gate arrays and application-specific integrated circuits. Finally, OpenCAPI architecturally ties together transaction layer, link layer, and physical layer attributes to optimally align to high serializer/deserializer (SerDes) ratios and enable high-bandwidth, highly parallel exploitation of attached silicon.

Proactive Retention-Aware Caching With Multi-Path Routing for Wireless Edge Networks

We consider the problem of proactive retention aware caching in a heterogeneous wireless edge network consisting of mobile users accessing content from a server and associated to one or more edge caches. Our goal is to design a caching policy that minimizes the sum of content storage costs and server access costs over two design variables: the retention time of each cached content and the probability that a user routes content requests to each of its associated caches. We develop a model that captures multiple aspects such as cache storage costs and several capabilities of modern wireless technologies, such as server multicast/unicast transmissions, device multi-path routing, and cache access constraints. We formulate the problem of Proactive Retention Routing Optimization as a non-convex, non-linear mixed-integer program. We prove that it is NP-hard under both multicast/unicast modes—even when the caches have a large capacity and storage costs are linear—and develop greedy algorithms that have provable performance bounds for the case of uncapacitated caches. Finally, we propose heuristics with low computational complexity for the capacitated cache case as well as for the case of convex storage costs. Systematic evaluations based on real-world data demonstrate the effectiveness of our approach, compared to the existing caching schemes.

A Low Overhead In-Network Data Compressor for the Memory Hierarchy of Chip Multiprocessors

Data compression has been intensively studied to increase the utility of cache, network-on-chip (NoC), and main memory in energy-efficient processors. However, prior solutions to data compression often add remarkable compression and decompression delay to the critical path of memory access, which is thought as the major factor limiting its application to commodity processors. Unlike prior work that deals with memory compression or network compression separately, this paper proposes a unified on-chip distributed data compressor (DISCO), to enable near-zero-latency cache and memory block compression for chip multiprocessors adopting nonuniform cache access. DISCO integrates a multimode cache compressor into the NoC routers and overlaps the de/compression latency with the queuing delay in the network. In addition, cache block evicted to or fetched from the main memory can also be compressed or decompressed during the network queuing time in this unified DISCO compressor. With the support of congestion-awareness, it is shown in the evaluation that DISCO, which unifies the compression solution of the memory hierarchy, dramatically decreases the compression overhead of isolated techniques, and significantly boosts the efficiency of data moving and store.

SPIN

Recent GPUs enable Peer-to-Peer Direct Memory Access ( p 2 p ) from fast peripheral devices like NVMe SSDs to exclude the CPU from the data path between them for efficiency. Unfortunately, using p 2 p to access files is challenging because of the subtleties of low-level non-standard interfaces, which bypass the OS file I/O layers and may hurt system performance. Developers must possess intimate knowledge of low-level interfaces to manually handle the subtleties of data consistency and misaligned accesses. We present SPIN , which integrates p 2 p into the standard OS file I/O stack, dynamically activating p 2 p where appropriate, transparently to the user. It combines p 2 p with page cache accesses, re-enables read-ahead for sequential reads, all while maintaining standard POSIX FS consistency, portability across GPUs and SSDs, and compatibility with virtual block devices such as software RAID. We evaluate SPIN on NVIDIA and AMD GPUs using standard file I/O benchmarks, application traces, and end-to-end experiments. SPIN achieves significant performance speedups across a wide range of workloads, exceeding p 2 p throughput by up to an order of magnitude. It also boosts the performance of an aerial imagery rendering application by 2.6× by dynamically adapting to its input-dependent file access pattern, enables 3.3× higher throughput for a GPU-accelerated log server, and enables 29% faster execution for the highly optimized GPU-accelerated image collage with only 30 changed lines of code.

SRAM- and STT-RAM-based hybrid, shared last-level cache for on-chip CPU–GPU heterogeneous architectures

Shared last-level cache (LLC) in on-chip CPU–GPU heterogeneous architectures is critical to the overall system performance, since CPU and GPU applications usually show completely different characteristics on cache accesses. Therefore, when co-running with CPU applications, GPU ones can easily occupy the majority of the LLC, making CPU applications starve severely. This imposes significant challenges to the design and management of the shared LLC in CPU–GPU heterogeneous architectures. To improve the overall system performance, we consider integrating conventional SRAM and a new memory technology (i.e., STT-RAM) to enlarge the shared LLC. Furthermore, we propose comprehensive management policies to reduce the contention between CPU and GPU units. Experimental results show that, compared with the conventional SRAM-only LLC design, our proposal improves the performance of CPU workloads by 17% while not hurting GPU ones and reduces the LLC energy consumption by 30% on average.

Access Types Effect on Internet Video Services and Its Implications on CDN Caching

Video providers heavily rely on geographically distributed content distribution networks (CDNs) to place video content as close to users as possible, with an aim of improving video quality and avoiding single point of failure at the server side. The effectiveness of CDNs is mostly dependent on the content consumption patterns. Currently, video providers are offering access to content from different platforms (e.g., mobile devices and PC clients), which might result in distinct video content consumption patterns and finally affect the efficiency of CDN caching. Nevertheless, the access type effect on Internet videos is not well understood. In this paper, using a data set consisting of 26 million video requests of a large-scale commercial video-on-demand system, we study the effect of three main access types, i.e., proprietary software on PC clients, Web browser, and mobile apps. Several observations suggest that access types should be considered carefully in CDN design. In particular, the user engagement, user interests in content, and video popularity dynamics patterns, three important factors for video caching, vary remarkably in the three access types. Leveraging off our findings, we propose an access type-aware CDN caching system that associates a cache for each access type and also several optimizations, including partial caching of videos based on chunk-level caching, cross-platform read-only cache access, and prefiltering of the least popular videos. Trace-driven simulations demonstrate that the access type-aware CDN caching achieves high cache hit rate and, more importantly, greatly reduces the disk load that is measured by the number of cache replacement operations.