Prefetch Engine Research Articles

PP-Bridge: Establishing a Bridge between the Prefetching and Cache Partitioning

— Modern computer processors are equipped with multiple cores, each boasting its own dedicated cache memory, while collectively sharing a generously sized Last Level Cache (LLC). To ensure equitable utilization of the LLC space and bolster system security, partitioning techniques have been introduced to allocate the shared LLC space among the applications running on different cores. This partition dynamically adapts to the requirements of these applications. Prefetching plays a vital role in enhancing cache performance by proactively loading data into the cache before it get requested explicitly by a core. Each core employs prefetch engines to decide which data blocks to fetch preemptively. However, a haphazard prefetcher may bring in more data blocks than necessary, leading to cache pollution and a subsequent degradation in system performance. To maximize the benefits of prefetching, it is essential to keep cache pollution to a minimum. Intriguingly, our research has uncovered that when existing prefetching techniques are combined with partitioning methods, they tend to exacerbate cache pollution within the LLC, resulting in a noticeable decline in system performance. In this paper, we present a novel approach aimed at mitigating cache pollution when combining prefetching with partitioning techniques.

Adaptive Regression Prefetching Algorithm by Using Big Data Application Characteristics

This paper presents an innovative prefetching algorithm for a hybrid main memory structure, which consists of DRAM and phase-change memory. To enhance the efficiency of hybrid memory hardware in serving big data technologies, the proposed design employs an application-adaptive algorithm based on big data execution characteristics. Specifically optimized for graph-processing applications, which exhibit complex and irregular memory access patterns, a dual prefetching scheme is proposed. This scheme comprises a fast-response model with low-cost algorithms for regular memory access patterns and an intelligent model based on an adaptive Gaussian-kernel-based machine-learning prefetch engine. The intelligent model can acquire knowledge from real-time data samples, capturing distinct memory access patterns via an adaptive Gaussian-kernel-based regression algorithm. These methods allow the model to self-adjust its hyperparameters at runtime, facilitating the implementation of locally weighted regression (LWR) for the Gaussian process of irregular access patterns. In addition, we introduced an efficient hybrid main memory architecture that integrates two different kinds of memory technologies, including DRAM and PCM, providing cost and energy efficiency over a DRAM-only memory structure. Based on the simulation-based experimental results, our proposed model achieved performance enhancement of 57% compared to the conventional DRAM model and of approximately 12% compared to existing prefetcher-based models.

HealthFetch: An Influence-Based, Context-Aware Prefetch Scheme in Citizen-Centered Health Storage Clouds

Over the past few years, increasing attention has been given to the health sector and the integration of new technologies into it. Cloud computing and storage clouds have become essentially state of the art solutions for other major areas and have started to rapidly make their presence powerful in the health sector as well. More and more companies are working toward a future that will allow healthcare professionals to engage more with such infrastructures, enabling them a vast number of possibilities. While this is a very important step, less attention has been given to the citizens. For this reason, in this paper, a citizen-centered storage cloud solution is proposed that will allow citizens to hold their health data in their own hands while also enabling the exchange of these data with healthcare professionals during emergency situations. Not only that, in order to reduce the health data transmission delay, a novel context-aware prefetch engine enriched with deep learning capabilities is proposed. The proposed prefetch scheme, along with the proposed storage cloud, is put under a two-fold evaluation in several deployment and usage scenarios in order to examine its performance with respect to the data transmission times, while also evaluating its outcomes compared to other state of the art solutions. The results show that the proposed solution shows significant improvement of the download speed when compared with the storage cloud, especially when large data are exchanged. In addition, the results of the proposed scheme evaluation depict that the proposed scheme improves the overall predictions, considering the coefficient of determination (R2 > 0.94) and the mean of errors (RMSE < 1), while also reducing the training data by 12%.

Dynamic recognition prefetch engine for DRAM-PCM hybrid main memory

Dynamic recognition prefetch engine for DRAM-PCM hybrid main memory

Adaptive Granularity Based Last-Level Cache Prefetching Method with eDRAM Prefetch Buffer for Graph Processing Applications

The emergence of big data processing and machine learning has triggered the exponential growth of the working set sizes of applications. In addition, several modern applications are memory intensive with irregular memory access patterns. Therefore, we propose the concept of adaptive granularities to develop a prefetching methodology for analyzing memory access patterns based on a wider granularity concept that entails both cache lines and page granularity. The proposed prefetching module resides in the last-level cache (LLC) to handle the large working set of memory-intensive workloads. Additionally, to support memory access streams with variable intervals, we introduced an embedded-DRAM based LLC prefetch buffer that consists of three granularity-based prefetch engines and an access history table. By adaptively changing the granularity window for analyzing memory streams, the proposed model can swiftly and appropriately determine the stride of memory addresses to generate hidden delta chains from irregular memory access patterns. The proposed model achieves 18% and 15% improvements in terms of energy consumption and execution time compared to global history buffer and best offset prefetchers, respectively. In addition, our model reduced the total execution time and energy consumption by approximately 6% and 2.3%, compared to those of the Markov prefetcher and variable-length delta prefetcher.

Bandwidth-Aware Dynamic Prefetch Configuration for IBM POWER8

Advanced hardware prefetch engines are being integrated in current high-performance processors. Prefetching can boost the performance of most applications, however, the induced bandwidth consumption can lead the system to a high contention for main memory bandwidth, which is a scarce resource in current multicores. In such a case, the system performance can be severely damaged. This article characterizes the applications’ behavior in an IBM POWER8 machine, which presents many prefetch settings, varying the bandwidth contention. The study reveals that the best prefetch setting for each application depends on the main memory bandwidth availability, that is, it depends on the co-running applications. Based on this study, we propose Bandwidth-Aware Prefetch Configuration (BAPC) a scalable adaptive prefetching algorithm that improves the performance of multi-program workloads. BAPC increases the performance of the applications in a 12, 15, and 16 percent of 6-, 8-, and 10-application workloads over the IBM POWER8 default configuration. In addition, BAPC reduces bandwidth consumption in 39, 42, and 45 percent, respectively.

Access pattern-based high-performance main memory system for graph processing on single machines

With the increasing complexity of graph structures, the current real-world large-scale graphs are being represented by a considerable amount of vertex and edge data. Furthermore, the analysis of a large number of computing nodes has become a very complicated job that requires a large amount of hardware resources. Moreover, in large-scale graph processing, the vertex and edge data show random and sequential memory access patterns at the same time, and this is a major bottleneck in graph processing. In this paper, we present a high-capacity main memory system with an intelligent pattern-aware prefetching engine to overcome the scalability problem and the memory inefficiency of single-machine graph processing. The proposed intelligent pattern-aware prefetching engine is designed to predict and handle sequential or regular patterns and random-access patterns simultaneously. Experimental results demonstrated that the proposed model exhibited performance improvements of 60% over conventional DRAM models, approximately 40% over the existing prefetch models, and about 12.5% over the latest prefetch models.

A self-learning pattern adaptive prefetching method for big data applications

In this study, we designed a KM-Cluster-based pattern adaptive prefetching mechanism for the last-level cache structure to support real time big data management. The goal is to predict future memory access patterns aggressively and accurately through a new self-learning prefetching engine model. The pattern adaptive last-level cache consisted of three major parts: the last-level cache, the first-level prefetching buffer (FLPB) and the second-level prefetching buffer (SLPB). The SLPB efficiently manages the history records of cache blocks evicted from the last-level cache through a self-learning mechanism. A K-means clustering algorithm is used as an SLPB prefetching scheme. Hybrid main memory is constructed using a small portion of the DRAM buffer space and primarily NAND-Flash memory space. The overall performance of our proposed model is evaluated for OpenStack Swift and in-memory database application-Redis. Experimental results show that the proposed architecture reduces the total execution time by 20.96% and power consumption by 31.9% compared to the same last-level cache size with no SLPB structure.

Adaptive Runtime-Assisted Block Prefetching on Chip-Multiprocessors

Memory stalls are a significant source of performance degradation in modern processors. Data prefetching is a widely adopted and well studied technique used to alleviate this problem. Prefetching can be performed by the hardware, or be initiated and controlled by software. Among software controlled prefetching we find a wide variety of schemes, including runtime-directed prefetching and more specifically runtime-directed block prefetching. This paper proposes a hybrid prefetching mechanism that integrates a software driven block prefetcher with existing hardware prefetching techniques. Our runtime-assisted software prefetcher brings large blocks of data on-chip with the support of a low cost hardware engine, and synergizes with existing hardware prefetchers that manage locality at a finer granularity. The runtime system that drives the prefetch engine dynamically selects which cache to prefetch to. Our evaluation on a set of scientific benchmarks obtains a maximum speed up of 32 and 10 % on average compared to a baseline with hardware prefetching only. As a result, we also achieve a reduction of up to 18 and 3 % on average in energy-to-solution.

Extreme Web Caching for Faster Web Browsing

Modern web pages are very complex; each web page consists of hundreds of objects that are linked from various servers all over the world. While mechanisms such as caching reduce the overall number of end-to-end requests saving bandwidth and loading time, there is still a large portion of content that is re-fetched -- despite not having changed. In this demo, we present Extreme Cache, a web caching architecture that enhances the web browsing experience through a smart pre-fetching engine. Our extreme cache tries to predict the rate of change of web page objects to bring cacheable content closer to the user.

Adaptive Prefetching on POWER7

Hardware data prefetch engines are integral parts of many general purpose server-class microprocessors in the field today. Some prefetch engines allow users to change some of their parameters. But, the prefetcher is usually enabled in a default configuration during system bring-up, and dynamic reconfiguration of the prefetch engine is not an autonomic feature of current machines. Conceptually, however, it is easy to infer that commonly used prefetch algorithms---when applied in a fixed mode---will not help performance in many cases. In fact, they may actually degrade performance due to useless bus bandwidth consumption and cache pollution, which in turn, will also waste power. We present an adaptive prefetch scheme that dynamically modifies the prefetch settings in order to adapt to workloads' requirements. We use a commercial processor, namely the IBM POWER7 as a vehicle for our study. First we characterize---in terms of performance and power consumption---the prefetcher in that processor using microbenchmarks and SPEC CPU2006. We then present our adaptive prefetch mechanism showing performance improvements with respect to the default prefetch setting up to 2.7X and 1.3X for single-threaded and multiprogrammed workloads, respectively. Adaptive prefetching is also able to reduce power consumption in some cases. Finally, we also evaluate our mechanism with SPECjbb2005, improving both performance and power consumption.

Co-design of the IBM Blue Gene/Q Level 1 prefetch engine with QCD

In order to optimize performance of the IBM Blue Gene®/Q memory system for real scientific applications, the VHDL (Very-high-speed integrated circuits Hardware Description Language) design and optimization of the Level 1 prefetch engine were undertaken in a co-design exercise with assembler kernels that form the critical loops in quantum chromodynamics delivering improved performance for both.

Location-based Application based on Prefetching on Mobile Applications: Case Study of Finding Interesting Restaurants

this work, we are interested in developing a mobile application which locates interesting points for a user. The interesting points are displayed in a map based on Google Map. We particularly consider the prediction of future points based on the user location, speed and direction. The data prefetching buffer is being used to store the future data. The prefetching engine is integrated in the mobile client.

ABS

Hardware data prefetch is a very well known technique for hiding memory latencies. However, in a multicore system fitted with a shared Last-Level Cache (LLC), prefetch induced by a core consumes common resources such as shared cache space and main memory bandwidth. This may degrade the performance of other cores and even the overall system performance unless the prefetch aggressiveness of each core is controlled from a system standpoint. On the other hand, LLCs in commercial chip multiprocessors are more and more frequently organized in independent banks. In this contribution, we target for the first time prefetch in a banked LLC organization and propose ABS, a low-cost controller with a hill-climbing approach that runs stand-alone at each LLC bank without requiring inter-bank communication. Using multiprogrammed SPEC2K6 workloads, our analysis shows that the mechanism improves both user-oriented metrics (Harmonic Mean of Speedups by 27% and Fairness by 11%) and system-oriented metrics (Weighted Speedup increases 22% and Memory Bandwidth Consumption decreases 14%) over an eight-core baseline system that uses aggressive sequential prefetch with a fixed degree. Similar conclusions can be drawn by varying the number of cores or the LLC size, when running parallel applications, or when other prefetch engines are controlled.

Energy-Efficient Hardware Data Prefetching

Extensive research has been done in prefetching techniques that hide memory latency in microprocessors leading to performance improvements. However, the energy aspect of prefetching is relatively unknown. While aggressive prefetching techniques often help to improve performance, they increase energy consumption by as much as 30% in the memory system. This paper provides a detailed evaluation on the energy impact of hardware data prefetching and then presents a set of new energy-aware techniques to overcome prefetching energy overhead of such schemes. These include compiler-assisted and hardware-based energy-aware techniques and a new power-aware prefetch engine that can reduce hardware prefetching related energy consumption by 7-11 ×. Combined with the effect of leakage energy reduction due to performance improvement, the total energy consumption for the memory system after the application of these techniques can be up to 12% less than the baseline with no prefetching.

Using current web page structure to improve prefetching performance

Web prefetching is a technique aimed at reducing user-perceived latencies in the World Wide Web. The spatial locality shown by user accesses makes it possible to predict future accesses from the previous ones. A prefetching engine uses these predictions to prefetch web objects before the user demands them. The existing prediction algorithms achieved an acceptable performance when they were proposed but the high increase in the number of embedded objects per page has reduced their effectiveness in the current web. In this paper, we show that most of the predictions made by the existing algorithms are not useful to reduce the user-perceived latency because these algorithms do not take into account the structure of the current web pages, i.e., an HTML object with several embedded objects. Thus, they predict the accesses to the embedded objects in an HTML after reading the HTML itself. For this reason, the prediction is not made early enough to prefetch the objects and, therefore, there is no latency reduction. In this paper we present the double dependency graph (DDG) algorithm that distinguishes between container objects (HTML) and embedded objects to create a new prediction model according to the structure of the current web. Results show that, for the same number of extra requests to the server, DDG reduces the perceived latency, on average, 40% more than the existing algorithms. Moreover, DDG distributes latency reductions more homogeneously among users.

An immune inspired co-evolutionary affinity network for prefetching of distributed object

In distributed object computing (DOC) containers, cache strategy as a passive approach improves the performance by caching the recently accessed objects. By the advent of large-scale enterprise applications of DOC, caching methods fail to keep pace with the increasing importance of performance and the increasing scale of DOC system. Prefetching is an effective approach to improve the performance. It generates and stores pages or objects in caches in advance, by predicting the requests. In the current DOC container, prefetching strategy is not supported and seldom studied in the literature. The immune system with faster secondary response and its affinity network inspire applying the immune mechanisms to build holistic model of DOC for performance improvement. In this study, a co-evolutionary affinity network (CEA-Net) is proposed for prefetching distributed objects. In CEA-Net, objects are antibodies and computation tasks are antigens. Invocation relations among object classes are modeled by the immune network of antibody genotypes. Multiple affinity measures among antibody, antigen, genotypes of antibody and antigen, genotype set and antibody population are defined to model the complex relations among distributed objects. Especially in the antibody population, immune principles including clonal proliferation, immune memory, immune toleration and elimination are designed to add evolutionary features to the antibody population. Based on CEA-Net, the prefetching architecture for DOC is built including 5 main procedures, Network Abstractor, Access Recorder, Object Factory, Cache Engine and Prefetch Engine. Finally, the experimental study shows the promising access performance and the evolutionary features of CEA-Net. CEA-Net is instructive for the future design of high performance DOC containers, such as WebSphere Application Server and BEA WebLogic Application Server.

A Rule-Based Prefetching Approach for Object-Based Storage Device

Capturing the advantages of NAS and SAN, Object-based Storage System (OBSS) is becoming an active research area. OBSS is composed of clients, metadata server, and Object-based Storage Devices (OSDs). Similar to the prefetching of file and block, object prefetching is also an effective method to reduce the object access latency in OSD. However, the existing methods focus on intra-object prefetching [ 1 ] and they limit the prefetching capability among objects to a certain extent. In this paper, we present a novel inter-object prefetching approach which considers the characteristics of the object access in OSD. This approach adopts a kind of data mining technique called sequential pattern mining to find the object access sequential patterns [ 2 ], then produces the object sequential rules according to the patterns, and finally prefetches objects according to the rules and the object access context. The whole process includes the following three modules: Object Request Service Module. It maintains a current object access context, which is a l-length recently accessed object sequence. Furthermore, it appends new object information to the historic object access stream when an object is accessed. Sequential Rules Mining Module. It is composed of the object sequences preprocessing, the sequential pattern mining and the sequential rules producing modules. The preprocessing module divides the object access stream into many subsequences. Base on the discovered object sequential patterns gained by the sequential pattern mining module, object access sequential rules table is constructed. Prefetching Engine Module. When the current requested object is not in the object cache, the prefetching engine selects a matched rule with the highest confidence from the rules table according to the context, then chooses the corresponding objects and preloads them into the object cache. Simulated experimental results show that both prediction accuracy and object hit rate can be improved comparing with the Group-based prefetching method [ 3 ] which can also be used for object prefetching.

Runtime Engine for Dynamic Profile Guided Stride Prefetching

Stride prefetching is recognized as an important technique to improve memory access performance. The prior work usually profiles and/or analyzes the program behavior offline, and uses the identified stride patterns to guide the compilation process by injecting the prefetch instructions at appropriate places. There are some researches trying to enable stride prefetching in runtime systems with online profiling, but they either cannot discover cross-procedural prefetch opportunity, or require special supports in hardware or garbage collection. In this paper, we present a prefetch engine for JVM (Java Virtual Machine). It firstly identifies the candidate load operations during just-in-time (JIT) compilation, and then instruments the compiled code to profile the addresses of those loads. The runtime profile is collected in a trace buffer, which triggers a prefetch controller upon a protection fault. The prefetch controller analyzes the trace to discover any stride patterns, then modifies the compiled code to inject the prefetch instructions in place of the instrumentations. One of the major advantages of this engine is that, it can detect striding loads in any virtual code places for both regular and irregular code, not being limited with plain loop or procedure scopes. Actually we found the cross-procedural patterns take about 30% of all the prefetchings in the representative Java benchmarks. Another major advantage of the engine is that it has runtime overhead much smaller (the maximal is less than 4.0%) than the benefits it brings. Our evaluation with Apache Harmony JVM shows that the engine can achieve an average 6.2% speed-up with SPECJVM98 and DaCapo on Intel Pentium 4 platform, in spite of the runtime overhead.

Server-Based Data Push Architecture for Multi-Processor Environments

Data access delay is a major bottleneck in utilizing current high-end computing (HEC) machines. Prefetching, where data is fetched before CPU demands for it, has been considered as an effective solution to masking data access delay. However, current client-initiated prefetching strategies, where a computing processor initiates prefetching instructions, have many limitations. They do not work well for applications with complex, non-contiguous data access patterns. While technology advances continue to increase the gap between computing and data access performance, trading computing power for reducing data access delay has become a natural choice. In this paper, we present a server-based data-push approach and discuss its associated implementation mechanisms. In the server-push architecture, a dedicated server called Data Push Server (DPS) initiates and proactively pushes data closer to the client in time. Issues, such as what data to fetch, when to fetch, and how to push are studied. The SimpleScalar simulator is modified with a dedicated prefetching engine that pushes data for another processor to test DPS based prefetching. Simulation results show that L1 Cache miss rate can be reduced by up to 97% (71% on average) over a superscalar processor for SPEC CPU2000 benchmarks that have high cache miss rates.