Prefetching Techniques Research Articles

PP-Bridge: Establishing a Bridge between the Prefetching and Cache Partitioning

— Modern computer processors are equipped with multiple cores, each boasting its own dedicated cache memory, while collectively sharing a generously sized Last Level Cache (LLC). To ensure equitable utilization of the LLC space and bolster system security, partitioning techniques have been introduced to allocate the shared LLC space among the applications running on different cores. This partition dynamically adapts to the requirements of these applications. Prefetching plays a vital role in enhancing cache performance by proactively loading data into the cache before it get requested explicitly by a core. Each core employs prefetch engines to decide which data blocks to fetch preemptively. However, a haphazard prefetcher may bring in more data blocks than necessary, leading to cache pollution and a subsequent degradation in system performance. To maximize the benefits of prefetching, it is essential to keep cache pollution to a minimum. Intriguingly, our research has uncovered that when existing prefetching techniques are combined with partitioning methods, they tend to exacerbate cache pollution within the LLC, resulting in a noticeable decline in system performance. In this paper, we present a novel approach aimed at mitigating cache pollution when combining prefetching with partitioning techniques.

Cache-oblivious Hilbert Curve-based Blocking Scheme for Matrix Transposition

This article presents a fast SIMD Hilbert space-filling curve generator, which supports a new cache-oblivious blocking-scheme technique applied to the out-of-place transposition of general matrices. Matrix operations found in high performance computing libraries are usually parameterized based on host microprocessor specifications to minimize data movement within the different levels of memory hierarchy. The performance of cache-oblivious algorithms does not rely on such parameterizations. This type of algorithm provides an elegant and portable solution to address the lack of standardization in modern-day processors. Our solution consists in an iterative blocking scheme that takes advantage of the locality-preserving properties of Hilbert space-filling curves to minimize data movement in any memory hierarchy. This scheme traverses the input matrix, in O(nm) time and space, improving the behavior of matrix algorithms that inherently present poor memory locality. The application of this technique to the problem of out-of-place matrix transposition achieved competitive results when compared to state-of-the-art approaches. The performance of our solution surpassed Intel MKL version after employing standard software prefetching techniques.

Stingy sketch

Recording the frequency of items in highly skewed data streams is a fundamental and hot problem in recent years. The literature demonstrates that sketch is the most promising solution. The typical metrics to measure a sketch are accuracy and speed, but existing sketches make only trade-offs between the two dimensions. Our proposed solution is a new sketch framework called Stingy sketch with two key techniques: Bit-pinching Counter Tree ( BCTree ) and Prophet Queue ( PQueue ) which optimizes both the accuracy and speed. The key idea of BCTree is to split a large fixed-size counter into many small nodes of a tree structure, and to use a precise encoding to perform carry-in operations with low processing overhead. The key idea of PQueue is to use pipelined prefetch technique to make most memory accesses happen in L2 cache without losing precision. Importantly, the two techniques are cooperative so that Stingy sketch can improve accuracy and speed simultaneously. Extensive experimental results show that Stingy sketch is up to 50% more accurate than the SOTA of accuracy-oriented sketches and is up to 33% faster than the SOTA of speed-oriented sketches.

Off-chip prefetching based on Hidden Markov Model for non-volatile memory architectures.

Non-volatile memory technology is now available in commodity hardware. This technology can be used as a backup memory for an external dram cache memory without needing to modify the software. However, the higher read and write latencies of non-volatile memory may exacerbate the memory wall problem. In this work we present a novel off-chip prefetch technique based on a Hidden Markov Model that specifically deals with the latency problem caused by complexity of off-chip memory access patterns. Firstly, we present a thorough analysis of off-chip memory access patterns to identify its complexity in multicore processors. Based on this study, we propose a prefetching module located in the llc which uses two small tables, and where the computational complexity of which is linear with the number of computing threads. Our Markov-based technique is able to keep track and make clustering of several simultaneous groups of memory accesses coming from multiple simultaneous threads in a multicore processor. It can quickly identify complex address groups and trigger prefetch with very high accuracy. Our simulations show an improvement of up to 76% in the hit ratio of an off-chip dram cache for multicore architecture over the conventional prefetch technique (g/dc). Also, the overhead of prefetch requests (failed prefetches) is reduced by 48% in single core simulations and by 83% in multicore simulations.

An Effective Techniques Using Apriori and Logistic Methods in Cloud Computing

This paper presents a creativity data prefetching scheme on the loading servers in distributed file systems for cloud computing. The server will get and piggybacked the frequent data from the client system, after analyzing the fetched data is forward to the client machine from the server. To place this technique to work, the data about client nodes is piggybacked onto the real client I/O requests, and then forwarded to the relevant storage server. Next, dual prediction algorithms have been proposed to calculation future block access operations for directing what data should be fetched on storage servers in advance. Finally, the prefetching data can be pressed to the relevant client device from the storage server. Over a series of evaluation experiments with a group of application benchmarks, we have demonstrated that our presented initiative prefetching technique can benefit distributed file systems for cloud environments to achieve better I/O performance. In particular, configuration-limited client machines in the cloud are not answerable for predicting I/O access operations, which can certainly contribute to preferable system performance on them.

Web Usage Mining by Neural Hybrid Prediction with Markov Chain Components

This paper presents and evaluates a two-level web usage prediction technique, consisting of a neural network in the first level and contextual component predictors in the second level. We used Markov chains of different orders as contextual predictors to anticipate the next web access based on specific web access history. The role of the neural network is to decide, based on previous behaviour, whose predictor’s output to use. The predicted web resources are then prefetched into the cache of the browser. In this way, we considerably increase the hit rate of the web browser, which shortens the load times. We have determined the optimal configuration of the proposed hybrid predictor on a real dataset and compared it with other existing web prefetching techniques in terms of prediction accuracy. The best configuration of the proposed neural hybrid method provides an average web access prediction accuracy of 86.95%.

Early Address Prediction

Achieving low load-to-use latency with low energy and storage overheads is critical for performance. Existing techniques either prefetch into the pipeline (via address prediction and validation) or provide data reuse in the pipeline (via register sharing or L0 caches). These techniques provide a range of tradeoffs between latency, reuse, and overhead. In this work, we present a pipeline prefetching technique that achieves state-of-the-art performance and data reuse without additional data storage, data movement, or validation overheads by adding address tags to the register file. Our addition of register file tags allows us to forward (reuse) load data from the register file with no additional data movement, keep the data alive in the register file beyond the instruction’s lifetime to increase temporal reuse, and coalesce prefetch requests to achieve spatial reuse. Further, we show that we can use the existing memory order violation detection hardware to validate prefetches and data forwards without additional overhead. Our design achieves the performance of existing pipeline prefetching while also forwarding 32% of the loads from the register file (compared to 15% in state-of-the-art register sharing), delivering a 16% reduction in L1 dynamic energy (1.6% total processor energy), with an area overhead of less than 0.5%.

Leveraging user access patterns and advanced cyberinfrastructure to accelerate data delivery from shared-use scientific observatories

With the growing number and increasing availability of shared-use instruments and observatories, observational data is becoming an essential part of application workflows and contributor to scientific discoveries in a range of disciplines. However, the corresponding growth in the number of users accessing these facilities coupled with the expansion in the scale and variety of the data, is making it challenging for these facilities to ensure their data can be accessed, integrated, and analyzed in a timely manner, and is resulting significant demands on their cyberinfrastructure (CI).In this paper, we present the design of a push-based data delivery framework that leverages emerging in-network capabilities, along with data pre-fetching techniques based on a hybrid data management model. Specifically, we analyze data access traces for two large-scale observatories, Ocean Observatories Initiative (OOI) and Geodetic Facility for the Advancement of Geoscience (GAGE), to identify typical user access patterns and to develop a model that can be used for data pre-fetching. Furthermore, we evaluate our data pre-fetching model and the proposed framework using a simulation of the Virtual Data Collaboratory (VDC) platform that provides in-network data staging and processing capabilities. The results demonstrate that the ability of the framework to significantly improve data delivery performance and reduce network traffic at the observatories’ facilities.

DeepPrefetcher: A Deep Learning Framework for Data Prefetching in Flash Storage Devices

In today's information-driven world, data access latency accounts for the expensive part of processing user requests. One potential solution to access latency is prefetching, a technique to speculate and move future requests closer to the processing unit. However, the block access requests received by the storage device show poor spatial locality because most file-related locality is absorbed in the higher layers of the memory hierarchy, including the CPU cache and main memory. Besides, the utilization of multithreading results in an interleaved access request making prefetching at the storage level more picky using existing prefetching techniques. Toward this, we propose and assess DeepPrefetcher, a novel deep neural network inspired context-aware prefetching method that adapts to arbitrary memory access patterns. DeepPrefetcher learns the block access pattern contexts using distributed representation and leverage long short-term memory learning model for context-aware data prefetching. Instead of using the logical block address (LBA) value directly, we model the difference between successive access requests, which contains more patterns than LBA value for modeling. By targeting access pattern sequence in this manner, the DeepPrefetcher can learn the vital context from a long input LBA sequence and learn to predict both the previously seen and unseen access patterns. The experimental result reveals that DeepPrefetcher can increase an average prefetch accuracy, coverage, and speedup by 21.5%, 19.5%, and 17.2%, respectively, contrasted with the baseline prefetching strategies. Overall, the proposed prefetching approach surpasses other schemes in all benchmarks, and the outcomes are promising.

Perceptron based filtering of futile prefetches in embedded VLIW DSPs

Abstract Prefetching has been widely used in general purpose computers, especially in high performance CPUs, but has been much less used in DSPs and embedded processors. The goal of this paper is to investigate adding a perceptron, a single layer neural network [1], to straight-forward hardware prefetching techniques for embedded DSPs, and assess their performance improvement. By using industry standard benchmarks we come to the conclusion that the use of a perceptron to improve the prefetch decision making significantly reduces the number of accesses from external memory via shared buses without any significant performance impact. We show that using a perceptron in addition to a prefetch mechanism results in a reduction of 70% on average in the number of accesses to external memory relative to accesses performed using the prefetch mechanism alone. The bandwidth reduction is achieved without any significant performance loss.

A Novel Location Aware Content Prefetching Technique for Mobile Adhoc Network

Abstract Numerous design has been presented in recent time for reducing data access, retrieval latency and improve data availability and accessibility in mobile adhoc network (MANET). Existing design used cooperative caching (CC) and content prefetching (CP) technique, allowing multiple mobile devices within a neighborhood to cache, prefetching and share data objects within their local caches. As mobile adhoc network is growing rapidly for the possessions of its capability in forming temporary network without the requirement of any centralized administration or predefined infrastructure or, Access rate of image data or Query latency has become a primary concern in mobile adhoc network This work focused on data prefetching techniques of geographically distributed data to improve or enhance the access rate of data there by reducing the query latency. Along with, our model can minimize processing time and cost for carrying out content prefetching operation. Experiment result shows the proposed content prefetching method improves throughput and latency performance.

Improved Particle Filter Resampling Architectures

The most challenging aspect of particle filtering hardware implementation is the resampling step. This is because of high latency as it can be only partially executed in parallel with the other steps of particle filtering and has no inherent parallelism inside it. To reduce the latency, an improved resampling architecture is proposed which involves pre-fetching from the weight memory in parallel to the fetching of a value from a random function generator along with architectures for realizing the pre-fetch technique. This enables a particle filter using M particles with otherwise streaming operation to get new inputs more often than 2M cycles as the previously best approach gives. Results show that a pre-fetch buffer of five values achieves the best area-latency reduction trade-off while on average achieving an 85% reduction in latency for the resampling step leading to a sample time reduction of more than 40%. We also propose a generic division-free architecture for the resampling steps. It also removes the need of explicitly ordering the random values for efficient multinomial resampling implementation. In addition, on-the-fly computation of the cumulative sum of weights is proposed which helps reduce the word length of the particle weight memory. FPGA implementation results show that the memory size is reduced by up to 50%.

Fork Path: Batching ORAM Requests to Remove Redundant Memory Accesses

Outsourcing data to a third-party cloud provider has become quite common with the increasing use of cloud computing. This brings convenience, as well as the concern for data security and privacy. It is believed that data encryption alone is often not enough to protect users' privacy from the cloud provider. According to previous work, the sequence of storage locations accessed by the client can leak up to 90% of the sensitive information, even with data encrypted. In this context, Oblivious RAM (ORAM) is proposed. ORAM algorithms allow the client to hide its access pattern from the service provider while introducing a lot of extra operations. Among all the prototypes, Path ORAM is one of the most promising designs. However, there are still redundant memory accesses that can be removed without harming the security of traditional ORAM as we observed. We came up with three optimization techniques, including path merging, ORAM request scheduling, and merging aware caching. We also propose a prefetching technique to further decreasing the access overhead. Moreover, we also illustrate the compatibility of Fork Path and some state-of-the-art Path ORAM optimizations. Compared to traditional Path ORAM approaches, our Fork Path ORAM can reduce overall performance overhead and power consumption of memory system by 65% and 44%, while the design overhead is trivial.

ID-Based Signcryption Authentication Algorithm for Intra- and Inter-domain Handoff in Wireless Mesh Networks

In wireless mesh networks, during handoff process, the mobile client needs to authenticate with other nodes. It is necessary to ensure secure communication between the mobile client and mesh nodes. However, using the conventional public key infrastructure requires a large overhead of storage requirement, since it has to deal with the management of the public key certifications. In the present paper, an ID-based signcryption authentication algorithm for intra- and inter-domain handoff has been proposed. In this solution, a proxy-based hierarchical network has been considered. To reduce the handoff latency, a caching-list-based pre-fetching technique has been applied. During inter-domain handoff, authentication messages are encrypted using ID signcryption and signed by the mesh clients. Thus, the proposed algorithm ensures high security along with fast handoff. It has been observed from simulation results that the proposed algorithm reduces the percentage of compromised nodes, authentication delay, and communication cost with increased packet delivery ratio.

ANALYSIS END USER BEHAVIOR OVER WEB SCENARIO FOR EFFICIENT PREFETCHING THROUGH BIG DATA ANALYSIS

Web caching and restoration are the most common techniques that play a key role in improving web performance by keeping web objects that are likely to be visited in the near future closer to the custo mer. Web caching can work independently or in combination with Web pre-fetching. Cache and Web fetching can complement each other as the temporary Log file is a crucial part of web application. The log analysis is an important issue for the web application. Log file is not to be over emphasized as a source of information in systems and network management, whereas conduct efficient investigation and gathering of useful information need to correlate different log file. Task of analyzing event log files with the ever-increasing size and complexity of today's event logs has become cumbersome to carry out manually. Nowadays latest spotlight is automatic analysis of these logs files. Analysis of Web log is an innovative and unique domain constantly formed and modified by the convergence of several emerging web technologies. Because of its interdisciplinary nature, the diversity of issues addressed, the variety and the number of Web applications, is subject to many different methodologies. Proposed methodology has been used as effective web pre-fetching technique for frequent page generation by using relative position by inter session gap. The fast and efficient web pre-fetching scheme has improved user response of web page and expedites users visiting speed. Keywords: Web mining, web caching, web prefetching, Clustering, Client server Architecture, Log File.

Interactive natural language question answering over knowledge graphs

As many real-world data are constructed into knowledge graphs, providing effective and convenient query techniques for end users is an urgent and important task. Although structured query languages, such as SPARQL, offer a powerful expression ability to query RDF datasets, they are difficult to use. Keywords are simple but have a very limited expression ability. Natural language question (NLQ) is promising for querying knowledge graphs. A huge challenge is how to understand the question clearly so as to translate the unstructured question into a structured query. In this paper, we present a data + oracle approach to answer NLQs over knowledge graphs. We let users verify the ambiguities during the query understanding. To reduce the interaction cost, we formalize an interaction problem and design an efficient strategy to solve the problem. We also propose a query prefetching technique by exploiting the latency in the interactions with users. Moreover, we devise a hybrid approach that incorporates NLP-based, data-driven, and interaction techniques together to complete the question understanding. Extensive experiments over real datasets demonstrate that our proposed approach is effective as it outperforms state-of-the-art methods significantly.

AVPP

Value prediction improves instruction level parallelism in superscalar processors by breaking true data dependencies. Although this technique can significantly improve overall performance, most of the state-of-the-art value prediction approaches require high hardware cost, which is the main obstacle for its wide adoption in current processors. To tackle this issue, we revisit load value prediction as an efficient alternative to the classical approaches that predict all instructions. By speculating only on loads, the pressure over shared resources (e.g., the Physical Register File) and the predictor size can be substantially reduced (e.g., more than 90% reduction compared to recent works). We observe that existing value predictors cannot achieve very high performance when speculating only on load instructions. To solve this problem, we propose a new, accurate and low-cost mechanism for predicting the values of load instructions: the Address-first Value-next Predictor with Value Prefetching (AVPP). The key idea of our predictor is to predict the load address first (which, we find, is much more predictable than the value) and to use a small non-speculative Value Table (VT)—indexed by the predicted address—to predict the value next. To increase the coverage of AVPP, we aim to increase the hit rate of the VT by predicting also the load address of a future instance of the same load instruction and prefetching its value in the VT. We show that AVPP is relatively easy to implement, requiring only 2.5% of the area of a 32KB L1 data cache. We compare our mechanism with five state-of-the-art value prediction techniques, evaluated within the context of load value prediction, in a relatively narrow out-of-order processor. On average, our AVPP predictor achieves 11.2% speedup and 3.7% of energy savings over the baseline processor, outperforming all the state-of-the-art predictors in 16 of the 23 benchmarks we evaluate. We evaluate AVPP implemented together with different prefetching techniques, showing additive performance gains (20% average speedup). In addition, we propose a new taxonomy to classify different value predictor policies regarding predictor update, predictor availability, and in-flight pending updates. We evaluate these policies in detail.

MTB-Fetch: Multithreading Aware Hardware Prefetching for Chip Multiprocessors

To fully exploit the scaling performance in Chip Multiprocessors, applications must be divided into semi-independent processes that can run concurrently on multiple cores within a system. One major class of such applications, shared-memory, multi-threaded applications, requires programmers insert thread synchronization primitives (i.e., locks, barriers, and condition variables) in their critical sections to synchronize data access between processes. For this class of applications, scaling performance requires balanced per-thread workloads with little time spent in critical sections. In practice, however, threads often waste significant time waiting to acquire locks/barriers in their critical sections, leading to thread imbalance and poor performance scaling. Moreover, critical sections often stall data prefetchers that mitigate the effects of long critical section stalls by ensuring data is preloaded in the core caches when the critical section is complete. In this paper we examine a pure hardware technique to enable safe data prefetching beyond synchronization points in CMPs. We show that successful prefetching beyond synchronization points requires overcoming two significant challenges in existing prefetching techniques. First, we find that typical data prefetchers are designed to trigger prefetches based on current misses. This approach this works well for traditional, continuously executing, single-threaded applications. However, when a thread stalls on a synchronization point, it typically does not produce any new memory references to trigger a prefetcher. Second, even in the event that a prefetch were to be correctly directed to read beyond a synchronization point, it will likely prefetch shared data from another core before this data has been written. While this prefetch would be considered “accurate” it is highly undesirable, because such a prefetch would lead to three extra “ping-pong” movements back and forth between private caches in the producing and consuming cores, incurring more latency and energy overhead than without prefetching. We develop a new data prefetcher, Multi-Thread B-Fetch (MTB-Fetch), built as an extension to a previous single-threaded data prefetcher. MTB-Fetch addresses both issues in prefetching for shared memory multi-threaded workloads. MTB-Fetch achieves a speedup of 9.3 percent for multi-threaded applications with little additional hardware.

CDPM: Context-Directed Pattern Matching Prefetching to Improve Coarse-Grained Reconfigurable Array Performance

Coarse-grained reconfigurable arrays (CGRAs) can be dynamically programmed by configuration contexts to concurrently run multiple operations on a processing elements array. This further widens the gap between off-chip memory bandwidth demand and the limited speed of off-chip memory access. Cache prefetching is widely used for mitigating off-chip memory latency. However, straightforwardly applying existing prefetching techniques (primarily focusing on instruction driven processors) to CGRA may induce inaccurate prefetching, thereby crippling CGRA performance. Based on repetitively executed context in CGRA computing, this paper proposes a context-directed pattern matching (CDPM) mechanism to improve prefetching accuracy for CGRAs. CDPM generates a prefetch pattern for an initially executed context, and then reuses the pattern to issue prefetch requests when the context is re-executed. In order to eliminate the outdated prefetch pattern, CDPM also evaluates the prefetching accuracy of the prefetch pattern at run-time by adding prefetch addresses to a Bloom filter. The distinguishing feature of CDPM is the employment of the CGRA configuration context as a guide to improving prefetching accuracy. Experimental results showed that CDPM prefetching averagely improved performance by 31.1% compared to tests without prefetching and by 7.7% compared to state-of-the-art cache prefetching techniques, while only incurring slight area and power overheads.

Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley

Burst Buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the Quality-of-Service (QoS) of HPC applications—whichare considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley, a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the QoS of HPC applications. To achieve this goal, Eley embraces interference-aware prefetching technique that makes reading data input faster while introducing low interference for HPC applications. Evaluations using a wide range of Big Data and HPC applications demonstrate that Eley improves the performance of Big Data applications by up to 30% compared to existing BBs while maintaining the QoS of HPC applications.