Prefetching Techniques Research Articles

An Event-Triggered Programmable Prefetcher for Irregular Workloads

Many modern workloads compute on large amounts of data, often with irregular memory accesses. Current architectures perform poorly for these workloads, as existing prefetching techniques cannot capture the memory access patterns; these applications end up heavily memory-bound as a result. Although a number of techniques exist to explicitly configure a prefetcher with traversal patterns, gaining significant speedups, they do not generalise beyond their target data structures. Instead, we propose an event-triggered programmable prefetcher combining the flexibility of a general-purpose computational unit with an event-based programming model, along with compiler techniques to automatically generate events from the original source code with annotations. This allows more complex fetching decisions to be made, without needing to stall when intermediate results are required. Using our programmable prefetching system, combined with small prefetch kernels extracted from applications, we achieve an average 3.0x speedup in simulation for a variety of graph, database and HPC workloads.

Accelerating BFS via Data Structure-Aware Prefetching on GPU

Breadth First Search (BFS) is a key graph traversing algorithm for many graph analytics applications. In recent decades, as the scale of the graph analytics problem has become larger and larger, it has raised many interests to accelerate graph traversing on GPU. However, due to the irregular memory access pattern of BFS, a great number of the memory divergent accesses harm the efficiency of GPU dramatically. Data prefetching can fetch useful data into the on-chip memory in advance to reduce the latency of accessing the off-chip memory. However, traditional prefetching techniques on GPU cannot deal with irregular memory accesses efficiently. By analyzing BFS algorithms for GPU, we find an opportunity to design an efficient prefetching mechanism by using the explicit information of the graph data structure. In this paper, we propose DSAP, a data structure-aware prefetcher on GPU that generates prefetching requests based on the well-defined data structure access pattern of BFS. Also, we introduce an adaptive fine-grain prefetching management to adjust the status of the prefetching granularity dynamically to balance the cache resource contention and data prefetching based on the utilization of the prefetched data. We implement DSAP on a GPGPU-sim simulator and evaluate six data sets from three different kinds of applications. DSAP can achieve a geometrical mean IPC improvement of 28%, up to 48.4%, compared with that of GPU with no prefetching technique, while in contrast, a stride-based global history buffer prefetching mechanism makes no effects on improving BFS performance for these data sets. Also, we use the GPUWattch to estimate the power consumption, and the power increases 8.3% in average and up to 11.8%, but the total energy cost drops 15.1% in average.

A Survey on Web Pre-Fetching and Web Caching Techniques for Latency Optimization in Mobile Environment

A Survey on Web Pre-Fetching and Web Caching Techniques for Latency Optimization in Mobile Environment

A Prefetching Technique Using HMM Forward and Backward Chaining for the DFS in Cloud

A general class of temporal probabilistic model have recently developed, which extends the Forward, Backward and Viterbi algorithm for hidden Markov models. The HMM (Midden Markov Model) is a probabilistic model of the joint probability of a collection of random variables with both observations and states. The algorithm is based on shrinking the state space of the HMM noticeably using such chains. The states through which the world passes are hidden, or unobserved. However, at each point in time also gets an observation that in some way reflects on the current state of the world. The Cloud Computing is a big deal for three reasons: It does not need any effort on Clients part to maintain or manage. It’s effectively infinite in size, so clients don’t need to worry about it running out of capacity. User can access cloud-based applications and services from anywhere all you need is a device with an Internet connection. In this Cloud Computing used the Distributed File Systems (DFS) for sharing and allocating the data during dynamic process .Those process are using some Prediction algorithms here using HMM Forward and Backward Chain. In this paper represents, Cloud Storage Server can Share the data among with the multiple users, using two prediction algorithms such as forward and backward chain in HMM.

Popularity based hierarchical prefetching technique for P2P video-on-demand

In Peer-to-Peer Video on Demand System like Video Cassette Recording (VCR) various operations (i.e. forward, backward, resume) are found to be used very frequently. The uncertainty of frequent VCR operations makes it difficult to provide services to play as download and also the workload on VoD server is very high because of it handles the overall network. So it may sometimes hinder the network performance. To overcome these issues we propose to develop popularity based hierarchical prefetching technique in this paper. In this technique, the network is considered in a hierarchical topology and then the popularity level of each video is estimated by the proxy and based on it, the popular video is cached in the proxy and streamed to the required peer nodes in the network. Simulation results of our work show that our proposed prefetching technique outperforms the delay of the previous technique.

ANALYSIS OF MARKOV MODEL AND ITS APPLICATION IN WEB PAGE ANTICIPATION

In digital era of information where accessing information in limited time frame is necessary, then Markov model helps in the anticipation of the web page. Various network obstacles and minimum bandwidth of network can be efficiently used by employing Markov model and its clustering and pre-fetching technique. Markov Model and its clustering mechanisms helps us to make clusters of similar web users having based on same specified constraints and after the application of the Anticipation framework necessary web page can be accessed.

Informed Prefetching for Distributed Multi-Level Storage Systems

In this paper, we present an informed prefetching technique called IPODS that makes use of application-disclosed access patterns to prefetch hinted blocks in distributed multi-level storage systems. We develop a prefetching pipeline in IPODS, where an informed prefetching process is divided into a set of independent prefetching steps and separated among multiple storage levels in a distributed system. In the IPODS system, while data blocks are prefetched from hard disks to memory buffers in remote storage servers, data blocks buffered in the servers are prefetched through networks to the clients' local cache. We show that these two prefetching steps can be handled in a pipelining manner to improve I/O performance of distributed storage systems. Our IPODS technique differs from existing prefetching schemes in two ways. First, it reduces applications' I/O stalls by keeping hinted data in clients' local caches and storage servers' fast buffers (e.g., solid state disks). Second, in a prefetching pipeline, multiple informed prefetching mechanisms coordinate semi-dependently to fetch blocks (1) from low-level (slow) to high-level (fast) storage devices in servers and (2) from high-level devices in servers to the clients' local cache. The prefetching pipeline in IPODS judiciously hides network latency in distributed storage systems, thereby reducing the overall I/O access time in distributed systems. Using a wide range of real-world I/O traces, our experiments show that IPODS can noticeably improve I/O performance of distributed storage systems by 6%.

WEB PAGE ANTICIPATION SYSTEM USING MARKOV MODEL

Web page prediction plays a vital role in anticipation and pre-fetching webpage in advance earlier than the user accesses that webpage in future, resulting in reducing the user latency and efficiently utilize the minimum bandwidth of the network. Process of accessing any information on the internet, clustering plays an important role. But clustering is done only by using the navigational behavior of the user. The clustering technique is done for the purpose of making certain groups of similar web pages. The whole process of anticipation of web page tries to use the pre-fetching techniques and Markov and hidden Markov model. On the basis of the similar behavior of web pages they are divided into various clusters and with the help of using the prediction algorithm we are able to anticipate the next most probable web page by using the user’s current accessible web page.

An Efficient Temporal Data Prefetcher for L1 Caches

Server workloads frequently encounter L1-D cache misses, and hence, lose significant performance potential. One way to reduce the number of L1-D misses or their effect is data prefetching. As L1-D access sequences have high temporal correlations, temporal prefetching techniques are promising for L1 caches. State-of-the-art temporal prefetching techniques are effective at reducing the number of L1-D misses, but we observe that there is a significant gap between what they offer and the opportunity. This work aims to improve the effectiveness of temporal prefetching techniques. To overcome the deficiencies of existing temporal prefetchers, we introduce Domino prefetching. Domino prefetcher is a temporal prefetching technique that looks up the history to find the last occurrence of the last one or two L1-D miss addresses for prefetching. We show that Domino prefetcher captures more than 87 percent of the temporal opportunity at L1-D. Through evaluation of a 16-core processor on a set of server workloads, we show that Domino prefetcher improves system performance by 26 percent (up to 56 percent).

A Prefetching Technique using Apriori and Logistic Methods for the DFS in Cloud

This paper presents an creativity data prefetching scheme on the loading servers in distributed file systems for cloud computing. The server will get and piggybacked the frequent data from the client system, after analyzing the fetched data is forward to the client machine from the server. To place this technique to work, the data about client nodes is piggybacked onto the real client I/O requests, and then forwarded to the relevant storage server. Next, dual prediction algorithms have been proposed to calculation future block access operations for directing what data should be fetched on storage servers in advance. Finally, the prefetching data can be pressed to the relevant client device from the storage server. Over a series of evaluation experiments with a group of application benchmarks, we have demonstrated that our presented initiative prefetching technique can benefit distributed file systems for cloud environments to achieve better I/O performance. In particular, configuration-limited client machines in the cloud are not answerable for predicting I/O access operations, which can certainly contribute to preferable system performance on them.

Live Prefetching for Mobile Computation Offloading

Mobile computation offloading refers to techniques for offloading computation intensive tasks from mobile devices to the cloud so as to lengthen the formers’ battery lives and enrich their features. The conventional designs fetch (transfer) user-specific data from mobiles to the cloud prior to computing, called offline prefetching . However, this approach can potentially result in excessive fetching of large volumes of data and cause heavy loads on radio-access networks. To solve this problem, the novel technique of live prefetching , which seamlessly integrates the task-level computation prediction and prefetching within the cloud-computing process of a large program with numerous tasks, is proposed in this paper. The technique avoids excessive fetching but retains the feature of leveraging prediction to reduce the program runtime and mobile transmission energy. By modeling the tasks in an offloaded program as a stochastic sequence, stochastic optimization is applied to design fetching policies to minimize mobile energy consumption under a deadline constraint. The policies enable real-time control of the prefetched-data sizes of candidates for future tasks. For slow fading, the optimal policy is derived and shown to have a threshold-based structure, selecting candidate tasks for prefetching and controlling their prefetched data based on their likelihoods. The result is extended to design close-to-optimal prefetching policies to fast fading channels. Compared with fetching without prediction, live prefetching is shown theoretically to always achieve reduction on mobile energy consumption.

An Adaptive IO Prefetching Approach for Virtualized Data Centers

Cloud and data center applications often make heavy use of virtualized servers, where flash-based solid-state drives (SSDs) have become popular alternatives over hard drives for data-intensive applications. Traditional data prefetching focuses on applications running on bare metal systems using hard drives. In contrast, virtualized systems using SSDs present different challenges for data prefetching. Most existing prefetching techniques, if applied unchanged in such environments, are likely to either fail to fully utilize SSDs, interfere with virtual machine I/O requests, or cause too much overhead if run in every virtualized instance.In this work, we demonstrate that data prefetching, when run in a virtualization-friendly manner can provide significant performance benefits for a wide range of data-intensive applications. We have designed and developed VIO-prefetching , consisting of accurate prediction of application needs in runtime and adaptive feedback-directed prefetching that scales with application needs, while being considerate to underlying storage devices and host systems. We have implemented a real system in Linux and evaluated it on different storage devices with the virtualization layer.Our comprehensive study provides insights of VIO-prefetching’s behavior at various virtualization system configurations, e.g., the number of VMs, in-guest processes, application types, etc. The proposed method improves virtual I/O performance up to 43 percent with the average of 14 percent for 1 to 12 VMs while running various applications on a Xen virtualization system.

Exact Response Time Analysis for Fixed Priority Memory-Processor Co-Scheduling

Recent technological advances have led to an increasing gap between memory and processor performance, since memory bandwidth is progressing at a much slower pace than processor bandwidth. Pre-fetching techniques are traditionally used to bridge this gap and achieve high processor utilization while tolerating high memory latencies. Following this trend, new computational models have been proposed to split task execution in two consecutive phases: a memory phase in which the required instructions and data are pre-fetched to local memory (M-phase), and an execution phase in which the task is executed with no memory contention (C-phase). Decoupling memory and execution phases not only simplifies the timing analysis, but also allows a more efficient (and predictable) pipelining of memory and execution phases through proper co-scheduling algorithms. This paper takes a further step towards the design of smart co-scheduling algorithms for sporadic real-time tasks complying with the memory-computation (M/C) model, by proposing a theoretical framework aimed at tightly characterizing the schedulability improvement obtainable with the adopted M/C task model on single-core systems. In particular, a critical instant is identified for M/C tasks scheduled with fixed priority and an exact response time analysis with pseudo-polynomial complexity is provided. Then, we investigate the problem of priority assignment for M/C tasks, showing that a necessary condition to achieve optimality is to allow different priorities for the two phases. Our experiments show that the proposed techniques provide a significant schedulability improvement with respect to classic execution models, placing an important building block towards the design of more efficient partitioned multi-core systems.

Hardware-friendly advanced motion vector prediction method and its architecture design for high efficiency video coding

High efficiency video coding (HEVC) has been standardized as a means of meeting the coding requirements of 4 K (3840 × 2160) video. However, HEVC has a high computational complexity and a challenging hardware implementation. As a result, 4 K video applications are still limited. Consequently, the present study proposes a hardware-friendly advanced motion vector prediction (AMVP) method for HEVC which avoids the data dependency problem during the hardware pipeline operation. In the proposed method, the motion vector relationship between the largest coding unit (LCU) and the smaller coding units (CUs) and prediction units (PUs) is observed first. Based on the observation results, a linear model is constructed to estimate the motion vectors of the CUs and PUs from the motion vector of the LCU. It is shown that the proposed prediction method improves the hardware coding throughput by at least 53.8% compared to a traditional AMVP hardware realization, and increases the BD-rate by no more than 0.99% on average. To reduce the hardware implementation costs, a coefficient approximation and control signal sharing technique are also proposed in this paper to realize the proposed linear model. In addition, since the motion vectors of the small CUs and PUs are estimated in advance, a data pre-fetch technique can be employed to further increase the hardware-coding throughput. The experimental results show that the proposed AMVP design has a gate count of just 10 k.

Exploiting I/O Reordering and I/O Interleaving to Improve Application Launch Performance

Application prefetchers improve application launch performance through either I/O reordering or I/O interleaving. However, there has been no proposal to combine the two techniques together, missing the opportunity for further optimization. We present a new application prefetching technique to take advantage of both the approaches. We evaluated our method with a set of applications to demonstrate that it reduces cold start application launch time by 50%, which is an improvement of 22% from the I/O reordering technique.

Leakage aware resource management approach with machine learning optimization framework for partially reconfigurable architectures

Shrinking size of transistors has enabled us to integrate more and more logic elements into FPGA chips leading to higher computing power. However, it also brings a serious concern to the leakage power dissipation of the FPGA devices. One of the major reasons for leakage power dissipation in FPGA is the utilization of prefetching technique to minimize the reconfiguration overhead (delay) in Partially Reconfigurable (PR) FPGAs. This technique creates delays between the reconfiguration and execution parts of a task, which may lead up to 38% leakage power of FPGA since the SRAM-cells containing reconfiguration information cannot be powered down. In this work, a resource management approach (RMA) containing scheduling, placement and post-placement stages has been proposed to address the aforementioned issue. In scheduling stage, a leakage-aware priority function is derived to cope with the leakage power. The placement stage uses a cost function that allows designers to determine the desired trade-off between performance and leakage-saving. The post-placement stage employs a heuristic approach to close the gaps between reconfiguration and execution of tasks, hence further reduce leakage waste. To further examine the trade-off between performance (schedule length) and leakage waste, we propose a framework to utilize the Genetic Algorithm (GA) for exploring the design space and obtaining Pareto optimal design points. Addressing the time-consuming limitation of GA, we apply Regression technique and Clustering algorithm to build predictive models for the Pareto fronts using a training task graph dataset. Experiments show that our approach can achieve large leakage savings for both synthetic and real-life applications with acceptable extended deadline. Furthermore, different variants of the proposed approach can reduce leakage power by 40–65% when compared to a performance-driven approach and by 15–43% when compared to state-of-the-art works. It’s also proven that our Machine Learning Optimization framework can estimate the Pareto front for new coming task graphs 10x faster than well-established GA approach with only 10% degradation in quality.

A Novel Probability based Approach for Optimized Prefetching

As the World Wide Web carries on to grow up rapidly in size and popularity, web traffic and network bottlenecks are more important issues in the networked world. The continued enhancement in demand for items on the World Wide Web causes severe overloading in many sites, network congestion, delay in perceived latency and network bottleneck. Many users have no patience in waiting more than a few seconds for downloading a web page, that's why Web traffic reduction system is very necessary in today World Wide Web for accessing the websites efficiently with the facility of existing networks. Web caching is an effective method to improve the performance of the World Wide Web but in today's World Wide Web caching method alone is not enough because of World Wide Web has grown quickly from a simple information-sharing mechanism to a rich collection of dynamic objects and multimedia data. The web prefetching is used to improve the performance of the proxy server. Prefetching predict web object that is expected to be requested in the near future and store them in advance, thus the response time of the user request is reduced. To improve the performance of the proxy server, this paper proposed a new framework which combines the caching system and prefetching technique and also optimize the prefetching with the help of probability. In this paper, we use the dataset for the experiment which is collected from ircache.net proxy server and give the result with the comparison of other technique of prefetching.

FPGA Implementation of an Algorithm for Automatically Detecting Targets in Remotely Sensed Hyperspectral Images

Timely detection of targets continues to be a relevant challenge for hyperspectral remote sensing capability. The automatic target-generation process using an orthogonal projection operator (ATGP-OSP) has been widely used for this purpose. Hyperspectral target-detection applications require timely responses for swift decisions, which depend upon (near) real-time performance of algorithm analysis. Reconfigurable field-programmable gate arrays (FPGAs) are promising platforms that allow hardware/software codesign and the potential to provide powerful onboard computing capabilities and flexibility at the same time. In this paper, we present an FPGA implementation for the ATGP-OSP algorithm. Our system includes a direct memory access module and implements a prefetching technique to hide the latency of the input/output communications. The proposed method has been implemented on a Virtex-7 XC7VX690T FPGA and tested using real hyperspectral data collected by NASA’s Airborne Visible Infra-Red Imaging Spectrometer (AVIRIS) over the Cuprite mining district in Nevada and the World Trade Center in New York. Experimental results demonstrate that our hardware version of the ATGP-OSP algorithm can significantly outperform a software version, which makes our reconfigurable system appealing for onboard hyperspectral data processing.

A Survey of Recent Prefetching Techniques for Processor Caches

As the trends of process scaling make memory systems an even more crucial bottleneck, the importance of latency hiding techniques such as prefetching grows further. However, naively using prefetching can harm performance and energy efficiency and, hence, several factors and parameters need to be taken into account to fully realize its potential. In this article, we survey several recent techniques that aim to improve the implementation and effectiveness of prefetching. We characterize the techniques on several parameters to highlight their similarities and differences. The aim of this survey is to provide insights to researchers into working of prefetching techniques and spark interesting future work for improving the performance advantages of prefetching even further.

A General Approach to Scalable Buffer Pool Management

In high-end data processing systems, such as databases, the execution concurrency level rises continuously since the introduction of multicore processors. This happens both on premises and in the cloud. For these systems, a buffer pool management of high scalability plays an important role on overall system performance. The scalability of buffer pool management is largely determined by its data replacement algorithm, which is a major component in the buffer pool management. It can seriously degrade the scalability if not designed and implemented properly. The root cause is its use of lock-protected data structures that incurs high contention with concurrent accesses. A common practice is to modify the replacement algorithm to reduce the contention on the lock(s), such as approximating the LRU replacement with the CLOCK algorithm or partitioning the data structures and using distributed locks. Unfortunately, the modification usually compromises the algorithm's hit ratio, a major performance goal. It may also involve significant effort on overhauling the original algorithm design and implementation. This paper provides a general solution to improve the scalability of a buffer pool management using any replacement algorithms for the data processing systems on physical on-premises machines and virtual machines in the cloud. Instead of making a difficult trade-off between the high hit ratio of a replacement algorithm and the low lock contention of its approximation, we design a system framework, called BP-Wrapper , that eliminates almost all lock contention without requiring any changes to an existing algorithm. In BP-Wrapper, we use a dynamic batching technique and a prefetching technique to reduce lock contention and to retain high hit ratio. The implementation of BP-Wrapper in PostgreSQL adds only about 300 lines of C code. It can increase the throughput by up to two folds compared with the replacement algorithms with lock contention when running TPC-C-like and TPC-W-like workloads.