Page Prefetching Research Articles

An Intelligent Framework for Oversubscription Management in CPU-GPU Unified Memory

This paper proposes a novel intelligent framework for oversubscription management in CPU-GPU UVM. We analyze the current rule-based methods of GPU memory oversubscription with unified memory, and the current learning-based methods for other computer architectural components. We then identify the performance gap between the existing rule-based methods and the theoretical upper bound. We also identify the advantages of applying machine intelligence and the limitations of the existing learning-based methods. This paper proposes a novel intelligent framework for oversubscription management in CPU-GPU UVM. It consists of an access pattern classifier followed by a pattern-specific Transformer-based model using a novel loss function aiming for reducing page thrashing. A policy engine is designed to leverage the model's result to perform accurate page prefetching and pre-eviction. We evaluate our intelligent framework on a set of 11 memory-intensive benchmarks from popular benchmark suites. Our solution outperforms the state-of-the-art (SOTA) methods for oversubscription management, reducing the number of pages thrashed by 64.4\% under 125\% memory oversubscription compared to the baseline, while the SOTA method reduces the number of pages thrashed by 17.3\%. Our solution achieves an average IPC improvement of 1.52X under 125\% memory oversubscription, and our solution achieves an average IPC improvement of 3.66X under 150\% memory oversubscription. Our solution outperforms the existing learning-based methods for page address prediction, improving top-1 accuracy by 6.45\% (up to 41.2\%) on average for a single GPGPU workload, improving top-1 accuracy by 10.2\% (up to 30.2\%) on average for multiple concurrent GPGPU workloads.

Deep learning based data prefetching in CPU-GPU unified virtual memory

Unified Virtual Memory (UVM) relieves the developers from the onus of maintaining complex data structures and explicit data migration by enabling on-demand data movement between CPU memory and GPU memory. However, on-demand paging soon becomes a performance bottleneck of UVM due to the high latency caused by page table walks and data migration over interconnect. Prefetching is considered a promising solution to this problem given its ability to leverage the locality of program memory access patterns. However, existing locality-based prefetching schemes can not handle all the situations. An ideal prefetcher should not only look at narrow regions of the requested address space but also capture global context to deliver a good prediction of the memory access pattern. This paper proposes a novel framework for page prefetching for UVM through deep learning. We first show that a powerful Transformer learning model can provide high accuracy for UVM page prefetching. We then perform analysis to interpret this Transformer model and derive several insights that allow us to design a simpler model to match the unconstrained model's accuracy with orders of magnitude lower cost. We use a pattern-based method to make the UVM page preditor general to different GPU workloads. We evaluate this framework on a set of 11 memory-intensive benchmarks from popular benchmark suites. Our solution outperforms the state-of-the-art (SOTA) UVM framework, improving the performance by 10.89%, improving the device memory page hit rate by 16.98% (89.02% vs. 76.10% for prior art), and reducing the CPU-GPU interconnect traffic by 11.05%. According to our proposed unified metric, which combines the accuracy, coverage, and page hit rate, our solution is approaching the ideal prefetching scheme more than the SOTA design (0.90 vs. 0.85, with the perfect prefetcher of 1.0).

Improved Post-Copy Live Migration with Memory Page Prefetching

Improved Post-Copy Live Migration with Memory Page Prefetching

PART2: using adaptive resonance theory for web caching prefetching

As the Web becomes the major source for information and services, fast access to relevant Web objects is a critical requirement for many applications. Various methods have been developed to achieve this goal. Web page prefetching is a commonly used technique that is highly effective in reducing user perceived delays. In this paper, we propose a new prefetching model pART2, which is based on the adaptive resonance theory (ART) for data clustering. A corresponding cache replacement policy (Probability-Based Replacement) is also proposed and developed. The new policy matches with the prefetching scheme and therefore produces a higher cache hit ratio compared with some of the traditional algorithms. To evaluate the new model, we conduct a series of experiments using data sets collected from a digital library system and Monte Carlo simulation techniques. Sensitivity of the parameters and statistical analysis are also presented. The proposed model using ART-type networks provides a promising avenue for constructing accurate caching prefetching systems that are flexible and adaptive.

A Vague Improved Markov Model Approach for Web Page Prediction

Today most of the information in all areas is available over the web. It increases the web utilization as well as attracts the interest of researchers to improve the effectiveness of web access and web utilization. As the number of web clients gets increased, the bandwidth sharing is performed that decreases the web access efficiency. Web page prefetching improves the effectiveness of web access by availing the next required web page before the user demand. It is an intelligent predictive mining that analyze the user web access history and predict the next page. In this work, vague improved markov model is presented to perform the prediction. In this work, vague rules are suggested to perform the pruning at different levels of markov model. Once the prediction table is generated, the association mining will be implemented to identify the most effective next page. In this paper, an integrated model is suggested to improve the prediction accuracy and effectiveness.

Access pattern based stream buffer management scheme for portable media players

Limited amount of main memories are often available on mobile phones and portable media players. In multitasking environments, however, programs often need to wait for pages to be loaded to the main memory, even for the pages once loaded but swapped out due to memory shortage. Such page fault mechanism provided by the operating system often becomes the main hindrance to the uninterrupted playback of multimedia players on portable electronic devices. In this paper, we propose a memory management scheme that helps multimedia players perform with a consistent quality, when other processes are concurrently running on memory limited environments. We devise a profile-based memory allocation scheme for multimedia stream buffers with the adoption of page prefetching and pinning. Using periodic monitoring points of main loops, we profile access patterns of memory, analyze the meaningful access patterns for stream buffers and provide the access pattern information for custom memory management. Our scheme helped reduce the number of page faults in heap. Reductions in page faults for a multimedia player are 60~80% under various memory-limited execution environments.

User behavior modeling and content based speculative web page prefetching

This paper provides a transparent and speculative algorithm for content based web page prefetching. The algorithm relies on a profile based on the Internet browsing habits of the user. It aims at reducing the perceived latency when the user requests a document by clicking on a hyperlink. The proposed user profile relies on the frequency of occurrence for selected elements forming the web pages visited by the user. These frequencies are employed in a mechanism for the prediction of the user’s future actions. For the anticipation of an adjacent action, the anchored text around each of the outbound links is used and weights are assigned to these links. Some of the linked documents are then prefetched and stored in a local cache according to the assigned weights. The proposed algorithm was tested against three different prefetching algorithms and yield improved cache–hit rates given a moderate bandwidth overhead. Furthermore, the precision of accurately inferring the user’s preference is evaluated through the recall–precision curves. Statistical evaluation testifies that the achieved recall–precision performance improvement is significant.

Improving page prefetching with prior knowledge

Prefetching is a technique applied to memory management policies in which pages are brought into memory before they are actually needed. In this study, prior knowledge of program behavior obtained through trace data is used to parameterize a variation of the working set memory management policy supporting demand page prefetching using one page lookahead. Two new algorithms supporting double page prefetching are proposed. A comparative analysis of them is made. Evaluation of the techniques through trace driven simulation shows that, in general, they are very effective in reducing the page fault rate and in some cases the space time product. As expected, memory occupancy increases, but usually by small amounts. The results are encouraging for their ability to improve performance for a variety of programs which exhibit some form of locality (spatial and/or temporal).