Optimal Prefetching Research Articles

RmdnCache: Dual-Space Prefetching Neural Network for Large-Scale Volume Visualization.

Volume visualization plays a significant role in revealing important intrinsic patterns of 3D scientific datasets. However, these datasets are often large, making it challenging for interactive visualization systems to deliver a seamless user experience because of high input latency that arises from I/O bottlenecks and limited fast memory resources with high miss rates. To address this issue, we have proposed a deep learning-based prefetching method called RmdnCache, which optimizes the data flow across the memory hierarchy to reduce the input latency of large-scale volume visualization. Our approach accurately prefetches the content of the next view to fast memory using learning-based prediction while rendering the current view. The proposed deep learning architecture consists of two networks, RNN and MDN in respective spaces, which work together to predict both the location and likelihood distribution of the next view for defining an optimal prefetching range. Our method outperforms existing state-of-the-art prefetching algorithms in reducing overall input latency for visualizing real-world large-scale volumetric datasets.

DeepP: Deep Learning Multi-Program Prefetch Configuration for the IBM POWER 8

Current multi-core processors implement sophisticated hardware prefetchers, that can be configured by application (PID), to improve the system performance. When running multiple applications, each application can present different prefetch requirements, hence different configurations can be used. Setting the optimal prefetch configuration for each application is a complex task since it does not only depend on the application characteristics but also on the interference at the shared memory resources (e.g., memory bandwidth). In his paper, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DeepP</i> , a deep learning approach for the IBM POWER8 that identifies at run-time the best prefetch configuration for each application in a workload. To this end, the neural network predicts the performance of each application under the studied prefetch configurations by using a set of performance events. The prediction accuracy of the network is improved thanks to a dynamic training methodology that allows learning the impact of dynamic changes of the prefetch configuration on performance. At run-time, the devised network infers the best prefetch configuration for each application and adjusts it dynamically. Experimental results show that the proposed approach improves performance, on average, by 5.8%, 6.7%, and 15.8% compared to the default prefetch configuration across different 6-, 8-, and 10-application workloads, respectively.

Optimal Prefetching in Random Trees

We propose and analyze a model for optimizing the prefetching of documents, in the situation where the connection between documents is discovered progressively. A random surfer moves along the edges of a random tree representing possible sequences of documents, which is known to a controller only up to depth d. A quantity k of documents can be prefetched between two movements. The question is to determine which nodes of the known tree should be prefetched so as to minimize the probability of the surfer moving to a node not prefetched. We analyzed the model with the tools of Markov decision process theory. We formally identified the optimal policy in several situations, and we identified it numerically in others.

Prefetching and Caching for Minimizing Service Costs

In practice, prefetching data strategically has been used to improve caching performance. The idea is that data items can either be cached upon request (traditional approach) or prefetched into the cache before the requests actually occur. The caching and prefetching operations compete for the limited cache space, whose size is typically much smaller than the number of data items. A key challenge is to design an optimal prefetching and caching policy, assuming that the future requests can be predicted to a certain extent. This is a non-trivial challenge even under the idealized assumption that future requests are precisely known.

Prefetching and caching for minimizing service costs: Optimal and approximation strategies

Strategically prefetching data has been utilized in practice to improve caching performance. Apart from caching data items upon requests, they can be prefetched into the cache before requests actually occur. The caching and prefetching operations compete for the limited cache space, whose size is typically much smaller than the number of data items. A key question is how to design an optimal prefetching and caching policy, assuming that the future requests can be predicted to certain extend. This question is non-trivial even under an idealized assumption that the future requests are precisely known.To investigate this problem, we propose a cost-based service model. The objective is to find the optimal offline prefetching and caching policy that minimizes the accumulated cost for a given request sequence. By casting it as a min-cost flow problem, we are able to find the optimal policy for a data trace of length N in expected time O(N3∕2) via flow-based algorithms. However, this requires the entire trace for each request and cannot be applied in real time. To this end, we analytically characterize the optimal policy by obtaining an optimal cache eviction mechanism. We derive conditions under which proactive prefetching is a better choice than passive caching. Based on these insights, we propose a lightweight approximation policy that only exploits predictions in the near future. Moreover, the approximation policy can be applied in real time and processes the entire trace in O(N) expected time. We prove that the competitive ratio of the approximation policy is less than 2. Extensive simulations verify its near-optimal performance, for both heavy and light-tailed popularity distributions.

Fractional Dynamic Caching: A Collaborative Design of Storage and Backhaul

To reduce the average file delivery time under limited backhaul bandwidth and cache storage, we propose a fractional dynamic caching scheme, that coordinates the utilization of the backhaul and the cache storage. In the proposed scheme, only part of each file is pre-fetched and stored in the static storage segment of small base stations (SBSs), while remaining part of the file will be fetched into the dynamic storage segment when requested by users. We aim to minimize the average file delivery time by investigating the optimal prefetching policy, i.e., the size of the pre-fetched part of each file and the size of the dynamic storage. We formulate the file delivery time minimization problem as a convex optimization formulation. We compute the closed-form of the optimal pre-fetching policy when the wireless rate is constant. Then, a heuristic pre-fetching policy is derived, using the conditional average wireless rate. To overcome the issue that file popularity may be unknown or time-varying, we further provide an approximation of the heuristic pre-fetching policy which is irrelevant to the file popularity. We also discuss on the potential application of the proposed caching scheme in scenarios with heterogeneous file sizes. Numerical results show that the proposed scheme can substantially improve the backhaul utilization efficiency, and reduce the average file delivery time. The heuristic pre-fetching policy has similar performance with the optimal pre-fetching policy yet with much lower complexity.

Software Prefetching for Unstructured Mesh Applications

This article demonstrates the utility and implementation of software prefetching in an unstructured finite volume computational fluid dynamics code of representative size and complexity to an industrial application and across a number of modern processors. We present the benefits of auto-tuning for finding the optimal prefetch distance values across different computational kernels and architectures and demonstrate the importance of choosing the right prefetch destination across the available cache levels for best performance. We discuss the impact of the data layout on the number of prefetch instructions required in kernels with indirect addressing patterns and show how to best implement them in an existing large-scale computational fluid dynamics application. Through this, we show significant full application speed-ups on a range of processors and realistic test cases in both single core/tile and full socket configurations, such as 1.14× on the Intel Xeon Sandy Bridge, 1.09× on the Intel Xeon Broadwell, 1.29× on the Intel Xeon Skylake, 1.99× on the in-order Intel Xeon Phi Knights Corner coprocessor, and 1.51× on the out-of-order Intel Xeon Phi Knights Landing many-core processor.

Three-level performance optimization for heterogeneous systems based on software prefetching under power constraints

High power consumption has become one of the critical problems restricting the development of high-performance computers. Recently, there are numerous studies on optimizing the execution performance while satisfying the power constraint in recent years. However, these methods mainly focus on homogeneous systems without considering the power or speed difference of heterogeneous processors, so it is difficult to apply these methods in the heterogeneous systems with an accelerator. In this paper, by abstracting the current execution model of a heterogeneous system, we propose a new framework for managing the system power consumption with a three-level power control mechanism. The three levels from top to bottom are: system-level power controller (SPC), group-level power controller (GPC) and unit-level power controller (UPC). The study establishes a power management method for software prefetch in UPC to scale frequency and voltage of programs, select the optimal prefetch distance and guide optimization process to satisfy the constraint boundary according to power constraints. The strategy for dividing power based on key threads is put forward in GPC to preferentially allocate power to threads in key paths. In SPC, a method for evaluating the performance of heterogeneous processing engines is designed for dividing power in order to improve the overall execution performance of the system while sustaining the fairness between concurrent applications. Finally, the proposed framework is verified on a central processing unit (CPU)-graphics processing unit (GPU) heterogeneous system.

Multimedia prefetching with optimal Markovian policies

Multimedia prefetching is able to reduce the end-to-end latency and improve the video quality perceived by users. Previous work modeled prefetching as applying sequential decisional policies under uncertainty, using a Markov Decision Process model that integrated both user behavior and resources availability, to achieve optimal prefetching policies under realistic assumptions. In this paper, we extended the existing MDP model by considering more complex and aggressive policies, while preserving the optimality of the prefetching policy. We further enriched the extended model by considering user's profile to provide finer prefetching policies. The proposed extensions and the associated policies are validated through comparison against the existing model and against heuristics found in related work. We showed that our profiles-aware optimal policies can be achieved up to 28% latency reduction with respect to known heuristics.

Adaptive Prefetching in Wireless Computing

In this paper, we consider a basic issue in wireless computing, where mobile devices (with relatively limited memory) fetch data (text, images, multimedia, etc.) from access points over wireless channels of fluctuating quality. When the channel quality is low, slow data downloads can contribute to application latency, and degradation of the user experience. To mitigate this issue, mobile devices can prefetch data during good quality channel periods preemptively, in anticipation of using them during low quality channel epochs. In other words, under favorable wireless channel conditions, the mobile terminal can prefetch data aggressively to reduce application latency when channel conditions have degraded. Considering channel fluctuations, memory constraints, and application latency, the issue at each point in time is whether to prefetch or not. A dynamic programming approach is taken here to determine the optimal prefetching policy. The latter is leveraged to develop a prefetching algorithm “Fetch-or-Not” (FON). We then design a randomized “Fetch-or-Not” (RFON) prefetching algorithm, which uses a randomized approximation of FON, thus lifting the need for online optimization and substantially reducing the computational complexity. Simulations are used to demonstrate that our low-complexity schemes perform well when compared to the optimal. In addition, our schemes outperform benchmark design techniques in realistic channel conditions.

Peer-Assisted On-Demand Streaming: Characterizing Demands and Optimizing Supplies

Nowadays, there has been significant deployment of peer-assisted on-demand streaming services over the Internet. Two of the most unique and salient features in a peer-assisted on-demand streaming system are the differentiation in the demand (or request) and the prefetching capability with caching. In this paper, we develop a theoretical framework based on queuing models, in order to 1) justify the superiority of service prioritization based on a taxonomy of requests, and 2) understand the fundamental principles behind optimal prefetching and caching designs in peer-assisted on-demand streaming systems. The focus is to instruct how limited uploading bandwidth resources and peer caching capacities can be utilized most efficiently to achieve better system performance. To achieve these objectives, we first use priority queuing analysis to prove how service quality and user experience can be statistically guaranteed, by prioritizing requests in the order of significance, including urgent playback (e.g., random seeks or initial startup), normal playback, and prefetching. We then proceed to construct a fine-grained stochastic supply-demand model to investigate peer caching and prefetching as a global optimization problem. This not only provides insights in understanding the fundamental characterization of demand, but also offers guidelines toward optimal prefetching and caching strategies in peer-assisted on-demand streaming systems.

Estimating Effective Prefetch Distance in Threaded Prefetching for Linked Data Structures

Helper threaded prefetching based on chip multiprocessor has been shown to reduce memory latency and improve overall system performance, and has been explored in linked data structures accesses. In our earlier work, we had proposed an effective threaded prefetching technique that balances delinquent loads between main thread and helper thread to improve effectiveness of prefetching. In this paper, we analyze memory access characteristic of specific application to estimate effective prefetch distance range for our proposed threaded prefetching technique. The effect of hardware prefetchers on the estimation is also exploited. We discuss key design issues of our proposed method and present preliminary experimental results. Our experimental evaluations indicated that the bounded range of effective prefetch distance can be determined using our method, and the optimal prefetch distances can be determined based on the estimated effective prefetch distance range by few trial runs.

Evaluation and Optimization of Kernel File Readaheads Based on Markov Decision Models

Readahead is an important technique to deal with the huge gap between disk drives and applications. It has become a standard in modern operating systems and advanced storage systems. However, it is difficult to develop the common kernel readahead and to achieve full testing coverage of all cases. In this paper, we formulate the kernel read handling, caching and readahead behavior as an absorbing Markov decision process, and present performance evaluations to compare or verify various readaheads. We also introduce algorithms to find optimal prefetching policies for specific read pattern and present the convergence analysis. By exploiting sample-path-based methods, it becomes much easier to evaluate and optimize prefetching policies, which can provide valuable informations to help the design and improvement of practical readaheads. For illustration and verification, we present two examples and some experiments on the readaheads in Linux kernel. The results show that the model-based evaluations agree with the practice and the improved prefetching policy significantly outperforms the original one in Linux kernel for the specified workloads.

Optimal Prefetching Scheme in P2P VoD Applications With Guided Seeks

Most existing peer-to-peer (P2P) video-on-demand (VoD) systems have been designed and optimized for the sequential playback. In practice, users often want to seek to the positions they are interested in. Such frequent seeks raise greater challenges to the design of the prefetching scheme. In this work, we first propose the concept of guided seeks. With the guidance, users can perform more efficient seeks to the desired positions. The guidance can be obtained from collective seeking statistics of other peers who have watched the same title in the previous and/or concurrent sessions. However, it is very challenging to aggregate the statistics efficiently, timely and in a completely distributed way. We design the hybrid sketches that not only capture the seeking statistics at significantly reduced space and time complexity, but also adapt to the popularity of the video. From the collected seeking statistics, we estimate the segment access probability, based on which we further develop an optimal prefetching scheme and an optimal cache replacement policy to minimize the expected seeking delay at every viewing position. Through extensive simulations, we demonstrate that the proposed prefetching framework significantly reduces the seeking delay compared to the sequential prefetching scheme.

Objective-optimal algorithms for long-term Web prefetching

Web prefetching is based on Web caching and attempts to reduce user-perceived latency. Unlike on-demand caching, Web prefetching fetches objects and stores them in advance, hoping that the prefetched objects are likely to be accessed in the near future and such accesses would be satisfied from the caches rather than by retrieving the objects from the Web server. This paper reviews the popular prefetching algorithms based on popularity, good fetch, API characteristic, and lifetime, and then makes the following contributions: 1) The paper proposes a family of linear-time prefetching algorithms, objective-greedy prefetching, wherein each algorithm greedily prefetches those Web objects that most significantly improve the performance as per the targeted metric. 2) The hit rate-greedy and bandwidth-greedy algorithms are shown to be optimal for their respective objective metrics. A linear-time optimal prefetching algorithm that maximizes the H/B metric as the performance measure is proposed. 3) The paper shows the results of a performance analysis via simulations, comparing the proposed algorithms with the existing algorithms in terms of the respective objectives - the hit rate, bandwidth, and the H/B metrics. The proposed prefetching algorithms are seen to provide better objective-based performance than any existing algorithms. Further, H/B-greedy performs almost as well as H/B-optimal.

Vers l'apprentissage de mécanismes d'adaptation multimédia

This paper argues that adaptation methods should not be fixed. We propose to use reinforcement learning in order to improve the behaviour of multimedia adaptation agents by taking into account the usage of a multimedia content. Some examples dealing with mobile services are presented. The Markov Decision Process (MDP) framework is briefly explained. We therefore present an application of MDP to the problem of an adaptation agent's learning: how to learn optimal prefetching policies. Experimental results show the benefits of the proposed method.

Optimal Read-Once Parallel Disk Scheduling

An optimal prefetching and I/O scheduling algorithm L-OPT, for parallel I/O systems, using a read-once model of block references is presented. The algorithm uses knowledge of the next $L$ references, $L$-block lookahead, to create a minimal-length I/O schedule. For a system with $D$ disks and a buffer of capacity $m$ blocks, we show that the competitive ratio of L-OPT is $\Theta(\sqrt{mD/L})$ when $L \geq m$, which matches the lower bound of any prefetching algorithm with $L$-block lookahead. Tight bounds for the remaining ranges of lookahead are also presented. In addition we show that L-OPT is the optimal offline algorithm: when the lookahead consists of the entire reference string, it performs the absolute minimum possible number of I/Os. Finally, we show that L-OPT is comparable with the best online algorithm with the same amount of lookahead; the ratio of the length of its schedule to the length of the optimal schedule is always within a constant factor.

Duality Between Prefetching and Queued Writing with Parallel Disks

Parallel disks promise to be a cost effective means for achieving high bandwidth in applications involving massive data sets, but algorithms for parallel disks can be difficult to devise. To combat this problem, we define a useful and natural duality between writing to parallel disks and the seemingly more difficult problem of prefetching. We first explore this duality for applications involving read-once accesses using parallel disks. We get a simple linear time algorithm for computing optimal prefetch schedules and analyze the efficiency of the resulting schedules for randomly placed data and for arbitrary interleaved accesses to striped sequences. Duality also provides an optimal schedule for prefetching plus caching, where blocks can be accessed multiple times. Another application of this duality gives us the first parallel disk sorting algorithms that are provably optimal up to lower-order terms. One of these algorithms is a simple and practical variant of multiway mergesort, addressing a question that had been open for some time.

Minimizing Average Schedule Length under Memory Constraints by Optimal Partitioning and Prefetching

Over the last 20 years, the performance gap between CPU and memory has been steadily increasing. As a result, a variety of techniques has been devised to hide that performance gap, from intermediate fast memories (caches) to various prefetching and memory management techniques for manipulating the data present in these caches. In this paper we propose a new memory management technique that takes advantage of access pattern information that is available at compile time by prefetching certain data elements before explicitly being requested by the CPU, as well as maintaining certain data in the local memory over a number of iterations. In order to better take advantage of the locality of reference present in loop structures, our technique also uses a new approach to memory by partitioning it and reducing execution to each partition, so that information is reused at much smaller time intervals than if execution followed the usual pattern. These combined approaches--using a new set of memory instructions as well as partitioning the memory--lead to improvements in total execution time of approximately 25% over existing methods.