Prefetching Scheme Research Articles

ACroSS: AI-driven cross-layer adaptive streaming for short video applications

As short video applications gain popularity, researchers are exploring ways to enhance Quality of Experience (QoE) for short videos while maximizing network bandwidth efficiency. Despite the growing interest, existing Adaptive Bitrate (ABR) algorithms primarily concentrate on content prefetching strategies and often overlook the dynamic interaction between network congestion control and ABR. This interaction is especially critical for short video streaming, where network conditions can fluctuate rapidly, and user expectations for seamless playback are high. To address these challenges, we propose aCroSS, an AI-driven framework for adaptive short video streaming that jointly optimizes both the application and transport layers to enhance QoE and bandwidth utilization. The aCroSS algorithm leverages advanced machine learning techniques to adapt in real time to fluctuating network conditions and dynamic user behaviors, delivering a more robust and responsive streaming experience. Our simulation results demonstrate that aCroSS consistently outperforms existing baseline algorithms, achieving more than a 10% improvement in utility scores across various network trace datasets. This highlights the effectiveness of aCroSS in delivering superior performance in diverse streaming environments.

WCET Analysis Based on Micro-Architecture Modeling for Embedded System Security

To ensure the timely execution of hard real-time applications, scheduling analysis techniques must consider safe upper bounds on the possible execution durations of tasks or runnables, which are referred to as Worst-Case Execution Times (WCET). Bounding WCET requires not only program path analysis but also modeling the impact of micro-architectural features present in modern processors. In this paper, we model the ARMv8 ISA and micro-architecture including instruction cache, branch predictor, instruction prefetching strategies, out-of-order pipeline. We also consider the complex interactions between these features (e.g., cache misses caused by branch predictions and branch misses caused by instruction pipelines) and estimate the WCET of the program using the Implicit Path Enumeration Technique (IPET) static WCET analysis method. We compare the estimated WCET of benchmarks with the observed WCET on two ARMv8 boards. The ratio of estimated to observed WCET values for all benchmarks is greater than 1, demonstrating the security of the analysis.

Exploration and optimization of novel replacement and prefetching strategies for inefficiencies of advanced MRAM-based hybrid cache systems

With the emergence of cutting-edge hardware systems such as cloud computing, edge computing, and on-chip neural network accelerators, how to design advanced memory strategies to substitute the traditional ones for maximizing the potential performance of non-volatile memory (NVM) under the existing hardware conditions, has become an urgent research issue for both academia and industrial communities. It is promising and innovative to improve computer systems in the layer of data exchanging with the emerging advanced semiconductor devices. In the paper, to address the inefficiencies of write-intensive, high power consumption, low hit rate and so on, which exist in hybrid magnetic random access memory cache systems, three novel cache replacement strategies and two cache prefetching strategies are put forward. The proposed triple novel replacement strategies, including historical frequency and time judgments, duplicate data-aware deletion, and dynamic relevance factors computing, can be utilized to compensate for the shortcomings of the traditional least recently used replacement strategy, respectively. In the two novel prefetching strategies, region distribution parameters and Listnet ranking network are imported into the caching process, respectively, to achieve optimized hitting performance. The simulation results demonstrate that the proposed replacement strategies can achieve up to 61.76%, 84.91%, 56.49%, and 53.21% optimization of write count, hit rate, dynamic power, and IPC compared to the conventional one. The proposed prefetching strategy can achieve up to 91.27%, 49.25% hit rate and IPC optimization. Meanwhile, the synthetic evaluation of the replacement and prefetching strategies are elaborated in the paper, including multi-core characteristics, information entropy, interplays and the performance constraints between replacement and prefetching mechanism, which would facilitate more credible ideas for future memory inefficiencies management and strategy design.

RISC-V-Based Evaluation and Strategy Exploration of MRAM Triple-Level Hybrid Cache Systems

Magnetic random access memory (MRAM) is considered one of the most promising memories among nonvolatile memories (NVMs), to replace static random access memory (SRAM) in the computing system to improve its cache performance. In this article, performance evaluation and strategy exploration of three-level hybrid cache systems are conducted under the framework of Reduced InstrucTIon Set Computer-Five (RISC-V) instruction set. First, system-level joint calls of Nvsim, Gem5, McPAT, and other cache emulation architectures are implemented based on the system-level simulator called MAGPIE tool. Based on the performance comparison of spin transfer torque (STT)-MRAM, spin-orbit torque (SOT)-MRAM, and SRAM with varied capacities, a variety of combined CPU three-level hybrid cache architectures are simulated. The performances of the proposed hybrid cache systems and the bus impact of interconnecting are evaluated, with better analysis of the cost of CPU system implementation and interactions among levels of cache. The power consumption of different processors with various instruction sets and cores is also evaluated. It is demonstrated that SOT-MRAM and STT-MRAM show great potential applications as L2 and L3 caches in the RISC-V system. RISC-V shows the potential benefits for future CPU cache. Meanwhile, an adaptive stride prefetching strategy is proposed to address the problem of delay hysteresis and high miss rate of STT-MRAM in the L3 cache. The applicability of this strategy to different storage technologies and the comparison with cutting-edge technologies are also illustrated. Simulation results show that the prefetching strategy can achieve at most 64.12% and 94.55% optimization effects on the miss latency and the miss rate of L3 cache, respectively.

Adaptive Selection and Clustering of Partial Reconfiguration Modules for Modern FPGA Design Flow

Dynamic Partially Reconfiguration (DPR) on FPGA has attracted significant research interest in recent years since it provides benefits such as reduced area and flexible functionality. However, due to the lack of supporting synthesis tools in the current DPR design flow, leveraging benefits from DPR requires specific design expertise with laborious manual design effort. Considering the complicated concurrency relations among various functions, it is challenging to select appropriate Partial Reconfiguration Modules (PR Modules) and cluster them into proper groups with a proper reconfiguration schedule so that the hardware modules can be swapped in and out correctly during the run time. Furthermore, the design of PR Modules also impacts reconfiguration latency and resource utilization greatly. In this paper, we propose a Maximum-Weight Independent Set model to formulate the PR Module selection and clustering problem so that the original manual exploration can be solved efficiently and automatically. We also propose a step-wise adjustment configuration prefetching strategy incorporated in our model to generate optimized reconfiguration schedules. Our proposed approach not only supports various design constraints but also can consider multiple objectives such as area and reconfiguration delay. Experimental results show that our approach can optimize resource utilization and reduce reconfiguration delay with good scalability. Especially, the implementation of the real design case shows that our approach can be embedded in Xilinx's DPR design flow successfully.

Adaptive Regression Prefetching Algorithm by Using Big Data Application Characteristics

This paper presents an innovative prefetching algorithm for a hybrid main memory structure, which consists of DRAM and phase-change memory. To enhance the efficiency of hybrid memory hardware in serving big data technologies, the proposed design employs an application-adaptive algorithm based on big data execution characteristics. Specifically optimized for graph-processing applications, which exhibit complex and irregular memory access patterns, a dual prefetching scheme is proposed. This scheme comprises a fast-response model with low-cost algorithms for regular memory access patterns and an intelligent model based on an adaptive Gaussian-kernel-based machine-learning prefetch engine. The intelligent model can acquire knowledge from real-time data samples, capturing distinct memory access patterns via an adaptive Gaussian-kernel-based regression algorithm. These methods allow the model to self-adjust its hyperparameters at runtime, facilitating the implementation of locally weighted regression (LWR) for the Gaussian process of irregular access patterns. In addition, we introduced an efficient hybrid main memory architecture that integrates two different kinds of memory technologies, including DRAM and PCM, providing cost and energy efficiency over a DRAM-only memory structure. Based on the simulation-based experimental results, our proposed model achieved performance enhancement of 57% compared to the conventional DRAM model and of approximately 12% compared to existing prefetcher-based models.

TVSR‐OR: Tile‐based 360‐degree video streaming over real time streaming protocol with optimized read

AbstractAmong video application use cases and scenarios, 360‐degree video applications gradually become significant. The state‐of‐the‐art solution for 360‐degree video streaming is field of view (FOV) transmission, whose quality of experience highly depends on system processing speed and network latency. The extant 360‐degree video FOV transmission profiles transmit multiple video tile streams to the client over either Hypertext Transfer Protocol (HTTP) or Real Time Streaming Protocol (RTSP). The client needs to merge all different tile streams after receiving them, which may cause a data synchronization problem of waiting for each tile to sync in the presents of packet losses. Therefore, we propose a system to select and merge tile streams on the Content Delivery Network (CDN) server and transmit the merged stream to the client. During the streaming service, the client can send a viewport switch request through RTSP signaling, and the server will subsequently deliver the merged video stream of the new viewport to the client after receiving the request. Also, to reduce disk overhead from parallel reads, we optimize the system with a file prefetch strategy to reduce unorganized, random, and parallel reads into serialized reads of large data trunks. We evaluate its performance through user request simulation experiments. Compared with the extant transmission solution, our system provides slightly lower video transmission latency, lower central processing unit (CPU) usage, and better disk performance under heavy service load.

ShortcutFusion++: Optimizing an End-to-End CNN Accelerator for High PE Utilization

ShorcutFusion [1] is an end-to-end framework that effectively maps many well-known deep neural networks (DNNs), such as MobileNet-v2, EfficientNet-B0, ResNet-50, and YOLO-v3, to a generic CNN accelerator on FPGA. Nevertheless, its processing elements are not fully utilized when supporting various networks, leading to relatively low hardware utilization (e.g., 68.42% for YOLO-v3). This study aimed to enhance the performance of ShortcutFusion and introduce ShortcutFusion++ by proposing two simple but effective techniques for eliminating unnecessary stalls in conventional design. First, the prefetching scheme was re-designed to avoid bubble cycles when feeding data to the PE array. Second, the output buffer was reconstructed to pipeline the operations of PEs and the process of writing output feature maps to off-chip memory. The experimental results show that ShortcutFusion++ achieves a PE utilization of 80.95% for the well-known object detection network YOLO-v3, outperforming its baseline by 12.53%.

Deep learning based data prefetching in CPU-GPU unified virtual memory

Unified Virtual Memory (UVM) relieves the developers from the onus of maintaining complex data structures and explicit data migration by enabling on-demand data movement between CPU memory and GPU memory. However, on-demand paging soon becomes a performance bottleneck of UVM due to the high latency caused by page table walks and data migration over interconnect. Prefetching is considered a promising solution to this problem given its ability to leverage the locality of program memory access patterns. However, existing locality-based prefetching schemes can not handle all the situations. An ideal prefetcher should not only look at narrow regions of the requested address space but also capture global context to deliver a good prediction of the memory access pattern. This paper proposes a novel framework for page prefetching for UVM through deep learning. We first show that a powerful Transformer learning model can provide high accuracy for UVM page prefetching. We then perform analysis to interpret this Transformer model and derive several insights that allow us to design a simpler model to match the unconstrained model's accuracy with orders of magnitude lower cost. We use a pattern-based method to make the UVM page preditor general to different GPU workloads. We evaluate this framework on a set of 11 memory-intensive benchmarks from popular benchmark suites. Our solution outperforms the state-of-the-art (SOTA) UVM framework, improving the performance by 10.89%, improving the device memory page hit rate by 16.98% (89.02% vs. 76.10% for prior art), and reducing the CPU-GPU interconnect traffic by 11.05%. According to our proposed unified metric, which combines the accuracy, coverage, and page hit rate, our solution is approaching the ideal prefetching scheme more than the SOTA design (0.90 vs. 0.85, with the perfect prefetcher of 1.0).

SMURF: Efficient and Scalable Metadata Access for Distributed Applications

In parallel with big data processing and analysis dominating the usage of distributed and Cloud infrastructures, the demand for distributed metadata access and transfer has increased. The volume of data generated by many application domains exceeds petabytes, while the corresponding metadata amounts to terabytes or even more. This article proposes a novel solution for efficient and scalable metadata access for distributed applications across wide-area networks, dubbed SMURF. Our solution combines novel pipelining and concurrent transfer mechanisms with reliability, provides distributed continuum caching and semantic locality-aware prefetching strategies to sidestep fetching latency, and achieves scalable and high-performance metadata fetch/prefetch services in the Cloud. We incorporate the phenomenon of semantic locality awareness for increased prefetch prediction rate using real-life application I/O traces from Yahoo! Hadoop audit logs and propose a novel prefetch predictor. By effectively caching and prefetching metadata based on the access patterns, our continuum caching and prefetching mechanism significantly improves the local cache hit rate and reduces the average fetching latency. We replay approximately 20 Million metadata access operations from real audit traces, where SMURF achieves 90% accuracy during prefetch prediction and reduced the average fetch latency by 50% compared to the state-of-the-art mechanisms.

Exploring Synchronous Page Fault Handling

The advance of nonvolatile memory in storage technology has presented challenges in redefining the ways in handling the main memory and the storage. This work is motivated by the strong demands in effective handling of page faults over ultralow-latency storage devices. In particular, we propose synchronous and asynchronous prefetching strategies to satisfy process executions with different memory demands in supporting of synchronous page fault handling. An adaptive CPU scheduling strategy is also proposed to cope with the needs of processes in maintaining their working sets in the main memory. Six representative benchmarks and applications were evaluated. It was shown that our strategy can effectively save 12.33% of the total execution time and reduce 13.33% of page faults, compared to the conventional demand paging strategy with nearly no sacrificing of process fairness.

HealthFetch: An Influence-Based, Context-Aware Prefetch Scheme in Citizen-Centered Health Storage Clouds

Over the past few years, increasing attention has been given to the health sector and the integration of new technologies into it. Cloud computing and storage clouds have become essentially state of the art solutions for other major areas and have started to rapidly make their presence powerful in the health sector as well. More and more companies are working toward a future that will allow healthcare professionals to engage more with such infrastructures, enabling them a vast number of possibilities. While this is a very important step, less attention has been given to the citizens. For this reason, in this paper, a citizen-centered storage cloud solution is proposed that will allow citizens to hold their health data in their own hands while also enabling the exchange of these data with healthcare professionals during emergency situations. Not only that, in order to reduce the health data transmission delay, a novel context-aware prefetch engine enriched with deep learning capabilities is proposed. The proposed prefetch scheme, along with the proposed storage cloud, is put under a two-fold evaluation in several deployment and usage scenarios in order to examine its performance with respect to the data transmission times, while also evaluating its outcomes compared to other state of the art solutions. The results show that the proposed solution shows significant improvement of the download speed when compared with the storage cloud, especially when large data are exchanged. In addition, the results of the proposed scheme evaluation depict that the proposed scheme improves the overall predictions, considering the coefficient of determination (R2 > 0.94) and the mean of errors (RMSE < 1), while also reducing the training data by 12%.

Probabilistic Data Prefetching for Data Transportation in Smart Cities

To deal with the ever increasing wireless traffic, we have recently designed a vehicular cognitive capability harvesting network (V-CCHN) architecture to leverage vehicles as an alternative “transmission medium” (i.e., an opportunistic data carrier), besides the wireless spectrum, to effectively transport data from the location where it is collected to the place where it is consumed or utilized in a smart city environment. In the V-CCHN, cognitive radio technologies are utilized so that a large amount of data can be exchanged between vehicles and roadside infrastructure through short-range high-speed transmissions. Considering the limited contact duration and the uncertain activities of primary users, how to facilitate efficient data exchange between vehicles and roadside infrastructure is very challenging. This problem is further complicated by the fact that the mobility of vehicles might not be accurately predicted. In this paper, we propose a probabilistic data prefetching (PDP) scheme for the V-CCHN to address these challenges. By considering the conditional value at risk, we formulate the PDP schematic design as an optimization problem which allows us to obtain the corresponding PDP scheme. Finally, we have conducted extensive study to evaluate the performance of the obtained PDP scheme under various parameter settings.

Pattern-Based Prefetching with Adaptive Cache Management Inside of Solid-State Drives

This article proposes a pattern-based prefetching scheme with the support of adaptive cache management, at the flash translation layer of solid-state drives ( SSDs ). It works inside of SSDs and has features of OS dependence and uses transparency. Specifically, it first mines frequent block access patterns that reflect the correlation among the occurred I/O requests. Then, it compares the requests in the current time window with the identified patterns to direct prefetching data into the cache of SSDs. More importantly, to maximize the cache use efficiency, we build a mathematical model to adaptively determine the cache partition on the basis of I/O workload characteristics, for separately buffering the prefetched data and the written data. Experimental results show that our proposal can yield improvements on average read latency by 1.8 %– 36.5 % without noticeably increasing the write latency, in contrast to conventional SSD-inside prefetching schemes.

A Cross-Prefetcher Schedule Optimization Methodology

Prefetching offers the potential to significantly improve performance by speculatively loading application data so that it is available before it is needed. By their very nature, prefetching techniques are application behavior dependant. This implies that no universal prefetching solution exists. A combination of prefetching strategies need to be used to target a diverse set of applications. In this work, we develop the first comprehensive mathematical framework that allows a designer to better understand the prefetching opportunities of an application. We first use dynamic analysis to study the memory access behavior of an application and measure a series of metrics to both identify the optimized schedule, and estimate its achievable performance. To validate our model, we implement and evaluate three different prefetching strategies: helper threads, software prefetching and FPGA prefetching. We show that, for each individual scenario, our framework correctly generates the optimized schedule of prefetches and predicts the performance improvement with an accuracy of more than 95%. Using our framework, developers can choose the best prefetching strategy and parameters for their specific workload and use case.

A Large-Scale Measurement and Optimization of Mobile Live Streaming Services

Mobile Live Streaming (MLS) services are one of the most popular types of mobile apps. They involve a (often amateur) user broadcasting content to a potentially large online audience via unreliable networks. Nevertheless, we still lack a deep understanding of MLS user behavior that is critical for optimizing MLS systems, despite some active measurements on viewer-side behavior. Using detailed logs obtained from a major MLS provider, this paper first conducts an in-depth measurement study of both viewer-side and broadcaster-side behavior. Key findings include large wasteful uploads, strong viewing locality, and traffic dominance of loyal viewers. Specifically, 33.3% of uploads go unwatched, and the viewership of broadcasters tends to be localized. Inspired by our findings, we propose EDGEOPT– a centralized control center for MLS services for optimizing both the first-mile and the last-mile transmission in MLS. Specifically, EDGEOPT reduces wasteful uploading by 71% through adaptive uploading and enhances the replay quality of popular video segments by 10% via highlights retransmission. EDGEOPT also uses a learning-based content pre-fetching scheme that boosts the viewing startup by 29.5% and offloads at most 80% of the viewing workload from the edge servers with peer-assisted delivery.

Correlation-aware replica prefetching strategy to decrease access latency in edge cloud

With the number of connected devices increasing rapidly, the access latency issue increases drastically in the edge cloud environment. Massive low time-constrained and data-intensive mobile applications require efficient replication strategies to decrease retrieval time. However, the determination of replicas is not reasonable in many previous works, which incurs high response delay. To this end, a correlation-aware replica prefetching (CRP) strategy based on the file correlation principle is proposed, which can prefetch the files with high access probability. The key is to determine and obtain the implicit high-value files effectively, which has a significant impact on the performance of CRP. To achieve the goal of accelerating the acquisition of implicit high-value files, an access rule management method based on consistent hashing is proposed, and then the storage and query mechanisms for access rules based on adjacency list storage structure are further presented. The theoretical analysis and simulation results corroborate that CRP shortens average response time over 4.8%, improves average hit ratio over 4.2%, reduces transmitting data amount over 8.3%, and maintains replication frequency at a reasonable level when compared to other schemes.

Optimizing the copy-on-write mechanism of docker by dynamic prefetching

Docker, as a mainstream container solution, adopts the Copy-on-Write (CoW) mechanism in its storage drivers. This mechanism satisfies the need of different containers to share the same image. However, when a single container performs operations such as modification of an image file, a duplicate is created in the upper read-write layer, which contributes to the runtime overhead. When the accessed image file is fairly large, this additional overhead becomes non-negligible. Here we present the concept of Dynamic Prefetching Strategy Optimization (DPSO), which optimizes the CoW mechanism for a Docker container on the basis of the dynamic prefetching strategy. At the beginning of the container life cycle, DPSO pre-copies up the image files that are most likely to be copied up later to eliminate the overhead caused by performing this operation during application runtime. The experimental results show that DPSO has an average prefetch accuracy of greater than 78% in complex scenarios and could effectively eliminate the overhead caused by the CoW mechanism.

Design and Analysis of Low Delay Deterministic Network Based on Data Mining Association Analysis

The purpose of this paper is to research on the design and analysis of low delay deterministic network based on data mining association. This paper studies and implements the algorithm of mining page association rules. A session recognition algorithm based on log reference page and request time is proposed by using the time probability relationship of continuous requests. This method improves the accuracy of log data preprocessing and page association rules mining. This paper studies and tests the efficiency, prefetch timing and cache organization of the two page association rules. The results show that in this prefetch scheme, the prefetch performance of the association rules of unordered pages is better than that of the association rules of ordered pages. The prefetch performance when cache hits is better than that when cache fails and the cache fails to hit has better performance. In the case of a certain size of cache space, reasonable organization of cache space can further improve the cache hit rate and reduce the network delay.

Learning-based Buffer Starvation Modeling for Packets Prefetching Strategies of Video Streaming Services

Improving the quality of experience (QoE) of video streaming is a significant task in the wireless network scenario. Buffer starvation in the transmission process will cause playback freeze, and a certain number of packets must be prefetched before the service restarts. Taking into account the shortcomings of buffer in video streaming services, this paper proposes a deep learning-based starvation probability calculation model and a reinforcement learning-based packet prefetching model. The deep learning approach extracts the correlation between different timing inputs through the recurrent neural network module to return an explicit result and the precise distribution of the number of buffer starvation. The reinforcement learning approach leverages a better trade-off between start-up/rebuffering delay and buffer starvation by adjusting the packet prefetching strategy, so that the long-term objective quality of experience (QoE) of the video stream is optimized. Our framework can be applied to actual scenarios including finite video streaming and long video streaming transmission.