Cache Usage Research Articles

Minimizing cache usage with fixed-priority and earliest deadline first scheduling

Minimizing cache usage with fixed-priority and earliest deadline first scheduling

Predicting Resource Utilization Trends with Southern California Petabyte Scale Cache

Large community of high-energy physicists share their data all around world making it necessary to ship a large number of files over wide- area networks. Regional disk caches such as the Southern California Petabyte Scale Cache have been deployed to reduce the data access latency. We observe that about 94% of the requested data volume were served from this cache, without remote transfers, between Sep. 2022 and July 2023. In this paper, we show the predictability of the resource utilization by exploring the trends of recent cache usage. The time series based prediction is made with a machine learning approach and the prediction errors are small relative to the variation in the input data. This work would help understanding the characteristics of the resource utilization and plan for additional deployments of caches in the future.

CR-5: Resource optimization opportunistic network routing algorithm based on node dynamic attributes

The opportunistic network routing algorithms are the core technology of intermittent communication of mobile nodes in opportunistic networks. However, the existing researches on opportunistic network routing algorithms mainly focus on how to improve the message delivery ratio and reduce the route overhead ratio, and often neglect managing energy consumption and cache usage of node. This results in problems such as communication chain easy to interruption and congestion control instability in opportunity networks. To solve the above problems, this paper proposes CR-5: resource optimization opportunistic network routing algorithm based on node dynamic attributes. CR-5 comprehensively considers node dynamic attributes such as historical encounter information, resource availability, and node social input degree, defines node communication relay degree as the basis for selecting relay nodes, and realizes node energy balance and cache optimization. CR-5 tends to choose relay nodes with strong communication ability, which solves the problem of communication chain easy to interruption to some extent. CR-5 also enhances the resource management and message processing ability of nodes through energy balance and cache optimization, reduces the phenomenon that some nodes process too many messages, which solves the problem of congestion control instability to some extent. Simulation results show that in opportunistic networks with low node initial cache, the CR-5 algorithm greatly improves the message delivery ratio and the node average remaining energy, and significantly reduces the route overhead ratio.

Web Caching Hybrid Algorithm by Semantic Similarity

Recently, most companies have been conducting their business through the cloud network, so increasing the speed of data access via the Internet has become very important. Therefore, the importance of cache algorithms lies in increasing the speed of data access. As we know, there are cache substitution algorithms that work well with the cache when applied to the processor cache, but they hardly work when applied to caching for web purposes. The reason for this discrepancy is that it is not intended to increase the complexity of this type of storage. Considering the challenges of cache usage in terms of the large discrepancy in file size and the heterogeneous model of user access to data in a web environment, especially with pages with dynamic content, there is a real need to develop 'web cache' algorithms. In this paper, a hybrid algorithm for web caching using semantic similarity is developed. The GDFS algorithm was developed using NGD (Normalized Google Distance) to determine the semantic similarity between cache objects, which resulted in better performance in comparison to other algorithms. The results showed that the web cache hybrid algorithm using semantic similarity increased the hit rate compared to the basic algorithms by up to 80.10%. The proposed hybrid algorithm was able to overcome the problem of the low byte hit rate in the GDFS algorithm. The improvement in the byte hit rate reached 65%. This indicates an increase in the byte hit rate. The results showed that the web cache hybrid algorithm using semantic similarity reduced the page load time using distance measured from Google compared to the page load time using other algorithms that do not use semantic similarity and do not use cache. The results showed that the web cache hybrid algorithm using semantic similarity reduced the page load time from 4.17 seconds using GDFS-Line to 2.16 seconds using GDFS-NGD.

Efficient thread‐to‐core mapping alternatives for application‐level redundant multithreading

SummaryRedundant multithreading (RMT) is an effective thread‐level replication method to improve the reliability requirements of applications. Although it significantly improves the robustness of applications, it comes with additional performance overhead since the redundant threads might share the same core resources. In our previous study [Efficient selective replication of critical code regions for SDC mitigation leveraging redundant multithreading.J Supercomput2021;77:14130‐14160], we presented an efficient software‐level RMT approach, where we execute the most critical code regions with three threads to correct errors. In this study, we focus on further improving the performance of our software‐level RMT method by presenting a set of different thread‐to‐core mapping alternatives. We provide different static mapping methods, which require preliminary information about the applications, such as execution time, instruction‐per‐cycle (IPC), or cache usage patterns, and a set of dynamic mapping methods, which map threads to cores dynamically at runtime without requiring any additional information. The dynamic mapping methods decide which threads are mapped to which cores at each scheduling point based on the IPC, cache miss rate, or cache access values of each thread as well as each core. Experimental results show that the dynamic mapping method, which maps threads to cores based on IPC values, outperforms all other static and dynamic methods. It also outperforms our baseline model, where the operating system handles the thread‐to‐core mappings by 8%, 7%, and 20% based on average speedup, harmonic speedup, and mean slowdown metrics.

Adaptively restarted block Krylov subspace methods with low-synchronization skeletons

With the recent realization of exascale performance by Oak Ridge National Laboratory’s Frontier supercomputer, reducing communication in kernels like QR factorization has become even more imperative. Low-synchronization Gram-Schmidt methods, first introduced in Świrydowicz et al. (Numer. Lin. Alg. Appl. 28(2):e2343, 2020), have been shown to improve the scalability of the Arnoldi method in high-performance distributed computing. Block versions of low-synchronization Gram-Schmidt show further potential for speeding up algorithms, as column-batching allows for maximizing cache usage with matrix-matrix operations. In this work, low-synchronization block Gram-Schmidt variants from Carson et al. (Linear Algebra Appl. 638:150–195, 2022) are transformed into block Arnoldi variants for use in block full orthogonalization methods (BFOM) and block generalized minimal residual methods (BGMRES). An adaptive restarting heuristic is developed to handle instabilities that arise with the increasing condition number of the Krylov basis. The performance, accuracy, and stability of these methods are assessed via a flexible benchmarking tool written in MATLAB. The modularity of the tool additionally permits generalized block inner products, like the global inner product.

A Low Memory Requirement MobileNets Accelerator Based on FPGA for Auxiliary Medical Tasks.

Convolutional neural networks (CNNs) have been widely applied in the fields of medical tasks because they can achieve high accuracy in many fields using a large number of parameters and operations. However, many applications designed for auxiliary checks or help need to be deployed into portable devices, where the huge number of operations and parameters of a standard CNN can become an obstruction. MobileNet adopts a depthwise separable convolution to replace the standard convolution, which can greatly reduce the number of operations and parameters while maintaining a relatively high accuracy. Such highly structured models are very suitable for FPGA implementation in order to further reduce resource requirements and improve efficiency. Many other implementations focus on performance more than on resource requirements because MobileNets has already reduced both parameters and operations and obtained significant results. However, because many small devices only have limited resources they cannot run MobileNet-like efficient networks in a normal way, and there are still many auxiliary medical applications that require a high-performance network running in real-time to meet the requirements. Hence, we need to figure out a specific accelerator structure to further reduce the memory and other resource requirements while running MobileNet-like efficient networks. In this paper, a MobileNet accelerator is proposed to minimize the on-chip memory capacity and the amount of data that is transferred between on-chip and off-chip memory. We propose two configurable computing modules: Pointwise Convolution Accelerator and Depthwise Convolution Accelerator, to parallelize the network and reduce the memory requirement with a specific dataflow model. At the same time, a new cache usage method is also proposed to further reduce the use of the on-chip memory. We implemented the accelerator on Xilinx XC7Z020, deployed MobileNetV2 on it, and achieved 70.94 FPS with 524.25 KB on-chip memory usage under 150 MHz.

Syllable-PBWT for space-efficient haplotype long-match query.

The positional Burrows-Wheeler transform (PBWT) has led to tremendous strides in haplotype matching on biobank-scale data. For genetic genealogical search, PBWT-based methods have optimized the asymptotic runtime of finding long matches between a query haplotype and a predefined panel of haplotypes. However, to enable fast query searches, the full-sized panel and PBWT data structures must be kept in memory, preventing existing algorithms from scaling up to modern biobank panels consisting of millions of haplotypes. In this work, we propose a space-efficient variation of PBWT named Syllable-PBWT, which divides every haplotype into syllables, builds the PBWT positional prefix arrays on the compressed syllabic panel, and leverages the polynomial rolling hash function for positional substring comparison. With the Syllable-PBWT data structures, we then present a long match query algorithm named Syllable-Query. Compared to the most time- and space-efficient previously published solution to the long match query problem, Syllable-Query reduced the memory use by a factor of over 100 on both the UK Biobank genotype data and the 1000 Genomes Project sequence data. Surprisingly, the smaller size of our syllabic data structures allows for more efficient iteration and CPU cache usage, granting Syllable-Query even faster runtime than existing solutions. https://github.com/ZhiGroup/Syllable-PBWT. Supplementary data are available at Bioinformatics online.

Evaluating execution time predictions on GPU kernels using an analytical model and machine learning techniques

Predicting the performance of applications executed on GPUs is a great challenge and is essential for efficient job schedulers. There are different approaches to do this, namely analytical modeling and machine learning (ML) techniques. Machine learning requires large training sets and reliable features, nevertheless it can capture the interactions between architecture and software without manual intervention.In this paper, we compared a BSP-based analytical model to predict the time of execution of kernels executed over GPUs. The comparison was made using three different ML techniques. The analytical model is based on the number of computations and memory accesses of the GPU, with additional information on cache usage obtained from profiling. The ML techniques Linear Regression, Support Vector Machine, and Random Forest were evaluated over two scenarios: first, data input or features for ML techniques were the same as the analytical model and, second, using a process of feature extraction, which used correlation analysis and hierarchical clustering. Our experiments were conducted with 20 CUDA kernels, 11 of which belonged to 6 real-world applications of the Rodinia benchmark suite, and the other were classical matrix-vector applications commonly used for benchmarking. We collected data over 9 NVIDIA GPUs in different machines.We show that the analytical model performs better at predicting when applications scale regularly. For the analytical model a single parameter λ is capable of adjusting the predictions, minimizing the complex analysis in the applications. We show also that ML techniques obtained high accuracy when a process of feature extraction is implemented. Sets of 5 and 10 features were tested in two different ways, for unknown GPUs and for unknown Kernels. For ML experiments with a process of feature extractions, we got errors around 1.54% and 2.71%, for unknown GPUs and for unknown Kernels, respectively.

Routing brain traffic through the von Neumann bottleneck: Efficient cache usage in spiking neural network simulation code on general purpose computers

Simulation is a third pillar next to experiment and theory in the study of complex dynamic systems such as biological neural networks. Contemporary brain-scale networks correspond to directed random graphs of a few million nodes, each with an in-degree and out-degree of several thousands of edges, where nodes and edges correspond to the fundamental biological units, neurons and synapses, respectively. The activity in neuronal networks is also sparse. Each neuron occasionally transmits a brief signal, called spike, via its outgoing synapses to the corresponding target neurons. In distributed computing these targets are scattered across thousands of parallel processes. The spatial and temporal sparsity represents an inherent bottleneck for simulations on conventional computers: irregular memory-access patterns cause poor cache utilization. Using an established neuronal network simulation code as a reference implementation, we investigate how common techniques to recover cache performance such as software-induced prefetching and software pipelining can benefit a real-world application. The algorithmic changes reduce simulation time by up to 50%. The study exemplifies that many-core systems assigned with an intrinsically parallel computational problem can alleviate the von Neumann bottleneck of conventional computer architectures.

Thwarting Cache Side Channel Attacks with Encryption Address-Based Set Balance Cache

The deterministic memory-to-cache mapping used by cache side channel attack causes the leakage of sensitive information such as secret keys, which seriously threatens user security and highlights the need to defend against this kind of attack. Recent table-based secure cache design requires more space to store the entry while a purely encryption-based design needs complex encryption units to ensure robustness. What is more, the newer attack algorithm enabling faster eviction set discovery may still break such defenses. Even though increasing the association of cache can be a potential solution, it introduces too much redundancy cache access and storage overhead. In this paper, we eliminate this problem. We present Encryption address-based Set Balance Cache (ESBC), a novel cache design to mitigate cache-based side channel attack. ESBC encrypts an address into two-level sets and displaces the data from a primary set into the secondary set when the primary set is full. The two-level mapping structure increases the complexity for attackers to build eviction sets, which is a vital step for the conflict-based attack. Furthermore, we adopt two different optimized variants from a temporal perspective, ESBC with remapping (ESBCR), which serves dynamic-remapping, and spatial perspective, ESBC with multi-mapping (ESBCM), which performs multi-mapping to improve the robustness. Our security analysis reveals that these designs can confuse the exploitation of conflicting addresses. Simulation-based evaluation on SPEC2017 shows 0.24% instruction per cycle (IPC) degradation for ESBCR with 1% remap rate. While the reduction of ESBCM that has five potential secondary sets is 0.69%. What is exciting is that both miss per kilo instructions (MPKI) and miss rates are even reduced because of the efficient cache usage. The storage overhead of these two variants is only 0.87%. By comparing the two schemes, we can observe that though ESBCR brings less performance overhead, the ESBCM has better scalability.

Location-Dependent Query Processing: Semantic Cache for Real-Time Smart City Analytics.

With the advent of wireless sensor networks and their deep integration with the world have enabled users worldwide to achieve benefits from location-based services through mobile applications, the problems such as low bandwidth, high network traffic, and disconnections issues are normally extracted from mobile services. An efficient database system is required to manage mentioned problems. Our research work finds the probability of user's next locations. A mobile user (query issuer) changes its position when performing a specific mobile search, where these queries change and repeat the search with the issuer position. Moreover, the query issuer can be static and may perform searches with varying conditions of queries. Data is exchanged with mobile devices and questions that are formulated during searching for query issuer locations. An aim of the research work is achieved through effectively processing of queries in terms of location-dependent, originated by mobile users. Significant studies have been performed in this field in the last two decades. In this paper, our novel approach comprise of usage of semantic caches with the Bayesian networks using a prediction algorithm. Our approach is unique and distinct from the traditional query processing system especially in mobile domain for the prediction of future locations of users. Consequently, a better search is analyzed using the response time of data fetch from the cache.

A Novel Hilbert Curve for Cache-Locality Preserving Loops

Modern microprocessors offer a rich memory hierarchy including various levels of cache and registers. Some of these memories (like main memory, L3 cache) are big but slow and shared among all cores. Others (registers, L1 cache) are fast and exclusively assigned to a single core but small. Only if the data accesses have a high locality, we can avoid excessive data transfers between the memory hierarchy. In this paper we consider fundamental algorithms like matrix multiplication, K-Means, Cholesky decomposition as well as the algorithm by Floyd and Warshall typically operating in two or three nested loops. We propose to traverse these loops whenever possible not in the canonical order but in an order defined by a space-filling curve. This traversal order dramatically improves data locality over a wide granularity allowing not only to efficiently support a cache of a single, known size (cache conscious) but also a hierarchy of various caches where the effective size available to our algorithms may even be unknown (cache oblivious). We propose a new space-filling curve called Fast Unrestricted (FUR) Hilbert with the following advantages: (1) we overcome the usual limitation to square-like grid sizes where the side-length is a power of 2 or 3. Instead, our approach allows arbitrary loop boundaries for all variables. (2) FUR-Hilbert is non-recursive with a guaranteed constant worst case time complexity per loop iteration (in contrast to O(log(grid-size)) for previous methods). (3) Our non-recursive approach makes the application of our cache-oblivious loops in any host algorithm as easy as conventional loops and facilitates automatic optimization by the compiler. (4) We demonstrate that crucial algorithms like Cholesky decomposition as well as the algorithm by Floyd and Warshall by can be efficiently supported. (5) Extensive experiments on runtime efficiency, cache usage and energy consumption demonstrate the profit of our approach. We believe that future compilers could translate nested loops into cache-oblivious loops either fully automatic or by a user-guided analysis of the data dependency.

Precise Cache Profiling for Studying Radiation Effects

Increased access to space has led to an increase in the usage of commodity processors in radiation environments. These processors are vulnerable to transient faults such as single event upsets that may cause bit-flips in processor components. Caches in particular are vulnerable due to their relatively large area, yet are often omitted from fault injection testing because many processors do not provide direct access to cache contents and they are often not fully modeled by simulators. The performance benefits of caches make disabling them undesirable, and the presence of error correcting codes is insufficient to correct for increasingly common multiple bit upsets. This work explores building a program’s cache profile by collecting cache usage information at an instruction granularity via commonly available on-chip debugging interfaces. The profile provides a tighter bound than cache utilization for cache vulnerability estimates (50% for several benchmarks). This can be applied to reduce the number of fault injections required to characterize behavior by at least two-thirds for the benchmarks we examine. The profile enables future work in hardware fault injection for caches that avoids the biases of existing techniques.

Data Layout Optimization for the LCC Compiler

In this research-in-progress report, we propose a novel approach to unified cache usage analysis for implementing data layout optimizations in the LCC compiler for the Elbrus and SPARC architectures. The approach consists of three parts. The first part is generalizing two methods of estimating cache miss amount and choosing more applicable one in the compiler. The second part is finding an applicable solution for the problem of cache miss amount minimization. The third part is implementing this analysis in the compiler and using analysis results for data layout transformations.

Partition placement by PR/SM

Every new machine generation of IBM Z brings with it an increase in number of physical processors and memory capacity. Some generations can also bring change in the physical configuration of the server. The z15 for example, can have from one to five drawers instead of a maximum of four on the z14. As another example, z15 has fixed two chips per node versus the two or three chips per node on z14. The logical partitions on the other hand can come in various configurations, including “Dedicated” logical partition, shared “Hiperdispatch = YES” logical partition, and shared “Hiperdispatch = NO” partition. Each of the partition types can request as many logical processors and memory as the machine generation will allow, which is usually less than the physical resources available on the machine. The optimal placement of logical partitions on the physical server, given its configuration, is an NP-hard problem. Memory access latency and cache usage play vital roles in the performance of logical partitions, and it is imperative that placement is optimal. Moreover, on z15, the integrated facility for linux processors and internal coupling facility processors can be moved from one chip to another, during reoptimization of partition placement, in addition to general-purpose and IBM Z integrated information processors that are already allowed to be moved, compounding the placement problem. This article describes the changes made to the Processor Resource/Systems Manager (PR/SM) heuristic placement algorithm for z15 and how it surmounts the problems inherent for optimal placement of logical partitions.

Learning-based dynamic cache management in a cloud

Caches are an important component of modern computing systems given their significant impact on performance. In particular, caches play a key role in the cloud due to the nature of large-scale, data-intensive processing. One of the key challenges for the cloud providers is how to share the caching capacity among tenants, under the circumstance that each often requires a different degree of quality of service (QoS) with respect to data access performance. The invariant is that the individual tenants’ QoS requirements should be satisfied while the cache usage is optimized in a system-wide manner. In this paper, we introduce a learning-based approach for dynamic cache management in a cloud, which is based on the estimation of data access pattern of a tenant and the prediction of cache performance for the access pattern in question. We consider a variety of probability distributions to estimate the data access pattern, and examine a set of learning-based regression techniques to predict the cache hit rate for the access pattern. The predicted cache hit rate is then used to make a decision whether reallocating cache space is needed to meet the QoS requirement for the tenant. Our experimental results with an extensive set of synthetic traces and the YCSB benchmark show that the proposed method consistently optimizes the cache space while satisfying the QoS requirement.

Ideal and Predictable Hit Ratio for Matrix Transposition in Data Caches

Matrix transposition is a fundamental operation, but it may present a very low and hardly predictable data cache hit ratio for large matrices. Safe (worst-case) hit ratio predictability is required in real-time systems. In this paper, we obtain the relations among the cache parameters that guarantee the ideal (predictable) data hit ratio assuming a Least-Recently-Used (LRU) data cache. Considering our analytical assessments, we compare a tiling matrix transposition to a cache oblivious algorithm, modified with phantom padding to improve its data hit ratio. Our results show that, with an adequate tile size, the tiling version results in an equal or better data hit ratio. We also analyze the energy consumption and execution time of matrix transposition on real hardware with pseudo-LRU (PLRU) caches. Our analytical hit/miss assessment enables the usage of a data cache for matrix transposition in real-time systems, since the number of misses in the worst case is bound. In general and high-performance computation, our analysis enables us to restrict the cache resources devoted to matrix transposition with no negative impact, in order to reduce both the energy consumption and the pollution to other computations.

Analysis of a New MPI Process Distribution for the Weather Research and Forecasting (WRF) Model

The standard method used in the Weather Research and Forecasting (WRF) model for distributing MPI processes across the processors is not always optimal. This circumstance affects performance, i.e., execution times, but also energy consumption, especially if the application is to be extended to exascale. The authors found that the reason why the standard method for process distribution is not always optimal was an imbalance between the orthogonality of the communication and the proper cache usage, and this affects energy consumption. We present an improved MPI process distribution algorithm that increases the performance. Furthermore, scalability analyses for the new algorithm are presented and the energy use of the system is evaluated. A solution for balancing energy use with performance is also proposed for cases where the former is a concern.

A Packet-Level Caching Algorithm for Mitigating Negative Effects Caused by Large Objects in ICN Networks

Increase in the size of content due to the emergence of bandwidth-intensive applications leads to a problematic phenomenon of packet-level caching which is called looped replacement . This causes a waste of cache usage and results in caching performance degradation. The goals of the paper are twofold: identifying the root cause of looped replacement and mitigating it. In order to achieve the goals, we analytically investigate not only how looped replacement occurs, but also how cache hit probability decreases. According to the analysis, we propose to use cache admission , which only admits packets arriving frequently to a cache, as a countermeasure against both problems. Finally, we develop an analytical model of packet-level caching with cache admission, and theoretically proves that cache admission mitigates these problems and improves cache hit probability.