Multi-level Cache Research Articles

SafeTI: a Hardware Traffic Injector for Complex MPSoC Platform Validation and Characterization

Functional and timing validation of high performing safety-related platforms requires testing specific traffic patterns in the network-on-chip interconnects. Generally, testing needs to be performed by using software tests whose degree of control on the traffic generated is indirect, and limited to behavior that can be triggered by software, thus often unable to produce traffic generated by peripherals. Therefore, untested traffic scenarios can be abundant and, to a large extent, it is hard to know what traffic scenarios have been effectively tested. This paper presents the safe traffic injector , SafeTI, which allows injecting programmable traffic in AMBA AHB interconnects with high flexibility and degree of control, thus easing achieving high coverage in terms of traffic scenarios tested, and mitigating the uncertainty due to the difficulties to relate software tests with actual traffic scenarios tested. We also integrate successfully the SafeTI in the safety open platform SELENE proving the effectiveness of the proposed traffic injector for characterizing the platform memory access capabilities under different patterns of contention on the multi-level cache system.

Designing a Multi-Level Caching Forward Index for Recommendation System Services

This paper introduces a forward index based on microservices, which effectively addresses the issue of forward read amplification in recommendation systems through a multi-layer caching mechanism. This allows the forward service to handle a large number of concurrent requests with limited resources while ensuring service stability and final value rate. The paper provides a detailed description of the forward index's data structure design, traffic penetration calculation, and overall service architecture design, dividing the system into three parts: SDK, proxy service, and storage, corresponding to data query, data processing, and data storage respectively. In the SDK design, three components are employed: Cache Slot, synchronous queue, and timer, supporting synchronous, asynchronous, and flexible request forms. The proxy service design handles heterogeneous data and online computation, supporting batch and asynchronous designs, and provides a dynamic data protocol to adapt to the characteristic needs of different microservices. The storage layer design considers performance, cost, and maintainability, utilizing a Cache & SSD model to cope with high-concurrency scenarios.

Multilayered Decentralized Coded Caching With Nonuniform Popularity and Multilevel Cache Capacity in Space–Air–Ground Integrated Networks

Multilayered Decentralized Coded Caching With Nonuniform Popularity and Multilevel Cache Capacity in Space–Air–Ground Integrated Networks

Design Scheme of High Concurrent and High Usable Cache based on TB Level Data Capacity

With the continuous development of the Internet in recent years, the Internet business has also changed greatly. In the past, when accessing the database, it usually takes the way of direct access, in order to resist the reading and writing traffic, often use a master and data fragmentation methods. However, the current traffic is more and more, and the amount of data is also accumulating. The data capacity of many e-commerce websites is getting bigger and bigger, even exceeding the TB level. If we still use the previous way, all the traffic using the database to bear, its stability will be greatly reduced, and it will also bring higher cost and lower efficiency, is extremely undesirable. Therefore, there is an urgent need to find technologies and architectures related to high concurrency and high availability. For the above reasons, this topic first understands and analyzes the previous application platform cache and Redis database cluster cache, and then builds a multi-level cache strategy based on Njinx local cache, which will be described from requirements analysis and architecture design.

Adaptive Streaming Transmission Optimization Method Based on Three-Dimensional Caching Architecture and Environment Awareness in High-Speed Rail

In high-mobility scenarios, a user’s media experience is severely constrained by the difficulty of network channel prediction, the instability of network quality, and other problems caused by the user’s fast movement, frequent base station handovers, the Doppler effect, etc. To this end, this paper proposes a video adaptive transmission architecture based on three-dimensional caching. In the temporal dimension, video data are cached to different base stations, and in the spatial dimension video data are cached to base stations, high-speed trains, and clients, thus constructing a multilevel caching architecture based on spatio-temporal attributes. Then, this paper mathematically models the media stream transmission process and summarizes the optimization problems that need to be solved. To solve the optimization problem, this paper proposes three optimization algorithms, namely, the placement algorithm based on three-dimensional caching, the video content selection algorithm for caching, and the bitrate selection algorithm. Finally, this paper builds a simulation system, which shows that the scheme proposed in this paper is more suitable for high-speed mobile networks, with better and more stable performance.

Design and Development of Effective Multi-Level Cache Memory Model

An algorithm to determine the effectiveness and efficiency of a multi-level cache was developed in this paper. The developed model was used to test the efficiency rate, the relationships and the performance output level of a computer concerning the cache properties. This research paper showed that the level of cache and access time increases with the absolute hit rate but decreases with the relative hit rate. The number of cache levels varies directly with absolute access time and inversely with relative access time. The level one cache with set associativity of one has the highest access time and as the associativity and cache levels increase the memory access time decreases. The increase in the number of set associativity leads to an increase in cache performance and as well increases the performance speed of a computer.

A Caching-Based Dual K-Anonymous Location Privacy-Preserving Scheme for Edge Computing

Location-based services have become prevalent and the risk of location privacy leakage increases. Most existing schemes use third-party-based or third-party-free system architectures; the former suffers from a single point of failure (SPOF) and the latter experiences a heavy load on user terminal equipment and higher communication costs. Ensuring location privacy while lowering system overhead becomes a challenge. Many existing schemes fail to leverage the responses from a LBS server; such responses can be cached to answer subsequent queries. As a result, providing users with a relatively comprehensive level of location privacy protection is troublesome. In this article, we propose a caching-based dual K-anonymous (CDKA) location privacy preserving scheme in edge computing environments. Our scheme employs an edge server to intercedes between a user and LBS the server. We reduce the load on the user device by applying multi-level caching and to protect location privacy through dual anonymity. To ensure the location privacy in our construction, we set mobile clients and edge servers as anonymous. We use caching to lower the communication overhead and enforce the location privacy. The security analysis of our scheme supports its robustness against the edge server and the LBS server privacy offenses. We rely on the computation time, the communication cost and the cache hit ratio to evaluate our work against existing constructions. The results are two-fold: our work possesses a better response rate down to 15 ms to 32.6 ms and exhibits lower communication cost requirements down to 6.2 and 38.9 KB compared to existing works. Our scheme witnesses a higher cache hit ratio up to 13.6% and 39.1% compared to the literature.

Highly Concurrent TCP Session Connection Management System on FPGA Chip

Transmission Control Protocol (TCP) is a connection-oriented data transmission protocol, and it is also the main communication protocol used for end-to-end data transmission in the current Internet. At present, the mainstream TCP protocol processing service is implemented by software running on the Central Processing Unit (CPU). However, with the rapid growth of transmission bandwidth and the number of connections, the software-based processing method is not ideal in terms of delay and throughput, and also affects the processing performance of the CPU in other applications such as virtualization services. Moreover, other hardware solutions can only support a limited number of TCP session connections. In order to improve the processing efficiency of the TCP protocol and achieve highly concurrent network services, this paper proposes a TCP offload engine (TOE) prototype system based on field programmable gate array (FPGA) chips. It not only provides hardware-based data path processing, but also realizes hardware management of large-scale TCP session connection status through a multi-level cache management mechanism. Studies have shown that this solution can support 100 Gbps high-performance throughput characteristics, and allow concurrent processing of hundreds to 250,000 TCP connection state hardware maintenance on a single network node, improving the overall performance of the network system.

Going fast on a small-size computing cluster

Fast turnaround times for LHC physics analyses are essential for scientific success. The ability to quickly perform optimizations and consolidation studies is critical. At the same time, computing demands and complexities are rising with the upcoming data taking periods and new technologies, such as deep learning. We present a show-case of the HH→bbWW analysis at the CMS experiment, where we process \U0001d4aa(1 − 10)TB of data on 100 threads in a few hours. This analysis is based on the columnar NanoAOD data format, makes use of the NumPy ecosystem and HEP specific tools, in particular Coffea and Dask. Data locality, especially IO latency, is optimized by employing a multi-level caching structure using local file storage and on-worker SSD caches. We process thousands of events simultaneously within a single thread, thus enabling straightforward use of vectorized operations. Resource intensive computing tasks, such as GPU accelerated DNN inference and histogram aggregation in the \U0001d4aa(10)GB regime, are offloaded to dedicated workers. The analysis consists of hundreds of distinctly different workloads and is steered through a workflow management tool ensuring reproducibility throughout the development process up to journal publication.

An Ageing-Aware and Temperature Mapping Algorithm for Multilevel Cache Nodes

Increase in chip inactivity in the future threatens the performance of many-core systems and therefore, efficient techniques are required for continuous scaling of transistors. As of a result of this challenge, future proposed many-core system designs must consider the possibility of a 50% functioning chip per time as well maintaining performance. Fortunately, this 50% inactivity can be increased by managing the temperature of active nodes and the placement of the dark nodes to leverage a balance working chip whilst considering the lifetime of nodes. However, allocating dark nodes inefficiently can increase the temperature of the chip and increase the waiting time of applications. Consequently, due to stochastic application characteristics, a dynamic rescheduling technique is more desirable compared to fixed design mapping. In this paper, we propose an Ageing Before Temperature Electromigration-Aware, Negative Bias Temperature Instability (NBTI) & Time-dependent Dielectric Breakdown (TDDB) Neighbour Allocation (ABENA 2.0), a dynamic rescheduling management system which considers the ageing and temperature before mapping applications. ABENA also considers the location of active and dark nodes and migrate task based on the characteristics of the nodes. Our proposed algorithm employ Dynamic Voltage Frequency Scaling (DVFS) to reduce the Voltage and Frequency (VF) of the nodes. Results show that, our proposed methods improve the ageing of nodes compared to a conventional round-robin management system by 10% in temperature, and 10% ageing.

Enterprise-Class Multilevel Cache Design: Low Latency, Huge Capacity, and High Reliability

The IBM Z computing platform is optimized for processing vast amounts of data and transactions with low latency in a highly virtualized and secured environment with sustained processor utilization of over 90%. The platform and its microprocessor chip are designed to deliver consistent system performance, throughput, and response times under all conditions. The innovative cache architecture of the IBM Telum Processor provides low latency, large capacity, and reliable L2 caches. Based on a novel horizontal cache persistence algorithm, these L2 caches also serves as system wide L3 and L4 caches delivering optimal enterprise application performance. When built into an IBM z16 system, these architectural features deliver 11% per-core performance improvement over the prior z15 hardware, running real-world enterprise applications.

Multi-level caching and data verification based on ethereum blockchain

Multi-level caching and data verification based on ethereum blockchain

An efficient multi-level cache system for geometrically interconnected many-core chip multiprocessor

<p><span lang="EN-US">Many-core chip multiprocessor offers high parallel processing power for big data analytics; however, they require efficient multi-level cache and interconnection to achieve high system throughput. Using on-chip first level L1 and second level L2 per core fast private caches is expensive for large number of cores. In this paper, for moderate number of cores from 16 to 64, we present a cost and performance efficient multi-level cache system with per core L1 and last level shared bus cache on each bus line of a cost-efficient geometrically bus-based interconnection. In our approach, we extracted cache hit and miss concurrencies and applied concurrent average memory access time to more accurately determine the cache system performance. We conducted least recently used cache policy-based simulation for cache system with L1, with L1/L2, and with L1/shared bus cache. Our simulation results show that an average system throughput improvement of 2.5x can be achieved by using system with L1/shared bus cache system compared to using only first level L1 or L1/L2. Further, we show that the throughput degradation for the proposed cache system is only within 5% for a single bus fault, suggesting a good bus fault tolerance.</span></p>

The Impact of Cache and Dynamic Memory Management in Static Dataflow Applications

Dataflow is a parallel and generic model of computation that is agnostic of the underlying multi/many-core architecture executing it. State-of-the-art frameworks allow fast development of dataflow applications providing memory, communicating, and computing optimizations by design time exploration. However, the frameworks usually do not consider cache memory behavior when generating code. A generally accepted idea is that bigger and multi-level caches improve the performance of applications. This work evaluates such a hypothesis in a broad experiment campaign adopting different multi-core configurations related to the number of cores and cache parameters (size, sharing, controllers). The results show that bigger is not always better, and the foreseen future of more cores and bigger caches do not guarantee software-free better performance for dataflow applications. Additionally, this work investigates the adoption of two memory management strategies for dataflow applications: Copy-on-Write (CoW) and Non-Temporal Memory transfers (NTM). Experimental results addressing state-of-the-art applications show that NTM and CoW can contribute to reduce the execution time to -5.3% and \(-15.8\%\), respectively. CoW, specifically, shows improvements up to -21.8% in energy consumption with -16.8% of average among 22 different cache configurations.

Derived blockchain architecture for security-conscious data dissemination in edge-envisioned Internet of Drones ecosystem

Internet of Drones (IoD) facilitates the autonomous operations of drones into every application (warfare, surveillance, photography, etc) across the world. The transmission of data (to and fro) related to these applications occur between the drones and the other infrastructure over wireless channels that must abide to the stringent latency restrictions. However, relaying this data to the core cloud infrastructure may lead to a higher round trip delay. Thus, we utilize the cloud close to the ground, i.e., edge computing to realize an edge-envisioned IoD ecosystem. However, as this data is relayed over an open communication channel, it is often prone to different types of attacks due to it wider attack surface. Thus, we need to find a robust solution that can maintain the confidentiality, integrity, and authenticity of the data while providing desired services. Blockchain technology is capable to handle these challenges owing to the distributed ledger that store the data immutably. However, the conventional block architecture pose several challenges because of limited computational capabilities of drones. As the size of blockchain increases, the data flow also increases and so does the associated challenges. Hence, to overcome these challenges, in this work, we have proposed a derived blockchain architecture that decouples the data part (or block ledger) from the block header and shifts it to off-chain storage. In our approach, the registration of a new drone is performed to enable legitimate access control thus ensuring identity management and traceability. Further, the interactions happen in the form of transactions of the blockchain. We propose a lightweight consensus mechanism based on the stochastic selection followed by a transaction signing process to ensure that each drone is in control of its block. The proposed scheme also handles the expanding storage requirements with the help of data compression using a shrinking block mechanism. Lastly, the problem of additional delay anticipated due to drone mobility is handled using a multi-level caching mechanism. The proposed work has been validated in a simulated Gazebo environment and the results are promising in terms of different metrics. We have also provided numerical validations in context of complexity, communication overheads and computation costs.

Tightening the CRPD bound for multilevel non-inclusive caches

Tasks running on microprocessors with cache memories are often subjected to cache related preemption delays (CRPDs). CRPDs may significantly increase task execution times, thereby, affecting their schedulability. Schedulability analysis accounting for the impact of CRPD has been extensively studied over the past two decades for systems with a single level of cache. Yet, the literature on CRPD for multilevel non-inclusive caches is relatively scarce . Two main challenges exist when analyzing multilevel caches: (1) characterization of the indirect effect of preemption , i.e., capturing the increase in cache interference at lower cache levels (e.g., level-two or L2 cache) due to the evictions of cache content from a higher cache level (e.g., level-one or L1 cache), and (2) upper bounding the maximum CRPD suffered by tasks at lower cache levels (e.g., L2 cache), i.e., determining the cache content of tasks that can be evicted from lower cache levels in case of preemptions. Existing analysis that focus on bounding CRPD for multilevel non-inclusive caches overestimate the values of (1) and (2) leading to pessimistic worst-case response time (WCRT) estimations. In this work, we reduce the excessive pessimism of the state-of-the-art CRPD analysis for multilevel non-inclusive caches by (i) introducing the notion of multi-level useful cache blocks, i.e., cache blocks that can cause CRPD at different cache levels, and use it to compute a tighter bound on the indirect effect of preemption of tasks; and (ii) deriving a new analysis to compute tighter bounds on the CRPD of tasks at lower cache levels (e.g., L2 cache). We performed a thorough experimental evaluation using benchmarks to compare the performance of our proposed CRPD analysis against the state-of-the-art CRPD analysis. Experimental results show that our proposed CRPD analysis dominates the existing analysis and improves task set schedulability by up to 20% percentage points.

Efficient Prefetching and Client-Side Caching Algorithms for Improving the Performance of Read Operations in Distributed File Systems

Modern web applications are deployed in cloud computing systems because they support unlimited storage and computing power. One of the main back-end storage components of this cloud computing system is the distributed file system which allows massive amounts of data to be stored and accessed. In most web applications deployed in such systems, read operations are performed more frequently than write operations. Consequently, increasing the efficiency of read operations in distributed file systems is a challenging and important research problem. The two main procedures used in distributed file systems to improve the performance of read operations are prefetching and caching. In this paper, we proposed novel prefetching and multi-level caching algorithms based on the Access-Frequency and Access-Recency ranking of file blocks that were previously accessed by client application programs. We also proposed new augmented ranking algorithms for prefetching file blocks by combining the Access-Frequency and Access-Recency ranking of the file blocks. We used rank-based replacement algorithms to replace file blocks in the cache. The simulation results show that, the proposed algorithms improve the performance of read operations on distributed file systems by 29% to 77% in comparison to algorithms proposed in the literature.

Multi-level cache management of quantitative trading platform

Abstract With the rapid development of quantitative trading business in the field of investment, quantitative trading platform is becoming an important tool for numerous investing users to participate in quantitative trading. In using the platform, return time of backtesting historical data is a key factor that influences user experience. In the aspect of optimising data access time, cache management is a critical link. Research work on cache management has achieved many referential results. However, quantitative trading platform has its special demands. (1) Data access of users has overlapping characteristics for time-series data. (2) This platform uses a wide variety of caching devices with heterogeneous performance. To address the above problems, a cache management approach adapting quantitative trading platform is proposed. It not only merges the overlapping data in the cache to save space but also places data into multi-level caching devices driven by user experience. Our extensive experiments demonstrate that the proposed approach could improve user experience up to >50% compared with the benchmark algorithms.

An FPGA‐Based Convolutional Neural Network Coprocessor

In this paper, an FPGA‐based convolutional neural network coprocessor is proposed. The coprocessor has a 1D convolutional computation unit PE in row stationary (RS) streaming mode and a 3D convolutional computation unit PE chain in pulsating array structure. The coprocessor can flexibly control the number of PE array openings according to the number of output channels of the convolutional layer. In this paper, we design a storage system with multilevel cache, and the global cache uses multiple broadcasts to distribute data to local caches and propose an image segmentation method that is compatible with the hardware architecture. The proposed coprocessor implements the convolutional and pooling layers of the VGG16 neural network model, in which the activation value, weight value, and bias value are quantized using 16‐bit fixed‐point quantization, with a peak computational performance of 316.0 GOP/s and an average computational performance of 62.54 GOP/s at a clock frequency of 200 MHz and a power consumption of about 9.25 W.

Analytical Modeling the Multi-Core Shared Cache Behavior With Considerations of Data-Sharing and Coherence

To mitigate the ever worsening “Power wall” and “Memory wall” problems, multi-core architectures with multi-level cache hierarchies have been widely accepted in modern processors. However, the complexity of the architectures makes modeling of shared caches extremely complex. In this article, we propose a data-sharing aware analytical model for estimating the miss rates of the downstream shared cache under multi-core scenarios. To avoid time-consuming full simulations of the cache architecture required by conventional approaches, the proposed model can also be integrated with our refined upstream cache analytical model, which also evaluates coherence misses with similar accuracies of state-of-the-art approach with only one tenth time overhead. We validate our analytical model against gem5 simulation results under 13 applications from PARSEC 2.1 benchmark suites. Compared to the results from gem5 simulations under 8 hardware configurations including dual-core and quad-core architectures, the average absolute error of the predicted shared L2 cache miss rates is less than 2% for all configurations. After integrated with the refined upstream model with coherence misses, the overall average absolute error in 4 hardware configurations is degraded to 4.82% due to the error accumulations. As an application case of the integrated model, we also evaluate the miss rates of 57 different multi-core and multi-level cache configurations.