Scratchpad Memory Space Research Articles

A Reconfigurable Cache for Efficient Use of Tag RAM as Scratch-Pad Memory

The cache memory has been a predominant component in modern chips, easily taking up more than 50% of the silicon area. It is then desirable to make the cache memory flexible for different needs. Therefore, many modern processor chips allow users to configure a part of the cache memory as the scratch-pad memory (SPM), a high-speed internal memory for rapid data access. However, such approach uses only the data RAM of the cache memory while leaving the tag RAM unused and thus wasting its capacity. This paper presents a cache organization, called Tag-SPM architecture, which allows the tag RAM to be used as the SPM and thus increases its capacity. It is accomplished with small Tag/Data-SPM controllers and four additional multiplexers in the cache organization. The proposed Tag-SPM architecture has been implemented with an academic ARM-based microprocessor with 4-/4-kB four-way set-associative instruction/data caches at the register transfer level level. Experiments show that the proposed architecture boosts the SPM capacity by 12.5% and requires only 0.08% area (434 gates) overhead without impairing the cache’s circuit speed in TSMC’s 90-nm standard cell implementation. Furthermore, the power overhead is negligible. When the Tag-SPM architecture is applied to typical cache systems, such as in ARM’s Cortex-A5 and Cortex-A53 processors, additional 12.5% SPM space per way can also be gained in both cases. The analyses show that our Tag-SPM architecture is a highly cost-effective way to boost the SPM space.

ROHOM: Requirement-aware Online Hybrid On-chip Memory Management for Multicore Systems

Many studies have shown that energy consumption of on-chip memory is a critical issue for multicore embedded systems. In order to reduce energy consumption, scratchpad memory (SPM), a software controlled on-chip memory, has been increasingly used. In a typical multicore embedded system that uses SPM in the on-chip memory, each core has local SPM and can access data in both local SPM and SPMs of other cores (i.e., remote SPM). Since the latency and energy of accessing remote SPMs is higher than accessing local SPM, how data are allocated in local and remote SPMs influences the performance and energy consumption of the system. This paper proposes a requirement-aware online hybrid on-chip memory management (ROHOM) method. This method contains an SPM supervisor that determines the SPM allocation of each task according to the dynamic access behavior and SPM requirements of the task. In addition, two policies: 1) free remote SPM space first and 2) get local SPM space first, are proposed in ROHOM to reduce the access frequency of remote SPMs. The experimental results show ROHOM can reduce energy delay product up to 69% (42% on average) in an 8-core system and up to 69% (50% on average) in a 16-core system when compared to a contention aware SPM allocation method. The hardware area overhead is insignificant (about 1.8%).

Dynamic Shared SPM Reuse for Real-Time Multicore Embedded Systems

Allocating the scratchpad memory (SPM) space to tasks is a challenging problem in real-time multicore embedded systems that use shared SPM. Proper SPM space allocation is important, as it considerably influences the application worst-case execution time (WCET), which is of great importance in real-time applications. To address this problem, in this article we present a dynamic SPM reuse scheme, where SPM space can be reused by other tasks during runtime without requiring any static SPM partitioning. Although the proposed scheme is applied dynamically at runtime, the required decision making is fairly complex and hence cannot be performed at runtime. We have developed techniques to perform the decision making offline at design time in the form of optimization problems combined with task scheduling/mapping. The proposed work is unlike previous works that either exploit static schemes for SPM space allocation or perform task scheduling/mapping and SPM space allocation incoherently. The experimental results show that our dynamic SPM reuse scheme can reduce WCET by up to 55% as compared to recent previous works on SPM allocation in real-time multicore embedded systems.

Virtualizing on-chip distributed ScratchPad memories for low power and trusted application execution

Emerging multicore platforms are increasingly deploying distributed scratchpad memories to achieve lower energy and area together with higher predictability; but this requires transparent and efficient software management of these critical resources. In this paper, we introduce the concept of ScratchPad Memory virtualization, a hardware/software run-time layer (called SPMVisor) that virtualizes the scratchpad memory space in order to facilitate the use of distributed SPMs in an efficient, transparent and secure manner. We introduce the notion of virtual scratchpad memories (vSPMs), which can be dynamically created and managed as regular SPMs. The SPMVisor exploits policy-driven allocation strategies based on application privilege levels and data level prioritization metrics (e.g., utilization) to efficiently manage the on-chip memory real-estate. Our experimental results on Mediabench/CHStone benchmarks running on various Chip-Multiprocessor configurations and software stacks (RTOS, virtualization, secure execution) showed that SPMVisor enhances performance by 71 % on average and reduces power consumption by 79 % on average with respect to traditional context switching schemes. We showed the benefits of using vSPMs in a various environments (a RTOS multi-tasking environment, a virtualization environment, and a trusted execution environment). Furthermore, we explored the effects of mapping instructions and data onto vSPMs, and showed that sharing on-chip space reduces both execution time and energy by an average 16 % and 12 % respectively. We then compared our priority-driven memory allocation scheme with traditional dynamic allocation and showed an average 54 % execution time reduction and 65 % energy savings. Finally, to further validate the SPMVisor’s benefits, we modified the initial bus-based architecture to include a mesh-based CMP with up to 4×4 nodes. We were able to observe that SPMVisor’s priority-driven allocator was able to reduce execution time by an average 17 % with respect to competing allocation policies, while saving an average 65 % across various architectural configurations. We were also able to observe that SPMVisor reduces execution time by an average 12.6 % with respect to competing allocation policies, while saving an average 63.5 % in total energy for various architectural configuration running 1024 jobs.

Static bus schedule aware scratchpad allocation in multiprocessors

Compiler controlled memories or scratchpad memories offer more predictable program execution times than cache memories. Scratchpad memories are often employed in multi-processor system-on-chip (MPSoC) platforms which seek to meet the performance needs of embedded applications while limiting power consumption and timing unpredictability. Scratchpad allocation schemes optimize performance while ensuring predictable execution times (as compared to caches). In this work, we develop a compile-time scratchpad allocation framework for multi-processor platforms, where the processors (virtually) share on-chip scratchpad space and external memory is accessed through a shared bus. Our allocation method considers the waiting time for bus access while deciding which memory blocks to load into the shared scratchpad memory space. Incorporating the bus schedule into our scratchpad allocation method leads to a global optimization of an application, as compared to employing local scratchpad allocation schemes in individual processors which locally optimize the per-processor execution time. We evaluate the efficacy, sensitivity and efficiency of our memory allocation scheme on two real-world embedded applications - an application controlling an Unmanned Aerial Vehicle (UAV), and a (fragment of) an in-orbit spacecraft software.

Partitioning and Allocation of Scratch-Pad Memory for Energy Minimization of Priority-Based Preemptive Multi-Task Systems

Energy minimization has become one of the primary goals in the embedded real-time domains. Consequently, scratch-pad memory has been employed as partial or entire replacement for cache memory due to its better energy efficiency. However, most previous approaches were not applicable to a preemptive multi-task environment. We propose three methods of partitioning and allocation of scratch-pad memory for fixed-priority-based preemptive multi-task systems. The three methods, i.e., spatial, temporal, and hybrid methods, achieve energy reduction in the instruction memory subsystems. With the spatial method, each task occupies its exclusive space in scratch-pad memory. With the temporal method, the running task uses entire scratch-pad space. The content of scratch-pad memory is swapped out as a task executes or gets preempted. The hybrid method is based on the spatial one but a higher priority task can temporarily use the space of lower priority task. The amount of space is prioritized for higher priority tasks. We formulate each method as an integer programming problem that simultaneously determines (1) partitioning of scratch-pad memory space for the tasks, and (2) allocation of program code to scratch-pad memory space for each task. Our methods not only support the real-time task scheduling but also consider aggressively the periods and priorities of tasks for the energy minimization. Additionally, we implement an RTOS-hardware cooperative support mechanism for runtime code allocation to the scratch-pad memory space. We have made the experiments with the fully functional real-time operating system. The experimental results have demonstrated the effectiveness of our techniques. Up to 73% energy reduction compared to a conventional method was achieved.

Dynamic and adaptive SPM management for a multi-task environment

In this paper, we present a dynamic and adaptive scratchpad memory (SPM) management strategy targeting a multi-task environment. It can be applied to a contemporary embedded processor that maps the physically addressed SPM into a virtual space with the help of an integrated memory management unit (MMU). Based on mass-count disparity, we introduce a hardware memory reference sampling unit (MRSU) that samples the memory reference stream with very low probability. The captured address is considered as one of the memory addresses contained in a frequently referenced memory block. A hardware interruption is generated by the MRSU, and the identified frequently accessed memory block is placed into the SPM space by software. The software also modifies the page table so that the follow-up memory accesses to the memory block will be redirected to the SPM. With no dependence on compiler and profiling information, our proposed strategy is specifically adequate for SPM management in a multi-task environment. In such an environment, a real-time operating system (RTOS) is usually hosted, and the behavior of the memory accesses cannot be predicted by static analysis or profiling. We evaluate our SPM allocation strategy by running several tasks on a tiny RTOS with preemptive scheduling. Experimental results show that our approach can achieve 10% reduction in energy consumption on average, with 1% performance degradation at runtime compared with a cache-only reference system.

Scratchpad allocation for concurrent embedded software

Software-controlled scratchpad memory is increasingly employed in embedded systems as it offers better timing predictability compared to caches. Previous scratchpad allocation algorithms typically consider single-process applications. But embedded applications are mostly multitasking with real-time constraints, where the scratchpad memory space has to be shared among interacting processes that may preempt each other. In this work, we develop a novel dynamic scratchpad allocation technique that takes these process interferences into account to improve the performance and predictability of the memory system. We model the application as a Message Sequence Chart (MSC) to best capture the interprocess interactions. Our goal is to optimize the Worst-Case Response Time (WCRT) of the application through runtime reloading of the scratchpad memory content at appropriate execution points. We propose an iterative allocation algorithm that consists of two critical steps: (1) analyze the MSC along with the existing allocation to determine potential interference patterns, and (2) exploit this interference information to tune the scratchpad reloading points and content so as to best improve the WCRT. We present various alternative scratchpad allocation heuristics and evaluate their effectiveness in reducing the WCRT. The scheme is also extended to work on Message Sequence Graph models. We evaluate our memory allocation scheme on two real-world embedded applications controlling an Unmanned Aerial Vehicle (UAV) and an in-orbit monitoring instrument, respectively.

Shared scratch pad memory space management across applications

Scratch Pad Memories (SPMs) have received considerable attention lately as on-chip memory building blocks. The main characteristic that distinguishes an SPM from a conventional cache memory is that the data flow is controlled by software. The main focus of this paper is the management of an SPM space shared by multiple applications that can potentially share data. The proposed approach has three major components; a compiler analysis phase, a runtime space partitioner, and a local partitioning phase. Our experimental results show that the proposed approach leads to minimum completion time among all alternate memory partitioning schemes tested.

Partitioning and Allocation of Scratch-Pad Memory in Priority-Based Multi-Task Systems

Energy consumption has become one of the major concerns in modern embedded systems. Recently memory subsystems have become consumed large amount of total energy in the embedded processors. This paper proposes partitioning and allocation approaches of scratch-pad memory in non-preemptive fixed-priority multi-task systems. We propose three approaches (i.e., spatial, temporal, and hybrid approaches) which enable energy efficient usage of the scratch-pad region. These approaches can reduce energy consumption of instruction memory. Each approach is formulated as an integer programming problem that simultaneously determines (1) partitioning of the scratch-pad memory space for the tasks, and (2) allocation of functions to the scratch-pad memory space for each task. Our formulations pay attention to the task periods for the purpose of energy minimization. The experimental results show up to 47% of energy reduction in the instruction memory subsystems can be achieved by the proposed approaches.