Memory Subsystem Research Articles

MCMG simulator: A unified simulation framework for CPU and graphic GPU

Accurate simulation is vital for the proper design and evaluation of any computing architecture. Researchers seek unified simulation frameworks that can model heterogeneous architectures like CPU and GPU devices and their interactions as computing patterns move toward heterogeneous era. In this paper, we introduce MCMG (Multi-CPU Multi-GPU) simulator, a cycle accurate, modular and open-source toolset that enables simulating x86 CPUs and Nvidia G80 like GPUs simultaneously. Targeting heterogeneous architectural exploration, MCMG supports fully configuration of multiple CPUs, GPUs and their memory sub-system. Not only CPUs, relative running frequency of each GPU can be also defined conveniently. Simulation validation is demonstrated with a preliminary architectural exploration study. Then we present shared LLC access results of heterogeneous cores and show reasonable explanation. Finally, we conclude the job.

GemDroid

As the demand for feature-rich mobile systems such as smartphones and tablets has outpaced other computing systems and is expected to continue at a faster rate, it is projected that SoCs with tens of cores and hundreds of IPs (or accelerator) will be designed to provide unprecedented level of features and functionality in future. Design of such mobile systems with required QoS and power budgets along with other design constraints will be a daunting task for computer architects since any ad hoc, piece-meal solution is unlikely to result in an optimal design. This requires early exploration of the complete design space to understand the system-level design trade-offs. To the best of our knowledge, there is no such publicly available tool to conduct a holistic evaluation of mobile platforms consisting of cores, IPs and system software. This paper presents GemDroid, a comprehensive simulation infrastructure to address these concerns. GemDroid has been designed by integrating the Android open-source emulator for facilitating execution of mobile applications, the GEM5 core simulator for analyzing the CPU and memory centric designs, and models for several IPs to collectively study their impact on system-level performance and power. Analyzing a spectrum of applications with GemDroid, we observed that the memory subsystem is a vital cog in the mobile platform because, it needs to handle both core and IP traffic, which have very different characteristics. Consequently, we present a heterogeneous memory controller (HMC) design, where we divide the memory physically into two address regions, where the first region with one memory controller (MC) handles core-specific application data and the second region with another MC handles all IP related data. The proposed modifications to the memory controller design results in an average 25% reduction in execution time for CPU bound applications, up to 11% reduction in frame drops, and on average 17% reduction in CPU busy time for on-screen (IP bound) applications.

V-SET CACHE: AN EFFICIENT ADAPTIVE SHARED CACHE FOR MULTI-CORE PROCESSORS

Development in VLSI design allows multi- to many-cores to be integrated on a single microprocessor chip. This increase in the core count per chip makes it more critical to design an efficient memory sub-system especially the shared last level cache (LLC). The efficient utilization of the LLC is a dominant factor to achieve the best microprocessor throughput. Conventional set-associative cache cannot cope with the new access pattern of the cache blocks in the multi-core processors. In this paper, the authors propose a new design for LLC in multi-core processor. The proposed v-set cache design allows an adaptive and dynamic utilization of the cache blocks. Unlike lately proposed design such as v-way caches, v-set cache design limits the serial access of cache blocks. In our paper, we thoroughly study the proposed design including area and power consumption as well as the performance and throughput. On eight-core microprocessor, the proposed v-set cache design can achieve a maximum speedup of 25% and 12% and an average speedup of 16% and 6% compared to conventional n-way and v-way cache designs, respectively. The area overhead of v-set does not exceed 7% compared to n-way cache.

SPMCloud

The era of cloud computing on-a-chip is enabled by the aggressive move towards many-core platforms and the rapid adoption of Network-on-Chips. As a result, there is a need for large-scale distributed on-chip shared memories that are reliable, low power, and seamlessly manageable. In this work, we propose SPMCloud , a novel scratchpad-memory-based cloud-inspired volatile storage subsystem designed to meet the needs of future-generation many-core platforms. SPMCloud is composed of several concepts, including: (1) a highly scalable data-center-like memory subsystem that exploits two enterprise-network-inspired memory configurations, namely, embedded Network Attached Storage ( eNAS ) and embedded Storage Area Network ( eSAN ), and (2) on-demand allocation of reliable memory space through memory virtualization and the use of embedded RAIDs. Our experimental results on Mediabench/CHStone benchmarks show that the SPMCloud 's fully distributed reliable memory subsystems can achieve 48% energy savings and 70% latency reduction on average over state-of-the-art NoC memory reliability techniques. We then evaluate the scalability of the SPMCloud and compare it with traditional SPM allocation policies. The SPMCloud 's dynamic allocator outperforms the best competition by an average 60% ( eNAS ) and 46% ( eSAN ) when the platform runs at 250 MHz and by an average 80% ( eNAS ) and 40% when running at 1 GHz. Moreover, the SPMCloud achieves an average 83% energy savings across all configurations (number of cores) with respect to the best competitors when running at 250 MHz and 1 GHz. We then studied the SPM hit ratio across the various allocation policies discussed in this article and showed that on average the SPMCloud 's priority-driven dynamic allocation policy achieves 93.5% SPM hit ratio, 0.6% higher hit ratio than the closest allocation policy. We then showed that the eNAS and eSAN achieve an average of 67.9% and 29% reduction in execution time, respectively, over the best competitor. Similarly, the eNAS and eSAN achieve an average of 82.7% and 82.3% energy savings, respectively, over the best competitor. Furthermore, we evaluated the scalability of the SPMCloud and its performance/energy efficiency when providing support for some of the heavier E-RAID levels, and showed that the eNAS / eSAN configurations with SECDED achieve an average of 51.5% and 34.9% reduction in execution time, respectively, over the best competitor with SECDED. Similarly, the eNAS / eSAN configurations with E-RAID Level 1, + SECDED achieve an average of 82.3% and 75.6% energy savings, respectively, over the best competitor.

Working Memory Subsystems and Receptive Vocabulary in Children with Specific Language Impairment

배경 및 목적: 작업기억 능력의 결함이 단순언어장애 아동의 지체된 어휘능력에 영향을 미치는 것으로 나타났다. 그러나 작업기억 하위체계별 수행능력과 수용어휘능력을 함께 비교한 연구는 미비한 실정이다. 따라서 본 논문에서는 작업기억 하위체계별 능력이 수용어휘에 미치는 영향을 살펴보고자 한다. 방법: 본 연구에서는 작업기억을 음운루프, 시공간스캐치패드 및...

Implementation of the Kinetic Plasma Code with Locally Recursive non-Locally Asynchronous Algorithms

Numerical simulation is presently considered impractical for several relevant plasma kinetics problems due to limitations of computer hardware even with the use of supercomputers. To overcome the existing limitations it is suggested to develop algorithms which would effectively utilize the computer memory subsystem hierarchy to optimize the dependency graph traversal rules. The ideas for general cases of numerical simulation and implementation of such algorithms to particle-in-cell code is discussed in the paper. This approach enables the simulation of previously unaccessible for modeling problems and the execution of series of numerical experiments in reasonable time. The latter is demonstrated on a multiscale problem of the development of filamentation instability in laser interaction with overdense plasma. One variant of the simulation with parameters typical for simulations on supercomputers is performed with the use of one cluster node. The series of such experiments revealed the dependency of energy loss on incoming laser pulse amplitude to be nonmonotonic and reach over 4%, an interesting result for research of fast ignition concept.

Superoptimization of memory subsystems

The disparity in performance between processors and main memories has led computer architects to incorporate large cache hierarchies in modern computers. Because these cache hierarchies are designed to be general-purpose, they may not provide the best possible performance for a given application. In this paper, we determine a memory subsystem well suited for a given application and main memory by discovering a memory subsystem comprised of caches,scratchpads, and other components that are combined to provide better performance. We draw motivation from the superoptimization of instruction sequences, which successfully finds unusually clever instruction sequences for programs. Targeting both ASIC and FPGA devices, we show that it is possible to discover unusual memory subsystems that provide performance improvements over a typical memory subsystem.

Embedded RAIDs-on-chip for bus-based chip-multiprocessors

The dual effects of larger die sizes and technology scaling, combined with aggressive voltage scaling for power reduction, increase the error rates for on-chip memories. Traditional on-chip memory reliability techniques (e.g., ECC) incur significant power and performance overheads. In this article, we propose a low-power-and-performance-overhead Embedded RAID (E-RAID) strategy and present Embedded RAIDs-on-Chip (E-RoC), a distributed dynamically managed reliable memory subsystem for bus-based Chip-Multiprocessors. E-RoC achieves reliability through redundancy by optimizing RAID-like policies tuned for on-chip distributed memories. We achieve on-chip reliability of memories through the use of Distributed Dynamic ScratchPad Allocatable Memories (DSPAMs) and their allocation policies. We exploit aggressive voltage scaling to reduce power consumption overheads due to parallel DSPAM accesses, and rely on the E-RoC Manager to automatically handle any resulting voltage-scaling-induced errors. We demonstrate how E-RAIDs can further enhance the fault tolerance of traditional memory reliability approaches by designing E-RAID levels that exploit ECC. Finally, we show the power and flexibility of the E-RoC concept by showing the benefits of having a heterogeneous E-RAID levels that fit each application's needs (fault tolerance, power/energy, performance). Our experimental results on CHStone/Mediabench II benchmarks show that our E-RAID levels converge to 100% error-free data rates much faster than traditional ECC approaches. Moreover, E-RAID levels that exploit ECC can guarantee 99.9% error-free data rates at ultra low Vdd on average, where as traditional ECC approaches were able to attain at most 99.1% error-free data rates. We observe an average of 22% dynamic power consumption increase by using traditional ECC approaches with respect to the baseline (non-voltage scaled SPMs), whereas our E-RAID levels are able to save dynamic power consumption by an average of 27% (w.r.t. the same non-voltage scaled SPMs baseline), while incurring worst-case 2% higher performance overheads than traditional ECC approaches. By voltage scaling the memories, we see that traditional ECC approaches are able to save static energy by 6.4% (average), where as our E-RAID approaches achieve 23.4% static energy savings (average). Finally, we observe that mixing E-RAID levels allows us to further reduce the dynamic power consumption by up to 55.5% at the cost of an average 5.6% increase in execution time over traditional approaches.

Cache-Hierarchy Contention-Aware Scheduling in CMPs

To improve chip multiprocessor (CMP) performance, recent research has focused on scheduling strategies to mitigate main memory bandwidth contention. Nowadays, commercial CMPs implement multilevel cache hierarchies that are shared by several multithreaded cores. In this microprocessor design, contention points may appear along the whole memory hierarchy. Moreover, this problem is expected to aggravate in future technologies, since the number of cores and hardware threads, and consequently the size of the shared caches increase with each microprocessor generation. This paper characterizes the impact on performance of the different contention points that appear along the memory subsystem. The analysis shows that some benchmarks are more sensitive to contention in higher levels of the memory hierarchy (e.g., shared L2) than to main memory contention. In this paper, we propose two generic scheduling strategies for CMPs. The first strategy takes into account the available bandwidth at each level of the cache hierarchy. The strategy selects the processes to be coscheduled and allocates them to cores to minimize contention effects. The second strategy also considers the performance degradation each process suffers due to contention-aware scheduling. Both proposals have been implemented and evaluated in a commercial single-threaded quad-core processor with a relatively small two-level cache hierarchy. The proposals reach, on average, a performance improvement by 5.38 and 6.64 percent when compared with the Linux scheduler, while this improvement is by 3.61 percent for an state-of-the-art memory contention-aware scheduler under the evaluated mixes.

Spatial Locality Speculation to Reduce Energy in Chip-Multiprocessor Networks-on-Chip

As processor chips become increasingly parallel, an efficient communication substrate is critical for meeting performance and energy targets. In this work, we target the root cause of network energy consumption through techniques that reduce link and router-level switching activity. We specifically focus on memory subsystem traffic, as it comprises the bulk of NoC load in a CMP. By transmitting only the flits that contain words predicted useful using a novel spatial locality predictor, our scheme seeks to reduce network activity. We aim to further lower NoC energy through microarchitectural mechanisms that inhibit datapath switching activity for unused words in individual flits. Using simulation-based performance studies and detailed energy models based on synthesized router designs and different link wire types, we show that 1) the prediction mechanism achieves very high accuracy, with an average rate of false-unused prediction of just 2.5 percent; 2) the combined NoC energy savings enabled by the predictor and microarchitectural support is 36 percent, on average, and up to 57 percent in the best case; and 3) there is no system performance penalty as a result of this technique.

A hardware–software co-design approach for implementing sparse matrix vector multiplication on FPGAs

The Field-Programmable Gate Array is an excellent match for the Sparse Matrix–Vector Multiply (SMVM) operation because of its enormous computational capacity and its ability to build a custom memory hierarchy that matches the memory access patterns of the operation. This paper describes a new sparse matrix storage format which works in conjunction with a custom memory subsystem which decodes the format on-the-fly. The SMVM operation is implemented on a single FPGA and a small parallel system of four FPGAs. The parameters that affect the performance of the sequential and parallel designs are investigated as well as the speedup for different matrices.

HMTT

DRAM access traces (i.e., off-chip memory references) can be extremely valuable for the design of memory subsystems and performance tuning of software. Hardware snooping on the off-chip memory interface is an effective and nonintrusive approach to monitoring and collecting real-life DRAM accesses. However, compared with software-based approaches, hardware snooping approaches typically lack semantic information, such as process/function/object identifiers, virtual addresses, and lock contexts, that is essential to the complete understanding of the systems and software under investigation. In this article, we propose a hybrid hardware/software mechanism that is able to collect off-chip memory reference traces with semantic information. We have designed and implemented a prototype system called HMTT (Hybrid Memory Trace Tool), which uses a custom-made DIMM connector to collect off-chip memory references and a high-level event-encoding scheme to correlate semantic information with memory references. In addition to providing complete, undistorted DRAM access traces, the proposed system is also able to perform various types of low-overhead profiling, such as object-relative accesses and multithread lock accesses.

Optimization of Application-Specific Memories

Memory access times are the primary bottleneck for many applications today. This “memory wall” is due to the performance disparity between processor cores and main memory. To address the performance gap, we propose the use of custom memory subsystems tailored to the application rather than attempting to optimize the application for a fixed memory subsystem. Custom subsystems can take advantage of application-specific properties as well as memory-specific properties to improve access times or write-backs given constraints on size or power.

A methodology to automatically optimize dynamic memory managers applying grammatical evolution

Modern consumer devices must execute multimedia applications that exhibit high resource utilization. In order to efficiently execute these applications, the dynamic memory subsystem needs to be optimized. This complex task can be tackled in two complementary ways: optimizing the application source code or designing custom dynamic memory management mechanisms. Currently, the first approach has been well established, and several automatic methodologies have been proposed. Regarding the second approach, software engineers often write custom dynamic memory managers from scratch, which is a difficult and error-prone work. This paper presents a novel way to automatically generate custom dynamic memory managers optimizing both performance and memory usage of the target application. The design space is pruned using grammatical evolution converging to the best dynamic memory manager implementation for the target application. Our methodology achieves important improvements (62.55% and 30.62% better on average in performance and memory usage, respectively) when its results are compared to five different general-purpose dynamic memory managers.

Analyzing Cross-Core Performance Interference on Multi-Core Processors Based on Statistical Learning

Cloud computing and multi-core processors are emerging to dominate the landscape of computing today. However, in terms of computing resources, the utilization of modern datacenters is rather low because of the potential negative and unpredictable cross-core performance interference. To provide QoS guarantees for some key applications, co-locations of such applications are disabled, causing computing resource overprovisioning. Therefore precise analysis for cross-core interference is a key challenge for improving resource utilization in datacenters. This study is motivated by the observation that the performance degradation of one application suffered from cross-core interference can be represented as a piecewise function of the aggregate pressures on memory subsystem from all cores, regardless of which applications are co-running and what their individual pressures are. The study results in a statistical learning-based method for predicting cross-core performance interference as well as predictor models using PCA linear regression, which can quantitatively and precisely predict performance degradation caused by memory subsystem contention in any applications. Experimental results show that the average prediction error of the proposed method is 1.1%.

Markov parameters tuning prediction to improve cache hit rate

Because the spatial and temporal locality of program codes, compiler could use heuristics and profile guided prediction to relocate the output of program codes to reduce the cache confliction. In this paper, for improving the average accessing time of memory subsystem by raising the instructions cache hit rate, compiler-assisted Markov parameters tuning MPT frequencies prediction techniques for codes relocation are proposed. Different with the traditional schemes that provide the fixed heuristics branch probability FHBP to calculate nodes frequencies NF for various kinds of programs, the Markov-based heuristics algorithm combining FHBP and parameters tuning techniques in MPT probability matrix is adopted for improving program NF prediction to take advantage of the probability matrix and model the control flow graph CFG in function more precision. The MPT model was simulated to illustrate the strengths of accurate expression of the program. Compare with the actual execution results of program with profile coverage tests, the experimental result is better fit to validate its feasibility.

Credit-based Runtime Placement of Virtual Machines on a Single NUMA System for QoS of Data Access Performance

While a NUMA system is being widely used as a target machine for virtualization, each data access request produced by a virtual machine (VM) on the NUMA system may have a different access time depending on not only remote access condition, but also shared resource contentions. Mainly due to this, each VM running on the NUMA system will have irregular data access performance at different times. Because existing hypervisors, such as KVM, VMware, and Xen, have yet to consider this, users of VMs cannot predict their data access performance or even recognize the data access performance they have experienced. In this paper, we propose a novel VM placement technique to resolve this issue pertaining to irregular data access performance of VMs running on the NUMA system. The hypervisor with our technique provides the illusion of a private memory subsystem to each VM, which guarantees the data access latency required by each VM on average. To enable this feature, we periodically evaluates the average data access latency of each VM using hardware performance monitoring units. After every evaluation, our Mcredit-based VM migration algorithm tries to migrate the VCPU or memory of the VM not meeting with its required data access latency to another node, giving the VM less data access latency. We implemented the prototype for KVM hypervisor on Linux 3.10.10. Experimental results show that, in the four-node NUMA system, our technique keeps the required data access performance levels of VMs running various workloads while it only consumes less than 1 percent of the cycles of a core and 0.3 percent of the system memory bandwidth.

CoMETC

We introduce a Coordinated Management of Energy, Thermal, and Cooling (CoMETC) technique to minimize cooling and memory energy of server machines. State-of-the-art solutions decouple the optimization of cooling energy costs and energy consumption of CPU and memory subsystems. This results in suboptimal solutions due to thermal dependencies between CPU and memory and the nonlinearity in energy costs of cooling. In contrast, we develop a unified solution that integrates energy, thermal, and cooling management for CPU and memory subsystems to maximize energy savings. CoMETC reduces the operational energy of the memory by clustering active memory pages to a subset of memory modules while accounting for thermal and cooling aspects. At the same time, CoMETC removes hotspots between and within the CPU sockets and reduces the effects of thermal coupling with memory in order to minimize cooling energy costs. We design CoMETC using a control-theoretic approach to guarantee meeting these objectives. We introduce a formal thermal and cooling model to be used for online decisions inside CoMETC. Our experimental results show that CoMETC achieves average cooling and memory energy savings of 58% compared to state-of-the-art techniques at a performance overhead of less than 0.3%.

Performance and energy impact of parallelization and vectorization techniques in modern microprocessors

While Moore's law states that the number of transistors is approximately doubled every 2 years, powering these transistors simultaneously is only possible as long as Dennard scaling continues. Unfortunately, voltage scaling has slowed down in recent years, and microprocessor designers have hit what is known as the "utilization wall" or the "dark silicon" effect. Vectorization, parallelization, specialization and heterogeneity are the key approaches to deal with this utilization wall. However, how software developers can maximize energy efficiency of these architectures remains an open question. This paper presents an energy evaluation of parallelization using both physical and logical cores (i.e., SMT/Hyper-Threading), vectorization (SSE, Advanced Vector Extensions and NEON) and dynamic core reconfiguration [ $$\hbox {Intel}^{\circledR }$$ Intel ® 's Turbo Boost Technology (TBT)]. The evaluation spans microprocessors for embedded, laptop, desktop and server markets, since there is a convergence among them towards energy efficiency. The analyzed processors include Intel's Core $$^\mathrm{TM}$$ TM i5 and i7 family and ARM $$^{\circledR }$$ ® 's Cortex $$^\mathrm{TM}$$ TM A9 and A15. Results show that software developers should prioritize vectorization over thread parallelism when possible, as it yields better energy efficiency, especially on the Intel platforms. Application scalability can be reduced drastically when using vectorization and threading simultaneously since vectorization increases pressure on the memory subsystem. Intel's TBT further improves energy efficiency by an additional 10---20 % depending on the number of active threads.

A comparison of instruction memories from the WCET perspective

Abstract Hard real-time systems demand high performance in combination with a timing predictable program execution. The performance of a system in the worst-case, represented by its worst case execution time (WCET), highly depends on the design of the memory subsystem. In this paper we focus on the instruction memory hierarchy and quantify the impact of different on-chip instruction memories on the worst-case timing of the system. A function-based dynamic instruction scratchpad (D-ISP), an instruction cache, and static instruction scratchpads using basic-block-based and function-based assignment algorithms are compared. Therefore, we provide WCET bounds for systems with different on-chip instruction memories and different off-chip memory timings. We show that for small memory sizes a static instruction scratchpad usually outperforms the other memories in terms of the WCET estimate. However, with increasing memory sizes the D-ISP is able to reach lower WCET bounds. An instruction cache can only provide lower WCET bounds than the other memories, if no suitable assignment for the static instruction scratchpads is found or if the D-ISP suffers from thrashing or frequently loads unused code.