Memory Design Space Research Articles

GAHLS: an optimized graph analytics based high level synthesis framework

The urgent need for low latency, high-compute and low power on-board intelligence in autonomous systems, cyber-physical systems, robotics, edge computing, evolvable computing, and complex data science calls for determining the optimal amount and type of specialized hardware together with reconfigurability capabilities. With these goals in mind, we propose a novel comprehensive graph analytics based high level synthesis (GAHLS) framework that efficiently analyzes complex high level programs through a combined compiler-based approach and graph theoretic optimization and synthesizes them into message passing domain-specific accelerators. This GAHLS framework first constructs a compiler-assisted dependency graph (CaDG) from low level virtual machine (LLVM) intermediate representation (IR) of high level programs and converts it into a hardware friendly description representation. Next, the GAHLS framework performs a memory design space exploration while account for the identified computational properties from the CaDG and optimizing the system performance for higher bandwidth. The GAHLS framework also performs a robust optimization to identify the CaDG subgraphs with similar computational structures and aggregate them into intelligent processing clusters in order to optimize the usage of underlying hardware resources. Finally, the GAHLS framework synthesizes this compressed specialized CaDG into processing elements while optimizing the system performance and area metrics. Evaluations of the GAHLS framework on several real-life applications (e.g., deep learning, brain machine interfaces) demonstrate that it provides 14.27× performance improvements compared to state-of-the-art approaches such as LegUp 6.2.

Analytical Derivation of Concurrent Reuse Distance Profile for Multi-Threaded Application Running on Chip Multi-Processor

Reuse distance has been shown to be a useful metric for performance analysis of caches and programs, locality analysis and compiler optimization. Concurrent reuse distance profile defined as the reuse distance profile of a thread sharing the cache with many other threads varies from its standalone reuse distance profile due to interference from other threads. Measurement of reuse distance profile through simulation, especially for multi-threaded applications, consumes lot of time. Analytical model based reuse distance prediction can reduce drastically the time taken for exploring the cache memory design space. The objective of this work is to propose an analytical model to find the concurrent reuse distance profile of a thread belonging to multi-threaded applications in a shared memory environment. Using the standalone reuse distance profile of each thread as input, we derive three other reuse distance profiles: 1) The concurrent reuse distance profile of a thread sharing the cache with other threads 2) The combined reuse distance profile of all threads sharing the cache and 3) The coherent reuse distance profile of each thread, considering the coherency effect when each thread runs with private cache. We use Markov chain besides combinatorics and basic probability theory as a main analytical tool for the model. We validate our analytical model against simulations, using the multi-core simulator Sniper for the benchmarks of the PARSEC and the SPLASH benchmark suites.

An Analytical Model for Performance and Lifetime Estimation of Hybrid DRAM-NVM Main Memories

Emerging Non-Volatile Memories (NVMs) have promising advantages (e.g., lower idle power, higher density, and non-volatility) over the existing predominant main memory technology, DRAM. Yet, NVMs also have disadvantages (e.g., longer latencies, higher active power, and limited endurance). System architects are therefore examining hybrid DRAM-NVM main memories to enable the advantages of NVMs while avoiding the disadvantages as much as possible. Unfortunately, the hybrid memory design space is very large and complex due to the existence of very different types of NVMs and their rapidly-changing characteristics. Therefore, optimization of performance and lifetime of hybrid memory based computing platforms and their experimental evaluation using traditional simulation methods can be very time-consuming and sometimes even impractical. As such, it is necessary to develop a fast and flexible analytical model to estimate the performance and lifetime of hybrid memories on various workloads. This paper presents an analytical model for hybrid memories based on Markov decision processes. The proposed model estimates the hit ratio and lifetime for various configurations of DRAM-NVM hybrid main memories. Our model also provides accurate estimation of the effect of data migration policies on the hybrid memory hit ratio (i.e., percentage of accesses supplied by either DRAM or NVM), one of the most important factors in hybrid memory performance and lifetime. Such an analytical model can aid designers to tune hybrid memory configurations to improve performance and/or lifetime. We present several optimizations that make our model more efficient while maintaining its accuracy. Our experimental evaluations conducted using the PARSEC benchmark suite show that the proposed model (a) accurately predicts the hybrid memory hit ratio compared to the state-of-the-art hybrid memory simulators with an average (maximum) error of 4.61 percent (13.6 percent) on a commodity server (equipped with 192 GB main memory and quad-core Xeon processor), (b) accurately estimates the NVM lifetime with an average (maximum) error of 2.93 percent (8.8 percent), and (c) is on average (up to) 4x (10x) faster than conventional state-of-the-art simulation platforms for hybrid memories.

Strategic Priming with Multiple Antigens can Yield Memory Cell Phenotypes Optimized for Infection with Mycobacterium tuberculosis: A Computational Study.

Lack of an effective vaccine results in 9 million new cases of tuberculosis (TB) every year and 1.8 million deaths worldwide. Although many infants are vaccinated at birth with BCG (an attenuated M. bovis), this does not prevent infection or development of TB after childhood. Immune responses necessary for prevention of infection or disease are still unknown, making development of effective vaccines against TB challenging. Several new vaccines are ready for human clinical trials, but these trials are difficult and expensive; especially challenging is determining the appropriate cellular response necessary for protection. The magnitude of an immune response is likely key to generating a successful vaccine. Characteristics such as numbers of central memory (CM) and effector memory (EM) T cells responsive to a diverse set of epitopes are also correlated with protection. Promising vaccines against TB contain mycobacterial subunit antigens (Ag) present during both active and latent infection. We hypothesize that protection against different key immunodominant antigens could require a vaccine that produces different levels of EM and CM for each Ag-specific memory population. We created a computational model to explore EM and CM values, and their ratio, within what we term Memory Design Space. Our model captures events involved in T cell priming within lymph nodes and tracks their circulation through blood to peripheral tissues. We used the model to test whether multiple Ag-specific memory cell populations could be generated with distinct locations within Memory Design Space at a specific time point post vaccination. Boosting can further shift memory populations to memory cell ratios unreachable by initial priming events. By strategically varying antigen load, properties of cellular interactions within the LN, and delivery parameters (e.g., number of boosts) of multi-subunit vaccines, we can generate multiple Ag-specific memory populations that cover a wide range of Memory Design Space. Given a set of desired characteristics for Ag-specific memory populations, we can use our model as a tool to predict vaccine formulations that will generate those populations.

An Efficient Compiler Framework for Cache Bypassing on GPUs

Graphics processing units (GPUs) have become ubiquitous for general purpose applications due to their tremendous computing power. Initially, GPUs only employ scratchpad memory as on-chip memory. Though scratchpad memory benefits many applications, it is not ideal for those general purpose applications with irregular memory accesses. Hence, GPU vendors have introduced caches in conjunction with scratchpad memory in the recent generations of GPUs. The caches on GPUs are highly configurable. The programmer or compiler can explicitly control cache access or bypass for global load instructions. This highly configurable feature of GPU caches opens up the opportunities for optimizing the cache performance. In this paper, we propose an efficient compiler framework for cache bypassing on GPUs. Our objective is to efficiently utilize the configurable cache and improve the overall performance for general purpose GPU applications. In order to achieve this goal, we first characterize GPU cache utilization and develop performance metrics to estimate the cache reuses and memory traffic. Next, we present efficient algorithms that judiciously select global load instructions for cache access or bypass. Finally, we present techniques to explore the unified cache and shared memory design space. We integrate our techniques into an automatic compiler framework that leverages parallel thread execution instruction set architecture to enable cache bypassing for GPUs. Experiments evaluation on NVIDIA GTX680 using a variety of applications demonstrates that compared to cache-all and bypass-all solutions, our techniques improve the performance from 4.6% to 13.1% for 16 KB L1 cache.

Design Space Exploration of Typical STT MTJ Stacks in Memory Arrays in the Presence of Variability and Disturbances

High leakage power in sub-100-nm memory technology nodes drives the need for nonvolatile memory devices to reduce power consumption and enhance battery life. Spin-transfer torque magnetic tunneling junction (STT-MTJ) is a promising nonvolatile memory device with comparable read and write performances as SRAM and eDRAM with almost zero standby power. In this paper, we present a simulation framework that can solve transport [using nonequilibrium Green's function (NEGF) formalism] and magnet dynamics [using Landau-Lifshitz-Gilbert (LLG) equation] self-consistently to study the read and write performances of STT-MTJ. Due to process variations, thermal disturbances, and stray fields, the performance of STT-MTJ degrades and results in one transistor-one STT-MTJ (IT-ISTTMTJ) memory failures. A thorough memory design space investigation can help us to reduce such failures. Hence, we present a design space exploration framework for IT-ISTTMTJ memory, which consists of magnetic materials with different RAPA products, different genres of MTJ stacks, and a transistor. A comprehensive study based on critical memory performance metrics such as tunneling magnetoresistance, JC, and write cycle shows the relative merits and demerits of each MTJ stack for embedded memory applications. Finally, the benefits of synthetic antiferromagnet free layer in providing immunity against stray fields are shown illustrating the need for coupled free-layer stacks in scaled technology nodes.

Local Memory Design Space Exploration for High-Performance Computing

The performance of high-performance computing (HPC) applications highly depends on the memory subsystem due to the huge data sets used that do not fit into the cache hierarchy. Besides, energy efficiency has become a main design factor and, consequently, both performance and energy efficiency are primary goals in HPC designs. As a result, energy-efficient high-performance memory subsystem designs should be explored. In this paper, we extend the architecture of general-purpose processors by adding a software-managed local memory (LM) and a very simple programmable DMA controller. We demonstrate that with these extensions—together with efficient run-time management—we improve performance and energy consumption factors. We perform an LM design space exploration study for an Intel® Pentium® 4 platform: we analyze the performance, energy and energy-delay product for a total of 27 computational loops of the NAS benchmarks. We show a 1.2x performance speedup factor and an energy reduction of 6.21% on average when using a constrained 32Â KB LM with commodity memory bandwidths (6.4Â GB/s). More aggressive configurations (i.e. 256Â KB LM + 12.8 GB/s) show at least 2.14x performance speedup factors and energy savings of 42.07% on average.

A Compiler-Based Approach for Dynamically Managing Scratch-Pad Memories in Embedded Systems

Optimizations aimed at improving the efficiency of on-chip memories in embedded systems are extremely important. Using a suitable combination of program transformations and memory design space exploration aimed at enhancing data locality enables significant reductions in effective memory access latencies. While numerous compiler optimizations have been proposed to improve cache performance, there are relatively few techniques that focus on software-managed on-chip memories. It is well-known that software-managed memories are important in real-time embedded environments with hard deadlines as they allow one to accurately predict the amount of time a given code segment will take. In this paper, we propose and evaluate a compiler-controlled dynamic on-chip scratch-pad memory (SPM) management framework. Our framework includes an optimization suite that uses loop and data transformations, an on-chip memory partitioning step, and a code-rewriting phase that collectively transform an input code automatically to take advantage of the on-chip SPM. Compared with previous work, the proposed scheme is dynamic, and allows the contents of the SPM to change during the course of execution, depending on the changes in the data access pattern. Experimental results from our implementation using a source-to-source translator and a generic cost model indicate significant reductions in data transfer activity between the SPM and off-chip memory.