DRAM Architecture Research Articles

Design Strategies for BCAT Structures: Enhancing DRAM Reliability and Mitigating Row Hammer Effect

This study investigates the impact of four parameters—gate angles, fin height controlled through gate overlaps and the distance from fin to source/drain, and substrate bottom doping concentration—on the row hammer effect (RHE) in DRAM cells. The influence of adjacent and passing gates on the DRAM cell body potential was identified as a key factor in D0 and D1 failures. The tolerance for D1 and D0 failures was analyzed, defined as the threshold number of pulses required to induce a 0.6 V change in the storage node voltage (from 1.2 V to 0.6 V for a D1 failure or from 0 V to 0.6 V for a D0 failure). D1 (D0) failure tolerances with the slope from the top of the top gate (θangle) of 3°, the height of the TiN gate covering the fin (Hfin_overlap) of 12.5 nm, and the height of the fin (Hfin) of 12.5 nm are 1.26 × 106 (4.8 × 106), 1.14 × 106 (4 × 107), and 7.5 × 105 (4.8 × 105), respectively. Higher θangles and smaller fin heights generally result in higher RHE tolerances. Although decreasing the fin height reduced the RHE, it also decreased the on-current and resulted in an increase in the threshold voltage (VT) and the subthreshold swing (SS). In addition, by increasing the substrate bottom doping concentration (Pdop_bot), we improve RHE tolerance twice its original level without reducing the on-current. Therefore, designing a buried channel array transistor (BCAT) structure requires careful consideration of these trade-offs, and a thorough understanding of the underlying mechanism is crucial to devising strategies that reduce RHE tolerance. The findings of this study are expected to contribute significantly to the development of next-generation DRAM architectures, enhancing stability and performance. By addressing the reliability challenges posed by advanced scaling, this study paves the way for the ongoing advancement of DRAM technology for high-density and high-performance applications.

Logic-Compatible Embedded DRAM Architecture for Multifunctional Digital Storage and Compute-in-Memory

The compute-in-memory (CIM) which embeds computation inside memory is an attractive scheme to circumvent von Neumann bottlenecks. This study proposes a logic-compatible embedded DRAM architecture that supports data storage as well as versatile digital computations. The proposed configurable memory unit operates in three modes: (1) memory mode in which it works as a normal dynamic memory, (2) logic–arithmetic mode where it performs bit-wise Boolean logic and full adder operations on two words stored within the memory array, and (3) convolution mode in which it executes digitally XNOR-and-accumulate (XAC) operation for binarized neural networks. A 1.0-V 4096-word × 8-bit computational DRAM implemented in a 45-nanometer CMOS technology performs memory, logic and arithmetic operations at 241, 229, and 224 MHz while consuming the energy of 7.92, 8.09, and 8.19 pJ/cycle. Compared with conventional digital computing, it saves energy and latency of the arithmetic operation by at least 47% and 46%, respectively. For VDD = 1.0 V, the proposed CIM unit performs two 128-input XAC operations at 292 MHz with an energy consumption of 20.8 pJ/cycle, achieving 24.6 TOPS/W. This marks at least 11.9× better energy efficiency and 38.8× better delay, thereby achieving at least 461× better energy-delay product than traditional 8-bit wide computing hardware.

Design and analysis of gate all around stacked nanosheet-DRAM for future technology node

This paper reports a stacked Dynamic random-access transistor (DRAM) memory structure that is based on gate all-around nanosheet access transistor. A TCAD study is done to compare nanosheet DRAM and conventional saddle fin recessed channel access transistor (SRCAT) in terms of DRAM electrical characteristics and its row hammer-induced leakage. The nanosheet DRAM shows superior characteristics in terms of current driving capability, speed, and refresh than SRCAT. The nanosheet DRAM also shows significantly lower hammer-induced failure as compared to SRCAT because the original leakage path from the cell to the neighboring cell gets blocked due to the nanosheet device structure. We also investigate the effect of spacer length on nanosheet DRAM characteristics and show that extended spacer length is favorable for having better DRAM characteristics due to the floating body effect. Our study demonstrates the potential, and advantages of nanosheet DRAM architecture compared to the conventional SRCAT DRAM.

A Highly Parallel DRAM Architecture to Mitigate Large Access Latency and Improve Energy Efficiency of Modern DRAM Systems

A Highly Parallel DRAM Architecture to Mitigate Large Access Latency and Improve Energy Efficiency of Modern DRAM Systems

CDAR-DRAM: Enabling Runtime DRAM Performance and Energy Optimization via In-Situ Charge Detection and Adaptive Data Restoration

With the increasing of DRAM’s capacity, the refresh operation rapidly becomes a major concern to the performance of the current computational system. Moreover, conservative timing parameters adopted for access operations make an increasing amount of negative impact on system performance and energy efficiency. In this paper, we propose an in-situ charge detection and adaptive data restoration DRAM (CDAR-DRAM) architecture, which can dynamically adjust the refresh rate and relax the constraints on access timing by removing pessimistic timing margins for PVT variations. CDAR-DRAM employs a lowcost skewed-inverter-based detector to monitor the bitline voltage in runtime and estimate real-time timing parameters of cells. Based on the detector, an adaptive refresh and restore scheme (CDAR-ref) is presented, which progressively reduces the refresh rate and partially restores cells’ voltage just enough for cells with sufficient charge, thereby optimizing both refresh and restoration operations. Moreover, a supplementary adaptive access scheme (CDAR-acc) is presented, which detects the runtime charge level of recently-accessed rows and reduces access latency aggressively, benefitting workloads in a single-core system and memory nonintensive workloads in a multi-core system. CDAR’s flexibility allows the two schemes to be combined. The evaluation shows that in an 8-core system, the combined scheme improves performance and energy efficiency by 15.2% and 22.6%, respectively.

A Low-Cost Reduced-Latency DRAM Architecture With Dynamic Reconfiguration of Row Decoder

DRAM latency has remained almost constant over decades and has become a performance bottleneck of computing systems. In this study, we propose a low-cost DRAM architecture enabling dynamic reconfiguring of row decoder to provide reduced latency with high flexibility and reliability. We apply minimum changes to row decoders and allow dynamic reconfiguration to switch array blocks between two modes: 1) normal mode, where the DRAM array behaves in the same manner as the conventional DRAM does and 2) low latency mode, where two DRAM cells in the neighbor array blocks are coupled to operate as a logical cell and reduce latency reliably according to the differential principle. On the basis of an industrial open bitline (BL) cell array, we only change the word-line decoding scheme but keep the cell array and sense amplifiers (SAs) untouched to avoid modifications to the DRAM process for cost and reliability considerations. Our circuit simulation shows that the low-latency mode can reduce row-to-column delay and row access strobe time by 25.7% and 23.2%, respectively. We evaluate the reduced-latency LPDDR4 DRAM on various workloads. Compared with the JEDEC standard DRAM, our proposal provides a maximum system performance improvement of 8.5%. We believe that our proposal is a reliable and cost-friendly solution to DRAM latency reduction.

A Weighted Current Summation Based Mixed Signal DRAM-PIM Architecture for Deep Neural Network Inference

Processing-in-Memory (PIM) is an emerging approach to bridge the memory-computation gap. One of the major challenges of PIM architectures in the scope of Deep Neural Network (DNN) inference is the implementation of area-intensive Multiply-Accumulate (MAC) units in memory technologies, especially for DRAM-based PIMs. The DRAM architecture restricts the integration of DNN computation units near the area optimized commodity DRAM Sub-Array (SA) or Primary Sense Amplifier (PSA) region, where the data parallelism is maximum and the data movement cost is minimum. In this paper, we present a novel DRAM-based PIM architecture that is based on bit-decomposed MAC operation and Weighted Current Summation (WCS) technique to implement the MAC unit with minimal additional circuitry in the PSA region by leveraging on mixed-signal design. The architecture presents a two-stage design that employs light-weight current mirror based analog units near the SAs in the PSA region, whereas all the other substantial logic is integrated near the bank peripheral region. Hence, our architecture attains a balance between the data parallelism, data movement energy and area optimization. For an 8-bit CNN inference, our novel 8Gb DRAM PIM device achieves a peak performance of 142.8GOPS while consuming a power of 756.76mW, which results in an energy efficiency of 188.8GOPS/W. The area overhead of such an 8Gb device for a 2ynm DRAM technology is 12.63% in comparison to a commodity 8Gb DRAM device.

An Approximate DRAM Design with an Adjustable Refresh Scheme for Low-power Deep Neural Networks

A DRAM device requires periodic refresh operations to preserve data integrity, which incurs significant power consumption. Slowing down the refresh rate can reduce the power consumption; however, it may cause a loss of data stored in a DRAM cell, which affects the correctness of computation. This paper proposes a new memory architecture for deep learning applications, which reduces the refresh power consumption while maintaining accuracy. Utilizing the error-tolerant property of deep learning applications, the proposed memory architecture avoids the accuracy drop caused by data loss by flexibly controlling the refresh operation for different bits, depending on their criticality. For data storage in deep learning applications, the approximate DRAM architecture reorganizes the data so that these data are mapped to different DRAM devices according to their bit significance. Critical bits are stored in more frequently refreshed devices while non-critical bits are stored in less frequently refreshed devices. Compared to the conventional DRAM, the proposed approximate DRAM requires only a separation of the chip select signal for each device in a DRAM rank and a minor change in the memory controller. Simulation results show that the refresh power consumption is reduced by 66.5 % with a negligible accuracy drop on state-of-the-art deep neural networks.

Amnesiac DRAM: A Proactive Defense Mechanism Against Cold Boot Attacks

DRAMs in modern computers or hand-held devices store private or often security-sensitive data. Unfortunately, one known attack vector, called a cold boot attack, remains threatening and easy-to-exploit, especially when attackers have physical access to the device. It exploits the fundamental property of current DRAMs: remanence effects that retain the stored contents for a certain period of time even after powering off. To magnify the remanence effect, cold boot attacks typically freeze the victim DRAM, thereby providing a chance to detach, move, and reattach it to an attacker's computer. Once power is on, attackers can steal all the security-critical information from the victim's DRAM, such as a master decryption key for an encrypted disk storage. Two types of defenses were proposed in the past: 1) CPU-bound cryptography, where keys are stored in CPU registers and caches instead of in DRAMs, and 2) full or partial memory encryption, where sensitive data are stored encrypted. However, both methods impose non-negligible performance or energy overheads to the running systems, and worse, significantly increase the hardware and software manufacturing costs. We found that these proposed solutions attempted to address the cold boot attacks passively: either by avoiding or by indirectly addressing the root cause of the problem, the remanence effect. In this article, we propose and evaluate a proactive defense mechanism, Amnesiac DRAM, that comprehensively prevents the cold boot attacks. The key idea is to discard the contents in the DRAM when attackers attempt to retrieve (i.e., power on) them from the stolen DRAM. When Amnesiac DRAM senses a physical separation, it locks itself and deletes all the remaining contents, making it amnesiac. The Amnesiac DRAM causes neither performance nor energy overhead in ordinary operations (e.g., load and store) and can be easily implemented with negligible area overhead in commodity DRAM architectures.

A Novel DRAM Architecture for Improved Bandwidth Utilization and Latency Reduction Using Dual-Page Operation

Emerging memory-intensive applications require a paradigm shift from processor-centric to memory-centric computing. The performance of state-of-the-art computing systems and accelerators designed for such applications is not limited by the processing speed but rather by the limited DRAM bandwidth and long DRAM latencies. Although, the interface frequency of new commodity DRAM memories is constantly increasing to achieve a higher peak bandwidth, this bandwidth cannot be fully utilized due to long internal latencies. Page-misses (i.e., change of active row) are one of the key contributors to long access latencies that result in a low bandwidth utilization. In this brief, we propose a novel DRAM sub-array level architecture referred to as Dual-Sense-Amplifier (DSA) that masks the page-miss latency by allowing individual sub-arrays or banks to flexibly open two rows concurrently. Additionally, the DSA architecture design aim to be fully compatible with any type of commodity DRAM architecture (e.g., HBM, GDDR, etc.) and to retain most of its circuit designs. The area overhead of DSA for an 8Gb device is 9.6% compared to a commodity DRAM device. On average, a DSA 8Gb device improves the bandwidth utilization by 17.53% and reduces the average response latency by 31.59 ns compared to commodity DRAM devices operated at 800 MHz. Moreover, we demonstrate that the energy consumption overhead of a DSA DRAM in comparison to a commodity DRAM is negligible.

An Approximate Memory Architecture for Energy Saving in Deep Learning Applications

DRAM devices require periodic refresh operations to preserve data integrity. Slowing down the refresh rate can reduce the energy consumption; however, it may cause a loss of data stored in the DRAM cell. This paper proposes a new memory architecture of soft approximation for deep learning applications, which reduces the refresh energy consumption while maintaining accuracy and high performance. Utilizing the error-tolerant property of deep learning applications, the proposed memory architecture avoids the accuracy drop caused by data loss by flexibly controlling the refresh operation for different bits, depending on their criticality. For data storage, the approximate DRAM architecture reorganizes the data so that these data are sorted according to their bit significance. Critical bits are stored in more frequently refreshed devices while non-critical bits are stored in less frequently refreshed devices. In addition, for further reduction of the DRAM energy consumption, this paper combines hard approximation, which reduces the number of accesses to DRAM, with soft approximation. Simulation results show that the refresh energy consumption is reduced by 69.71%, and the total energy consumption is reduced by 26.0 % for the hybrid memory with a negligible drop in both training and testing phases on state-of-the-art deep networks.

NoM: Network-on-Memory for Inter-Bank Data Transfer in Highly-Banked Memories

Data copy is a widely-used memory operation in many programs and operating system services. In conventional computers, data copy is often carried out by two separate read and write transactions that pass data back and forth between the DRAM chip and the processor chip. Some prior mechanisms propose to avoid this unnecessary data movement by using the shared internal bus in the DRAM chip to directly copy data within the DRAM chip (e.g., between two DRAM banks). While these methods exhibit superior performance compared to conventional techniques, data copy across different DRAM banks is still greatly slower than data copy within the same DRAM bank. Hence, these techniques have limited benefit for the emerging 3D-stacked memories (e.g., HMC and HBM) that contain hundreds of DRAM banks across multiple memory controllers. In this paper, we present Network-on-Memory (NoM), a lightweight inter-bank data communication scheme that enables direct data copy across both memory banks of a 3D-stacked memory. NoM adopts a TDM-based circuit-switching design, where circuit setup is done by the memory controller. Compared to state-of-the-art approaches, NoM enables both fast data copy between multiple DRAM banks and concurrent data transfer operations. Our evaluation shows that NoM improves the performance of data-intensive workloads by 3.8X and 75%, on average, compared to the baseline conventional 3D-stacked DRAM architecture and state-of-the-art techniques, respectively.

DR Refresh: Releasing DRAM Potential by Enabling Read Accesses Under Refresh

Emerging data analytic workloads such as graph processing, neural network and edge data preprocesing desire efficient memory read operations. Unfortunately, due to the necessity of dynamic refresh, modern DRAM systems have to stall access during refresh cycles. As DRAM device density continues to grow, refresh operations can be a crucial throughput bottleneck. To fully unleash memory access performance, we revisit conventional refresh mechanism and DRAM architecture. We propose DR refresh, a specific refresh mechanism that enable read and refresh operations to be done simultaneously. We devise DR DRAM, a specific memory hardware system that can efficiently deploy DR refresh. Unlike traditional refresh, DR explores device refresh that only refreshes a designated device at a time. Meanwhile, DR increases read efficiency by recovering the inaccessible data that resides on a device under refreshing. We also propose Hybrid Refresh Main Memory (HRMM) which can designate refresh schemes (DR or traditional refresh) in a specific memory space. We expect that our design can benefit many real-life tasks such as SPEC CPU2006, CNN, LLT and PageRank.

Vehicle data management with specific wear-levelling and fault tolerance for hybrid DRAM-NVM memory

Abstract The emerging Non-Volatile Memory (NVM) attracts attentions in recent years because of its fast reading/writing speed, high density, low idle power and persistence. With such characteristics, NVM can be applied to application-specific embedded systems to improve performance such as automotive systems. However, NVM has limitation on writing operations and is consequently not suitable to be applied in writing sensitive systems. In this paper, we propose a Vehicle Data Management (VDM) approach based on a hybrid DRAM and NVM architecture for automotive systems. VDM approach includes automotive dynamic data management, vehicle-specific wear-levelling and vehicle fault tolerance. Automotive dynamic data management can distinguish the hot or cold data from vehicle Electronic Control Units (ECUs) and sensors, and then manage the data to reduce the writing operations on NVM for the hybrid main memory. Vehicle-specific wear-levelling can balance the wear to prolong the lifetime of NVM. A fault tolerance mechanism is also provided for NVM to recover the fault blocks. The experimental results show that both automotive dynamic data management and vehicle-specific wear-levelling can effectively reduce the writing operations compared with other approaches. Vehicle fault tolerance can achieve a high recovery rate and a low time overhead when error rate is not very high.

Exploiting OS-Level Memory Offlining for DRAM Power Management

Power and energy consumed by main memory systems in data-center servers have increased as the DRAM capacity and bandwidth increase. Particularly, background power accounts for a considerable fraction of the total DRAM power consumption; the fraction will increase further in the near future, especially when slowing-down technology scaling forces us to provide necessary DRAM capacity through plugging in more DRAM modules or stacking more DRAM chips in a DRAM package. Although current DRAM architecture supports low power states at rank granularity that turn off some components during idle periods, techniques to exploit memory-level parallelism make the rank-granularity power state become ineffective. Furthermore, the long wake-up latency is one of obstacles to adopting aggressive power management (PM) with deep power-down states. By tackling the limitations, we propose OffDIMM that is a software-assisted DRAM PM collaborating with the OS-level memory onlining/offlining. OffDIMM maps a memory block in the address space of the OS to a subarray group or groups of DRAM, and sets a deep power-down state for the subarray group when offlining the block. Through the dynamic OS-level memory onlining/offlining based on the current memory usage, our experimental results show OffDIMM reduces background power by 24 percent on average without notable performance overheads.

WIRD: An Efficiency Migration Scheme in Hybrid DRAM and PCM Main Memory for Image Processing Applications

Using a hybrid main memory in embedded systems to process image processing applications has become an irresistible trend. However, the performance deficiencies (less write endurance and relative longer write latency) in phase change memory (PCM) have become the bottleneck for performance improvement in the whole system. To further improve the performance in the hybrid memory system, data migration schemes have been put forward to migrate the sequential data between DRAM and PCM. Nevertheless, the existing schemes are always universal type page migration schemes, which cannot be aware of the write hot pages in image processing applications accurately. In addition, a large number of unnecessary page migrations occurring has the potential to increase the latency overhead of page management and destroy the locality of the applications leading to performance degradation. In this paper, focusing on the image processing applications, we present a better estimator in predicting the future memory writes: inter-reference distance with history writes frequency. Based on this observation, we present a novel page migration scheme for hybrid DRAM and PCM memory architecture called write frequency with inter-reference distance (WIRD). The scheme monitors access patterns, migrates pages between DRAM and PCM relying on the memory controller. The experimental results show that our scheme minimizes the unnecessary page migrations which make most of the write hot pages absorbed by DRAM efficiently. Meanwhile, the high-efficiency character of WIRD has decreased the write to read switching ratio in the system which makes the PCM effective page write access time reduced to a large extent.

What Your DRAM Power Models Are Not Telling You

Main memory (DRAM) consumes as much as half of the total system power in a computer today, due to the increasing demand for memory capacity and bandwidth. There is a growing need to understand and analyze DRAM power consumption, which can be used to research new DRAM architectures and systems that consume less power. A major obstacle against such research is the lack of detailed and accurate information on the power consumption behavior of modern DRAM devices. Researchers have long relied on DRAM power models that are predominantly based off of a set of standardized current measurements provided by DRAM vendors, called IDD values. Unfortunately, we find that state-of-the-art DRAM power models are often highly inaccurate, as these models do not reflect the actual power consumed by real DRAM devices. To build an accurate model and provide insights into DRAM power consumption, we perform the first comprehensive experimental characterization of the power consumed by modern real-world DRAM modules. Our extensive characterization of 50 DDR3L DRAM modules from three major vendors yields four key new observations about DRAM power consumption that prior models cannot capture: (1) across all IDD values that we measure, the current consumed by real DRAM modules varies significantly from the current specified by the vendors; (2) DRAM power consumption strongly depends on the data value that is read or written; (3) there is significant structural variation, where the same banks and rows across multiple DRAM modules from the same model consume more power than other banks or rows; and (4) over successive process technology generations, DRAM power consumption has not decreased by as much as vendor specifications have indicated. Because state-of-the-art DRAM power models do not account for any of these four key characteristics, they are highly inaccurate compared to the actual, measured power consumption of 50 real DDR3L modules. Based on our detailed analysis and characterization data, we develop the Variation-Aware model of Memory Power Informed by Real Experiments (VAMPIRE). VAMPIRE is a new, accurate power consumption model for DRAM that takes into account (1) module-to-module and intra-module variations, and (2) power consumption variation due to data value dependency. We show that VAMPIRE has a mean absolute percentage error of only 6.8% compared to actual measured DRAM power. VAMPIRE enables a wide range of studies that were not possible using prior DRAM power models. As an example, we use VAMPIRE to evaluate the energy efficiency of three different encodings that can be used to store data in DRAM. We find that a new power-aware data encoding mechanism can reduce total DRAM energy consumption by an average of 12.2%, across a wide range of applications. We have open-sourced both VAMPIRE and our extensive raw data collected during our experimental characterization.

Reducing DRAM Latency at Low Cost by Exploiting Heterogeneity.

In modern systems, DRAM-based main memory is signi?cantly slower than the processor.Consequently, processors spend a long time waiting to access data from main memory, makingthe long main memory access latency one of the most critical bottlenecks to achieving highsystem performance. Unfortunately, the latency of DRAM has remained almost constant inthe past decade. This is mainly because DRAM has been optimized for cost-per-bit, ratherthan access latency. As a result, DRAM latency is not reducing with technology scaling, andcontinues to be an important performance bottleneck in modern and future systems.This dissertation seeks to achieve low latency DRAM-based memory systems at low costin three major directions. The key idea of these three major directions is to enable and ex-ploit latency heterogeneity in DRAM architecture. First, based on the observation that longbitlines in DRAM are one of the dominant sources of DRAM latency, we propose a newDRAM architecture, Tiered-Latency DRAM (TL-DRAM), which divides the long bitline intotwo shorter segments using an isolation transistor, allowing one segment to be accessed withreduced latency. Second, we propose a ?ne-grained DRAM latency reduction mechanism,Adaptive-Latency DRAM, which optimizes DRAM latency for the common operating conditions for individual DRAM module. We observe that DRAM manufacturers incorporate a very large timing margin as a provision against the worst-case operating conditions, whichis accessing the slowest cell across all DRAM products with the worst latency at the highesttemperature, even though such a slowest cell and such an operating condition are rare. Ourmechanism dynamically optimizes DRAM latency to the current operating condition of theaccessed DRAM module, thereby reliably improving system performance. Third, we observethat cells closer to the peripheral logic can be much faster than cells farther from the peripherallogic (a phenomenon we call architectural variation). Based on this observation, we propose anew technique, Architectural-Variation-Aware DRAM (AVA-DRAM), which reduces DRAMlatency at low cost, by pro?ling and identifying only the inherently slower regions in DRAMto dynamically determine the lowest latency DRAM can operate at without causing failures.This dissertation provides a detailed analysis of DRAM latency by using both circuit-levelsimulation with a detailed DRAM model and FPGA-based pro?ling of real DRAM modules.Our latency analysis shows that our low latency DRAM mechanisms enable significant latencyreductions, leading to large improvement in both system performance and energy e?fficiencyacross a variety of workloads in our evaluated systems, while ensuring reliable DRAM operation.

What Your DRAM Power Models Are Not Telling You

Main memory (DRAM) consumes as much as half of the total system power in a computer today, due to the increasing demand for memory capacity and bandwidth. There is a growing need to understand and analyze DRAM power consumption, which can be used to research new DRAM architectures and systems that consume less power. A major obstacle against such research is the lack of detailed and accurate information on the power consumption behavior of modern DRAM devices. Researchers have long relied on DRAM power models that are predominantly based off of a set of standardized current measurements provided by DRAM vendors, called IDD values. Unfortunately, we find that state-of-the-art DRAM power models are often highly inaccurate when compared with the real power consumed by DRAM. This is because existing DRAM power models (1) are based off of the worst-case power consumption of devices, as vendor specifications list the current consumed by the most power-hungry device sold; (2) do not capture variations in DRAM power consumption due to different data value patterns; and (3) do not account for any variation across different devices or within a device.

Bank-Group Level Parallelism

DDR4 SDRAM introduced a new hierarchy in DRAM organization: bank-group (BG). The main purpose of BG is to increase I/O bandwidth without growing DRAM-internal bus-width. We, however, found that other benefits can be derived from the new hierarchy. To achieve the benefits, we propose a new DRAM architecture using the BG-hierarchy, leading to a creation of BG-Level Parallelism (BGLP). By exploiting BGLP, the overall parallelism grows in DRAM operations. We also argue that BGLP is a feasible solution in the cost-sensitive DRAM industry because the additional cost is negligible and only cost-insensitive area needs to be modified.