3D DRAM Research Articles

NeuroTAP: Thermal and Memory Access Pattern-aware Data Mapping on 3D DRAM for Maximizing DNN Performance

Deep neural networks (DNNs) have been widely adopted, owing to break-through performance and high accuracy. DNNs exhibit varying memory behavior involving specific and recognizable memory access patterns and access intensity, depending on the selected data reuse in different layers. Such applications have high memory bandwidth demands due to aggressive computations, performing several billion-floating-point-operations-per-second (BFLOPs). 3D DRAMs, providing very high memory access bandwidth, are extensively employed to break the memory wall , bridging the gap between compute and memory while running DNNs. However, the vertical integration in 3D DRAM introduces serious thermal issues, resulting from high power density and close proximity of memory cells, and requires dynamic thermal management (DTM). To unleash the true potential of 3D DRAM and exploit the enormous bandwidth under thermal constraints, there is a need to intelligently map the DNN application’s data across memory channels, pseudo-channels, and banks, minimizing the effective memory latency and reducing the thermal-induced application slowdown. The specific memory access patterns exhibited by a DNN layer execution are crucial to determine a favourable data mapping method for 3D DRAM dies that potentially causes minimal thermal impact, and also maximize DRAM bandwidth utilization. In this work, we propose an application-aware and thermal-sensitive data mapping that intelligently assigns portions of the 3D DRAM to DNN layers, leveraging the knowledge about layer’s memory access patterns and minimizing DTM-induced performance overheads. Additionally, we also deploy a DRAM low-power states based DTM mechanism to keep the 3D DRAM within safe thermal limits. Using our proposal, we observe a performance improvement of 1% to 61%, and memory energy savings of 1% to 55% for popular deep neural networks over state-of-the-art DTM strategies while running DNN inference.

Partially Isolated Dual Work Function Gate IGZO TFT With Obviously Reduced Leakage Current for 3D DRAMs

Partially Isolated Dual Work Function Gate IGZO TFT With Obviously Reduced Leakage Current for 3D DRAMs

3-Terminal Igzo FET Based 2T0C DRAM Combined Bit-Line Structure

The IGZO (InGaZnO)-based two-transistor zero-capacitor(2T0C) DRAM has attracted much attention as an alternative memory to overcome the scale-down limit of current 1T1C DRAM due to its low power consumption and monolithic 3D stacking capability. In particular, its low power consumption and the feasibility of low temperature process make it highly implementable for 3D DRAM. For operating the 2T0C DRAM, four metal lines are necessary, i.e., write word line and write bit line (WBL) for operating write transistor (WTR), and read word line and read bit line (RBL) for operating read transistor (RTR).In this study, we propose a 3-terminal 2T0C DRAM to enhance its memory characteristics, where RBL was combined with WBL, enabling 2T0C memory operation with only three metal lines as shown in Fig. 1(b). In addition, we investigated the dependency of retenetion time of the proposed 2T0C DRAM on channel length. In particular we evaluated the dependency of the sneak current of 2T0C DRAM on off-current of transistor. For the fabrication of 3-terminal IGZO 2T0C DRAM, a top gate IGZO transistor was fabricated using RF sputtering and two transistors were connected by storage node and bit line bridge as shown in Fig. 1(d).The storage node was formed by connecting the gate of RTR to drain of WTR. And source electrodes of both transistors were coupled. To determine the voltage values for memory operation, the transfer curve of each transistor was characterized and the threshold voltages for both RTR and WTR were around 1 V. The write and read voltages were set to 3 and 2 V, respectively. The retention time was measured by performing periodic read operations after writing and measuring the RTR currents. In addition, the retention time of transistors for varying channel length and gate capacitance was characterized. The retention time was defined as the time to be taken for decreasing the currents by 90 %. Furthermore, the disturbance of 2T0C DRAM with 2x2 array structure was investigated. The RTR currents of selected cells were measured depending on the memory state of adjacent cells.It was found that the retention times of transistors with 10- and 100-um RTR channel lengths were approximately 15 and 30 seconds, respectively. It was observed that the memory state was not changed from “0“ to “1“ by disturbant currents, but, RTR currents of “0“ state cell increased slightly when an adjacent cell was turned on. However, as shown in Fig.1(i) and (j), when off-currents of transistor of 2T0C DRAM decreased from 20 nA to 200 pA, it was confirmed the sneak current of 2T0C decreased. Finally, several technical apporaches how to suppress the disturbant currents will be presented in detail. Acknowledgement This research was supported by BrainKorea21 Four. Figure 1

NeuroCool: Dynamic Thermal Management of 3D DRAM for Deep Neural Networks through Customized Prefetching

Deep neural network (DNN) implementations are typically characterized by huge datasets and concurrent computation, resulting in a demand for high memory bandwidth due to intensive data movement between processors and off-chip memory. Performing DNN inference on general-purpose cores/edge is gaining attraction to enhance user experience and reduce latency. The mismatch in the CPU and conventional DRAM speed leads to under-utilization of the compute capabilities, causing increased inference time. 3D DRAM is a promising solution to effectively fulfill the bandwidth requirement of high-throughput DNNs. However, due to high power density in stacked architectures, 3D DRAMs need dynamic thermal management (DTM), resulting in performance overhead due to memory-induced CPU throttling. We study the thermal impact of DNN applications running on a 3D DRAM system, and make a case for a memory temperature-aware customized prefetch mechanism to reduce DTM overheads and significantly improve performance. In our proposed NeuroCool DTM policy, we intelligently place either DRAM ranks or tiers in low power state, using the DNN layer characteristics and access rate. We establish the generalization of our approach through training and test datasets comprising diverse data points from widely used DNN applications. Experimental results on popular DNNs show that NeuroCool results in a average performance gain of 44% (as high as 52%) and memory energy improvement of 43% (as high as 69%) over general-purpose DTM policies.

Amorphous oxide semiconductor for monolithic 3D DRAM: an enabler or passer-by?

Whether amorphous oxide semiconductor (AOS) is an enabler or pass-by for monolithic 3D DRAM is discussed, with current challenges and future directions proposed in this perspective.

The intensity enhancement of transmission small angle x-ray scattering from nanostructures with a high aspect ratio

Transmission small angle x-ray scattering (tSAXS) has been developed as a metrology for the critical dimension (CD) measurements to facilitate integrated circuit (IC) chip fabrications. Synchrotron x-ray sources were used for their high brilliance and a wavelength less than one tenth of a nanometer was chosen for its high penetration power to enable transmission measurements through a silicon wafer with a nominal thickness of 0.7 mm. A major hold back preventing tSAXS from reaching wide applications in IC fabrication is the lack of high brilliance laboratory-based x-ray sources. Within the last few years, even without any major breakthrough in x-ray source technology, this tSAXS metrology has finally been used for 3D NAND and DRAM, i.e., memory chips with tall or high aspect ratio (HAR) architectures. The scattering intensities from HAR structures will be discussed quantitatively in terms of the sample height and the effective longitudinal coherence length of the incident x-ray.

Surface reaction modelling of thermal atomic layer etching on blanket hafnium oxide and its application on high aspect ratio structures

Thermal atomic layer etching is rapidly becoming an important complementary processing technology in the manufacturing of 5 and 3 nm devices in the semiconductor industry. Critically, architectures such as 3D NAND and 3D DRAM require conformal isotropic etching to remove material such as HfO2 in hard-to-reach locations with aspect ratios that can be greater than 50:1. To achieve repeatable device performance throughout a 3D stack, the removal rate (etch per cycle) of the etched material during an etch process needs to be controlled such that the overall etch amount is the same from top to bottom of the device stack. In this work, the reaction kinetics of reactants and byproducts during a cyclical ligand exchange-based atomic layer etching (ALE) process have been modelled. This ALE process consists of two steps: a fluorination step followed by a fluorine-to-chlorine ligand exchange-based removal step. Modeling was performed for each of those steps separately. Experimental data revealed that the fluorine dosing during the fluorination step was predominantly responsible for controlling the etch rate of the ALE process but had only a minimal impact on the etch profile inside high aspect ratio holes. The ligand exchange dosing, on the other hand, predominantly controlled the etch profile (depth loading) with equal etch rates from top-to-bottom, obtained when the step was operated close to saturation. The model predicts that the chemical reaction rate of dimethylaluminum chloride (DMAC) on a fluorinated surface during the ligand exchange step is 9.1 s−1, about 46 times greater than the reaction rate of hydrogen fluoride (HF) on the hafnium oxide surface during the fluorination step (only 0.2 s−1). Furthermore, modeling results revealed that the sticking coefficient for DMAC on a hafnium fluoride surface far exceeded that of HF on a hafnium oxide surface in the conditions modelled (0.94 s−1 for DMAC vs 0.0058 s−1 for HF). With these modeling results, the different roles fluorination and ligand exchange steps have regarding the control of etch rate per cycle and profile inside high aspect ratio holes can be explained.

Capacitor-Less 4F DRAM Using Vertical InGaAs Junction for Ultimate Cell Scalability

In this work, we demonstrated capacitor-less 4F2 2-terminal InGaAs npn junction DRAM through careful device design. Using epitaxially grown InGaAs which have a steep junction, fabricated InGaAs bistable resistor (biristor) DRAM showed low voltage operation (~2 V), fast switching speed (<20 ns), long-term retention (103 s at 85 °C), and high endurance (>1010 cycles) with a high sensing margin. Considering this feasibility study, we believe that InGaAs <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{n}^{+}$ </tex-math></inline-formula> pn+ junction DRAM could be a good technological option for future scalable 3D DRAM.

(Invited) Transport and Reaction Kinetics of Thermal ALE in High Aspect Ratio Hafnium Oxide Structures

Thermal atomic layer etching is rapidly becoming an important complementary processing technology in manufacturing 5 and 3 nm devices in the semiconductor industry. Critically, stacked chip architectures such as 3D NAND and 3D DRAM require conformal isotropic etching to remove material such as HfO2 in hard-to-reach locations with aspect ratios that can be >50. To achieve repeatable device performance throughout a 3D stack, the removal rate (etch per cycle) of the etched material during an etch process need to be controlled such that the overall etch is the same from top to bottom of the device stack. In this work, we have modelled the reaction kinetics and transport processes of reactants and by-products during a cyclical ligand exchange-based ALE process. This ALE process consists of two steps: a fluorination step, followed by a fluorine-to-chlorine ligand exchange-based removal step. Experimental data revealed that the fluorine dosing during the fluorination step was predominantly responsible for controlling the etch rate of the ALE process but had only a minimal impact on the etch profile inside these holes. The ligand exchange dosing, on the other hand, predominantly controlled the etch profile (depth loading) with equal etch rates top-to-bottom obtained when the step was operated close to saturation. Our model predicts, in agreement with the experiment, that adsorption and reaction rates during fluorination on HfO2 surfaces are significantly slower than transport times inside these deep holes leading to essentially flat fluorination profiles even if the fluorination step is not operated in saturation mode. In contrast, transport rates with the ligand exchange molecule are slow in comparison but adsorption and ligand exchange rates with the fluorinated hafnium appear to be significantly higher than for fluorine during the fluorination step. Slow transport in combination with high surface reaction rates for the ligand exchange step led to an etch rate that was dependent on aspect ratio (feature depth) in processes that used sub-saturation exposures.

EBrainII: a 3\xa0kW Realtime Custom 3D DRAM Integrated ASIC Implementation of a Biologically Plausible Model of a Human Scale Cortex

The Artificial Neural Networks (ANNs), like CNN/DNN and LSTM, are not biologically plausible. Despite their initial success, they cannot attain the cognitive capabilities enabled by the dynamic hierarchical associative memory systems of biological brains. The biologically plausible spiking brain models, e.g., cortex, basal ganglia, and amygdala, have a greater potential to achieve biological brain like cognitive capabilities. Bayesian Confidence Propagation Neural Network (BCPNN) is a biologically plausible spiking model of the cortex. A human-scale model of BCPNN in real-time requires 162 TFlop/s, 50 TBs of synaptic weight storage to be accessed with a bandwidth of 200 TBs. The spiking bandwidth is relatively modest at 250 GBs/s. A hand-optimized implementation of rodent scale BCPNN has been done on Tesla K80 GPUs require 3 kWs, we extrapolate from that a human scale network will require 3 MWs. These power numbers rule out such implementations for field deployment as cognition engines in embedded systems.The key innovation that this paper reports is that it is feasible and affordable to implement real-time BCPNN as a custom tiled application-specific integrated circuit (ASIC) in 28 nm technology with custom 3D DRAM - eBrainII - that consumes 3 kW for human scale and 12 watts for rodent scale. Such implementations eminently fulfill the demands for field deployment.

DRAMsim3: A Cycle-Accurate, Thermal-Capable DRAM Simulator

DRAM technology has developed rapidly in recent years. Several industrial solutions offer 3D packaging of DRAM and some are envisioning the integration of CPU and DRAM on the same die. These solutions allow higher density and better performance and also lower power consumption in DRAM designs. However, accurate simulation tools have not kept up with DRAM technology, especially for the modeling of 3D DRAMs. In this letter we present a cycle-accurate, validated DRAM simulator, and DRAMsim3, which offers the best simulation performance and feature sets among existing cycle-accurate DRAM simulators. DRAMsim3 is also the first DRAM simulator to offer runtime thermal modeling alongside with performance modeling.

High-voltage CD-SEM-based application to monitor 3D profile of high-aspect-ratio features

Background: In-line metrology for three-dimensional (3D) profiling high-aspect-ratio (HAR) features is highly important for manufacturing semiconductor devices, particularly for memory devices, such as 3D NAND and DRAM. Aim: Our purpose was to obtain the cross-sectional profiles of the HAR features from top-view critical dimension scanning electron microscopy (CD-SEM) images. Approach: Based on Monte Carlo simulation results, we proposed a method for 3D profiling of HAR features using backscattered electron (BSE) signal intensities. Several kinds of HAR holes with different taper angles and bowing geometries were fabricated. High-voltage CD-SEM was used for experiments to determine the feasibility of our approach. Results: Using the BSE line-profile, we constructed cross sections of the taper holes and estimated sidewall angles (SWAs), which were approximately the same as those observed using field-emission scanning electron microscopy (FE-SEM). The constructed cross sections of the bowing holes and the trends of the geometric variance, which were estimated by the middle CD and its depth, were consistent with the cross sections observed by FE-SEM. Conclusions: The results demonstrate that the variation in the HAR holes, such as SWA and bowing geometry, can be measured and monitored using the BSE images.

Stress-Induced Performance Shifts in 3D DRAMs

3D-stacked DRAMs can significantly increase cell density and bandwidth while also lowering power consumption. However, 3D structures experience significant thermomechanical stress due to the differential rate of contraction of the constituent materials, which have different coefficients of thermal expansion. This impacts circuit performance. This article develops a procedure that performs a performance analysis of 3D DRAMs, capturing the impact of both layout-aware stress and layout-independent stress on parameters such as latency, leakage power, refresh power, area, and bus delay. The approach first proposes a semianalytical stress analysis method for the entire 3D DRAM structure, capturing the stress induced by through-silicon-vias (TSVs), micro bumps, package bumps, and warpage. Next, this stress is translated to variations in device mobility and threshold voltage, after which analytical models for latency, leakage power, and refresh power are derived. Finally, a complete analysis of performance variations is performed for various 3D DRAM layout configurations to assess the impact of layout-dependent stress. We explore the use of alternative flexible package substrate options to mitigate the performance impact of stress. Specifically, we explore the use of an alternative bendable package substrate made of polyimide to reduce warpage-induced stress, and show that it reduces stress-induced variations and improves the performance metrics for stacked 3D DRAMs.

Parana: A Parallel Neural Architecture Considering Thermal Problem of 3D Stacked Memory

Recent advances in deep learning (DL) have stimulated increasing interests in neural networks (NN). From the perspective of operation type and network architecture, deep neural networks can be categorized into full convolution-based neural network (ConvNet), recurrent neural network (RNN), and fully-connected neural network (FCNet). Different types of neural networks are usually cascaded and combined as a hybrid neural network (Hybrid-NN) to complete real-life cognitive tasks. Such hybrid-NN implementation is memory-intensive with large number of memory accesses, hence the performance of hybrid-NN is often limited by the insufficient memory bandwidth. A “3D + 2.5D” integration system, which integrates a high-bandwidth 3D stacked DRAM side-by-side with a highly-parallel neural processing unit (NPU) on a silicon interposer, overcomes the bandwidth bottleneck in hybrid-NN acceleration. However, intensive concurrent 3D DRAM accesses produced by the NPU lead to a serious thermal problem in 3D DRAM. In this paper, we propose a neural processor called Parana for hybrid-NN acceleration in consideration of thermal problem of 3D DRAM. Parana solves the thermal problem of 3D memory by optimizing both the total number of memory accesses and memory accessing behaviors. For memory accessing behaviors, Parana balances the memory bandwidth by spatial division mapping hybrid-NN onto computing resources, which efficiently avoids that masses of memory accesses are issued in a short time period. To reduce the total number of memory accesses, we design a new NPU architecture and propose a memory-oriented tiling and scheduling mechanism to exploit the maximum utilization of on-chip buffer. Experimental results show that Parana reduces the peak temperature by up to 54.72 $^\circ$ C and the steady temperature by up to 32.27 $^\circ$ C over state-of-the-art accelerators with 3D memory without performance degradation.

Trade-off between Hit Rate and Hit Latency for Optimizing DRAM Cache

Due to the large storage capacity, high bandwidth and low latency, 3D DRAM is proposed to be the last level cache, referred to as DRAM cache. The hit rate and hit latency are two conflicting optimization goals for DRAM cache. To address this issue, we design a new DRAM organization that trades the lower hit rate for shorter hit latency by way-locator cache and novel cache set layout. We have designed a novel DRAM cache organization to simultaneously achieve a good hit rate and shorter latency, referred to as SODA-cache. The SODA-cache adapts 2-way set associate cache motivated by the observation that 2-way set associative cache provides the most hit rate improvement from the direct-mapped cache to highly associative cache. The proposed way-locator cache and a novel set layout effectively reduce the cache-hit latency. We use SPEC 2006 CPU benchmark to evaluate our design and two stat-of-art DRAM cache designs. Experimental results show that SODA-cache can improve hit rate by 8.1 percent compared with Alloy-cache and reduce average access latency by 23.1, 13.2 and 8.6 percent compared with LH-cache, Alloy-cache and ATCache respectively on average. Accordingly, SODA-cache outperforms over LH-cache, Alloy-cache and ATCache by on average 17, 12.8 and 8.4 percent respectively in term of weighted speedup.

A Built-Off Self-Repair Scheme for Channel-Based 3D Memories

Redundancy repair is a commonly used technique for memory yield improvement. In order to ensure high repair rate and final product yield, it is necessary to develop a repair scheme for the coming three-dimensional (3D) architecture of stacked DRAM. According to the JEDEC mobile memory technology roadmap, the interface of 3D DRAM, including the Wide I/O and High-Bandwidth Memory (HBM), is mainly classified as channel-based memories. In this paper, we propose a built-off self-test (BOSR) scheme at the controller level for channel-based 3D memory to enhance final product yield after the bonding of a memory cube to its corresponding logic die. The logic die contains the Channel controller, in which the BOSR circuit resides. Experimental results show that the repair rate is high with higher cluster failure ratio due to the flexible algorithm we choose. The area overhead is low and it decreases significantly when the memory size or channel count increases. The performance penalty is also low due to the parallel execution of address comparison and repair. Moreover, the manufacture cost is lower than conventional DRAM architecture due to allocator-based redundancies. Finally, the proposed scheme can easily be applied to other channel-based 3D memories.

Improving 3D DRAM Fault Tolerance Through Weak Cell Aware Error Correction

Although the emerging 3D DRAM products can significantly improve the computing system performance, the relatively high cost is one of the most critical issues that prevent their wide real-life adoption. Intuitively, a strong memory fault tolerance can be leveraged to reduce the fabrication cost of DRAM dies, and the total cost will reduce if the fabrication cost saving can off-set the cost overhead of memory fault tolerance. Nevertheless, such a simple concept can be a practically viable option only for 3D DRAM because: (1) The stacked logic die can solely implement memory fault tolerance inside 3D DRAM chips, obviating any changes on the host CPUs and CPU-DRAM interfaces. (2) With the total ownership of both the logic die and DRAM dies inside 3D DRAM chips, DRAM manufacturers can fully exploit the potential to truly minimize the 3D DRAM bit cost. Following this intuition, we developed a 3D DRAM fault tolerance design strategy. It can achieve a very strong tolerance to weak DRAM cells at very small redundancy and latency overhead. The key is to cohesively leverage the detectability of weak cells and runtime configurability of error correction code (ECC) decoding. In addition, this design strategy can gracefully embrace the inaccuracy of weak cell detection (e.g., weak cell miss-detection and false-detection). We carried out thorough mathematical analysis, and the results show that, under the redundancy overhead of 1:8 (same as today’s ECC DIMM), this design strategy can tolerate the weak cell rate of as high as $10^{-4}$ and 6 $\times 10^{-5}$ if 100 and 90 percent of all the weak cells are known in prior. Using Micron’s hybrid memory cube (HMC) 3D DRAM chips as the test vehicle, we evaluated the implementation cost and the results show that it only consumes less than 0.4 mm2 (45 nm node) on the logic die. Using CPU and DRAM simulators, we further carried out simulations over a variety of computing benchmarks and the results show that this design solution only incurs less than 2 percent performance degradation on average.

RATT-ECC

This article proposes a rate-adaptive, two-tiered error-correction scheme (RATT-ECC) that provides strong reliability (10 10 x reduction in raw FIT rate) for an HBM-like 3D DRAM system. The tier-1 code is a strong symbol-based code that can correct errors due to small granularity faults and detect errors caused by large granularity faults; the tier-2 code is an XOR-based code that corrects errors detected by the tier-1 code. The rate-adaptive feature of RATT-ECC enables permanent bank failures to be handled through sparing. It can also be used to significantly reduce the refresh power consumption without decreasing reliability and timing performance.

3D-ProWiz: An Energy-Efficient and Optically-Interfaced 3D DRAM Architecture with Reduced Data Access Overhead

This paper introduces 3D-ProWiz , which is a high-bandwidth, energy-efficient, optically-interfaced 3D DRAM architecture with fine grained data organization and activation. 3D-ProWiz integrates sub-bank level 3D partitioning of the data array to enable fine-grained activation and greater memory parallelism. A novel method of routing the internal memory bus to individual subarrays using TSVs and fanout buffers enables 3D-ProWiz to use smaller dimension subarrays without significant area overhead. The use of TSVs at subarray-level granularity eliminates the need to use slow and power hungry global lines, which in turn reduces the random access latency and activation-precharge energy. 3D-ProWiz yields the best latency and energy consumption values per access among other well-known 3D DRAM architectures. Experimental results with PARSEC benchmarks indicate that 3D-ProWiz achieves 41.9 percent reduction in average latency, 52 percent reduction in average power, and 80.6 percent reduction in energy-delay product (EDP) on average over DRAM architectures from prior work.

Process Variation-Aware Nonuniform Cache Management in a 3D Die-Stacked Multicore Processor

Process variations in integrated circuits have significant impact on their performance, leakage, and stability. This is particularly evident in large, regular, and dense structures such as DRAMs. DRAMs are built using minimized transistors with presumably uniform speed in an organized array structure. Process variation can introduce latency disparity among different memory arrays. With the proliferation of 3D stacking technology, DRAMs become a favorable choice for stacking on top of a multicore processor as a last level cache for large capacity, high bandwidth, and low power. Hence, variations in bank speed create a unique problem of nonuniform cache accesses in 3D space. In this paper, we investigate cache management techniques for tolerating process variation in a 3D DRAM stacked onto a multicore processor. We modeled the process variation in a four-layer DRAM memory, including cell transistor, capacitor trench, and peripheral circuit, to characterize the latency and retention time variations among different banks. As a result, the notion of fast and slow banks from the core's standpoint is no longer associated with their physical distances with the banks. They are determined by the different bank latencies due to process variation. We develop cache migration schemes that utilize fast banks while limiting the cost due to migration. Our experiments show that there is a great performance benefit in exploiting fast memory banks through migration. On average, a variation-aware management can improve the performance of a workload over the baseline (where one of the slowest bank speed is assumed for all banks) by 16.5 percent. We are also only 0.8 percent away in performance from an ideal memory where no process variation is present.