Large DRAM Research Articles

HCFTL: A Locality-Aware Flash Translation Layer for Efficient Address Translation

In modern solid-state drives (SSDs), a flash translation layer (FTL) is the core engine performing the logical-to-physical address translations. However, the increasing capacity of SSDs requires a large DRAM space to hold the mapping table in a page-level FTL. Due to the limited size of built-in DRAM, existing FTL schemes selectively cache some active mapping entries in DRAM, while store the entire mapping table on flash. However, the low cache hit ratio introduced by the traditional mapping entry eviction policy degrades the access performance of SSDs. In this article, we propose a novel FTL, hot-clusterity FTL (HCFTL), to improve the hit ratio of cached mapping table (CMT) of SSDs with limited cache space. HCFTL clusters mapping entries, which are newly evicted from the cache, into dynamic translation pages (DTPs). Reading entries from DTPs will increase the CMT hit ratio due to spatial and temporal localities. Because the logical page numbers (LPNs) in a DTP are not consecutive, we introduce two different efficient index structures to speedup the lookup of mapping entries in DTPs. Furthermore, to make the DTP index highly accurate, we additionally allocate an auxiliary cache to buffer the mapping entries newly evicted from CMT, and produce dynamic pages with minimum difference between the maximal and the minimal LPNs of entries in those pages. Our experiments show that HCFTL can improve the CMT hit ratio by up to 41.1% and decrease the system response time by up to 33.3%, compared to baseline FTL schemes.

Power-optimized Deployment of Key-value Stores Using Storage Class Memory

High-performance flash-based key-value stores in data-centers utilize large amounts of DRAM to cache hot data. However, motivated by the high cost and power consumption of DRAM, server designs with lower DRAM-per-compute ratio are becoming popular. These low-cost servers enable scale-out services by reducing server workload densities. This results in improvements to overall service reliability, leading to a decrease in the total cost of ownership (TCO) for scalable workloads. Nevertheless, for key-value stores with large memory footprints, these reduced DRAM servers degrade performance due to an increase in both IO utilization and data access latency. In this scenario, a standard practice to improve performance for sharded databases is to reduce the number of shards per machine, which degrades the TCO benefits of reduced DRAM low-cost servers. In this work, we explore a practical solution to improve performance and reduce the costs and power consumption of key-value stores running on DRAM-constrained servers by using Storage Class Memories (SCM). SCMs in a DIMM form factor, although slower than DRAM, are sufficiently faster than flash when serving as a large extension to DRAM. With new technologies like Compute Express Link, we can expand the memory capacity of servers with high bandwidth and low latency connectivity with SCM. In this article, we use Intel Optane PMem 100 Series SCMs (DCPMM) in AppDirect mode to extend the available memory of our existing single-socket platform deployment of RocksDB (one of the largest key-value stores at Meta). We first designed a hybrid cache in RocksDB to harness both DRAM and SCM hierarchically. We then characterized the performance of the hybrid cache for three of the largest RocksDB use cases at Meta (ChatApp, BLOB Metadata, and Hive Cache). Our results demonstrate that we can achieve up to 80% improvement in throughput and 20% improvement in P95 latency over the existing small DRAM single-socket platform, while maintaining a 43–48% cost improvement over our large DRAM dual-socket platform. To the best of our knowledge, this is the first study of the DCPMM platform in a commercial data center.

Storage Class Memory Based Hybrid Memory System for Practical Remote Sensing

Koo, S.M.; Seo, J.M.; Song. Y.J., and Baek, S.J., 2020. Storage class memory based hybrid memory system for practical remote sensing. In: Jung, H.-S.; Lee, S., and Ryu, J.-H. (eds.), In: Jung, H.-S.; Lee, S.; Ryu, J.-H., and Cui, T. (eds.), Advances in Geospatial Research of Coastal Environments. Journal of Coastal Research, Special Issue No. 102, pp. 254-260. Coconut Creek (Florida), ISSN 0749-0208.In the remote sensing domain, large-sized, files such as high-resolution satellite images or sonar video clips from unmanned underwater vehicles are very common. For processing big files, a large main memory is necessary. The main memory capacity highly influences the performance of computer systems. The demand for DRAM capacity has never been satisfied, and more importantly, large DRAM systems suffer from significant power consumption. To ease the problem, a promising solution is to build a hybrid main memory (HMM) system composed of a small number of fast DRAMs and many inexpensive devices. By mimicking large and fast main memory capacity, HMM allows computer systems to run applications that require more DRAM than is installed on the system. In this paper, a novel HMM management scheme for a storage class memory (SCM)-based HMM was introduced. As all the data stored on SCM are already non-volatile, the overall performance of the computer system is enhanced further by not flushing them periodically. The proposed idea was implemented on Linux and its performance was measured using an SCM emulation system. It was shown that HMM efficiently improves performance by up to 77.9 %, compared with a conventional operating system. Additionally, it was demonstrated that the proposed idea's fault recovery mechanisms could restore dirty data that are not yet synchronized with storage, within 91 % of the test time.

DSM

The number of cores and the capacities of main memory in modern systems have been growing significantly. Specifically, memory scaling, although at a slower pace than computation scaling, provided opportunities for very large DRAMs with Terabytes (TBs) capacity. Consequently, addressing the performance and energy consumption bottlenecks of DRAMs is more important than ever. DRAM memory refresh operation is one of the main contributing factors to the memory overheads, especially for large capacity DRAMs used in modern servers and emerging large-scale data centers. This paper addresses the memory refresh problem by leveraging the fact that most cloud servers host virtualized systems that use similar kernels, libraries, etc. We propose and experimentally evaluate a novel approach that exploits this observation to address the DRAM refresh overhead in such systems. More specifically, in this work, we present DSM, a light-weight hardware extension in memory controller to detect the pages with same content in memory and refresh only one of them and redirect the requests to the others to this page. Our detailed experimental analysis shows that the proposed DSM design can reduce 99th percentile memory access latency by up to 2.01x, and it also reduces the overall memory energy consumption by up to 8.5%.

DSM

The number of cores and the capacities of main memory in modern systems have been growing significantly. Specifically, memory scaling, although at a slower pace than computation scaling, provided opportunities for very large DRAMs with Terabytes (TBs) capacity. Consequently, addressing the performance and energy consumption bottlenecks of DRAMs is more important than ever. DRAM memory refresh operation is one of the main contributing factors to the memory overheads, especially for large capacity DRAMs used in modern servers and emerging large-scale data centers. This paper addresses the memory refresh problem by leveraging the fact that most cloud servers host virtualized systems that use similar kernels, libraries, etc. We propose and experimentally evaluate a novel approach that exploits this observation to address the DRAM refresh overhead in such systems. More specifically, in this work, we present DSM, a light-weight hardware extension in memory controller to detect the pages with same content in memory and refresh only one of them and redirect the requests to the others to this page. Our detailed experimental analysis shows that the proposed DSM design can reduce 99\textsuperscriptth percentile memory access latency by up to 2.01x, and it also reduces the overall memory energy consumption by up to 8.5%.

McDRAM v2: In-Dynamic Random Access Memory Systolic Array Accelerator to Address the Large Model Problem in Deep Neural Networks on the Edge

The energy efficiency of accelerating hundreds of MB-large deep neural networks (DNNs) in a mobile environment is less than that of a server-class big chip accelerator because of the limited power budget, silicon area, and smaller buffer size of static random access memory associated with mobile systems. To address this challenge and provide powerful computing capability for processing large DNN models in power/resource-limited mobile systems, we propose McDRAM v2, which is a novel in-dynamic random access memory (DRAM) systolic array accelerator architecture. McDRAM v2 makes the best use of large in-DRAM bandwidths for accelerating various DNN applications. It can handle large DNN models without off-chip memory accesses, in a fast and efficient manner, by exposing the large DRAM capacity and large in-DRAM bandwidth directly to an input systolic array of a processing element matrix. Additionally, it maximizes data reuse using a systolic multiply-accumulate (MAC) structure. The proposed architecture maximizes the utilization of large-scale MAC units by judiciously exploiting the DRAM's internal bus and buffer structure. An evaluation of large DNN models in the fields of image classification, natural language processing, and recommendation systems shows that it achieves 1.7 times tera operations per second (TOPS), 3.7 times TOPS/watt, and 8.6 times TOPS/mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> improvements over a state-of-the-art mobile graphics processing unit accelerator, and 4.1 times better energy efficiency over a state-of-the-art server-class accelerator. Moreover, it incurs a minimal overhead, i.e., a 9.7% increase in area, and uses less than 4.4 W of peak operating power.

Correctable and uncorrectable errors using large scale DRAM DIMMs in replacement network servers

This paper investigated DRAM DIMM errors using field records in replacement network servers. Large DRAM samples of about 40 K were collected over a 2.5 years period from 23 different server types, included various DIMMs from three different DRAM manufacturers with densities between 4 and 128 GB, and speeds between 1066 and 2400 Mbps. Errors that occurred during system operation were classified as either correctable (CE) or uncorrectable (UE) errors based on error correction code (ECC) schemes built into the servers. Of the collected DIMMS, 24% had recorded errors, where CE-only, UE-only, and UE and CE together comprised 28%, 43%, and 29% of recorded errors, respectively. Since UEs can cause large-scale failures, systems are replaced upon any UE occurrence. Approximately half UE-only DIMMs had 1 UE error. In contrast, many DIMMs had billions of CE errors, where a faulty location may be repetitively accessed. Such drastic differences in UE and CE counts help explain the importance of ECC and error mitigation schemes. Comparative analyses of errors were made over the manufacturers and operating speeds. After reasonable adjustments for repetitive counts of errors, failure in time (FIT) differences were up to 38% over manufacturers. Higher speed DIMMs generally had higher FIT with 2400 Mbps DIMMs exhibiting 6.7 times FIT of 1066 Mbps DIMMs.

DOMe: A deduplication optimization method for the NewSQL database backups.

Reducing duplicated data of database backups is an important application scenario for data deduplication technology. NewSQL is an emerging database system and is now being used more and more widely. NewSQL systems need to improve data reliability by periodically backing up in-memory data, resulting in a lot of duplicated data. The traditional deduplication method is not optimized for the NewSQL server system and cannot take full advantage of hardware resources to optimize deduplication performance. A recent research pointed out that the future NewSQL server will have thousands of CPU cores, large DRAM and huge NVRAM. Therefore, how to utilize these hardware resources to optimize the performance of data deduplication is an important issue. To solve this problem, we propose a deduplication optimization method (DOMe) for NewSQL system backup. To take advantage of the large number of CPU cores in the NewSQL server to optimize deduplication performance, DOMe parallelizes the deduplication method based on the fork-join framework. The fingerprint index, which is the key data structure in the deduplication process, is implemented as pure in-memory hash table, which makes full use of the large DRAM in NewSQL system, eliminating the performance bottleneck problem of fingerprint index existing in traditional deduplication method. The H-store is used as a typical NewSQL database system to implement DOMe method. DOMe is experimentally analyzed by two representative backup data. The experimental results show that: 1) DOMe can reduce the duplicated NewSQL backup data. 2) DOMe significantly improves deduplication performance by parallelizing CDC algorithms. In the case of the theoretical speedup ratio of the server is 20.8, the speedup ratio of DOMe can achieve up to 18; 3) DOMe improved the deduplication throughput by 1.5 times through the pure in-memory index optimization method.

SPCM

Phase Change Memory (PCM) devices are one of the known promising technologies to take the place of DRAM devices with the aim of overcoming the obstacles of reducing feature size and stopping ever growing amounts of leakage power. In exchange for providing high capacity, high density, and nonvolatility, PCM Multilevel Cells (MLCs) impose high write energy and long latency. Many techniques have been proposed to resolve these side effects. However, read performance issues are usually left behind the great importance of write latency, energy, and lifetime. In this article, we focus on read performance and improve the critical path latency of the main memory system. To this end, we exploit striping scheme by which multiple lines are grouped and lie on a single MLC line array. In order to achieve more performance gain, an adaptive ordering mechanism is used to sort lines in a group based on their read frequency. This scheme imposes large energy and lifetime overheads due to its intensive demand for higher write bandwidth. Thus, we equipped our design with a grouping/pairing write queue to synchronize write-back requests such that all updates to an MLC array occur at once. The design is also augmented by a directional write scheme that takes benefits of the uniformity of accesses to the PCM device—caused by the large DRAM cache—to determine the writing mode (striped or nonstriped). This adaptation to write operations relaxes the energy and lifetime overheads. We improve the read latency of a 2-bit MLC PCM memory by more than 24% (and Instructions Per Cycle (IPC) by about 9%) and energy-delay product by about 20% for a small lifetime degradation of 8%, on average.

A fully associative, tagless DRAM cache

This paper introduces a tagless cache architecture for large in-package DRAM caches. The conventional die-stacked DRAM cache has both a TLB and a cache tag array, which are responsible for virtual-to-physical and physical-to-cache address translation, respectively. We propose to align the granularity of caching with OS page size and take a unified approach to address translation and cache tag management. To this end, we introduce cache-map TLB (cTLB), which stores virtual-to-cache, instead of virtual-to-physical, address mappings. At a TLB miss, the TLB miss handler allocates the requested block into the cache if it is not cached yet, and updates both the page table and cTLB with the virtual-to-cache address mapping. Assuming the availability of large in-package DRAM caches, this ensures that an access to the memory region within the TLB reach always hits in the cache with low hit latency since a TLB access immediately returns the exact location of the requested block in the cache, hence saving a tag-checking operation. The remaining cache space is used as victim cache for memory pages that are recently evicted from cTLB. By completely eliminating data structures for cache tag management, from either on-die SRAM or in-package DRAM, the proposed DRAM cache achieves best scalability and hit latency, while maintaining high hit rate of a fully associative cache. Our evaluation with 3D Through-Silicon Via (TSV)-based in-package DRAM demonstrates that the proposed cache improves the IPC and energy efficiency by 30.9% and 39.5%, respectively, compared to the baseline with no DRAM cache. These numbers translate to 4.3% and 23.8% improvements over an impractical SRAM-tag cache requiring megabytes of on-die SRAM storage, due to low hit latency and zero energy waste for cache tags.

Die-stacked DRAM caches for servers

Recent research advocates using large die-stacked DRAM caches to break the memory bandwidth wall. Existing DRAM cache designs fall into one of two categories --- block-based and page-based. The former organize data in conventional blocks (e.g., 64B), ensuring low off-chip bandwidth utilization, but co-locate tags and data in the stacked DRAM, incurring high lookup latency. Furthermore, such designs suffer from low hit ratios due to poor temporal locality. In contrast, page-based caches, which manage data at larger granularity (e.g., 4KB pages), allow for reduced tag array overhead and fast lookup, and leverage high spatial locality at the cost of moving large amounts of data on and off the chip. This paper introduces Footprint Cache, an efficient die-stacked DRAM cache design for server processors. Footprint Cache allocates data at the granularity of pages, but identifies and fetches only those blocks within a page that will be touched during the page's residency in the cache --- i.e., the page's footprint. In doing so, Footprint Cache eliminates the excessive off-chip traffic associated with page-based designs, while preserving their high hit ratio, small tag array overhead, and low lookup latency. Cycle-accurate simulation results of a 16-core server with up to 512MB Footprint Cache indicate a 57% performance improvement over a baseline chip without a die-stacked cache. Compared to a state-of-the-art block-based design, our design improves performance by 13% while reducing dynamic energy of stacked DRAM by 24%.

High-Performance Packet Classification for Network-Device Platforms

Recently, many network-device platforms provide a fast SRAM interface for short access latency. However, existing packet classification algorithms achieve high-speed performance by sacrificing the table size; consequently, the entire table is stored in a large DRAM instead of a small SRAM. In this letter, we propose a new partitioning-based packet classification algorithm that generates very small tables and enables the tables to be stored in an SRAM to achieve a very fast speed. Moreover, by removing overhead for the partitioning search, our algorithm provides excellent performance even with DRAMs as compared to other algorithms.

Supporting Very Large DRAM Caches with Compound-Access Scheduling and MissMap

This work efficiently enables conventional block sizes for very large die-stacked DRAM caches with two innovations: it makes hits faster with compound-access scheduling and misses faster with a MissMap. The combination of these mechanisms enables the new organization to deliver performance comparable to that of an idealistic DRAM cache that employs an impractically large SRAM-based on-chip tag array.

CHOP: Integrating DRAM Caches for CMP Server Platforms

Integrating large DRAM caches is a promising way to address the memory bandwidth wall issue in the many-core era. However, organizing and implementing a large DRAM cache imposes a trade-off between tag space overhead and memory bandwidth consumption. CHOP (Caching Hot Pages) addresses this trade-off through three filter-based DRAM-caching techniques.

Comment: US – Countervailing Duty Investigation of DRAMS

AbstractThere are two compelling issues involved in US – Countervailing Duty Investigation of DRAMS. First, there is the question of whether creditors were ‘entrusted or directed’ to make financial contributions to Hynix, a large South Korean DRAM producer. This was the main topic addressed by the Appellate Body (AB) and is the focus of the François–Palmeter paper. Second, there is there is the question of injury assessment and nonattribution. Because the US did not appeal the Panel's decision, this aspect of the WTO dispute was not considered by the AB.

Characteristics of PECVD grown tungsten nitride films as diffusion barrier layers for ULSI DRAM applications

We have developed tungsten nitride (W-Nitride) films grown by plasma enhanced chemical vapor deposition (PECVD) for barrier material applications in ultra large scale integration DRAM devices. As-deposited W-Nitride films show an amorphous structure, which transforms into crystalline, β-W2N and α-W phases upon annealing at 800°C. The resistivity of the as-deposited films grown at the NH3/WF6 gas flow ratio of 1 is about 160 μω-cm, which decreases to 50 μω-cm after an rapid thermal annealing treatment at 800°C. In the contact holes with the size of 0.35 μm and aspect ratio of 3.5, the bottom step coverage of the tungsten nitride films is about 60%, which is about three times higher than that of collimated-TiN films. We obtained contact resistance and leakage current with the tungsten nitride barrier layer comparable to those with conventional collimated TiN films. The contact resistance and leakage current are stable upon thermal stressing at 450°C up to 48 h.