Flash Devices Research Articles

FlashKV

As the cost-per-bit of solid state disks is decreasing quickly, SSDs are supplanting HDDs in many cases, including the primary storage of key-value stores. However, simply deploying LSM-tree-based key-value stores on commercial SSDs is inefficient and induces heavy write amplification and severe garbage collection overhead under write-intensive conditions. The main cause of these critical issues comes from the triple redundant management functionalities lying in the LSM-tree, file system and flash translation layer, which block the awareness between key-value stores and flash devices. Furthermore, we observe that the performance of LSM-tree-based key-value stores is improved little by only eliminating these redundant layers, as the I/O stacks, including the cache and scheduler, are not optimized for LSM-tree’s unique I/O patterns. To address the issues above, we propose FlashKV, an LSM-tree based key-value store running on open-channel SSDs. FlashKV eliminates the redundant management and semantic isolation by directly managing the raw flash devices in the application layer. With the domain knowledge of LSM-tree and the open-channel information, FlashKV employs a parallel data layout to exploit the internal parallelism of the flash device, and optimizes the compaction, caching and I/O scheduling mechanisms specifically. Evaluations show that FlashKV effectively improves system performance by 1.5× to 4.5× and decreases up to 50% write traffic under heavy write conditions, compared to LevelDB.

Reliability of nand-Based SSDs: What Field Studies Tell Us

Solid-state drives (SSDs) based on NAND flash are making deep inroads into data centers as well as the consumer market. In 2016, manufacturers shipped more than 130 million units totaling around 50 Exabytes of storage capacity. As the amount of data stored on solid state drives keeps increasing, it is important to understand the reliability characteristics of these devices. For a long time, our knowledge about flash reliability was derived from controlled experiments in lab environments under synthetic workloads, often using methods for accelerated testing. However, within the last two years, three large-scale field studies have been published that report on the failure behavior of flash devices in production environments subjected to real workloads and operating conditions. The goal of this paper is to provide an overview of what we have learned about flash reliability in production, and where appropriate contrasting it with prior studies performing controlled experiments.

Adaptive hybrid storage systems leveraging SSDs and HDDs in HPC cloud environments

Cloud computing should inherently support various types of data-intensive workloads with different storage access patterns. This makes a high-performance storage system in the Cloud an important component. Emerging flash device technologies such as solid state drives (SSDs) are a viable choice for building high performance computing (HPC) cloud storage systems to address more fine-grained data access patterns. However, the bit-per-dollar SSD price is still higher than the prices of HDDs. This study proposes an optimized progressive file layout (PFL) method to leverage the advantages of SSDs in a parallel file system such as Lustre so that small file I/O performance can be significantly improved. A PFL can dynamically adjust chunk sizes and stripe patterns according to various I/O traffics. Extensive experimental results show that this approach (i.e. building a hybrid storage system based on a combination of SSDs and HDDs) can actually achieve balanced throughput over mixed I/O workloads consisting of large and small file access patterns.

Analysis of 3D NAND technologies and comparison between charge-trap-based and floating-gate-based flash devices

NAND flash chips have been innovated from two-dimension (2D) design which is based on planar NAND cells to three-dimension (3D) design which is based on vertical NAND cells. Two types of NAND flash technologies–charge-trap (CT) and floating-gate (FG) are presented in this paper to introduce NAND flash designs in detail. The physical characteristics of CT-based and FG-based 3D NAND flashes are analyzed. Moreover, the advantages and disadvantages of these two technologies in architecture, manufacture, interference and reliability are studied and compared.

Effect of temperature on deposition layer formation in HBr/N2/fluorocarbon-based plasma

The effects of wafer temperature on etching rate and surface composition were investigated to clarify the surface reaction mechanism under HBr/N2/fluorocarbon-based gas plasma for developing a process for three-dimensional NAND flash devices. The etching rates of both polycrystalline silicon (poly-Si) and SiO2 were found to increase at a wafer temperature of 20 °C as compared with those at 60 °C. Comparing the gas combination of fluorocarbon/N2 and HBr/N2 mixtures, the temperature dependence of SiO2 etching rates was considered to relevant to the sticking probability of fluorocarbon polymers. To determine the cause of the temperature dependence of the poly-Si etching rate, surface composition was evaluated by thermal-desorption-spectroscopy and laser-sputtered-neutral-mass-spectrometry analyses. Ammonium bromide was confirmed in the deposition film at a wafer temperature of 20 °C. The observed increase in poly-Si etching rate at lower temperatures was possibly caused by increased amounts of nitrogen, hydrogen, and bromine fixed to the surface with the formation of ammonium bromide.

Electrical Characteristics Improvement of NAND Flash Memory on Experiment to Replace 49BF2+ with 11B+ in Implantation of Ions

As device size of NAND Flash is shrinking, the program and erase operations need the smaller number of electrons. So the importance of decreasing electron loss is getting increase. Because electron loss causes degradation of the device reliability. As has been discussed before, ion-implantation damage is important factor of NAND Flash fabrication process and device characteristics. Ion implantation damage makes interface trap so it causes electron loss in the device. The heavy ion mass (amu) ion-implantation process makes inevitably not only severe surface lattice damage and more defects in silicon (Si) surface but more thermal budget to recrystallizing. After RTA (Rapid Thermal Annealing) process, the unrestored defects exist at the silicon (Si) surface by heavy ion mass (amu) ion-implantation process damage. In this paper, we focus on endurance improvement characteristics between 49BF2+ ion implantation and 11B+ ion-implantation in 1X floating-gate (FG) NAND Flash cell. The interface trap density (Dit) decreases during initial and stress status by replacing 49BF2+ with 11B+. and erase Vth shift improves after 50k program/erase cycles. From experimental results, we found that the improvement by replacing 49BF2+ with 11B+ reduces interface trap at Si/SiO2 and improves endurance by reducing erase Vth shift in the program/erase cycles.

ReFlex

Remote access to NVMe Flash enables flexible scaling and high utilization of Flash capacity and IOPS within a datacenter. However, existing systems for remote Flash access either introduce significant performance overheads or fail to isolate the multiple remote clients sharing each Flash device. We present ReFlex, a software-based system for remote Flash access, that provides nearly identical performance to accessing local Flash. ReFlex uses a dataplane kernel to closely integrate networking and storage processing to achieve low latency and high throughput at low resource requirements. Specifically, ReFlex can serve up to 850K IOPS per core over TCP/IP networking, while adding 21us over direct access to local Flash. ReFlex uses a QoS scheduler that can enforce tail latency and throughput service-level objectives (SLOs) for thousands of remote clients. We show that ReFlex allows applications to use remote Flash while maintaining their original performance with local Flash.

ReFlex

Remote access to NVMe Flash enables flexible scaling and high utilization of Flash capacity and IOPS within a datacenter. However, existing systems for remote Flash access either introduce significant performance overheads or fail to isolate the multiple remote clients sharing each Flash device. We present ReFlex, a software-based system for remote Flash access, that provides nearly identical performance to accessing local Flash. ReFlex uses a dataplane kernel to closely integrate networking and storage processing to achieve low latency and high throughput at low resource requirements. Specifically, ReFlex can serve up to 850K IOPS per core over TCP/IP networking, while adding 21us over direct access to local Flash. ReFlex uses a QoS scheduler that can enforce tail latency and throughput service-level objectives (SLOs) for thousands of remote clients. We show that ReFlex allows applications to use remote Flash while maintaining their original performance with local Flash.

ZWR: Combining wear‐leveling with reclamation for flash‐memory‐based storage systems of embedded systems

Flash memory is widely adopted in embedded applications since it has several unique advantages. However, due to its hardware characteristic, uneven erasing and inefficient reclamation have been major concerns. These may not only severely degrade the overall performance but also damage the reliability. In order to solve these problems, we propose a novel scheme, which we call ZWR, for zoning, wear‐leveling, and reclamation. The storage zone is divided into the following three levels: an active zone for storing hot data, an inactive zone for storing cold data, and a transitional zone for storing the reclamation block. A function layer and a flash transformation layer are added to flash memory, which are located between the memory technology device layer and the file system. According to the combined wear‐leveling and efficient reclamation, in the function layer, the erase–write cycles are evenly distributed around the entire flash devices and the invalid blocks can be reclaimed properly. Finally, we implement an accurate flash simulator to evaluate the efficacy of ZWR and compare it with those of other flash schemes. We demonstrate that ZWR can effectively solve uneven erasing and inefficient reclamation problems. It can also greatly prolong the service life of flash devices and improve the efficiency of the erasing operation of blocks. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

PPFS: A Scale-out Distributed File System for Post-petascale Systems

The convergence of high-performance computing and big data, which has become known as the field of extreme big data, is problematic in that file creation in storage systems such as distributed file systems is not optimized. That is, the large workload leads to the simultaneous creation of many files by many processes when creating checkpoints. The need to improve the file creation processes prompted us to design a scale-out distributed file system for post-petascale systems named PPFS. PPFS consists of PPMDS, a scale-out distributed metadata server, and PPOSS, a scalable distributed storage server for flash storage. High file creation performance of PPMDS was achieved by using a key-value store for metadata storage and non-blocking distributed transactions to update multiple entries simultaneously. PPOSS depends on PPOST, an object storage system that manages underlying low-level storage, such as Fusion IO ioDrive, a flash device connected through PCI express supporting OpenNVM. High file creation performance was attained by implementing the PPFS prototype using file creation optimization, termed bulk creation, to reduce the amount of communication between PPMDS and PPOSS. Moreover, to enhance the I/O performance of PPOSS when the client process and PPOSS run on the same node, PPOSS accesses the local storage device directly. The prototype implementation of PPFS with file creation optimization achieves 119,000 operations per second for file creation when using five metadata servers and 128 client processes, thereby exceeding the performance of IndexFS 2.17 times. With local access optimization, PPOSS reached its limit at a block size of 16 KiB, an improvement of 1.5 times compared to before optimization. Furthermore, this evaluation indicates that PPFS has scalability on file creation and IO performance, that is required for post-petascale systems.

Operation characteristics of Poly-Si nanowire charge-trapping flash memory devices with SiGe and Ge buried channels

Operation characteristics of polycrystalline silicon (poly-Si) nanowire (NW) charge-trapping (CT) flash memory devices with a SiGe and a Ge buried channel are studied and compared in this work. The poly-Si NW devices with a Ge buried channel show faster programming and erasing speeds as compared to those with a SiGe one due to a lower energy barrier in tunneling layer with more Ge composition. The retention and endurance characteristics of devices with a Ge buried channel are similar to those with a SiGe one. Ge buried channel is promising to CT flash device for 3D nonvolatile memory applications.

Overlay improvement in nanoimprint lithography for 1×-nm patterning

Nanoimprint lithography (NIL) is becoming a promising technique for fine-patterning with a lower cost than other lithography techniques. High overlay accuracy is one of the issues in NIL. Using die-by-die alignment with moiré fringe detection, an NIL alignment measurement accuracy below 1 nm and an overlay accuracy below 5 nm have been reported. On the other hand, the requirement for overlay in 2020 is estimated to be 3–4 nm for dynamic random access memory, flash and logic devices. In order to make the overlay accuracy requirement qualify from the semiconductor industry, a lot of technology enhancements, such as the improvement of overlay control accuracy for NIL-tools, image placement accuracy improvement for NIL templates, mix and match technique of NIL, and other lithography tools such as immersion exposure ones, are needed. In this paper, the authors describe the evaluation of the NIL overlay performance using up-to-date NIL tools, and discuss the potentials of NIL overlay in the future. Alignment accuracy, precision, and overlay correction performance of NIL tools, NIL-to-NIL and NIL-to-optical lithography tool distortion matching performance and overlay error structure analysis, are discussed. From these analyses based on NIL overlay data, the authors discuss the possibility of NIL overlay evolution to realize the adaptation to mass production for the 1×-nm device.

Capacity of the MLC NAND Flash Channel

In this paper, we develop a framework for evaluating the symmetric capacity of multilevel-cell (MLC) NAND flash devices while making very few assumptions regarding the underlying device physics. A set of recursive equations are derived that allow one to measure the symmetric capacity for any given page in a flash device using simple conditional statistics that can be extracted experimentally. Using data captured from two different 1y nm MLC devices, we demonstrate that the symmetric capacity of a flash page not only depends on the amount of program/erase cycling and data retention stress that has accumulated, but also on the position of the page within the flash block. We then study the effect on symmetric capacity of using optimized read-back schemes (both hard and soft) and show that while there is significant benefit, not all pages in the block are improved by the same amount. Finally, we show that it is possible to design error correction architectures that harness the inherent variation of symmetric capacity within a flash block to dramatically extend the program/erase cycling endurance of flash-based storage systems.

Graceful Space Degradation: An Uneven Space Management for Flash Storage Devices

The high cell density, multilevel-cell programming, and manufacturing process variance force the new coming flash memory to have large bit-error-rate variance among blocks and pages, where a flash chip consists of multiple blocks and each block consists of a fixed number of pages. In order to avoid storing the crucial user data in more fragile pages, conventional flash management software tends to aggressively discard the high bit-error-rate area in the unit of a block. However, together with the aggressive discarding strategies and the enlarging sizes of pages/blocks of next generation flash memory, the available space of flash devices might encounter a very sharp degradation and therefore result in rapidly-shortened device lifespan. Thus, we advocate the concept of “graceful space degradation” to mitigate this problem by discarding the high bit-error-rate (or worn-out) area in the unit of pages (instead of blocks). To furthermore realize this concept, we are the pioneer to put forward an “uneven space management” to manage flash blocks containing different number of bad pages. Our design especially focuses on placing data with different access behaviors to make the best uses of blocks with different available space so as to ultimately prolong the device lifespan with good access performance. The experiments were conducted based on representative realistic workloads, and the results reveal that the proposed design can extend the device lifetime by at least 2.38 times of that of existent approaches, with very limited performance overheads.

Impact of LDD Depth Variations on the Performance Characteristics of SONOS NAND Flash Device

In this paper, we investigate a new physical phenomenon for short-channel NAND Flash memory devices. Herein, we intend to examine the effect of lightly doped drain (LDD) depth variations on the “ ON ” current after the erase operation and the erase speed of the short-channel SONOS device. Additionally, we also investigate the electron trap density (residual charge) in charge trap (CT) layer after erase operation and its effect on the device erase characteristics. Furthermore, we carry out the simulation to find the effect of residual charge on the device endurance and the program performance of a SONOS device. Based on our results, the effect of residual charge on the “ ON ” current after the erase operation that becomes smaller as the LDD depth increases. This phenomenon is contrary to the result of higher residual charge in the CT layer with a higher LDD depth and explained from device physics. Furthermore, program performance is found to be degraded with the presence of high residual charge.

The Origin of Traps Responsible for Localization and Charge Transport in Memory Devices

High-κ dielectrics are used now in ReRAM, FeRAM and charge trap (TANOS) flash devises. In present report the nature of deep traps responsible for the charge transport in the most important high-κ dielectrics of flash devices is considered [K.A. Nasyrov and V.A. Gritsenko, J. Appl. Phys. 109, 097705 (2011); V.A. Gritsenko, T.V. Perevalov and D.R. Islamov, Physics Reports 613, 1-20 (2016)]. The charge transport in high-κ dielectrics is controlled by electron and hole traps. In HfO2 charge transport is controlled by trap assisted tunneling between traps [D.R. Islamov, V.A. Gritsenko, C.H. Cheng and A. Chin, Appl. Phys. Lett. 105, 222901 (2014)] (Fig.1). Thermal trap energy in HfO2 is equal to 1.25 eV. The trap nature in HfO2 was studied with luminescence experiments. Blue luminescence band near 2.7 eV has excitation energy at 5.2 eV (Fig.2). Absorption peak of oxygen vacancy in HfO2 has equal energy as blue band excitation. Hence, the 5.2 eV excitation peak and 2.7 eV luminescence band are related to oxygen vacancies [T.V. Perevalov, V.Sh. Aliev, V.A. Gritsenko et al., Appl. Phys. Lett. 104, 071904 (2014)]. The photoluminescence Stokes shift (5.2-2.7)/2=1.25 eV is equal to thermal trap energy (Fig.3). So, the oxygen vacancy in HfO2 is the electron trap [V.A. Gritsenko, T.V. Perevalov and D.R. Islamov, Physics Reports 613, 1-20 (2016)]. Similar experiments were made for Al2O3, ZrO2, Hf0.5Zr0.5O2, Si3N4. It was concluded that in HfO2, Al2O3, ZrO2, HfZrO2 the oxygen vacancies are electron and hole traps. In Si3N4 the Si-Si bond is responsible for electron and hole localization and the charge transport [V.A. Gritsenko, Electronic structure of traps in silicon nitride for flash memory devices, chapter in book“Thin Films on Si: Electronic and Photonic Applications”, accepted for publication by “World Scientific Press”, 2016]. Figure 1

Virtual Flash Chips: Reinforcing the Hardware Abstraction Layer to Improve Data Recoverability of Flash Devices

The market trend of flash memory chips has been toward high density but with low reliability. The rapidly increasing bit error rates and emerging reliability issues of the coming triple-level cell and even three-dimensional flash chips will expose users to extremely high risks for storing data in such low reliability storage media. With these concerns in mind, this paper rethinks the layer design of flash devices and proposes a complete paradigm shift to re-configure physical flash chips of potentially massive parallelism into better “virtual chips”, in order to improve the data recoverability in a modular and low-cost way. The concept of virtual chips is realized by reinforcing the hardware abstraction layer without continually complicating the conventional flash management software of the flash translation layer. The capability and compatibility of the proposed design were verified by both property analysis and a series of experiments with encouraging results.

Modeling of Threshold Voltage Distribution in NAND Flash Memory: A Monte Carlo Method

NAND flash reliability (both endurance and retention) degrades with the scaling of NAND flash memory technology and the introduction of multilevel cell (MLC) and triple-level cell (TLC) technologies. As a result, stronger error correction code (ECC) methods are required in the solid-state drive application, especially for MLC and TLC flash memory devices. Modeling of the threshold voltage ( $V_{t}$ ) distribution in the NAND flash memory can make the ECC simulations more effective and efficient. In this paper, we show a semiphysical and Monte Carlo method to generate a reallike $V_{t}$ distribution by mimicking the NAND flash programming and data retention (DR) processes. Probability distributions used in the Monte Carlo method are obtained by the measurements of the NAND devices and data interpolation or extrapolation. Simulations by this model can obtain reallike $V_{t}$ distribution for any given Program/Erase cycle and DR time. Excellent results have been obtained and confirmed by comparing the simulations with the measured data from 16-nm NAND flash devices.

멀티 레벨 셀 플래시 메모리 신뢰성 분석을 통한 항공 전자장비용 내결함성 메모리 설계 연구

일반적으로 MLC NAND 플래시 저장장치는 SLC NAND 플래시 기반의 장치에 비해 정보 신뢰성이 낮은 것으로 평가된다. MLC 플래시는 SLC 플래시 보다 약 1000배 이상의 RBER (raw bit error rate)을 갖는다고 평가되나 최근 Google 데이터 센터에서 수집된 결과로부터 수행된 연구를 통해 실제 RBER은 이보다 훨씬 낮은 것으로 확인되었다. 이런 연구 결과를 바탕으로 우리는 MLC 플래시의 여유 저장 공간과 SSD 내부에 존재하는 Firmware 층을 활용하여 하드웨어적 구조 복잡성의 증가 없이 정보 신뢰성을 향상시키는 방법인 IDDD (in drive data duplication) 방식을 고안하였고 실 측정결과와 계산을 통해 MLC 플래시의 정보 신뢰성이 SLC 플래시 대비 상당히 높아질 수 있음을 보였다. 우리가 연구한 총 48개 상황 중 44개의 상황에서 SLC 플래시의 RBER이 MLC 플래시보다 낮았음에도 불구하고 IDDD방식을 적용함으로써 48개의 모든 상황에서 MLC 플래시의 RBER이 SLC 플래시보다 낮으며, 43개의 상황에서 UBER (uncorrectable bit error rate) 또한 SLC 플래시 대비 낮음을 보였다. Typical MLC NAND flash devices are considered less reliable than SLC NAND flash devices. Although raw bit error rate (RBER) of MLC flash had been considered approximately 1000times or more higher than that of SLC flash, recent research conducted on Google's data center shows that it is much lower than such expectation. Based on the research, we devised In Drive Data Duplication (IDDD) scheme that efficiently exploit MLC flash's sufficient capacity to improve its data reliability without structural complexity increment using SSD intrinsic firmware layer, and showed the data reliability expectation of MLC flash could be significantly higher than that of SLC flash from measured and calculated error rates. Even though RBER of SLC flash was lower than that of MLC flash in 44 out of 48 cases we studied, applying IDDD scheme, RBER of MLC flash was became lower than that of SLC in all 48 cases and uncorrectable bit error rate (UBER) of MLC flash was became lower than that of SLC flash in 45 out of 48 cases.

TreeFTL: An Efficient Workload-Adaptive Algorithm for RAM Buffer Management of NAND Flash-Based Devices

NAND flash memory is widely used for the secondary storage of computer systems. The flash translation layer (FTL) is the firmware that manages and operates a flash-based storage device. One of the FTL's modules manages the RAM buffer of the flash device. Now this RAM buffer is sufficient to be used for both address mapping and data buffering. As the fastest component of the flash layer interface, effective management of this buffer has a significant impact on the performance of data storage and access. This paper proposes a novel scheme called TreeFTL for this purpose. TreeFTL organizes address translation pages and data storage pages in a tree-like structure in the RAM buffer. The tree enables TreeFTL to adapt to the access behaviors of workloads by dynamically adjusting the partitions for address mapping and data buffering. Furthermore, TreeFTL employs a lightweight mechanism to evict the least-recently-used victim pages when the need arises. Our experiments show that TreeFTL is able to spend 46.6 and 49.0 percent less service time over various workloads than two state-of-the-art algorithms, respectively, for a 64 MB RAM buffer.