Bad Block Research Articles

Design and implementation of an efficient and parallel bad block checker for parallelism of storage devices

Design and implementation of an efficient and parallel bad block checker for parallelism of storage devices

An SVM-Based NAND Flash Endurance Prediction Method.

NAND flash memory is widely used in communications, commercial servers, and cloud storage devices with a series of advantages such as high density, low cost, high speed, anti-magnetic, and anti-vibration. However, the reliability is increasingly getting worse while process improvements and technological advancements have brought higher storage densities to NAND flash memory. The degradation of reliability not only reduces the lifetime of the NAND flash memory but also causes the devices to be replaced prematurely based on the nominal value far below the minimum actual value, resulting in a great waste of lifetime. Using machine learning algorithms to accurately predict endurance levels can optimize wear-leveling strategies and warn bad memory blocks, which is of great significance for effectively extending the lifetime of NAND flash memory devices and avoiding serious losses caused by sudden failures. In this work, a multi-class endurance prediction scheme based on the SVM algorithm is proposed, which can predict the remaining P-E cycle level and the raw bit error level after various P-E cycles. Feature analysis based on endurance data is used to determine the basic elements of the model. Based on the error features, we present a variety of targeted optimization strategies, such as extracting the numerical features closely related to the endurance, and reducing the noise interference of transient faults through short-term repeated operations. Besides a high-parallel flash test platform supporting multiple protocols, a feature preprocessing module is constructed based on the ZYNQ-7030 chip. The pipelined module of SVM decision model can complete a single prediction within 37 us.

State Estimation for Hybrid VSC Based HVDC/AC: Unified Bad Data Detection Integrated With Gaussian Mixture Model

The High Voltage Direct Current (HVDC) is an emerging technology that transmits power over long distances and at a higher capacity than the traditional AC systems. Integration of HVDC into modern power networks requires vital modification to the Supervisory, Control and Data Acquisition (SCADA) system, particularly in power system applications. For instance, the state estimator toolbox is an essential software to estimate the network AC and DC systems states. However, the state estimator may fail due to severely corrupted data, a.k.a bad data; hence, an additional data treatment is needed. This paper presents a unified bad data detection block for Weighted Least Squares (WLS) state estimation algorithm. The bad data detection block works for hybrid Voltage Source Converter (VSC)-HVDC/AC transmission systems. It improves the traditional Largest Normalized Residual (LNR) method by integrating the Gaussian Mixture Model (GMM) algorithm. This method reduces the time needed for bad data detection, increases the algorithm robustness, and enhances estimation accuracy. The Cigre B4 network is used as a test case to validate this work on a hybrid VSC-HVDC/AC network. Also, grid load and generation data from the UK is used to construct the simulation measurements and the GMM model. Simulation results show that the modified GMM-LNR has considerably reduced the detection time and improved the WLS accuracy.

Improving SSD Read Latency via Coding

We study the potential enhancement of the read access speed in high-performance solid-state drives (SSDs) by coding, given speed variations across the multiple flash interfaces and assuming occasional local memory failures. Our analysis is based on a queuing model that incorporates both read request failures and NAND element failures. The NAND element failure in the present context reflects various limitations on the memory element level such as bad blocks, dies or chips that cannot be corrected by error control coding (ECC) typically employed to protect pages read off the NAND cells. Our analysis provides a clear picture of the storage-overhead and read-latency trade-offs given read failures and NAND element failures. We investigate two different ways to mitigate the effect of NAND element failures using the notion of multi-class jobs with different priorities. A strong motivation for this work is to understand the reliability requirement of NAND chip components given an additional layer of failure protection, under the latency/storage-overhead constraints.

A Modeling Framework for Reliability of Erasure Codes in SSD Arrays

To help reliability of SSD arrays, Redundant Array of Independent Disks (RAID) are commonly employed. However, the conventional reliability models of HDD RAID cannot be applied to SSD arrays, as the nature of failures in SSDs are different from HDDs. Previous studies on the reliability of SSD arrays are based on the deprecated SSD failure data, and only focus on limited failure types, device failures, and page failures caused by the bit errors, while recent field studies have reported other failure types including bad blocks and bad chips, and a high correlation between failures. In this paper, we explore the reliability of SSD arrays using field storage traces and real-system implementation of conventional and emerging erasure codes. The reliability is evaluated by statistical fault injections that post-process the usage logs from the real-system implementation, while the fault/failure attributes are obtained from field data. As a case study, we examine conventional and emerging erasure codes in terms of both reliability and performance using Linux MD RAID and commercial SSDs. Our analysis shows that a) emerging erasure codes fail to replace RAID6 in terms of reliability, b) row-wise erasure codes are the most efficient choices for contemporary SSD devices, and c) previous models overestimate the SSD array reliability by up to six orders of magnitude, as they focus on the coincidence of bad pages and bad chips that roots the minority of Data Loss (DL) in SSD arrays. Our experiments show that the combination of bad chips with bad blocks is the major source of DL in RAID5 and emerging codes (contributing more than 54% and 90% of DL in RAID5 and emerging codes, respectively), while RAID6 remains robust under these failure combinations. Finally, the fault injection results show that SSD array reliability, as well as the failure breakdown is significantly correlated with SSD type.

Analysis and Solution of Abnormal Fault of Data Storage Based on NAND-flash

NAND-flash memory has the advantages of large capacity and fast rewriting speed. It is suitable for the storage of large amounts of data and is often used as an online storage device for embedded products. However, NAND-flash has the problems of bad blocks and other insufficient reliability.In a certain aerospace model temperature/strain measurement system, the NAND-flash memory is erased and written through DSP software, and the strain/temperature measurement data collected by the sensor is stored in real time. After the system was powered on many times and completed data collection and decoding, it was found that the data stored in the NAND-flash had abnormal faults. By analyzing the test phenomenon and failure mechanism, the failure problem is attributed to the fact that the data in the NAND-flash is not erased, and it is coupled with the newly written data, and it is finally located because the data in the original data address cannot be effectively erased when the data address is stored. Coupling also occurred, resulting in an error in the data address, and the data was coupled after power-on again. Based on the above-mentioned reasons, this paper proposed a troubleshooting method and conducted a test. The verification was successfully passed and the problem was resolved.This method has high reference significance in the large-capacity and high-reliability data storage of NAND-flash in the field of aerospace models.

OrcFS

In this work, we develop the Orchestrated File System (OrcFS) for Flash storage. OrcFS vertically integrates the log-structured file system and the Flash-based storage device to eliminate the redundancies across the layers. A few modern file systems adopt sophisticated append-only data structures in an effort to optimize the behavior of the file system with respect to the append-only nature of the Flash memory. While the benefit of adopting an append-only data structure seems fairly promising, it makes the stack of software layers full of unnecessary redundancies, leaving substantial room for improvement. The redundancies include (i) redundant levels of indirection (address translation), (ii) duplicate efforts to reclaim the invalid blocks (i.e., segment cleaning in the file system and garbage collection in the storage device), and (iii) excessive over-provisioning (i.e., separate over-provisioning areas in each layer). OrcFS eliminates these redundancies via distributing the address translation, segment cleaning (or garbage collection), bad block management, and wear-leveling across the layers. Existing solutions suffer from high segment cleaning overhead and cause significant write amplification due to mismatch between the file system block size and the Flash page size. To optimize the I/O stack while avoiding these problems, OrcFS adopts three key technical elements. First, OrcFS uses disaggregate mapping , whereby it partitions the Flash storage into two areas, managed by a file system and Flash storage, respectively, with different granularity. In OrcFS, the metadata area and data area are maintained by 4Kbyte page granularity and 256Mbyte superblock granularity. The superblock-based storage management aligns the file system section size, which is a unit of segment cleaning, with the superblock size of the underlying Flash storage. It can fully exploit the internal parallelism of the underlying Flash storage, exploiting the sequential workload characteristics of the log-structured file system. Second, OrcFS adopts quasi-preemptive segment cleaning to prohibit the foreground I/O operation from being interfered with by segment cleaning. The latency to reclaim the free space can be prohibitive in OrcFS due to its large file system section size, 256Mbyte. OrcFS effectively addresses this issue via adopting a polling-based segment cleaning scheme. Third, the OrcFS introduces block patching to avoid unnecessary write amplification in the partial page program. OrcFS is the enhancement of the F2FS file system. We develop a prototype OrcFS based on F2FS and server class SSD with modified firmware (Samsung 843TN). OrcFS reduces the device mapping table requirement to 1/465 and 1/4 compared with the page mapping and the smallest mapping scheme known to the public, respectively. Via eliminating the redundancy in the segment cleaning and garbage collection, the OrcFS reduces 1/3 of the write volume under heavy random write workload. OrcFS achieves 56% performance gain against EXT4 in varmail workload.

P-BMS

Flash memory is used as a main data storage medium in increasingly large areas of applications, rapidly replacing hard disk drives because of its low power consumption, fast random access, and high shock resistance. Such flash-based storage devices generally incorporate multiple flash memory chips to meet the ever growing capacity demands. Using multiple chips in a single storage device, at the same time, opens an opportunity to boost the performance based on multi-unit parallelism. However, parallel execution of multiple flash operations introduces complications when bad blocks occur, which is unavoidable due to flash memory’s physical characteristics. The situation gets even worse when bad block occurrences are accompanied by sudden power failures. We propose a bad block management scheme called P-BMS that can fully utilize flash-level parallelism, while guaranteeing provably correct block replacement. Experiments show that our P-BMS achieves a throughput that is more than 95% of the maximum bandwidth of the flash controller, even with bad block occurrences far heavier than in real flash memory.

Mitigating Stuck Cell Failures in MLC NAND Flash Memory via Inferred Erasure Decoding

The multilevel-cell NAND flash memory experiences permanent hard errors due to cell defects (stuck cells). To overcome this problem, stuck cells are either regarded as erasures by the decoder based upon the knowledge of stuck cell location, or the entire memory block containing the stuck cells is marked as bad block and made unavailable for future usage. In this paper, a multiround inferred stuck-cell erasure belief-propagation (BP) decoding (ISED) is proposed in which the stuck cell locations are assumed to be unknown to the decoder. To perform the inferred erasure decoding, the input channel log-likelihood ratio (LLR) information is attenuated before BP decoding by modifying the threshold voltage distribution functions. In case of decoding failure, the probable stuck cell locations are inferred by using the flash’s read-back voltage signal and the decoded code-word bits. For all such inferred stuck cells, the input LLRs are set to zero for subsequent rounds of BP decoding. As the likely incorrect LLRs corresponding to the stuck cells are erased, the performance of BP decoder is substantially improved. Simulation results show that the error-rate performance is improved by more than two orders of magnitude with a moderate increase in decoding complexity under the proposed ISED scheme.

NAND Flash Memory의 초기 Bad Block 정보 물리주소를 이용한 보안키 설계와 암호화 기법 제안

NAND Flash Memory의 초기 Bad Block 정보 물리주소를 이용한 보안키 설계와 암호화 기법 제안

Improving Productivity through Reducing MBT Application Time Using Internal Bad Block Management DFT

Recently, storage market has been shifting from hard disk to SSD, and the importance of NAND flash memory, which plays a key role as a storage media in the SSD, is also increasing. Accordingly, in the manufacturing phase, the test cost of NAND flash memory is getting larger as the density of NAND flash memory is increasing. In this paper, we propose a scheme for DFT(Design for Testability) that can reduce the NAND flash test cost and apply it to the actual MBT(Monitoring Burn-in Test) manufacturing process to prove improvements in the manufacturing cost.

PEVA: A Page Endurance Variance Aware Strategy for the Lifetime Extension of NAND Flash

With aggressive scaling and multilevel cell technology, the reliability of NAND flash continuously degrades. The lifetime of NAND flash is highly restricted by the bit error rate (BER), and error-correcting codes (ECCs) can provide only limited error correction capability to tolerate increasing bit errors. To cope with this issue, a novel page endurance variance aware (PEVA) strategy is proposed to extend the lifetime of NAND flash based on the experimental observations from our hardware–software codesigned experimental platform. The experimental observations indicate that the BER distribution of retention error shows distinct variances in different pages. The key purpose of PEVA is to exploit the lifetime potency of every page in a block by introducing fine-grained bad page management instead of coarse-grained bad block management (BBM). The experimental results show that the PEVA can extend the lifetime of $2\times $ -nm NAND flash by $9.8\times $ compared with the conventional BBM and that there is at most an 8.7% degradation in writing speed compared with the traditional sector mapping technology. In addition, the maximum writing response time increased by at most 5.9% during the operation of the PEVA strategy.

Improve Productivity through Reducing MBT Application Time Using Multi Bad Block Management DFT

Recently, storage market has been shifting from hard disk to SSD, and the importance of NAND flash memory, which plays a key role as a storage media in the SSD, is also increasing. Accordingly, in the manufacturing phase, the test cost of NAND flash memory is getting larger as the density of NAND flash memory is increasing. In this paper, we propose a scheme for DFT(Design for Testability) that can reduce the NAND flash test cost and apply it to the actual MBT(Monitoring Burn-in Test) manufacturing process to prove improvements in the manufacturing cost. There are 3 steps (DC/MBT/POB) between package process and shipment. The purpose of 3 steps is to evaluate the characteristics and to ensure the reliability. MBT equipment stresses to induce early failure of marginal devices in amount of time at hot temperature. Therefore we can reduce defective products in early stage. MBT is essential process in manufacturing NAND flash memory. Therefore, reducing MBT process cost has been a big issue. Using the characteristics of MBT application and NAND flash memory, the DFT method is devised. NAND flash memory always has bad blocks as part of the total blocks. User can not use bad blocks. Therefore it is necessary routine to manage bad blocks when devices are tested. MBT equipment consists of 48 slots and each slot has 64ea I/O channels. If a device has 8ea I/O pins, it can be tested just 8ea devices per slot concurrently. In order to test more than 8ea devices each slot, the test should be conducted in several times. This concept is called “scan”. Increasing test time is caused by increasing the total scan times in accordance with MBT structural problems. Scan and slot structure acts as a factor enabling the massive quantity test. Thus the total scan loop times take up a large portion MBT application time, we are studying ways to reduce the total scan loop times. We devised a DFT method for managing bad block efficiently. This DFT method is called multi bad block management, the details are as follows. Previously in order to find bad blocks, MBT equipment loops scan at every single block to determine pass or fail. We suggest that the newly proposed method is storing pass/fail data of multiple blocks in NAND flash memory internal register. And it is added to the accumulator to determine pass/fail of every multiple blocks rather than single block. It is possible that scan doesn`t loop not single block but every multiple blocks. As a result, it can reduce total scan loop times as follows. As is : (The Number of Total Blocks) x Scan Loop To be : (The Number of Total Blocks/64) x Scan Loop There are a large number of internals latches for pass /fail in NAND flash memory. Toggling RE(read-enable) signal outputs continuously multiple blocks Pass/Fail bit which is stored in the latch. It reduces the scan loop times of MBT, which can reduce system time. It enables test cost reduction and increases productivity due to reduced overall test time, and improves test efficiency. Status parity of 64ea blocks pass/fail parity are stored in chip for shorten the test time in MBT process. When applied to 128Gb NAND flash memory, we could reduce from 21.5 hours to 18.5 hours about 14% test time. If there are more the pass/fail latches, it can further reduce the test time. However because of the area overhead, we limit to status of 64ea blocks. With multi bad block management DFT techniques, we can reduce NAND flash memory test cost of the SSD. Also, if applied products are developed in the future, it is expected to continue to be able to see the effect.

A page-granularity wear-leveling (PGWL) strategy for NAND flash memory-based sink nodes in wireless sensor networks

Sink nodes are the data centers of wireless sensor networks (WSNs), and the storage management scheme for such nodes is vital, particularly in applications such as wireless multimedia sensor networks that involve the collection of massive amounts of data. NAND flash memory is often employed in sink nodes because of its excellent characteristics. Because the lifetime of NAND flash memory is highly restricted by the bit error rate (BER), we present a novel page-granularity wear-leveling (PGWL) strategy to extend the lifetime of NAND flash memory. The concept of PGWL is motivated by two main experimental observations obtained from our own experimental platform for NAND flash memory: first, the raw bit error rate (RBER) distribution exhibits a distinct variance in endurance among different pages, and this variance is more significant than that among different blocks; second, programming relief operations (consisting of only erasing, not programming) can clearly reduce both program-disturb and retention errors. In this study, we first present a practical average RBER prediction model to evaluate the reliability of flash pages using the system clock of the sink node. Thus, the PGWL strategy enables self-adaptive leveling of the RBER growths of different pages in real time by introducing page-granularity wear leveling instead of block-granularity wear leveling to exploit the lifetime potency of each page in a block. Experimental results show that PGWL can extend the lifetime of 2×-nm NAND flash memory by 88.3% compared with traditional bad block management (BBM), while experiencing at most a 0.85% degradation in data throughput speed compared with the conventional sector mapping scheme.

Design of FPGA-Based Missile-Borne Image Acquisition and Storage Recorder

The high-altitude missile and other special application occasions have requirements on image storage system, such as small size, high storage speed, low temperature resistance, etc. Commonly used image storage system in the market cannot meet such requirement. In the paper, real-time image storage system solutions on missile based on FPGA should be proposed. The system mainly consists of acquisition module and memory reading module. The whole system adopts FPGA as main control chip for mainly completing real-time decoding and acquisition on one path of PAL format video images, reading and writing of NandFlash chipset, erasure, bad block management and so on. The solution has passed various environmental tests with stable performance, large data storage capacity and easy expansion, which has been used in engineering practice.

Enhanced Bad Data Processing by Phasor-Aided State Estimation

The utilization of all the available data sources is a permanent objective of state estimation (SE). In this sense, forecasting-aided state estimation (FASE) is an important alternative to conventional state estimation, especially regarding its expeditious data validation scheme, which comprises bad data (BD) smearing effect elimination, block identification, and adequate replacement. Also, with the availability of synchrophasor measurements, there has been a growing interest in building a phasor-aided state estimation (PHASE) process. This paper presents a novel way of processing BD, whose features are similar to those found in the data validation routines of FASE. The proposed PHASE approach has the advantage of leaving the existing SE application software intact, complementing it with an extra estimation module, capable of processing phasor measurements separately and judging whether the measurement set contains anomalies. The results of a proof of concept study performed on the IEEE 14-bus benchmark system demonstrate the application of the proposed methodology. Also, PMU-observability issues are addressed and illustrated through simulation studies conducted on the IEEE 118-bus system.

NAND flash service lifetime estimate with recovery effect and retention time relaxation

A service life model of NAND flash and threshold voltage shift process is proposed to calculate the service life and endurance. The relationships among achievable program/erase (P/E) cycles, recovery time, bad block rate and storage time are analyzed. The achievable endurance and service life of a NAND flash are evaluated based on a flash cell degradation and recovery model by varying recovery time, badblock rate, and storage time. It is proposed to improve the service lifetime of solid state disk by both relaxing the bad block rate limitation and retention time while extending the recovery time. The results indicate that endurance can be improved by 17 times if the storage time guarantee is reduced from 10 a to 1 a with 105 s recovery time inserted between cycles.

Methodologies for Geotagging in FAT and ExFAT File Systems for Smart Phones

Methodologies for Geotagging in FAT and ExFAT File Systems for Smart Phones

The Design and Implementation of a High-Speed and Large-Capacity NAND Flash Storage System

Now, the quality of higher speed and larger capacity are required to the real-time storage system. This paper designs a high-speed and large-capacity storage system which uses FPGA as the master of SOPC system controlling NAND Flash chips. This system puts forward an advanced storage structure which has several NAND Flashes with multi-buses, forming a parallel pipeline design. By using the key technologies of bad block management and the ECC algorithm, which greatly avoids the influence of the invalid block to the storage system and reduces the probability of error data as well. It can not only improve the storage bandwidth and capacity substantially, but also ensure the reliability of the storage system effectively. As a result, the storage system achieves the capacity of 1.5TB and the bandwidth of 1280MBps. Also, this system uses high-speed exchange interface to link to the external network, which achieve the real-time transmission and control of data, and make the storage system standard, universal, and scalable.

The Design of an Embedded File System Based on the Flash

Analyzed the unique characteristics of reading and writing the flash memory, and researched the ways of designing a file system. The target embedded file system has the advantages of the traditional file system and the sequential file system. It is able to adapt to the unique characteristics of reading and writing the flash memory. It can solve the inconsistent data problem in the embedded device when power down. And it can solve the bad blocks of the flash which are generated by erase operations also.