Flash File System Research Articles

Development of Smart Earth Leakage Circuit Breaker Using IoT and Power Electronics

The Smart Earth Leakage Circuit Breaker research is based on IoT& power electronics-based protective switchgear for domestic consumers for the protection against earth leakage faults. In this paper, we have shown how to create a custom web server using HTML and CSS language and hosting this webserver to ESP module using Serial Peripheral Flash File System (SPIFFS). Along with protection against earth fault it provides, controlling of tripping current or fault current, monitoring voltage, power, current, power factor parameters of the connected load, monitoring fault parameters, Indicating the device status whether the ELCB is on or off, and the condition of ELCB it is in faulty condition or healthy condition and to let the user set the value of the tripping current on the webserver at which the ELCB must trip.

Mitigating Synchronous I/O Overhead in File Systems on Open-Channel SSDs

Synchronous I/O has long been a design challenge in file systems. Although open-channel solid state drives (SSDs) provide better performance and endurance to file systems, they still suffer from synchronous I/Os due to the amplified writes and worse hot/cold data grouping. The reason lies in the controversy design choices between flash write and read/erase operations. While fine-grained logging improves performance and endurance in writes, it hurts indexing and data grouping efficiency in read and erase operations. In this article, we propose a flash-friendly data layout by introducing a built-in persistent staging layer to provide balanced read, write, and garbage collection performance. Based on this, we design a new flash file system (FS) named StageFS , which decouples the content and structure updates. Content updates are logically logged to the staging layer in a persistence-efficient way, which achieves better write performance and lower write amplification. The updated contents are reorganized into the normal data area for structure updates, with improved hot/cold grouping and in a page-level indexing way, which is more friendly to read and garbage collection operations. Evaluation results show that, compared to recent flash-friendly file system (F2FS), StageFS effectively improves performance by up to 211.4% and achieves low garbage collection overhead for workloads with frequent synchronization.

Source Code Verification for Embedded Systems using Prolog

System relevant embedded software needs to be reliable and, therefore, well tested, especially for aerospace systems. A common technique to verify programs is the analysis of their abstract syntax tree (AST). Tree structures can be elegantly analyzed with the logic programming language Prolog. Moreover, Prolog offers further advantages for a thorough analysis: On the one hand, it natively provides versatile options to efficiently process tree or graph data structures. On the other hand, Prolog's non-determinism and backtracking eases tests of different variations of the program flow without big effort. A rule-based approach with Prolog allows to characterize the verification goals in a concise and declarative way. In this paper, we describe our approach to verify the source code of a flash file system with the help of Prolog. The flash file system is written in C++ and has been developed particularly for the use in satellites. We transform a given abstract syntax tree of C++ source code into Prolog facts and derive the call graph and the execution sequence (tree), which then are further tested against verification goals. The different program flow branching due to control structures is derived by backtracking as subtrees of the full execution sequence. Finally, these subtrees are verified in Prolog. We illustrate our approach with a case study, where we search for incorrect applications of semaphores in embedded software using the real-time operating system RODOS. We rely on computation tree logic (CTL) and have designed an embedded domain specific language (DSL) in Prolog to express the verification goals.

A Methodology for Estimating Performance and Power Consumption of Embedded Flash File Systems

In the embedded systems domain, obtaining performance and power consumption estimations is extremely valuable in numerous cases. This is particularly true during the design stage, as designers of complex embedded systems face an increasingly large design space. Secondary storage is a well-known performance bottleneck and has also been reported as an important factor of power consumption. Flash memory is the main secondary storage media in an embedded system and exhibits specific constraints in its usage. One popular way to manage these constraints is to use dedicated Flash File Systems (FFS). In this article, we propose a methodology to estimate the performance and power consumption of applicative I/Os on an FFS-based storage system within embedded Linux. The methodology is divided into three sequential steps. In the exploration phase, the main factors of an FFS storage system impacting performance and power consumption are identified. In the modeling phase, this impact is formalized into models. Finally, in the last phase, the models are implemented in a simulator named OpenFlash. OpenFlash allows obtaining performance and power consumption estimations for an applicative workload processed by the Linux FFS storage stack on an embedded platform. The simulator is validated against real measurements and the estimation error stays below 10%.

Modular, crash-safe refinement for ASMs with submachines

In this paper we define a formal refinement theory for a variant of Abstract State Machines (ASMs) with submachines and power cuts. The theory is motivated by the development of a verified flash file system. Different components of the system are modeled as submachines and refined individually. We define a non-atomic semantics that is suitable for considering power cuts in the middle of operations. We prove that refinement is compositional with respect to submachines and crashes. We give a criterion “crash-neutrality” and corresponding proof obligations that are sufficient to reduce non-atomic reasoning to standard pre/post verification in the context of power failures in file systems.

High reliability flash file system used into data unit of flight data recorder

High reliability flash file system used into data unit of flight data recorder

Specifying a Realistic File System

We present the most interesting elements of the correctness specification of BilbyFs, a performant Linux flash file system. The BilbyFs specification supports asynchronous writes, a feature that has been overlooked by several file system verification projects, and has been used to verify the correctness of BilbyFs's fsync() C implementation. It makes use of nondeterminism to be concise and is shallowly-embedded in higher-order logic.

O(1) 크래시 복구 수행시간을 갖는 FTL의 설계와 구현

The capacity of flash-based storage such as Solid State Drive(SSD) and embedded Multi Media Card(eMMC) is ever-increasing because of the needs from the end-users. However, if a flash-based storage crashes, such as during power failure, the flash translation layer(FTL) is responsible for the crash recovery based on the entire flash memory. The recovery time increases as the capacity of the flash-based storages increases. We propose O1FTL with O(1) crash recovery time that is independent of the flash capacity. O1FTL adopts the working area technique suggested for the flash file system and evaluates the design on a real hardware platform. The results show that O1FTL achieves a crash recovery time that is independent of the capacity and the overhead, in terms of I/O performance, and achieves a low P/E cycle.

스마트 기기의 효율적인 I/O를 위한 플래시 파일 시스템 성능 분석

Recently NAND flash memory has been found to be the primary cause of low performance in the smart device. NAND flash memory is different from each other the execution time of I/O operations that flash file system is required. Therefore, it is necessary to compare and analyze the flash file system I/O performance for the efficient I/O on smart device. In this paper, it was tested and analyzing the I/O performance of the YAFFS2, JFFS2, UBIFS. Experimental results most read I/O performance is good, but the writing I/O performance is not good. For UBIFS, showed a more good I/O performance compared to other flash file system.

컬럼-기반 데이터베이스를 위한 그림자 복구

컬럼-기반 데이터베이스 저장소는 우수한 입출력 성능으로 대용량 데이터 트랜잭션을 위한 매우 진보적인 모델이다. 전통적인 데이터 저장소는 빠른 쓰기 연산을 위하여 한 레코드의 속성들을 하드 디스크에 연속적으로 배치되어 있는 가로-지향 저장 모델을 활용하였다. 하지만 검색이 대부분인 데이터웨어하우스 시스템을 위해서는 월등한 판독 성능 때문에 컬럼-지향 저장소가 더 적합한 모델이 되고 있다. 또한 최근에는 플래시 메모리를 사용한 SSD가 고속 데이터 분석 시스템을 위한 적합한 저장 매체로 인식되고 있다. 본 연구에서는 플래시 미디어 파일 시스템을 기반으로 하는 컬럼-기반 데이터베이스 환경을 위한 새로운 트랜잭션 회복기법(CoSR)을 제안한다. 제안 기법은 기존의 쉐도우 페이징 기법을 개선하여 플래시 파일 시스템에서 새로운 블록에 데이터를 저장할 경우 무효화되어 폐기되는 이전 데이터 블록을 재활용하였다. 이를 위하여 제안된 컬럼-기반 쉐도우 복구 기법에 재활용 쉐도우 리스트 구조를 활용하였다. 제안 기법은 기존 쉐도우 페이징기법의 최대 단점인 쉐도우 페이지 관련 추가 저장공간의 부담을 최소화하고, 기존 복구 기법에서 컬럼 데이터 압축에 기인한 입출력 성능저하를 최소화 할 수 있다. 실험 분석결과를 통하여 CoSR기법이 기존 기법보다 17% 더 우수함을 확인하였다. The column-oriented database storage is a very advanced model for large-volume data transactions because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly data warehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. In this research, we propose a new transaction recovery scheme for a column-oriented database environment which is based on a flash media file system. We improved traditional shadow paging schemes by reusing old data pages which are supposed to be invalidated in the course of writing a new data page in the flash file system environment. In order to reuse these data pages, we exploit reused shadow list structure in our column-oriented shadow recovery(CoSR) scheme. CoSR scheme minimizes the additional storage overhead for keeping shadow pages and minimizes the I/O performance degradation caused by column data compression of traditional recovery schemes. Based on the results of the performance evaluation, we conclude that CoSR outperforms the traditional schemes by 17%.

A Tracing Toolset for Embedded Linux Flash File Systems

NAND flash memory integration in the traditional I/O software stack of Unix-like operating systems (OS) was achieved without modifying most of the OS layers. In fact, one can dissociate two categories of flash memory devices: (1) those which intricacies are abstracted to the OS (e.g. SSDs, USB sticks), and (2) raw flash memory chips driven by a specific Flash File System (FFS) such as JFFS2 and UBIFS. In the latter case, the operating system I/O software stack low level layers (i.e. file system and driver) were upgraded while the higher levels (virtual file system and related buffers) were not. In order to optimize the system behavior in such a case, one must understand the interactions between the different I/O software management layers, and the performance impact of each layer for a given I/O workload. For this sake, we developed a tracing toolset allowing to understand the impact of each layer on the I/O request flow, for instance: caching, overheads, and fragmentation. The developed framework proved to be precious to apprehend the interactions between OS flash specific layers and traditional layers for a better system performance understanding.

Improving Performance and Capacity of Flash Storage Devices by Exploiting Heterogeneity of MLC Flash Memory

The multi-level cell (MLC) NAND flash memory technology enables multiple bits of information to be stored in a memory cell, thus making it possible to increase the density of flash memory without increasing the die size. In MLC NAND flash memory, each memory cell can be programmed as a single-level cell or a multi-level cell at runtime because of its performance/capacity asymmetric programming property, which is called flexible programming in this paper. Therefore, MLC flash memory has a potential to achieve the high performance of SLC flash memory while preserving its maximum capacity. In this paper, we present a flexible flash file system, called FlexFS, which takes advantage of flexible programming. FlexFS divides a flash memory medium into SLC and MLC regions, and then dynamically changes two different types of regions to provide an optimal storage solution to end-users in terms of performance and capacity. FlexFS also provides a reasonable storage lifetime by managing the wearing rate of NAND flash memory, which is accelerated by the use of flexible programming. Our implementation of FlexFS in the Linux 2.6 kernel shows that it achieves the I/O performance comparable to SLC flash memory while guaranteeing the capacity of MLC flash memory in various real-world workloads.

O1FS: Flash file system with O(1) crash recovery time

The crash recovery time of NAND flash file systems increases with flash memory capacity. Crash recovery usually takes several minutes for a gigabyte of flash memory and becomes a serious problem for mobile devices. To address this problem, we propose a new flash file system, O1FS. A key concept of our system is that a small number of blocks are modified exclusively until we change the blocks explicitly. To recover from crashes, O1FS only accesses the most recently modified blocks rather than the entire flash memory. Therefore, the crash recovery time is bounded by the size of the blocks. We develop mathematical models of crash recovery techniques and prove that the time complexity of O1FS is O(1), whereas that of other methods is proportional to the number of blocks in the flash memory. Our evaluation shows that the crash recovery of O1FS is about 18.5 times faster than that of a state-of-the-art method.

File systems deserve verification too!

File systems are too important, and current ones are too buggy, to remain unverified. Yet the most successful verification methods for functional correctness remain too expensive for current file system implementations-we need verified correctness but at reasonable cost. This paper presents our vision and ongoing work to achieve this goal for a new high-performance flash file system, called BilbyFs. BilbyFs is carefully designed to be highly modular, so it can be verified against a high-level functional specification one component at a time. This modular implementation is captured in a set of domain specific languages from which we produce the design-level specification, as well as its optimised C implementation. Importantly, we also automatically generate the proof linking these two artefacts. The combination of these features dramatically reduces verification effort. Verified file systems are now within reach for the first time.

Application of constraints to formal verification and artificial intelligence

We are excited to present the AMAI special issue on Application of Constraints to Formal Verification and Artificial Intelligence. The issue contains three papers: In the first paper, entitled Establishing Flight Software Reliability: Testing, Model Checking, Constraint-Solving, Monitoring and Learning, Alex Groce from Oregon State University, Klaus Havelund, Gerard Holzmann, and Rajeev Joshi from the NASA Jet Propulsion Laboratory (JPL), and Ru-Gang Xu from the University of California at Los Angeles discuss the application of a range of techniques to the verification of missioncritical flight software at JPL. They present insights on the combination of random testing, model checking, constraint-solving, monitoring, and finite-state machine learning, in addition to static code analysis for the efficient verification of properties of a flash file system and other software for space missions. In the second paper, entitled Polynomial Function Intervals for Floating-Point Software Verification, Jan Duracz from Halmstad University in Sweden and Michal Konecny from Aston University in the U.K. present a new method for verification of numerical software, supporting a substantially more expressive language for specifications than other publicly available automated tools. Their experiments indicate that the additional expressivity does not come at a performance cost when comparing the results with those obtained with other publicly available state-of-the-art provers. They also include a scalability study of their algorithm when proving tight functional specifications of progressively larger randomly generated programs. In the third paper, entitled Tradeoffs in the Complexity of Backdoors to Satisfiability: Dynamic Sub-Solvers and Learning During Search, Bistra Dilkina from the Georgia Institute of Technology, Carla P. Gomes from Cornell University, and Ashish Sabharwal from IBM study the potential to capture succinctly hidden problem structure in Boolean Satisfiability (SAT) problems using different notions of backdoors—a set of key variables, such

Flashmon V2

This paper presents Flashmon version 2, a tool for monitoring embedded Linux NAND flash memory I/O requests. It is designed for embedded boards based devices containing raw flash chips. Flashmon is a kernel module and stands for "flash monitor". It traces flash I/O by placing kernel probes at the NAND driver level. It allows tracing at runtime the 3 main flash operations: page reads/writes and block erasures. Flashmon is (1) generic as it was successfully tested on the three most widely used flash file systems that are JFFS2, UBIFS and YAFFS, and several NAND chip models. Moreover, it is (2) non intrusive, (3) has a controllable memory footprint, and (4) exhibits a low overhead (< 6%) on the traced system. Finally, it is (5) simple to integrate and used as a standalone module or as a built-in function/module in existing kernel sources. Monitoring flash memory operations allows a better understanding of existing flash management systems by studying and analyzing their behavior. Moreover it is useful in development phase for prototyping and validating new solutions.

리눅스 환경에서 임베디드 시스템을 위한 플래시 파일 시스템의 성능 비교 분석

Recently the operating system share of linux on embedded system is increasing. The embedded systems on linux environment, commonly equip a file system as mini hard disk or flash memory to keep data. The types of the file system of the system are various according to it's operating system. Anyway, the more embedded system depends on the file system, the selection of the type of the file system effects more on the performance of the system. This thesis performs the performance benchmark of a FAT and Ext file systems which are most popular in embedded system. As the result, it is discussed that what file system is better at which case. These results will be a index at the selection of flash file system of the embedded systems on linux environment.

Performance analysis and modeling of SQLite embedded databases on flash file systems

Databases are more and more used in embedded system applications and especially in consumer electronics. This comes from the need to structure user and/or system data to be more efficiently managed and accessed. The transactional database management systems widely used in embedded systems have been designed considering hard disk drives as the storage device. In embedded systems, NAND flash memory is the main storage media and its intricacies make it fundamentally different from hard drives on many points of view, in particular in terms of performance and access mode. The performance behavior of on-flash database applications remains largely unknown, and we believe a better adequacy between those applications and flash management systems would lead to strong optimizations. A first step in that process is to assess and understand the performance behavior of database applications on flash memory. For this sake, this paper presents a micro benchmarking and modeling methodology, as well as the associated results and analysis for SQLite database queries on embedded flash specific file systems. SQLite is one of the most used database application in embedded systems and especially consumer electronics. This methodology has been applied and validated on two embedded hardware platforms. Flash file systems (FFS) behavior are very specific to flash memory intricacies and the objective of this study is to highlight the interactions between flash memory, FFSs, and SQLite based applications. The model proved to be very accurate in predicting flash I/O performance for a given workload.

FlashLight

A very promising approach for using NAND flash memory as a storage medium is a flash file system. In order to design a higher-performance flash file system, two issues should be considered carefully. One issue is the design of an efficient index structure that contains the locations of both files and data in the flash memory. For large-capacity storage, the index structure must be stored in the flash memory to realize low memory consumption; however, this may degrade the system performance. The other issue is the design of a novel garbage collection (GC) scheme that reclaims obsolete pages. This scheme can induce considerable additional read and write operations while identifying and migrating valid pages. In this article, we present a novel flash file system that has the following features: ( i ) a lightweight index structure that introduces the hybrid indexing scheme and intra-inode index logging , and ( ii ) an efficient GC scheme that adopts a dirty list with an on-demand GC approach as well as fine-grained data separation and erase-unit data allocation . We implemented FlashLight in a Linux OS with kernel version 2.6.21 on an embedded device. The experimental results obtained using several benchmark programs confirm that FlashLight improves the performance by up to 27.4% over UBIFS by alleviating index management and GC overheads by up to 33.8%.

On benchmarking embedded Linux flash file systems

Due to its attractive characteristics in terms of performance, weight and power consumption, NAND flash memory became the main non volatile memory (NVM) in embedded systems. Those NVMs also present some specific characteristics/constraints: good but asymmetric I/O performance, limited lifetime, write/erase granularity asymmetry, etc. Those peculiarities are either managed in hardware for flash disks (SSDs, SD cards, USB sticks, etc.) or in software for raw embedded flash chips. When managed in software, flash algorithms and structures are implemented in a specific flash file system (FFS). In this paper, we present a performance study of the most widely used FFSs in embedded Linux: JFFS2, UBIFS, and YAFFS. We show some very particular behaviors and large performance disparities for tested FFS operations such as mounting, copying, and searching file trees, compression, etc.