Virtual File System Research Articles

Sequentialized Virtual File System: A Virtual File System Enabling Address Sequentialization for Flash-Based Solid State Drives

Solid-state drives (SSDs) are widely adopted in mobile devices, desktop PCs, and data centers since they offer higher throughput, lower latency, and lower power consumption to modern computing systems and applications compared with hard disk drives (HDDs). However, the performance of the SSDs can be degraded depending on the I/O access pattern due to the unique characteristics of SSDs. For example, random I/O operation degrades the SSD performance since it reduces the spatial locality and induces garbage collection (GC) overhead. In this paper, we present an address reshaping scheme in a virtual file system (VFS) called sVFS for improving performance and easy deployment. To do this, it first sequentializes a random access pattern in the VFS layer which is an abstract layer on top of a more concrete file system. Thus, our scheme is independent and easily deployed on any concrete file systems, block layer configuration (e.g., RAID), and devices. Second, we adopt a mapping table for managing sequentialized addresses, which guarantees correct read operations. Third, we support transaction processing for updating the mapping table to avoid sacrificing the consistency. We implement our scheme at the VFS layer in Linux kernel 5.15.34. The evaluation results show that our scheme improve the random write throughput by up to 27%, 36%, 34%, and 2.35× using the microbenchmark and 25%, 22%, 20%, and 3.51× using the macrobenchmark compared with the existing scheme in the case of EXT4, F2FS, XFS, and BTRFS, respectively.

A STUDY ON LINUX KERNEL MODIFICATION FOR PERFORMANCE IMPROVEMENT OF SSD WEAR-OUT-AWARE SYSTEM

This study aims to improve the solid-state drive (SSD) wear-out-aware system by modifying the virtual file system (VFS) layer of the Linux kernel. Recently, according to the usage of big data and the development of artificial intelligence, the public can access it from a general PC rather than a research center. They installed several large-capacity SSDs required for learning individually or in a redundant array of inexpensive disks (RAID) 0 format on a PC to store the desired large amount of data. When multiple SSDs are connected and used, if one SSD suffers from wear-out, the input/output performance of other SSDs is adjusted to the corresponding device. Therefore, it is essential to ensure that wear out occurs evenly across multiple SSDs to prevent performance deterioration of the entire system. In this paper, we propose a method of distributing I/O to each SSD by modifying VFS layer of the Linux kernel to improve performance when the SSDs with the same performance but different performance due to wear-out. In other words, VFS was changed to detect each SSD wear out and distribute I/O to SSDs according to the performance ratio in a user-transparent way. Through the experimental results, when using the modified Linux kernel proposed in the paper, we found that the performance improved by 52% in the read and write functions compared to the original Linux kernel. Additionally, compared to RAID 0, it was found that there was a performance improvement of 26% in the read function and 16.7% in the write function. Keywords: Linux Kernel, Solid-State Drive, Redundant Array of Inexpensive Disks, Virtual File System DOI： https://doi.org/10.35741/issn.0258-2724.58.1.40

Verification of Crashsafe Caching in a Virtual File System Switch

When developing file systems, caching is a common technique to achieve a performant implementation. Integrating write-back caches is not primarily a problem for functional correctness, but is critical for proving crash safety. Since parts of written data are stored in volatile memory, special care has to be taken when integrating write-back caches to guarantee that a power cut during a running operation leads to a consistent state. This article shows how non-order-preserving caches can be added to a virtual file system switch (VFS) and gives a novel crash-safety criterion matching the characteristics of such caches. Broken down to individual files, a power cut can be explained by constructing an alternative run, where all writes since the last synchronization of that file have written a prefix. VFS caches have been integrated modularly into Flashix, a verified file system for flash memory, and both functional correctness and crash-safety of this extension have been verified with the interactive theorem prover KIV.

Virtual Consolidation of Resource Sharing in Green Cloud using an Efficient Meta Scheduling Algorithm

In modern researchers, cloud parallel data processing has emerging resource that to be one of the problematic application for Infrastructure-as-a-Service (IaaS)clouds. Major Cloud processing companies include starting incorporate frameworks using VM models for parallel data processing in their resource portfolio creation too easy for a client to access these services and to set out their programs. The growing computing requires from multiple requests on the main server has led to excessive power utilization. The waiting resource in the long-term sustainability of Cloud like infrastructures in provisions of energy cost but also from cloud environmental perspective. The trouble can be addressed to require with high energy consumption resource sharing infrastructures, but in the process of resources are dynamically switch to new infrastructure. Switching is not enough to cost efficient and also need time sharing green consuming. Cloud being consists of several virtual centers like VM's under the different administrative domain, make a problem more difficult. Thus, for the reduction in energy consumption, this propose address the challenge by effectively distributing compute-intensive parallel applications on the cloud. To propose a Meta-scheduling algorithm, this exploits the heterogeneous nature of Cloud to achieve the reduction in energy consumption as the green cloud. This intent addresses these challenges by proposing a virtual file system specifically optimized for virtual machine image storage. It is based on a lazy transfer scheme coupled with object versioning that handles snapshot transparently in a hypervisor independent fashion, ensuring high portability for different configurations.

Optimization of Small Sized File Access Efficiency in Hadoop Distributed File System by Integrating Virtual File System Layer

Storage for large datasets, handling data in different formats and data getting generated with high speed are the major highlights of the Hadoop because of which the Hadoop got invented. Hadoop is the solution for the big data problems as discussed above. In order to give the improved solution (in terms of access efficiency and time) for small sized files, this solution is proposed. A novel approach called VFS-HDFS architecture is designed in which the focus is on optimization of small sized files access problems with significant development compared with the existing solutions i.e. HDFS sequence files, HAR, NHAR. In the proposed work a Virtual file system layer has been added as a wrapper over the top of existing HDFS architecture. However, the research work is carried out without altering the existing HFDS architecture. In this paper drawbacks of existing techniques i.e. Flat File Technique and Table Chain Technique which are implemented in HDFS HAR, NHAR, sequence file is overcome by using Bucket Chain Technique. The files to merge in a single bucket are selected using ensemble classifier which is a combination of different classifiers. Combination of multiple classifiers gives the better accurate results. Using this proposed system, better results are obtained compared with the existing system in terms of access efficiency of small sized files in HDFS.

FASTAFS: file system virtualisation of random access compressed FASTA files

BackgroundThe FASTA file format, used to store polymeric sequence data, has become a bioinformatics file standard used for decades. The relatively large files require additional files, beyond the scope of the original format, to identify sequences and to provide random access. Multiple compressors have been developed to archive FASTA files back and forth, but these lack direct access to targeted content or metadata of the archive. Moreover, these solutions are not directly backwards compatible to FASTA files, resulting in limited software integration.ResultsWe designed a linux based toolkit that virtualises the content of DNA, RNA and protein FASTA archives into the filesystem by using filesystem in userspace. This guarantees in-sync virtualised metadata files and offers fast random-access decompression using bit encodings plus Zstandard (zstd). The toolkit, FASTAFS, can track all its system-wide running instances, allows file integrity verification and can provide, instantly, scriptable access to sequence files and is easy to use and deploy. The file compression ratios were comparable but not superior to other state of the art archival tools, despite the innovative random access feature implemented in FASTAFS.ConclusionsFASTAFS is a user-friendly and easy to deploy backwards compatible generic purpose solution to store and access compressed FASTA files, since it offers file system access to FASTA files as well as in-sync metadata files through file virtualisation. Using virtual filesystems as in-between layer offers format conversion without the need to rewrite code into different programming languages while preserving compatibility.

HAL-ASOS Accelerator Model: Evolutive Elasticity by Design

To address the integration of software threads and hardware accelerators into the Linux Operating System (OS) programming models, an accelerator architecture is proposed, based on micro-programmable hardware system calls, which fully export these resources into the Linux OS user-space through a design-specific virtual file system. The proposed HAL-ASOS accelerator model is split into a user-defined Hardware Task and a parameterizable Hardware Kernel with three differentiated transfer channels, aiming to explore distinct BUS technology interfaces and promote the accelerator to a first-class computing unit. This paper focuses on the Hardware Kernel and mainly its microcode control unit, which will leverage the elasticity to naturally evolve with Linux OS through key differentiating capabilities of field programmable gate arrays (FPGAs) when compared to the state of the art. To comply with the evolutive nature of Linux OS, or any Hardware Task incremental features, the proposed model generates page-faults signaling runtime errors that are handled at the kernel level as part of the virtual file system runtime. To evaluate the accelerator model’s programmability and its performance, a client-side application based on the AES 128-bit algorithm was implemented. Experiments demonstrate a flexible design approach in terms of hardware and software reconfiguration and significant performance increases consistent with rising processing demands or clock design frequencies.

VFS over the Years: An Efficient Change Log and System Call for Kernel Developers

“The Linux Kernel is getting bloated and huge, it’s a problem. Sometimes it's a bit sad that we are definitely not the streamlined, small, hyper-efficient kernel that I envisioned 15 years ago … The kernel is huge and bloated” – Linus Torvalds, Linuxcon2009 (Roundtable - The Linux Kernel: Straight From the Source). [1] The Linux Kernel is getting large day by day, both in terms of lines of code and complexity of features and functions. This includes rapid changes in naming conventions of kernel functions, variables and even the flow of data within the code. Moreover, these changes are not well supported by adequate documentation. These issues account to incremental difficulties faced by developers of system code across the globe. This paper is a study of internal working and changes of virtual File System specific code of Linux kernel versions 2.4, 2.6. and 3.0. We have tried to introduce various changes and new addition of several features of the Linux kernel. This survey regarding virtual file system among various kernel versions may play an important role for system developers during system development to gather information about file system in any specific or several kernel versions.

Security control heterogeneous big data cloud storage system based on adaptive cache

In order to improve enterprise information security and the performance of storage system in data processing process, the application of adaptive memory cache mechanism in remote sensing data storage is studied. Firstly, alluxio is used to build a unified virtual file system, and the caching mechanism based on spatial relationship and historical data access inference is studied. Secondly, Presto based on memory computation is introduced as the storage component of remote sensing data meta-data, providing real-time read-write support for massive remote sensing data meta-data and advanced semantic information. Through experiments, the process of adaptive memory buffer remote sensing data storage system takes less time than the traditional storage method.

VFS_CS: a light-weight and extensible virtual file system middleware for cloud storage system

In cloud environments, data-intensive applications have been widely deployed to solve non-trivial applications, while cloud-based storage systems usually fail to provide desirable performance and efficiency when running those data-intensive applications. To address the problems of storage capacity and performance when executing data-intensive applications, we design and implement a light-weight distributed file system middleware, namely virtual file system for cloud storage, which allows other storage-level services to be easily incorporated into an integrated framework in a plug-in manner. In the proposed middleware, we implement three effective mechanisms: disk caching, file striping, and metadata management. The implementation of the proposed middleware has been deployed in a realistic cloud platform, and its performance has been thoroughly investigated under various workloads. Experimental results show that it can significantly improve I/O performance compared with existing approaches. In addition, it also exhibits better robustness when the cloud system is facing intensive workloads.

Dynamic Analysis for IoT Malware Detection With Convolution Neural Network Model

Internet of Things (IoT) technology provides the basic infrastructure for a hyper connected society where all things are connected and exchange information through the Internet. IoT technology is fused with 5G and artificial intelligence (AI) technologies for use various fields such as the smart city and smart factory. As the demand for IoT technology increases, security threats against IoT infrastructure, applications, and devices have also increased. A variety of studies have been conducted on the detection of IoT malware to avoid the threats posed by malicious code. While existing models may accurately detect malicious IoT code identified through static analysis, detecting the new and variant IoT malware quickly being generated may become challenging. This paper proposes a dynamic analysis for IoT malware detection (DAIMD) to reduce damage to IoT devices by detecting both well-known IoT malware and new and variant IoT malware evolved intelligently. The DAIMD scheme learns IoT malware using the convolution neural network (CNN) model and analyzes IoT malware dynamically in nested cloud environment. DAIMD performs dynamic analysis on IoT malware in a nested cloud environment to extract behaviors related to memory, network, virtual file system, process, and system call. By converting the extracted and analyzed behavior data into images, the behavior images of IoT malware are classified and trained in the Convolution Neural Network (CNN). DAIMD can minimize the infection damage of IoT devices from malware by visualizing and learning the vast amount of behavior data generated through dynamic analysis.

Recent evolutions in the LTDP CDF project

In the latest years, CNAF (the national center of the Italian Institute for Nuclear Physics INFN dedicated to Research and Development on Information and Communication Technologies) has been working on the Long Term Data Preservation (LTDP) project for the CDF experiment, active at Fermilab from 1990 to 2011. The main aims of the project are to protect the most relevant part of the CDF RUN-2 data collected between 2001 and 2011 and already stored on tape at CNAF (4 PB), as well as to ensure the availability and the access to the analysis facility to those data over time. Lately, the CDF database, hosting information about CDF datasets such as their structure, file locations and metadata, has been imported from Fermilab to CNAF. Also, the Sequential Access via Metadata (SAM) station data handling tool for CDF data management, that allows to manage data transfers and to retrieve information from the CDF database, has been properly installed and configured at CNAF. This was a fundamental step in the perspective of a complete decommissioning of CDF services on Fermilab side. An access system has been designed and tested to submit CDF analysis jobs, using CDF software distributed via CERN Virtual Machine File System (CVMFS) and requesting delivery of CDF files stored on CNAF tapes, as well as data present only on Fermilab storage archive. Moreover, the availability and the correctness of all CDF data stored on CNAF tapes has been verified. This paper describes all these recent evolutions in detail, presenting the future plans for the LTDP project at CNAF.

VFS_CS: a light-weight and extensible virtual file system middleware for cloud storage system

VFS_CS: a light-weight and extensible virtual file system middleware for cloud storage system

Reference Exascale Architecture (Extended Version)

While political commitments for building exascale systems have been made, turning these systems into platforms for a wide range of exascale applications faces several technical, organisational and skills-related challenges. The key technical challenges are related to the availability of data. While the first exascale machines are likely to be built within a single site, the input data is in many cases impossible to store within a single site. Alongside handling of extreme-large amount of data, the exascale system has to process data from different sources, support accelerated computing, handle high volume of requests per day, minimize the size of data flows, and be extensible in terms of continuously increasing data as well as an increase in parallel requests being sent. These technical challenges are addressed by the general reference exascale architecture. It is divided into three main blocks: virtualization layer, distributed virtual file system, and manager of computing resources. Its main property is modularity which is achieved by containerization at two levels: 1) application containers - containerization of scientific workflows, 2) micro-infrastructure - containerization of extreme-large data service-oriented infrastructure. The paper also presents an instantiation of the reference architecture - the architecture of the PROCESS project (PROviding Computing solutions for ExaScale ChallengeS) and discusses its relation to the reference exascale architecture. The PROCESS architecture has been used as an exascale platform within various exascale pilot applications. This paper also presents performance modelling of exascale platform with its validation.

GRID-system Based on European EGI Standards for Large-scale Calculations Using the Original Accelerated Method of Quantum Chemistry

Based on the analysis of modern tools for creating GRID-type information systems that are part of the European EGI “standard” – UMD repository (including new versions of Globus Toolkit, ARC, dCache, etc.), the applying of GRID systems for computational chemistry is briefly discussed. The GRID system created by the authors combines two clusters with Linux CentOS 7 and is based on software from UMD-4. The relevance and effectiveness of batch processing systems (we use Torque 4.2.10) in quantum chemical calculations is increased for mass calculations of docking complexes (including for drug modeling problems), for which an improved semiempirical method with more efficient approximations was proposed, implemented in the Fortran-95 LSSDOCK software package. For such calculations, new approximation methods have been developed, including for DFT functionals, and their software implementation is carried out. Converters of calculation results by LSSDOCK into a natural for GRID XML-based format CML version 3 are developed. Using the CML format based on dCache software, a single tree of a virtual GRID filesystem distributed between heterogeneous nodes is used to store the results of LSSDOCK calculations.

Convoider: A Concurrency Bug Avoider Based on Transparent Software Transactional Memory

Software transactional memory is an effective mechanism to avoid concurrency bugs in multi-threaded programs. However, two problems hinder the adoption of traditional such systems in wild world: high human cost for equipping programs with transaction functionality and low compatibility with I/O calls and conditional variables. This paper presents Convoider to try to solve these problems. By intercepting inter-thread operations and designating code among them as transactions in each thread, Convoider automatically transactionalizes target programs without any source code modification and recompiling. By saving/restoring stack frames and CPU registers on beginning/aborting a transaction, Convoider makes execution flow revocable. By turning threads into processes, leveraging virtual memory protection and customizing memory allocation/deallocation, Convoider makes memory manipulations revocable. By maintaining virtual file systems and redirecting I/O operations onto them, Convoider makes I/O effects revocable. By converting lock/unlock operations to no-ops, customizing signal/wait operations on condition variables and committing memory changes transactionally, Convoider makes deadlocks, data races and atomicity violations impossible. Experimental results show that Convoider succeeds in transparently transactionalizing twelve real-world applications and perfectly avoid 94% of thirty-one concurrency bugs used in our experiments. This study can help efficiently transactionalize legacy multi-threaded applications and effectively improve the runtime reliability of them.

Stateless model checking of the Linux kernel\u2019s read\u2013copy update (RCU)

Read–copy update (RCU) is a synchronization mechanism used heavily in key components of the Linux kernel, such as the virtual filesystem (VFS), to achieve scalability by exploiting RCU’s ability to allow concurrent reads and updates. RCU’s design is non-trivial, requires a significant effort to fully understand it, let alone become convinced that its implementation is faithful to its specification and provides its claimed properties. The fact that as time goes by Linux kernels are becoming increasingly more complex and are employed in machines with more and more cores and weak memory does not make the situation any easier. This article presents an approach to systematically test the code of the main implementation of RCU used in the Linux kernel (Tree RCU) for concurrency errors, both under sequentially consistent and weak memory. Our modeling allows Nidhugg, a stateless model checking tool, to reproduce, within seconds, safety and liveness bugs that have been reported for RCU. Additionally, we present the real cause behind some failures that have been observed in production systems in the past. More importantly, we were able to verify both the publish–subscribe and the grace-period guarantee, with the latter being the basic and most important guarantee that RCU offers, on several Linux kernel versions, for particular configurations. Our approach is effective, both in dealing with the increased complexity of recent Linux kernels and in terms of time that the process requires. We hold that our effort constitutes a good first step toward making tools such as Nidhugg part of the standard testing infrastructure of the Linux kernel.

Concurrency Bug Avoiding Based on Optimized Software Transactional Memory

Software transactional memory is an effective mechanism to avoid concurrency bugs in multithreaded programs. However, two problems hinder the adoption of such traditional systems in the wild world: high human cost for equipping programs with transaction functionality and low compatibility with I/O calls and conditional variables. This paper presents Convoider to solve these problems. By intercepting interthread operations and designating code among them as transactions in each thread, Convoider automatically transactionalizes target programs without any source code modification and recompiling. By saving/restoring stack frames and CPU registers on beginning/aborting a transaction, Convoider makes execution flow revocable. By turning threads into processes, leveraging virtual memory protection and customizing memory allocation/deallocation, Convoider makes memory manipulations revocable. By maintaining virtual file systems and redirecting I/O operations onto them, Convoider makes I/O effects revocable. By converting lock/unlock operations to no-ops, customizing signal/wait operations on condition variables, and committing memory changes transactionally, Convoider makes deadlocks, data races, and atomicity violations impossible. Experimental results show that Convoider succeeds in transparently transactionalizing twelve real-world applications with averagely incurring only 28% runtime overhead and perfectly avoid 94% of thirty-one concurrency bugs used in our experiments. This study can help efficiently transactionalize legacy multithreaded applications and effectively improve the runtime reliability of them.

Advancing distributed data management for the HydroShare hydrologic information system

HydroShare (https://www.hydroshare.org) is an online collaborative system to support the open sharing of hydrologic data, analytical tools, and computer models. Hydrologic data and models are often large, extending to multi-gigabyte or terabyte scale, and as a result, the scalability of centralized data management poses challenges for a system such as HydroShare. A distributed data management framework that enables distributed physical data storage and management in multiple locations thus becomes a necessity. We use the iRODS (Integrated Rule-Oriented Data System) data grid middleware as the distributed data storage and management back end in HydroShare. iRODS provides a unified virtual file system for distributed physical storages in multiple locations and enables data federation across geographically dispersed institutions around the world. In this paper, we describe the iRODS-based distributed data management approaches implemented in HydroShare to provide a practical demonstration of a production system for supporting big data in the environmental sciences.

A LIGHTWEIGHT APPROACH TO ACCESS TO WIRELESS NETWORK WITHOUT OPERATING SYSTEM SUPPORT

AbstractWireless network is crucial for the Mobile Transparent Computing (MTC), in which a mobile device without any Operating System (OS) support needs to load the demanded OSes and applications through accessing the wireless network connection. In this paper, a lightweight approach based on the Boot Management System (BMS) was proposed to ensure the wireless network connection before booting OS. In BMS, the Virtual File System (VFS) technology was used to drive the wireless network card and establish a stable network connection. A prototype of the BMS was tested on ARM11 hardware platform and the results demonstrate the validity of the BMS.