Network File System Research Articles

미디어 스트리밍 시스템에서의 상태 천이 모델을 활용한 고속 분산 네트워크 파일 시스템

네트워크를 통해 전송되는 스트리밍 미디어의 대용량화로 인해 기존의 전송 방법은 최적의 성능을 제시하지 못하고 있다. 이를 위해 대역폭의 소비와 네트워크 혼잡 및 트래픽을 감소시키는 비디오 프록시 서버가 운용된다. 본 논문은 비디오 프록시 서버의 효율적인 활용을 위해 미디어 스트리밍 시스템에서의 상태 천이 모델을 활용한 고속 분산 네트워크 파일 시스템을 제안한다. 제안하는 방법은 상태 천이 모델을 활용한 학습 과정, 기본 확률과 결정 확률의 생성, 그리고 확률을 기반으로 한 저장과 삭제의 3단계로 구성된다. 또한 비디오 프록시 서버의 저장 공간에서 발생되는 단편화를 막기 위하여 해당 공간을 세그먼트 별로 영역을 구분한다. 실험을 통해 제안하는 방법이 기존의 방법들에 비해 보다 높은 적중률을 보이는 동시에 보다 적은 삭제 횟수를 보임을 확인한다. 이를 통해 제안하는 방법이 초기 지연시간을 최소화하는 동시에 네트워크 대역폭을 효율적으로 활용하는 것을 보인다. Due to the large sizes of streaming media, previous delivery techniques are not providing optimal performance. For this purpose, video proxy server is employed for reducing the bandwidth consumption, network congestion, and network traffic. This paper proposes a fast distributed network file system using state transition model in the media streaming system for efficient utilization of video proxy server. The proposed method is composed of three steps: step 1. Training process using state transition model, step 2. base and decision probability generation, and step 3. storing and deletion based on probability. In addition, storage space of video proxy server is divided into each segment area in order to store the segments efficiently and to avoid the fragmentation. The simulation results show that the proposed method performs better than other methods in terms of hit rate and number of deletion. Therefore, the proposed method provides the lowest user start-up latency and the highest bandwidth saving significantly.

Improving Mandatory Access Control for HPC clusters

hpc clusters are costly resources, hence nowadays these structures tend to be co-financed by several partners. A cluster administrator has to be designated, whose duties include, amongst others, the prevention of accidental data leakage or theft. Linux has been chosen as an operating system for the CEA’s computing platforms. However, strong system security solutions such as SELinux are usually difficult to set up in large environments.This article presents how we have adapted a mac mechanism in order to enforce confidentiality and integrity between a large number of users. First we define our security objectives, and show how they direct our technical choices. Then we present how confinement was achieved using the SELinux security mechanism, and how various attack scenarios were addressed. We then focus on the use of Mandatory Categories, access control on high bandwidth network filesystems and the integration of new users and applications. We discuss some residual technical challenges. Finally, we present benchmark results and validate the acceptable performance impact of our deployment on a modern cluster.

A Localized Parallel Trace I/O Method for Trace-Driven Simulator

Trace-driven simulation is widely used in performance evaluation of computer architectures. Efficient trace accesses are crucial for trace-driven parallel simulators, as trace size grows rapidly with the number of target processors. This article presents a method named LPIO (Localized Parallel trace I/O ) in the context of the BigSim parallel simulator. It utilizes the local disks of the host machines to store traces generated in the emulation phase. It proposes a task-local mapping algorithm that maps each simulation process to the node with the traces it needs. Hence trace I/O operations are performed in a scalable parallel manner with the support of all host nodes. Another benefit is that trace accesses are localized, avoiding data transfer through the interconnection network. Experiments with four parallel benchmarks on a SMP-cluster show that LPIO outperforms NFS (Network File System)-based method by 2.8% to 46.8% in the emulation phase. It also improves simulation performance up to 7.0% in the simulation phase. LPIO changes little of the simulator code or host-machine configuration and is easy to implement.

An Overview of File Server Group in Distributed Systems

The file server is a key factor to accomplish the data sharing essential in distributed systems. The file server is perhaps the most heavily used resource of the distributed systems and as an outcome; its performance is vital and critical to the victory of the system. The explosive growth of the Web contents and Internet users has led to increasing concentration on two major challenges in these systems: scalability and high availability of network file system. A simple load-sharing mechanism for the NFS client to switch to a lightly-load server based on the number of NFS client's RPC requests for a period of time, making these systems more efficient and scalable. The replication techniques are used for improving the availability of Distributed systems in FSG. We also discussed distributed application in terms of java programming language. In this paper we have proposed a new architecture for Distributed Systems that provides better security. Our proposed solution is more secure & efficient in the sense that a client can not search the whole group for a specific file, because the group header maintains a huge database of all the group members along with their specific services they offers. Also load balancing mechanism is improved where the group header keeps a record of connected users to a specific server. Every group member is not required to advertise their state.

Non-POSIX File System for LHCb Online Event Handling

LHCb aims to use its O(20000) CPU cores in the high level trigger (HLT) and its 120 TB Online storage system for data reprocessing during LHC shutdown periods. These periods can last a few days for technical maintenance or only a few hours during beam interfill gaps. These jobs run on files which are staged in from tape storage to the local storage buffer. The result are again one or more files. Efficient file writing and reading is essential for the performance of the system. Rather than using a traditional shared file-system such as NFS or CIFS we have implemented a custom, light-weight, non-Posix network file-system for the handling of these files. Streaming this file-system for the data-access allows to obtain high performance, while at the same time keep the resource consumption low and add nice features not found in NFS such as high-availability, transparent fail-over of the read and write service. The writing part of this streaming service is in successful use for the Online, real-time writing of the data during normal data acquisition operation. The network file system relies on a commercial file system which manages the data on persistent disk storage. The implementation together with performance figures are presented.

ReadDB Provides Efficient Storage for Mapped Short Reads

BackgroundThe advent of high-throughput sequencing has enabled sequencing based measurements of cellular function, with an individual measurement potentially consisting of more than 108 reads. While tools are available for aligning sets of reads to genomes and interpreting the results, fewer tools have been developed to address the storage and retrieval requirements of large collections of aligned datasets. We present ReadDB, a network accessible column store database system for aligned high-throughput read datasets.ResultsReadDB stores collections of aligned read positions and provides a client interface to support visualization and analysis. ReadDB is implemented as a network server that responds to queries on genomic intervals in an experiment with either the set of contained reads or a histogram based interval summary. Tests on datasets ranging from 105 to 108 reads demonstrate that ReadDB performance is generally within a factor of two of local-storage based methods and often three to five times better than other network-based methods.ConclusionsReadDB is a high-performance foundation for ChIP-Seq and RNA-Seq analysis. The client-server model provides convenient access to compute cluster nodes or desktop visualization software without requiring a shared network filesystem or large amounts of local storage. The client code provides a simple interface for fast data access to visualization or analysis. ReadDB provides a new way to store genome-aligned reads for use in applications where read sequence and alignment mismatches are not needed.

RFS

Due to the increasing number of applications (and their data) being placed on mobile devices, access to dependable storage is becoming a key issue in mobile system design -- and cloud storage is becoming an attractive solution. However, this introduces a number of new issues related to unpredictable wireless network connectivity and data privacy over the network. In this article we present RFS, a wireless-friendly network file system for mobile devices and the cloud. RFS provides deviceaware cache management and client-driven data security and privacy protection. We implement the RFS client in the Linux kernel and the RFS server with Amazon S3 cloud storage, and we employ two new optimizations: server prepush (a server-side data pre-fetching mechanism) and client reintegration (synchronizing a mobile device's cache with the cloud). The empirical results over wired, WiFi and 3G networks show that RFS achieves good performance compared to Coda and FScache, and it reduces network activity visibly. Further, the privacy overhead is acceptable when RFS is run over wireless networks. We present a case study of booting Android over RFS, thereby demonstrating the ability for RFS to host a full mobile system. Overall, RFS can deliver a good user experience under undependable network conditions, allowing mobile users to seamlessly, and safely, use the cloud for data storage.

BitNBD: BitTorrent-Based Network Block Device for Provisioning Virtual Machines in IaaS Clouds

Infrastructure-as-a-Service (IaaS) cloud computing is emerging as a viable alternative to the acquisition and management of physical resources. The new main feature of IaaS cloud computing is the virtual machine (VM) technology which improves the flexibility of resource management. VMs use virtual machine images that are preconfigured and ready to run. Typically, VM image management uses local file copy and distribution via a network file system (NFS). This potentially means that a more efficient method can be used for VM image distribution. For efficient VM image management, we have designed and implemented a BitTorrent-based network block device (namely, BitNBD) for provisioning VM images in IaaS clouds. The BitNBD mainly provides a ‘split read/write mechanism’ to deal with concurrent VM instances where the same pieces of a VM are shared. With respect to the legacy BitTorrent protocol, the BitNBD enhances the piece picker policy and energy-saving mode. It is very effective in minimizing VM startup delays and providing a hibernating capability.

Bound by the Speed of Light

I’ve been asked to optimize our NFS (network file system) set up for a global network, but NFS doesn’t work the same over a long link as it does over a LAN. Management keeps yelling that we have a multigigabit link between our remote sites but what our users experience when they try to access their files over the WAN link is truly frustrating. Is this just an impossible task?

Virus: A Menace for Information Security

Computer virus is a program that copy itself to harm the computer without the knowledge of user. A virus can spread from one computer to another through some executable code. The user can sent it over a network or the Internet, or carried it on a removable medium such as a floppy disk, CD, DVD, or USB drive. Its chances of spreading from one computer to other increases by infecting files on a network file system or a file system that is accessed by another computer.

CernVM – a virtual software appliance for LHC applications

CernVM is a Virtual Software Appliance capable of running physics applications from the LHC experiments at CERN. It aims to provide a complete and portable environment for developing and running LHC data analysis on any end-user computer (laptop, desktop) as well as on the Grid, independently of Operating System platforms (Linux, Windows, MacOS). The experiment application software and its specific dependencies are built independently from CernVM and delivered to the appliance just in time by means of a CernVM File System (CVMFS) specifically designed for efficient software distribution. The procedures for building, installing and validating software releases remains under the control and responsibility of each user community. We provide a mechanism to publish pre-built and configured experiment software releases to a central distribution point from where it finds its way to the running CernVM instances via the hierarchy of proxy servers or content delivery networks. In this paper, we present current state of CernVM project and compare performance of CVMFS to performance of traditional network file system like AFS and discuss possible scenarios that could further improve its performance and scalability.

Development of Ethernet emulation driver for reflective memory

Reflective memory (RFM) is adopted as a real time network in the KSTAR plasma control system (PCS). Since the data uploaded from any computer are automatically shared among all the computers on the RFM network, the design of a distributed control system based on RFM is easily implemented through the management of memory mapping. The data providers and consumers are logically well seperated so that, if memory mapping information is given, a new control unit can be added without any modification to the existing system except connecting a new RFM module through an optical cable. The KSTAR PCS is also connected with the Ethernet in addition to the RFM because the RFM does not support the Transmission Control Protocol/Internet Protocol (TCP/IP) and many network services of the operating system such as the Network File System (NFS) and the Secure Shell (SSH) are based on the TCP/IP. Therefore we developed an Ethernet emulation driver for the RFM to eliminate the need for a separate Ethernet network. The driver was tested on the Linux kernel 2.6.31. The algorithm of the emulation driver is explained and the experimental setup is presented.

Implementation of a stackable file system for real-time network backup

We propose a backup system based on a stackable mirroring file system, general-purpose mirroring file system (GMFS). This file system mirrors data in real-time on the file system layer. It uses the typical network file system (NFS) and backs up data to one or more NFS servers. Therefore, an existing empty resource can be used. It is necessary to backup data to two or more servers in case a backup server fails. Moreover, the throughput of read and write operations has been improved by adopting a method of calling a system function inside NFS. Experimental results shows that the throughput of read and write operations of GMFS is 2.4 times faster than that of conventional mirroring file systems. In addition, even when the number of backup servers increases, the decrease in throughput is small.

CA-NFS

We develop a holistic framework for adaptively scheduling asynchronous requests in distributed file systems. The system is holistic in that it manages all resources, including network bandwidth, server I/O, server CPU, and client and server memory utilization. It accelerates, defers, or cancels asynchronous requests in order to improve application-perceived performance directly. We employ congestion pricing via online auctions to coordinate the use of system resources by the file system clients so that they can detect shortages and adapt their resource usage. We implement our modifications in the Congestion-Aware Network File System (CA-NFS), an extension to the ubiquitous network file system (NFS). Our experimental result shows that CA-NFS results in a 20% improvement in execution times when compared with NFS for a variety of workloads.

대용량 이메일 서비스를 위한 분산 구조 기반의 SMTP 서버

SMTP(Simple Mail Transfer Protocol) 서버는 네트워크를 기반으로 사용자의 메일을 전달 해주거나 저장하기 위해 만들어졌다. SMTP 서버는 대규모 서비스를 운영하거나 사용자가 많은 경우 또는 많은 양의 메일을 처리해야 하는 경우에는 SMTP 서버들을 복수로 구성할 수 있어야 한다. SMTP 서버의 메일 저장 공간인 파일시스템을 분리함으로써 복수의 SMTP 서버로 구성하는 것이 가능하며, 이때 분리된 파일시스템을 각각의 SMTP 서버들이 공유하기 위해 NFS(Network File System)를 사용하게 된다. 그러나 NFS는 네트워크 기반의 파일시스템으로써 파일시스템이 가지는 특성을 모두 가지고 있기 때문에, SMTP 서버들이 메일 저장 공간 공유를 위해 사용할 경우 불필요한 작업들로 인해 오버헤드가 발생하게 된다. 본 논문에서는 소켓(Socket)을 통해 직접 작업하는 방식을 사용하여 NFS를 사용함으로 인해 발생하는 작업 오버헤드를 줄이는 방식을 제안한다. NFS를 사용함으로 인해 발생하는 오버헤드를 최소화하기 위해 직접 소켓 작업을 할 수 있도록, 정보 기반의 저장 공간 구조 및 메일 저장을 위한 프로토콜을 정의하였다. 제안된 방식은 Netscape에서 만든 Mailstone을 이용하여 실험을 수행하였고, 실험을 통하여 제안된 방식이 기존 방식에 비해 성능이 향상되었음을 확인하였다. An SMTP(Simple Mail Transfer Protocol) server was designed for delivering and storing user's email across a network. An SMTP server can be distributed as multiple servers for large service, huge users or massive emails. An SMTP server can be constructed by multiple servers with separating file system as email storing space, and each SMTP server can usually share each file system by using the NFS(Network File system). However the NFS is originally designed for sharing each file system across a network, and contains all attributes and features of regular file system. Using this NFS for email storing space of SMTP servers, it makes overhead due to unnecessary work of regular file system. In this paper, we propose a method to do directly operation with socket for reducing work overhead caused by the NFS. For doing directly operation with socket, this paper defines information based storing space structure and a protocol for storing emails. We performed experiments using Mailstone made by Netscape. The experimental results show the performance improvement of the proposed method compared to the existing method.

CRUST: cryptographic remote untrusted storage without public keys

This paper presents CRUST, a stackable file system layer designed to provide secure file sharing over remote untrusted storage systems. CRUST is intended to be layered over insecure network file systems without changing the existing systems. In our approach, data at rest is kept encrypted, and data integrity and access control are provided by cryptographic means. Our design completely avoids public-key cryptography operations and uses more efficient symmetric-key alternatives to achieve improved performance. As a generic and self-contained system, CRUST includes its own in-band key distribution mechanism and does not rely on any special capabilities of the server or the clients. We have implemented CRUST as a Linux file system and shown that it performs comparably with typical underlying file systems, while providing significantly stronger security.

Eight Times Acceleration of Geospatial Data Archiving and Distribution on the Grids

A grid-powered Web Geographical Information Science (GIS)/Web Processing Service (WPS) system has been developed for archiving and distributing large volumes of geospatial data. However, users, WPS servers, and data resources are always distributed across different locations, attempting to access and archive geospatial data from a GIS survey via conventional Hypertext Transport Protocol, Network File System Protocol, and File Transfer Protocol, which often encounters long waits and frustration in wide area network (WAN) environments. To provide a ldquolocal-likerdquo performance, a WAN/grid-optimized protocol known as ldquoGridJetrdquo developed at our lab was used as the underlying engine between WPS servers and clients, which utilizes a wide range of technologies including the one of paralleling the remote file access. No change in the way of using software is required since the multistreamed GridJet protocol remains fully compatible with the existing IP infrastructures. Our recent progress includes a real-world test that PyWPS and Google Earth over the GridJet protocol beat those over the classic ones by a factor of two to eight, where the distribution/archiving distance is over 10 000 km.

Mitigating Denial-of-Service Attacks on the Chord Overlay Network: A Location Hiding Approach

Serverless distributed computing has received significant attention from both the industry and the research community. Among the most popular applications are the wide area network file systems, exemplified by CFS, Farsite and OceanStore. These file systems store files on a large collection of untrusted nodes that form an overlay network. They use cryptographic techniques to maintain file confidentiality and integrity from malicious nodes. Unfortunately, cryptographic techniques cannot protect a file holder from a Denial-of-Service (DoS) or a host compromise attack. Hence, most of these distributed file systems are vulnerable to targeted file attacks, wherein an adversary attempts to attack a small (chosen) set of files by attacking the nodes that host them. This paper presents LocationGuard - a location hiding technique for securing overlay file storage systems from targeted file attacks. LocationGuard has three essential components: (i) location key, (ii) routing guard, a secure algorithm that protects accesses to a file in the overlay network given its location key, and (iii) a set of location inference guards. Our experimental results quantify the overhead of employing LocationGuard and demonstrate its effectiveness against DoS attacks, host compromise attacks and various location inference attacks.

Deconstructing Network Attached Storage systems

Network Attached Storage (NAS) has been gaining general acceptance, because it can be managed easily and files shared among many clients, which run different operating systems. The advent of Gigabit Ethernet and high speed transport protocols further facilitates the wide adoption of NAS. A distinct feature of NAS is that NAS involves both network I/O and file I/O. This paper analyzes the layered architecture of a typical NAS and the data flow, which travels through the layers. Several benchmarks are employed to explore the overhead involved in the layered NAS architecture and to identify system bottlenecks. The test results indicate that a Gigabit network is the system bottleneck due to the performance disparity between the storage stack and the network stack. The tests also demonstrate that the performance of NAS has lagged far behind that of the local storage subsystem, and the CPU utilization is not as high as imagined. The analysis in this paper gives three implications for the NAS, which adopts a Gigabit network: (1) The most effective method to alleviate the network bottleneck is increasing the physical network bandwidth or improving the utilization of network. For example, a more efficient network file system could boost the NAS performance. (2) It is unnecessary to employ specific hardware to increase the performance of the storage subsystem or the efficiency of the network stack because the hardware cannot contribute to the overall performance improvement. On the contrary, the hardware methods could have side effect on the throughput due to the small file accesses in NAS. (3) Adding more disk drives to an NAS when the aggregate performance reaches the saturation point can only contribute to storage capacity, but not performance. This paper aims to guide NAS designers or administrators to better understand and achieve a cost-effective NAS.

Performance Comparison of IP-Networked Storage

Dramatically increasing amounts of digital data are placing huge requirements on storage systems. IP-networked storage systems, such as the network file system (NFS)-based network-attached storage (NAS) systems and the iSCSI-storage area network (SAN) systems, have become increasingly common in today's local area network (LAN) environments. The emergence of new storage techniques, such as object-based storage (OBS) and content aware storage (CAS), significantly improves the functionality of storage devices to meet further needs for storage sub-systems. However, these may impact system performance. This paper compares the performance of NFS, iSCSI storage, object-based storage devices (OSDs), and CAS-based storage systems in an environment with no data sharing across host machines. A gigabit ethernet network is used as the storage network. Test results demonstrate that the performances of these systems are comparable with CAS being much better than the others for write operations. The performance bottlenecks in these systems are analyzed to provide insight into how future storage systems may be improved and possible optimization methods. The analysis shows how the I/O interfaces in these systems affect the application performance and that network-based storage systems require optimized I/O latency and reduced network and buffer processing in the servers.