A Metadata Redundancy Approach for File Systems over Lossy Packet-based Network

Abstract

Wide Area Network (WAN) has the characteristics of high latency and packet loss. Traditionally, WAN-based file systems use TCP to transfer data and metadata. Due to the properties of TCP, if packet loss occurs during transmission, the sender has to resend the lost contents after timeout, which incurs delay, resulting low performance. In this paper, we propose a metadata redundancy approach to address the issue. We use replication as well as erasure coding to produce redundant data at the server side and send it with the original data using UDP, if the client detects metadata loss and the loss is recoverable, it can recover the metadata locally at client side, which greatly improves file system performance. We carry out performance tests in a simulated WAN environment to evaluate our method. Compared with TCP, test results show our approach achieve good performance in severe delay and persistent packet loss environment.