Improving the System Capacity by Client Cooperation in Distributed File Service

Abstract

We have investigated the workload of a campus wide distributed file service. The observation is that hot files are downloaded many times within a short period of time. To reduce the server overhead, we employed a client side cooperation module in the system. The client side cooperation module can get file data from other clients instead of directly download data from data servers. The applied mechanism can significantly reduce server load that is critical to system scalability. Thus, this distributed file service can support more users with the same amount of servers while keeping or improving user experiences. We verified the proposed mechanism in the file service system with over 5000 registered users. Results show that, in the environment of campus network, server load can be reduced by up to 30%. The chunk response latency can be reduced by 90% percent and gives a significant boost to the client side download speed.