Data redundancy and compression methods for a disk-based network backup system

Abstract

Traditionally backup has been performed on tapes. With the price of disk becoming cheaper and CPUs becoming faster, the use of disk-based network backup system to replace or augment tape-based backup becomes realistic. More interestingly, random access nature of disks allows redundancy of data to be taken into consideration for more efficient network transmissions and data storage at the backup system. In this paper, we consider the type of redundancy that can be exploited and study different types of compression techniques suitable to achieve improved network efficiency and storage efficiency during the backup. One form of redundancy we exploit is the similarities between different versions. Such efficiencies are never before achievable using tape-based solutions. We also implemented a working disk-based network backup system called A-Backup system, which applies the technologies discussed in the paper. Such a system is capable of achieving over l0x compression in both network utilization and data storage during backup.