Understanding peer distribution in the global internet

Abstract

The fast-growing traffic of peer-to-peer applications, most notably BitTorrent, is putting unprecedented pressure to Internet service providers. Understanding the peer distribution over the global Internet thus becomes critical toward building new generation of ISP-friendly peer-to-peer systems. There are unfortunately significant scalability and representability challenges in measuring and understanding realworld peer distribution. In this article we demonstrate a novel hybrid measurement methodology that uses the PlanetLab as a distributed probing platform to interact with BitTorrent trackers and peers in the global Internet. Our design achieves fast real-time scanning of genuine online peers; yet we carefully avoid the potential copyright infringement and traffic overhead for PlanetLab. From three months¿ data of over 9 million peers, we identify fundamental issues in conventional traffic locality designs, and also shed new light on re-engineering trackers and reusing historically downloaded data to make BitTorrent a better storage system.