Critical link-aware P2P search for internet videos in semantic overlay network

Abstract

Search is fundamentally important for Internet video systems, especially for user generated content (UGC) video systems (like YouTube and YouKu). Currently, these systems rely on centralized servers for video search. In this paper, we study the P2P (Peer-to-Peer) search in UGC video systems, to achieve low latency and low overhead search. To this end, we propose to build semantic overlays for users in UGC video systems. Each node (user) designates most of its connections to link with semantically relevant nodes, resulting in an overlay network with large but tight clusters. A critical link-aware search algorithm (CLA) is further proposed for query forwarding. CLA prefers loose connections that are critical for the connectivity of semantic clusters to forward queries if local clusters are not the relevant ones. The algorithm enables queries to quickly reach the semantically relevant clusters, while large clusters make it possible that queries are satisfied within the relevant clusters once reaching them. We adopt the Bloom filter to reduce maintenance cost. Finally, we use the YouKu trace for experiments. The results show the efficiency and effectiveness of CLA.