Network Coordinate System Research Articles

Constructing a Two-Level Topology-Aware Distributed Hash Table with a Hierarchical Network Coordinate System

Distributed Hash Tables (DHTs) may suffer from a high lookup latency due to the issue of topology mismatching between the overlay and the physical underlying network. To handle this problem, recently, some researchers use network coordinates (NCs) to construct topology-aware DHTs(e.g. Canary). However, these proposed DHTs still have room to be improved due to the inaccuracy of NCs. To construct a more efficient DHT, in this paper, we propose hCanary: a two-level topology-aware CAN. The novelty of hCanary is that it uses a hierarchical NC system Pharos to generate more accurate NCs to construct multiple local topology-aware overlays over a global topology-aware CAN to reduce lookup latency. Meanwhile, we further use expressway technique to improve the ability of query forwarding of hCanary. Experiments show local overlays of hCanary can reduce average lookup latency by up to 60% and hCanary with expressways can reduce average lookup latency by up to 77% compared with Canary.

On Suitability of Euclidean Embedding for Host-Based Network Coordinate Systems

In this paper, we investigate the suitability of embedding Internet hosts into a Euclidean space given their pairwise distances (as measured by round-trip time). Using the classical scaling and matrix perturbation theories, we first establish the (sum of the) magnitude of negative eigenvalues of the (doubly centered, squared) distance matrix as a measure of suitability of Euclidean embedding. We then show that the distance matrix among Internet hosts contains negative eigenvalues of large magnitude, implying that embedding the Internet hosts in a Euclidean space would incur relatively large errors. Motivated by earlier studies, we demonstrate that the inaccuracy of Euclidean embedding is caused by a large degree of triangle inequality violation (TIV) in the Internet distances, which leads to negative eigenvalues of large magnitude. Moreover, we show that the TIVs are likely to occur locally; hence the distances among these close-by hosts cannot be estimated accurately using a global Euclidean embedding. In addition, increasing the dimension of embedding does not reduce the embedding errors. Based on these insights, we propose a new hybrid model for embedding the network nodes using only a two-dimensional Euclidean coordinate system and small error adjustment terms. We show that the accuracy of the proposed embedding technique is as good as, if not better than, that of a seven-dimensional Euclidean embedding.

Exploiting Location-aware Mechanism for Distributed Web Crawling over DHTs

Inspired by the concept of internet computing, DHT-based distributed Web crawling model is proposed to solve the bottlenecks of the traditional Web crawling systems. Based on this system model, we propose optimizations to reduce the download time of the Web crawling tasks in order to increase the efficiency of the system. The improvement on the download time is achieved by shortening the crawler-crawlee network distance. By utilizing the mapping mechanism of Content Addressable Network (CAN) over Network Coordinate System (NC), the issue can be mapped onto a problem of minimizing the distances between peers and resources on the DHT overlay. This paper focuses on reducing such distances, seeking to provide an improved location-aware infrastructure for distributed Web crawling. A new DHT-based distributed Web crawling model is proposed first. Then, under this model, a new method based on CAN’s splitting schemes is proposed which shows a significant decrease in crawler-crawlee distance against existing schemes. In addition, the issue of load balancing is also solved by combining the new method with old ones.

Efficient Distributed Web Crawling Utilizing Internet Resources

Internet computing is proposed to exploit personal computing resources across the Internet in order to build large-scale Web applications at lower cost. In this paper, a DHT-based distributed Web crawling model based on the concept of Internet computing is proposed. Also, we propose two optimizations to reduce the download time and waiting time of the Web crawling tasks in order to increase the system's throughput and update rate. Based on our contributor-friendly download scheme, the improvement on the download time is achieved by shortening the crawler-crawlee RTTs. In order to accurately estimate the RTTs, a network coordinate system is combined with the underlying DHT. The improvement on the waiting time is achieved by redirecting the incoming crawling tasks to light-loaded crawlers in order to keep the queue on each crawler equally sized. We also propose a simple Web site partition method to split a large Web site into smaller pieces in order to reduce the task granularity. All the methods proposed are evaluated through real Internet tests and simulations showing satisfactory results.

Matchmaking for online games and other latency-sensitive P2P systems

The latency between machines on the Internet can dramatically affect users' experience for many distributed applications. Particularly, in multiplayer online games, players seek to cluster themselves so that those in the same session have low latency to each other. A system that predicts latencies between machine pairs allows such matchmaking to consider many more machine pairs than can be probed in a scalable fashion while users are waiting. Using a far-reaching trace of latencies between players on over 3.5 million game consoles, we designed Htrae, a latency prediction system for game matchmaking scenarios. One novel feature of Htrae is its synthesis of geolocation with a network coordinate system. It uses geolocation to select reasonable initial network coordinates for new machines joining the system, allowing it to converge more quickly than standard network coordinate systems and produce substantially lower prediction error than state-of-the-art latency prediction systems. For instance, it produces 90th percentile errors less than half those of iPlane and Pyxida. Our design is general enough to make it a good fit for other latency-sensitive peer-to-peer applications besides game matchmaking.

Handling node churn in decentralised network coordinate system

A Network Coordinate (NC) system is an efficient mechanism to predict Internet distance with scalable measurements. In this paper, we focus on the node churn problem – the continuous process of nodes arrival and departure – in distributed applications. Studies on Vivaldi, a representative distributed NC system, show that under node churn the prediction accuracy of the NC system will be seriously impaired. In this paper, we focus on how to handle the impact of node churn in Vivaldi. Firstly, we propose a simple solution by directly increasing the measurement frequency. Our experiments have demonstrated that this approach can reduce the harm of node churn. However, it increases the communication overhead as the measurement frequency grows. To avoid such expensive solution, we propose the design and implementation of Myth, a decentralised and fast convergence NC system. It introduces the merit of Landmark-based NC system to shorten convergence time in Vivaldi with slight extra overhead. Our experimental results show that Myth outperforms Vivaldi a lot under node churn, without compromising the performance under stable environment. Moreover, we have found that the use of Myth is a cost-effective way to achieve higher prediction accuracy; it will not only improve the prediction accuracy but also save the communication overhead.

Pharos: accurate and decentralised network coordinate system

Network coordinates (NC) system is an efficient mechanism for Internet distance prediction with scalable measurements. The intrinsical cause for the unsatisfactory accuracy of the simulation-based NC algorithms has been identified. Then Pharos, a fully decentralised and hierarchical scheme, is proposed to solve this problem. Pharos leverages multiple coordinate sets at different distance scales, with the right scale being chosen for prediction each time. We evaluate the performance of Pharos system with the King data set and latency data from PlanetLab, and compare it with the representative NC system, Vivaldi. The experimental results show that Pharos greatly outperforms Vivaldi in Internet distance prediction without adding any significant overhead. Our extensive evaluation results also demonstrate that Pharos can significantly improve the performance in distributed Internet applications, such as overlay multicast and server selection.

A study of distorsions of the primary triangulation network of Slovenia

The paper presents some methods of transforming ETRS89 co-ordinates to official Slovenian datum (D48) co-ordinates. Together with traditional transformation methods of scaling, translation and rotation least squares collocation is also used. The results of different transformation procedures between the astrogeodetic network and global co-ordinate system (ETRS89) show a lot of the drawbacks of existing national co-ordinate frame.