Data Distribution Management in Large-Scale Distributed Environments

Abstract

Data Distribution Management (DDM) deals with two basic problems: how to distribute data generated at the application layer among underlying nodes in a distributed system and how to retrieve data back whenever it is necessary. This thesis explores DDM in two different network environments: peer-to-peer (P2P) overlay networks and cluster-based network environments. DDM in P2P overlay networks is considered a more complete concept of building and maintaining a P2P overlay architecture than a simple data fetching scheme, and is closely related to the more commonly known associative searching or queries. DDM in the cluster-based network environment is one of the important services provided by the simulation middle-ware to support real-time distributed interactive simulations. The only common feature shared by DDM in both environments is that they are all built to provide data indexing service. Because of these fundamental differences, we have designed and developed a novel distributed data structure, Distributed Tree (HD Tree), to support range queries in P2P overlay networks. All the relevant problems of a distributed data structure, including the scalability, self-organizing, fault-tolerance, and load balancing have been studied. Both theoretical analysis and experimental results show that the Tree is able to give a complete view of system states when processing multi-dimensional range queries at different levels of selectivity and in various error-prone routing environments. On the other hand, a novel DDM scheme, Adaptive Grid-based DDM scheme, is proposed to improve the DDM performance in the cluster-based network environment. This new DDM scheme evaluates the input size of a simulation based on probability models. The optimum DDM performance is best approached by adapting the simulation running in a mode that is most appropriate to the size of the simulation. List of Publications Related to Thesis Refereed Journal Papers • Yunfeng Gu, A. Boukerche: HD Tree: A Novel Data Structure to Support MultiDimensional Range Query for P2P Networks. Journal of Parallel and Distributed Computing, Volume 71, Issue 8 (Aug. 2011), pp. 1111-1124 • Yunfeng Gu, A. Boukerche: Efficient Adaptive Transmission Control Scheme for Large-scale Distributed Simulation System. Parallel and Distributed Systems, IEEE Transactions on, Volume 20, Issue 2 (Feb 2009), pp. 246-260 • Yunfeng Gu, A. Boukerche, and Regina B. Araujo: Performance Analysis of An Adaptive Dynamic Grid-based Approach to Data Distribution Management. Journal of Parallel and Distributed Computing, Vol. 68, Issue 4 (Apr. 2008), pp. 536-547 • A. Boukerche, Yunfeng Gu, and Regina B. Araujo: Adaptive Dynamic GridBased approach to DDM for Large-scale Distributed Simulation Systems. Journal of Computer and System Sciences, Volume 74 (Sep. 2008), pp. 1043-1054 Refereed Conference papers • Yunfeng Gu, A. Boukerche: Error-Resilient Routing for Supporting Multidimensional Range Query in Tree. Distributed Simulation and Real Time Applications (DS-RT’11), 2011 IEEE/ACM 15th International Symposium on, 4 7 Sep. 2011, MediaCITYUK, Salford, UK. pp. 44-51 • Yunfeng Gu, A. Boukerche: Hierarchically Distributed Tree. Computers and Communications (ISCC’11), 2011 IEEE 16th Symposium on, 28 Jun. 1 Jul. 2011, Kerkyra, Greece. pp. 91-96