The optimal query plan selection based on the network and remote server analysis

Abstract

Query processing over the Internet involving multiple data sources is one of the most difficult and important problems. In this new data processing environment, three major factors affect the cost of a query; network congestion situation, server states (server workload), and data/query complexity. In this paper, we study the behavior of the wide area network and derive the server states from the system contention states and network congestion situation. We also analyze two cases of changes in network behavior and propose different methods to deal with them respectively. The first case is that the changes in the network are regular. We propose a time slide method to deal with this situation. The second case is that the changes in the network are irregular. We use a real discrete Fourier transform method to filter the noise in the main trend of network. So we can choose query plan much accurately by estimating the cost of servers and the network respectively at the query optimization stage.