Abstract
The problems of difference of nodes capabilities and the unevenly-distributed bandwidth of the network, widespread exist in the heterogeneous clouding environment. Together with the users randomness of submitting jobs, the problems above lead to server synchronization problems.Under the platform of Hadoop and the situations mentioned above, we come up with a method which is based on the native hadoop speculative algorithm to solve the problems. Through monitoring the load-balance in realtime, dynamically assessing the performance of the node and making the speculative tasks happened in high-performance node which meantime is the nearest node from input split, the algorithm effectively reduces the occupation of the network and accelerates executing speed. The experiment result shows that the method in the execution of which the speculative tasks has a high ratio, significantly improved the efficiency and throughput of the cluster.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
