Abstract
Background: Extreme growth of data necessitates the need for high-performance computing. MapReduce is among the most sought-after platform for processing large-scale data. Research work and analysis of the existing system has revealed its performance bottlenecks and areas of concern. MapReduce has the problem of skew on its processing nodes. This paper proposes an algorithm for MapReduce to balance the load and eliminate the skew on Map tasks. It reduces the execution time of job by lowering the completion time of the slowest task. Methods: The proposed method performs one-time settlement of load balancing among the Map tasks by analyzing the expected completion time of the Map tasks and redistributes the load. It uses intervals to migrate the overloaded or slows tasks and append them on the under loaded tasks. Results: Experiments revealed an improvement of up to 1.3x by implementing the proposed strategy. Comparison of the proposed technique with other relevant strategies exhibits a better distribution of load among Map tasks and lower level of the skew. Evaluation is done using different workloads. Conclusion: A significant improvement is observed in the performance and reduced completion time of job.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Recent Advances in Computer Science and Communications
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
