Abstract
MapReduce is a programming model for data-intensive applications. The performance of MapReduce is greatly affected by the scheduling efficiency, which needs to consider both the parallelism of processing and overhead of data transmission. Lots of researches only focus on optimizing scheduling mechanisms or data storage strategies to reduce the overhead of data transmission in big data applications. The analysis of MapReduce carried out doesn't consider the overhead of data transmission sufficiently. This paper proposes a divisible load scheduling model to describe the data transmission and task execution in MapReduce. The task allocation and corresponding data transmission are abstracted as scheduling divisible loads from multiple input sources. With the established model, the optimal distribution of input data and scheduling of loads in map phase are presented with linear programming. The performance of map phase is evaluated and analyzed under different environments.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
