MapReduce Algorithm on a Serverless Platform

Abstract

Using the MapReduce (MR) programming technique, large-scale data processing tasks can be divided into tasks that are easier to manage and independent. The serverless implementation's performance is better than the non-serverless MR model when the model parameters are altered in the experiment. Optimizing the use of the MR model on the serverless platform can be achieved by taking into account the relationship between implementation efficiency and platform settings. Additionally, it can act as a source of inspiration for future serverless hardware support configurations. Also, the serverless platform demonstrates how it improves the effectiveness of resource utilization in machine learning training. The intended result can then be achieved by combining the results of these concurrently operating jobs on server clusters. This work adapts MR, a popular big data processing framework, to the serverless platform, emphasizing realization simulation principles and services, and then uses the results in a word count experiment. The experiment uses a word count of about eleven thousand words to evaluate how well MR is implemented on Alibaba Cloud. It is validated for execution time on the platform with varying Central Processing Unit (CPU) core counts, memory configurations, and worker counts. By testing several platform configurations, it is found that the memory configuration has very little impact on the model's execution time, while the size of the CPU core has a considerable impact on reaction time relative to the number of workers.