Performance optimization of Hadoop cluster using linux services

Abstract

Hadoop is an open source tool. It enables the processing and distributed storage of big data sets using commodity cluster computing. With Hadoop occupying a core status in the current processing era, its performance optimization is also being heavily studied. This paper introduces one such method to improve Hadoop cluster performance by using a Remote Procedure Call (RPC), rpcbind service of the Linux system. The comparison is done by executing multiple Hadoop benchmarks on a configured multi-node Hadoop cluster. The final outcome turns in rpcbind favor depicting how the service improves the cluster performance by reducing the elapsed time of the benchmark executed.