Research on the Performance Optimization of Hadoop in Big Data Environment

Abstract

In the age of Internet, the data transmission and storage got rapid progress, however, data processing and information extraction is still exist many problems to solve. Under the condition of so much data, processing data, get useful information; In cloud computing, big data environment to adopt the method of distributed computing, such a large complex networks, however, requires a simulation environment, for comparison and optimization platform, it can save development costs. Hadoop can evaluate the performance of distributed cloud computing platform, so the Hadoop performance directly affects the evaluation on the performance of the big data cloud computing, which fully show the importance of performance of Hadoop. Algorithm is improved based on Hadoop platform, using the particle swarm optimization algorithm improved the calculation and implementation of the Hadoop platform, so as to improve its ability to execute and compute, the calculation results and analysis show that the proposed scheme is effective.