Abstract
Big Data is currently a hot topic for companies and scientists around the world, due to the emergence of new technologies, devices and communication means like social network sites, which led to a noticeable increase of the amount of data produced every year, even every day. In addition, traditional algorithms and technologies are inefficient to process, analyze and store this vast amount of data. So, to solve this problem, Big Data frameworks are needed. In this paper, we present and discuss a performance comparison between two popular Big Data frameworks. Hadoop and Spark, which are used to efficiently process vast amount of data in parallel and distributed mode on a large clusters. Hibench benchmark suite is used to compare the performance of these two frameworks based on the criteria as execution time, throughput and speedup. Our experimental results show that Spark is more efficient than Hadoop to deal with large amount of data. However, spark requires higher memory allocation, since it loads processes into memory and keeps them in caches for a while, just like standard databases. So the choice depends on performance level and memory constraints.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
