A Comparative Study of Indexing Techniques Effect in Big Data System Storage Optimization

Abstract

The growth of the information and networking technologies such as mobile devices, social media, and cloud applications required an efficient management big data system. Moreover, especially storage system because of its great importance in terms of the cost of big data analysis to provide techniques which can search, access, process, retrieve, and then distribute data item in speed manner. So, the most important technique for dealing with big data storage in the cloud is Indexing and distribution. The continuous increase in big data requires improved indexing and distribution systems, which motivate new optimized indexing schemes. In general, most of these techniques categories the indexing techniques of big data to three general categories: Artificial Intelligent indexing (AI), Non-Artificial Intelligent indexing (NAI), and Collaborative Artificial Intelligent (CAI) indexing. The goal of this paper is to provide readers with a systematic understanding of insights the emerging techniques. This article provides a thorough survey of the storage in a big data system regarding the two most important terms in the storage (Indexing and Distribution) to gain in-depth knowledge for the existing hashing techniques with clarification of the main characteristics and how can use hash indexing. It also provides an overview of Indexing in the Hadoop framework, especially in HDFS, for future development.