Data structures for big data

Abstract

Listen

Translate article

The present day world dealing with big data (expanding very fast in 4Vs: Volume, Varity, Velocity and Veracity) needs New advanced kind of logical and physical storage structures, New advanced kind of heterogeneous data structures, New mathematical theories and New models for processing giant big data in 4Vs. In the literature, there is no appropriate data structure for big data, no appropriate network topology for big data, no appropriate distributed system for big data. The existing data structures, network topologies and type of distributed system are not sufficiently rich to deal with big data. The existing network topologies like tree topology or bus/ring/star/mesh/hybrid topologies seem to be weak topologies for big data processing. For a success, there is no other way but to develop ‘new data structures’, ‘new type of distributed systems’ having a very fast and tremendous extent of mutual compatibility and mutual understanding with the new data structures, ‘new type of network topologies’ to support the new distributed system, and of course ‘new mathematical/logical theory’ models. Needless to mention that the next important issue is how to integrate all these ‘new’ to make ultimately a single and simple scalable system to the laymen users. With these views in mind, Biswas [11, 12, 15] introduced a new special type of fundamental data structure ‘atrain’ exclusively for heterogeneous big data (where, the data structure ‘train’ is exclusively for homogeneous big data) and then introduced a new distributed system called by ‘Atrain Distributed System’ (ADS) which can process big data of any 4V of any momentum. The rich merit of ADS [15] is due to its layout architecture making it infinitely scalable both in breadth (horizontally) and depth (vertically). The ‘Atrain Distributed System’ (ADS) could be unitier or multitier having a single unique Pilot Computer (PC) and several Distributed Computers (DCs) connected by new