Big Data Analytical Architecture Using Divide-and-Conquer Approach in Machine-to-Machine Communication

Abstract

Machine-to-Machine (M2M) technology unremit-tingly motivates any time-place-objects connectivity of the devices in and around the world. Every day, a rapid growth of large M2M networks and digital storage technology, lead to a massive heterogeneous data depository, in which the M2M data are captured and warehoused in the diverse database frameworks as a magnitude of heterogeneous data sources. Hence, the M2M that handles Big Data might perform poorly or not according to the goals of their operator due to massive heterogeneous data sources may face various incompatibilities, such as data quality, processing and computational efficiency, analysis and feature extraction applications. Therefore, to address the aforementioned constraints, this paper presents a Big Data Analytical architecture based on Divide-and-Conquer approach. The designed system architecture exploits divide-and-conquer approach, where big data sets are first transformed into a several data blocks that can be quickly processed, then it classifies and reorganizes these data blocks from the same source. In addition, the data blocks are aggregated in a sequential manner based on a machine ID, and equally partitions the data using filtration and load balancing algorithms. The feasibility and efficiency of the proposed system architecture are implemented on Hadoop single node setup. The results show that the proposed system architecture efficiently extract various features (such as River) from the massive volume of satellite data.