Case study on: Scalability of preprocessing procedure of remote sensing in Hadoop

Abstract

In the field of remote sensing, the recent increase in image sizes has drawn a significant attention on processing these files in a fault tolerant distributed architecture In this regard, Apache Hadoop architecture becomes an efficient MapReduce model. In the satellite image processing, large scale images put the limitation on the single computer analysis. Whereas, Hadoop Distributed File System (HDFS) gives a remarkable solution to handle these files through its inherent data parallelism technique. This architecture is well suited for structured data, as the structured data can be equally distributed easily and access the relevant data. Images are considered as unstructured matrix data in Hadoop and the whole part of the data is relevant for any processing. It leads to a challenge to maintain the data locality with the equal data distribution. In this paper, we introduce a novel technique, which decrypts the standard format of raw satellite data and localizes the distributed preprocessing step on the equal split of datasets in Hadoop. For this purpose, a suitable modification on the Hadoop interface is proposed. For the case study on the scalability of preprocessing steps, Synthetic Aperture Radar (SAR) and Multispectral (MS), are used in a distributed environment.