DEVELOPMENT OF AUTOMATED SATELLITE DATA DOWNLOADING AND PROCESSING PIPELINE ON AWS CLOUD FOR NEAR-REAL-TIME AGRICULTURE APPLICATIONS

Abstract

Abstract. Remote sensing satellites allow users to acquire detailed information about the Earth's surface on a temporal basis. Widen time-series analysis at a large geographical scale involves a huge amount (in Terabytes) of satellite data downloading and processing operations. Such processes need good computational power, large storage, and sophisticated tools. Maintaining such infrastructure can cost heavily to the research/commercial enterprises. To overcome such issues, Amazon Web Service (AWS) offers a sophisticated cloud computing environment. We developed an in-house automated satellite data downloading and processing (ADDPro) pipeline on the AWS platform. The ADDPro pipeline employed Sentinel-2 satellite data to offer current and relative vegetation health information of the agriculture region on a temporal basis at the pan-India scale. Image compositing and multi-sensor data fusion technique have been incorporated into the ADDPro pipeline to produce cloud-free raster (GeoTIFF) outputs. ADDPro pipeline also facilitates lossless raster data compression, which reduces AWS data transfer costs between regions. Data compression also aids in reducing raster publishing time on GeoServer. Operationally, AWS allows users to download only the bands required to generate a certain index (e.g. NDVI) rather than the entire Sentinel-2 data package. The entire ADDPro pipeline is extremely cost-effective, efficient, and scalable.