Development of a big data system to assess ecosystem services of surface soil

Abstract

<p>A big data system plays a significant role in various fields. This technology has also been applied to environment fields because it can discover hidden patterns between environmental factors. As the massive data set was constructed for several decades, big data analysis has widely been using for extracting useful information by analyzing different types of big data sets. In this study, we developed a big data system frame to assess the ecosystem service provided from surface soil. Among big data platforms, we used the Amazon Web Service (AWS) due to their cost-efficiency and hardware flexibility. There are five stages of the big data system (i.e. data acquisition– data storage – data processing – data analysis – visualization). In the data acquisition step, the soil sensor and Internet of Things (IoT) system were used, and we collected existing soil properties data provided by national institutes such as Rural Development Administration (RDA), Ministry of Environment (MOE), and Ministry of Land, Infrastructure, and Transport (MOLIT). AWS S3 platform, which is an object storage service and provides easy-to-use management features to users, was accepted as the data storage platform of the big data system. Amazon EMR, Amazon SageMaker, and Amazon QuickSight were used for the step of data processing, data analysis, and visualization of the big data system respectively. We tested that the developed system could predict soil bulk density and able to replace a typical environmental model by using models based on machine learning and deep learning. The results of the two tests showed positive results that the developed models could predict soil properties and simulate natural phenomena as much as the typical environmental model could.  However, since the system is at an early development stage, it needs repetitive tests in the future considering various soil properties. If this system becomes fully functional, the system will be helpful to improve soil environments.</p>