Assessing the Effect of Training Sampling Design on the Performance of Machine Learning Classifiers for Land Cover Mapping Using Multi-Temporal Remote Sensing Data and Google Earth Engine

Shobitha Shetty,S K Srivastav,Mariana Belgiu,Prasun Kumar Gupta

doi:10.3390/rs13081433

Shobitha Shetty, S K Srivastav + Show 2 more

Open Access

https://doi.org/10.3390/rs13081433

Copy DOI

Abstract

Machine learning classifiers are being increasingly used nowadays for Land Use and Land Cover (LULC) mapping from remote sensing images. However, arriving at the right choice of classifier requires understanding the main factors influencing their performance. The present study investigated firstly the effect of training sampling design on the classification results obtained by Random Forest (RF) classifier and, secondly, it compared its performance with other machine learning classifiers for LULC mapping using multi-temporal satellite remote sensing data and the Google Earth Engine (GEE) platform. We evaluated the impact of three sampling methods, namely Stratified Equal Random Sampling (SRS(Eq)), Stratified Proportional Random Sampling (SRS(Prop)), and Stratified Systematic Sampling (SSS) upon the classification results obtained by the RF trained LULC model. Our results showed that the SRS(Prop) method favors major classes while achieving good overall accuracy. The SRS(Eq) method provides good class-level accuracies, even for minority classes, whereas the SSS method performs well for areas with large intra-class variability. Toward evaluating the performance of machine learning classifiers, RF outperformed Classification and Regression Trees (CART), Support Vector Machine (SVM), and Relevance Vector Machine (RVM) with a >95% confidence level. The performance of CART and SVM classifiers were found to be similar. RVM achieved good classification results with a limited number of training samples.

Highlights

Land Use and Land Cover (LULC) maps are vital for landscape monitoring, planning, and management and for studying the impact of climate change and human interventions on the ecosystem processes and services [1,2,3]
The accuracy of classified land cover maps from remote sensing data is affected by various factors including the choice of classifiers, quality of training data, heterogeneity of the landscape, or characteristics of the input remote sensing datasets
This paper aimed at analyzing the impact of various sampling strategies upon the performance of the Random Forest (RF)

Summary

Introduction

Land Use and Land Cover (LULC) maps are vital for landscape monitoring, planning, and management and for studying the impact of climate change and human interventions on the ecosystem processes and services [1,2,3]. While land cover is directly amenable to remote sensing, land use can be derived by using ancillary data and expert knowledge on the characteristics of the classes available in the study area. The most common procedure for identifying the land cover types is by classifying the remote sensing images collected by spaceborne or aerial platforms [4]. According to Yu et al [7], the parametric Maximum Likelihood Classifier (MLC) has been the most popular technique for image classification. In recent times, the non-parametric machine learning (ML) classifiers have been reported to achieve better classification results for LULC [8]. Among these classifiers Random Forest (RF), Remote Sens.

Objectives

Methods

Results

Discussion

Conclusion