Evaluating the impact of sampling designs on the performance of machine learning techniques for land use land cover classification using Sentinel-2 data

Abstract

ABSTRACT In today’s world, by integrating remote sensing technology and modern state-of-the-art machine learning techniques, obtaining Land Use Land Cover (LULC) maps has become easier in comparison to traditional manual methods. The performance of a Machine Learning classifier is influenced by various factors. The objective of this study is to evaluate the impact of sampling design in rough complex terrain located in the Northern Himalayan region in Uttarakhand state, India, where reference data is often limited due to the geographical characteristics of the study area. Three sampling design strategies have been incorporated in this study, namely, stratified random sampling with a proportional number of samples (SRS)proportional, stratified random sampling with an equal number of samples (SRS)equivalent and stratified systematic sampling with an equal number of samples with a minimum distance of 10 m between the consecutive samples (SSS)D = 10 m for the LULC classification. In this study, Sentinel-2 data of 10 m spatial resolution for the study area of Dehradun district, Uttarakhand, India, has been selected. The following conclusions can be drawn from the results of this study (i) (SRS)proportional achieved the highest Overall Accuracy (OA) among all the three sampling techniques. The OA and kappa score (ka) using (SRS)proportional are OA = 90.25 and ka = 0.874 by Random Forest, OA = 88.84 and ka = 0.856 by Support Vector Machine and k Nearest Neighbours (kNN) obtained OA = 87.72 and ka = 0.842, respectively. (ii) It was found that in the case of (SRS)proportional, the majority classes like the deciduous forest, evergreen forest and cropland achieved higher recall and precision values in comparison to those obtained from the other two sampling strategies, i.e. (SRS)equivalent and (SSS)D = 10 m. (iii) The results showed that while switching from (SRS)proportional to (SRS)equivalent or from (SRS)proportional to (SSS)D = 10 m, there was a slight reduction in the precision and recall values for the majority classes and a slight increase for a few of the minority classes.