Effectiveness of the integration of data balancing techniques and tree-based ensemble machine learning algorithms for spatially-explicit land cover accuracy prediction

Abstract

Classification of remote sensing images has become a common practice in providing land use/land cover (LULC) maps. Among the main elements of a successful LC mapping task, accuracy assessment significantly influences the quality of decisions taken by the end users and prevents potential misinterpretations. The use of conventional accuracy assessment approaches often overlooks the spatial variation of LC accuracy. In this manner, per-pixel land cover accuracy (PLCA) mapping which provides spatially-explicit accuracy information of classified remote sensing images, emerged as a viable alternative to the traditional accuracy assessment approaches. This study comprehensively compared three tree-based ensemble machine learning algorithms including random forest (RF), rotation forest (RoF), and stochastic gradient boosting (SGB) for PLCA mapping. Moreover, considering the efficiency of the LC mapping practices, there are normally more instances linked with correctly classified pixels than incorrectly classified pixels in a given LC map. This condition leads to a serious challenge for PLCA mapping known as the class imbalance problem. Accordingly, this study also examined the feasibility of coupling three data balancing (DB) techniques (random over-sampling (ROS), synthetic minority oversampling technique (SMOTE), and adaptive synthetic sampling approach for unbalanced learning (ADASYN)) with the selected PLCA mapping algorithms for addressing the class imbalance problem. The experiments were employed on five different Sentinel-2 images with 20 km × 20 km dimensions acquired at different times and parts of Iran. The area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), geometric mean (GM), and overall accuracy (OA) were used to assess performance of the employed PLCA mapping methods. Among the PLCA mapping methods, RF-SMOTE was ranked first (with an average, AUROC = 0.84, AUPRC = of 0.91, GM = 0.81, and OA = 0.82), and was followed by RF-ADASYN and RoF-SMOTE. The results showed that integration of the SMOTE technique with the RF algorithm was the most successful practice for improving the performance of PLCA mapping and significantly outperformed all the algorithms without integration of DB techniques. As the PLCA maps provide the LULC's accuracy locally, they can be of significant use in refining the LULC mapping practices.