Deep forest (DF), an alternative to neural networks (NNs) based deep learning (DL), has gained increasing attentions in recent years. Despite its remarkable advantages, the original multi-Grained Cascade Forest (gcForest) is limited by the high time cost and memory requirement. To overcome this limitation, gcForest with confidence screening (gcForestCS) and feature screening (gcForestFS) were proposed with the proven improvements. But they were not comparatively investigated for remote sensing (RS) image classification. Furthermore, gcForest, gcForestCS and gcForestFS could be further improved by introducing patch-based pooling (PP), morphological profiling (MP) and pseudo labeling (PL) techniques. In this sense, DF algorithms are introduced and comparatively studied for hyperspectral and polarimetric synthetic aperture radar (PolSAR) image classification first. To further foster the classification performance from accurate, efficient and effective feature abstraction viewpoints, improved versions of gcForest, gcForestCS and gcForestFS, are proposed by adopting PP, MP PL techniques. To evaluate the performance of the introduced and proposed DF algorithms, 6 state-of-the-art spectral-spatial features aware NNs based DL algorithms are selected. Experimental results on three widely acknowledged hyperspectral and PolSAR benchmarks showed that: 1) gcForest, gcForestCS and gcForestFS are also advanced algorithms for remote sensing (RS) image classification; 2) mixed pooling with larger patch size set is always the best option in contrast with average, maximum, minimum and median pooling strategies; and 3) positive improvements on gcForest, gcForestCS and gcForestFS are clear using PP, MP and PL techniques, and the best improvements can always be obtained by fused usage of PP and MP with PL features.