crop classification is conducive to precision agriculture. Due to the cost of high-resolution image collection, it is uneasy to conduct crop classification in remotely sensed scenes using deep networks, which have become increasingly popular in remote sensing. This work combines geographical-based image analysis (GEOBIA) with deep learning for crop classification in a small area. An image classifier network is designed by using multi-scale CNN and transformer modules. The network input is an image transformed from a segment obtained using multi-resolution segmentation (MRS). An iterative optimization framework is developed to correct the segments with under-segmentation errors (USE). Two scenes of high-resolution images are employed for the experiment. The proposed optimization algorithm leads to superior performance to competitors. By using the proposed classifier network as a baseline, the optimization approach can improve overall accuracy (OA) by 4.33 % and 1.29 % respectively for the first and second dataset.