Serious farmland abandonment in hilly areas, and the resolution of commonly used satellite-borne remote sensing images are insufficient to meet the needs of identifying abandoned farmland in such regions. Furthermore, addressing the problem of identifying abandoned farmland in hilly areas with a certain level of accuracy is a crucial issue in the research of extracting information on abandoned farmland patches from remote sensing images. Taking a typical hilly village as an example, this study utilizes airborne multispectral remote sensing images, incorporating various feature factors such as spectral characteristics and texture features. Aiming at the issue of identifying abandoned farmland in hilly areas, a method for extracting abandoned farmland based on the OVR-FWP-RF algorithm is proposed. Furthermore, two machine learning algorithms, Random Forest (RF) and XGBoost, are also utilized for comparison. The results indicate that the overall accuracy (OA) of the OVR-FWP-RF, Random Forest, and XGboost classification algorithms have reached 92.66%, 90.55%, and 90.75%, respectively, with corresponding Kappa coefficients of 0.9064, 0.8796, and 0.8824. Therefore, by combining spectral features, texture features, and vegetation factors, the use of machine learning methods can improve the accuracy of identifying ground objects. Moreover, the OVR-FWP-RF algorithm outperforms the Random Forest and XGboost. Specifically, when using the OVR-FWP-RF algorithm to identify abandoned farmland, its producer accuracy (PA) is 3.22% and 0.71% higher than Random Forest and XGboost, respectively, while the user accuracy (UA) is also 5.27% and 6.68% higher, respectively. Therefore, OVR-FWP-RF can significantly improve the accuracy of abandoned farmland identification and other land use type recognition in hilly areas, providing a new method for abandoned farmland identification and other land type classification in hilly areas, as well as a useful reference for abandoned farmland identification research in other similar areas.