China is suffering from serious soil and water loss due to improper land use, leading to flood and drought disasters, and industrial and agricultural reduction. The commonly used monitoring methods of soil and water loss rely on the visual interpretation of remote sensing images, which is time-consuming and laborious. In this study, deep learning semantic segmentation technology was applied to monitoring land use for soil and water conservation. Manual visual interpretation mark samples and land use products collected in recent years were combined to construct training samples for deep learning. Deep convolution neural network model based on ResNet152 and DeeplabV3+ was selected for performing land use classification. Then the soil and water loss were quantified using the Chinese Soil Loss Equation (CSLE) based on the hydrological, geographic, vegetation cover and land use information. The experimental results show that the deep learning model can quickly extract robust edge features from remote sensing images and has high pixel accuracy (83.4%). Although the model accuracy was affected by the land cover types and image quality in different study area, it can still achieve an accuracy higher than 70% in other counties through our experiment. It can further improve the information service level of soil and water conservation product, and provide a useful guideline for automatic land use interpretation and identification of soil and water conservation status based on high resolution remote sensing images.