Remote sensing is an important tool for monitoring soil information. However, accurate spatial modeling of soil organic matter (SOM) in areas with high vegetation coverage, typically represented by agroecosystems, remains a challenge for field-scale estimation using remote sensing. To date, studies have focused on using single-period or multi-temporal vegetation information to characterize SOM. Thus, the relationship between SOM content and time-series vegetation biomass has not yet been fully explored. In addition, most studies have ignored the effects of critical soil properties and human activities (e.g., soil salinization, soil particle size fractions, history of land-use changes) on SOM. By integrating information on vegetation, soil, and human activities, we propose a novel framework for assessing SOM in cotton fields of artificial oases in northwest China, where returned straw is one of the primary sources of SOM coming from vegetation. We developed an Annual Maximum Biomass Accumulation Index (AMBAI) using time-series Landsat images from 1990 to 2019. Subsequently, we quantified the information of the planting years (PY) of cropland using spectral index threshold and incorporated proximal sensing data (soil hyperspectral and apparent conductivity data) and soil particle size fractions to establish a predictive model of SOM using partial least squares regression (PLSR), random forest (RF), and convolutional neural network (CNN). The results revealed that AMBAI had the highest correlation coefficient (r) with SOM (0.76, P < 0.01). AMBAI, soil hyperspectral data, and PY were the most relevant predictors for estimating SOM. The CNN model integrating vegetation, soil, and human activity information performed best, with coefficient of determination (R2), relative analysis error (RPD), and root mean square error (RMSE) of 0.83, 2.38 and 1.38 g kg−1, respectively. This study confirmed that AMBAI and PY had great potential for characterizing SOM in arid and semi-arid regions, providing a reference for other relevant studies.