Urban soils and cultural layers may accumulate carbon over a long period, so certain carbon stocks may be stored beneath cities. Studies have shown that the turnover rate of soil inorganic carbon (SIC) is also fast, and its role in global carbon pool and regulation of atmospheric CO2 should be noticed. Visible and near infrared (vis–NIR) spectroscopy, as a rapid and cost-effective approach, has demonstrated its huge potential for soil characterization. However, little has been reported on the potential of vis–NIR spectra for estimating SIC in urban and suburban areas that are greatly affected by human activities. Here, an urban soil spectral library (SSL) with 3492 samples in Wuhan City, China was established. Our main objective was to evaluate whether stratification of an urban SSL by land-use/land-cover (LULC) strategy can generate better estimation, compared with non-stratified (global) model for predicting SIC. Five modeling approaches were developed for SIC prediction: linear regression using an optimal spectral ratio index, and partial least squares regression, Cubist, convolutional neural network (CNN), and autoencoder-based residual deep network using full-spectrum. Raw absorbance was processed by continuous wavelet transform (CWT), thus decomposing spectral data at six scales. Results demonstrated that in almost all cases, CWT scale 1 obtained better performances in predicting SIC than raw absorbance and other five CWT scales. Stratification by LULC resulted in improved accuracy, compared with non-stratified model, with CWT scale 1 obtaining the highest validation R2 of 0.73. Among the non-stratified modeling, CNN based on CWT scale 1 generated the optimal validation result with an R2 value of 0.69. The most important spectral absorption for SIC was peaked around 2310 nm. This study confirms that vis–NIR technique combined with LULC stratification and machine/deep learning has a huge potential for SIC prediction in urban soils.