Fabric smoothness appearance assessment plays an important role in the textile and apparel industry. It remains a challenging task to evaluate the fabric smoothness objectively. In the existing objective methods based on the image analysis, features describing the fabric surface structure were used widely without considering the fact that fabric smoothness is a subjective concept concerned by the human vision perception. This paper proposes an effective method to realize the objective fabric smoothness assessment by analyzing the relationship between the spatial masking effect in the human vision system (HVS) and the fabric smoothness perceived by the human. The spatial masking effect model was improved and extended into the scale space model to imitate the human perception of the fabric smoothness appearance. A set of features was extracted from the fabric images by the model and trained by the support vector machine (SVM) classifier. The method was tested on the fabric image data set, including 385 manually labeled specimens captured by the proposed image acquisition system, and reached 82.60%, 95.84%, and 100% accuracies under the errors of 0°, 0.5°, and 1°, respectively. In the experiments, based on the discussion of the effect of the system parameters, i.e., classification models, training data size, feature scales, and illumination direction number, to the performance of the proposed method, and a set of best system parameters was verified. The final assessment results of the proposed method were illustrated and outperformed the state-of-the-art methods for fabric smoothness assessment and a series of widely used deep learning methods. Promisingly, the proposed method can provide novel research ideas for the image-based fabric smoothness assessment that considering the human perception mechanism.