When measuring the indentation hardness of a material, the hardness number is generally not constant in the micro- and nano-hardness ranges but varies with the applied load. In the present work, we examine the role of surface modifications produced by different levels of polishing on this load dependence. The Vickers hardness of four stainless steel samples, three polished and one ground were measured in the micro- (0·1–10 N load) and macro-hardness (10–2500 N load) ranges. At loads exceeding ∼10 N, a constant, low bulk hardness that is the same for all polished samples, is found. We also find that each of the four methods of surface preparation leads to different rates of increase in microhardness with decreasing load. Although this pattern of hardness change is consistent with real variations of microhardness with depth, we examine various models that have been used to describe the indentation size effect to interpret and model the measurements. We note that most of the existing models can be fitted to the data but do not always allow for a reasonable physical interpretation. We propose a modification to the model of Abu Al-Rub and Voyiadjis (2004) to best capture the pattern of hardness variation. This model can be considered a hybrid of models based on dislocation or strain gradient plasticity theory.