Susceptibility-sensitive magnetic resonance imaging (MRI) has gained importance in multiple sclerosis (MS) research because of its versatility, high resolution and excellent sensitivity to changes in tissue structure and composition. In particular, mapping of the resonance frequency of the MR signal and quantitative susceptibility mapping (QSM) have been explored for the description of MS lesions. Many current studies utilizing these techniques attribute increases in the MR frequency or QSM to elevated tissue iron content, in addition to myelin loss. However, this common interpretation is inconsistent with recent histopathological studies. Here, we investigate the nature of MR frequency shifts related to MS lesions by comparing post-mortem MRI data with histology, and contrast them with numerical simulations of the MR signal. We demonstrate that iron accumulation is not the driving source of the MR frequency or QSM image contrast in our sample; rather, most chronic MS lesions are characterized by advanced loss of both myelin and iron. Moreover, our results suggest that the appearance of MS lesions on MR frequency maps and QSM depends on changes in the non-lesional white matter surrounding the lesions. Understanding and accounting for these changes is essential for the quantitative interpretation of MR frequency or QSM data in white matter.