The relationship between lateral erosion of salt marshes and wind waves is studied in Hog Island Bay, Virginia USA, with high-resolution field measurements and aerial photographs. Marsh retreat is compared to wave climate calculated in the bay using the spectral wave-model Simulating Waves Nearshore (SWAN). We confirm the existence of a linear relationship between long-term salt marsh erosion and wave energy, and show that wave power can serve as a good proxy for average salt-marsh erosion rates. At each site, erosion rates are consistent across several temporal scales, ranging from months to decades, and are strongly related to wave power. On the contrary, erosion rates vary in space and weakly depend on the spatial distribution of wave energy. We ascribe this variability to spatial variations in geotechnical, biological, and morphological marsh attributes. Our detailed field measurements indicate that at a small spatial scale (tens of meters), a positive feedback between salt marsh geometry and wave action causes erosion rates to increase with boundary sinuosity. However, at the scale of the entire marsh boundary (hundreds of meters), this relationship is reversed: those sites that are more rapidly eroding have a marsh boundary which is significantly smoother than the marsh boundary of sheltered and slowly eroding marshes.