Surface roughening of polycrystalline Al–Mg alloys during tensile deformation is investigated using white light confocal microscopy. Materials are tested that differ only in grain size. A height–height correlation technique is used to analyze the data. The surface obeys self-affine scaling on length scales up to a correlation length which approximately equals the grain size and above which no height correlation is present. The self-affine scaling exponent increases initially with strain and saturates at a value around 0.9. A linear relation is observed between root-mean-square roughness and both strain and grain size. The observed roughness is explained as the result of the combined effect of a self-affine roughening on a subgrain scale and a grain scale roughening caused by orientation differences between neighboring grains.