Observation of backscattering induced by magnetism in a topological edge state

Abstract

The boundary modes of topological insulators are protected by the symmetries of the nontrivial bulk electronic states. Unless these symmetries are broken, they can give rise to novel phenomena, such as the quantum spin Hall effect in one-dimensional (1D) topological edge states, where quasiparticle backscattering is suppressed by time-reversal symmetry (TRS). Here, we investigate the properties of the 1D topological edge state of bismuth in the absence of TRS, where backscattering is predicted to occur. Using spectroscopic imaging and spin-polarized measurements with a scanning tunneling microscope, we compared quasiparticle interference (QPI) occurring in the edge state of a pristine bismuth bilayer with that occurring in the edge state of a bilayer, which is terminated by ferromagnetic iron clusters that break TRS. Our experiments on the decorated bilayer edge reveal an additional QPI branch, which can be associated with spin-flip scattering across the Brioullin zone center between time-reversal band partners. The observed QPI characteristics exactly match with theoretical expectations for a topological edge state, having one Kramer's pair of bands. Together, our results provide further evidence for the nontrivial nature of bismuth and in particular, demonstrate backscattering inside a helical topological edge state induced by broken TRS through local magnetism.