Maximally localized Wannier functions, interaction models, and fractional quantum anomalous Hall effect in twisted bilayer MoTe2

Abstract

We investigate the moiré band structures and the strong correlation effects in twisted bilayer MoTe[Formula: see text] for a wide range of twist angles, employing a combination of various techniques. Using large-scale first-principles calculations, we pinpoint realistic continuum modeling parameters, subsequently deriving the maximally localized Wannier functions for the top three moiré bands. Simplifying our model with reasonable assumptions, we obtain a minimal two-band model, encompassing Coulomb repulsion, correlated hopping, and spin exchange. Our minimal interaction models pave the way for further exploration of the rich many-body physics in twisted MoTe[Formula: see text]. Furthermore, we explore the phase diagrams of the system through Hartree-Fock approximation and exact diagonalization (ED). Our two-band ED analysis underscores significant band-mixing effects in this system, which enlarge the optimal twist angle for fractional quantum anomalous Hall states.