SU(4) Heisenberg model on the honeycomb lattice with exchange-frustrated perturbations: Implications for twistronics and Mott insulators

Abstract

The SU(4)-symmetric spin-orbital model on the honeycomb lattice was recently studied in connection to correlated insulators such as the $e_{g}$ Mott insulator Ba$_{3}$CuSb$_{2}$O$_{9}$ and the insulating phase of magic-angle twisted bilayer graphene at quarter filling. Here we provide a unified discussion of these systems by investigating an extended model that includes the effects of Hund's coupling and anisotropic, orbital-dependent exchange interactions. Using a combination of mean-field theory, linear flavor-wave theory, and variational Monte Carlo, we show that this model harbors a quantum spin-orbital liquid over a wide parameter regime around the SU(4)-symmetric point. For large Hund's coupling, a ferromagnetic antiferro-orbital ordered state appears, while a valence-bond crystal combined with a vortex orbital state is stabilized by dominant orbital-dependent exchange interactions.