Interplay of magnetic field and trigonal distortion in the honeycomb Γ model: Occurrence of a spin-flop phase

Abstract

In candidate Kitaev materials, the off-diagonal $\Gamma$ and $\Gamma'$ interactions are identified to come from the spin-orbit coupling and trigonal distortion, respectively. They have generated intense research efforts because of their intimate relation to the field-induced magnetically disordered state reported in $\alpha$-RuCl$_3$. Theoretically, while a plethora of field-induced phases has been proposed in the honeycomb lattice, a stable intermediate phase that can survive in a wide parameter region regardless of the underlying phases is still lacking. Here we focus on the interplay of an out-of-plane magnetic field and a symmetry-allowed $\Gamma'$ term due to trigonal distortion in the dominant antiferromagnetic $\Gamma$ region. By using multifaceted approaches ranging from classical Monte Carlo and semiclassical spin-wave theory to density-matrix renormalization group, we identify an intriguing spin-flop phase in the presence of magnetic field and antiferromagnetic $\Gamma'$ interaction, before it eventually enters into a fully polarized state. As the $\Gamma'$ interaction approaches the size of $\Gamma$ one, the $\Gamma$-$\Gamma'$ model maps to the easy-axis XXZ antiferromagnet, where the spin-flop phase can be understood as a superfluid phase in the extended Bose-Hubbard model. Our finding thus demonstrates an exciting path from the honeycomb $\Gamma$ model towards a $U(1)$-symmetric XXZ antiferromagnet in a magnetic field.