Quantum spin liquid ground states of the Heisenberg-Kitaev model on the triangular lattice

Abstract

We study quantum disordered ground states of the two-dimensional Heisenberg-Kitaev model on the triangular lattice using a Schwinger boson approach. Our aim is to identify and characterize potential gapped quantum spin liquid phases that are stabilized by anisotropic Kitaev interactions. For antiferromagnetic Heisenberg and Kitaev couplings and sufficiently small spin $S$, we find three different symmetric ${Z}_{2}$ spin liquid phases, separated by two continuous quantum phase transitions. Interestingly, the gap of elementary excitations remains finite throughout the transitions. The first spin liquid phase corresponds to the well-known zero-flux state in the Heisenberg limit, which is stable with respect to small Kitaev couplings and develops ${120}^{\ensuremath{\circ}}$ order in the semiclassical limit at large $S$. In the opposite Kitaev limit, we find a different spin liquid ground state, which is a quantum disordered version of a magnetically ordered state with antiferromagnetic chains, in accordance with results in the classical limit. Finally, at intermediate couplings, we find a spin liquid state with unusual spin correlations. Upon spinon condensation, this state develops Bragg peaks at incommensurate momenta in close analogy to the magnetically ordered Z2 vortex crystal phase, which has been analyzed in recent theoretical works.