Phase diagram of a frustrated quantum antiferromagnet on the honeycomb lattice: Magnetic order versus valence-bond crystal formation

Abstract

We present a comprehensive computational study of the phase diagram of the frustrated S=1/2 Heisenberg antiferromagnet on the honeycomb lattice, with second-nearest (J2) and third-neighbor (J3) couplings. Using a combination of exact diagonalizations of the original spin model, of the Hamiltonian projected into the nearest neighbor short range valence bond basis, and of an effective quantum dimer model, as well as a self-consistent cluster mean-field theory, we determine the boundaries of several magnetically ordered phases in the region J2,J3\in [0,1], and find a sizable magnetically disordered region in between. We characterize part of this magnetically disordered phase as a plaquette valence bond crystal phase. At larger J2, we locate a sizable region in which staggered valence bond crystal correlations are found to be important, either due to genuine valence bond crystal ordering or as a consequence of magnetically ordered phases which break lattice rotational symmetry. Furthermore we find that a particular parameter-free Gutzwiller projected tight-binding wave function has remarkably accurate energies compared to finite-size extrapolated ED energies along the transition line from conventional N\'eel to plaquette VBC phases, a fact that points to possibly interesting critical behavior - such as a deconfined critical point - across this transition. We also comment on the relevance of this spin model to model the spin liquid region found in the half-filled Hubbard model on the honeycomb lattice.