Plaquette resonating valence bond state in a frustrated honeycomb antiferromagnet

Abstract

We study the proposed plaquette-RVB (pRVB) state in the honeycomb lattice $J_1-J_2$ model with frustration arising from next-nearest neighbour interactions. Starting with the limit of decoupled hexagons, we develop a plaquette operator approach to describe the pRVB state and its low energy excitations. Our calculation clarifies that the putative pRVB state necessarily has f-wave symmetry - the plaquette wavefunction is an antisymmetric combination of the Kekul\'e structures. We estimate the plaquette ordering amplitude, ground state energy and spin gap as a function of $J_2/J_1$. The pRVB state is most stable around $J_2/J_1 \sim 0.25$. We identify the wavevectors of the lowest triplet excitations, which can be verified using exact diagonalization or DMRG studies. When $J_2$ is reduced, we can have either a deconfined Quantum Phase Transition (QPT) or a first-order transition into a N\'eel state. When $J_2$ is increased, we surmise that the system undergoes a first order phase transition into a state which breaks lattice rotational symmetry.