Exotic quantum phase transitions of strongly interacting topological insulators

Abstract

Using determinant quantum Monte Carlo (d-QMC) simulations, we demonstrate that an extended Hubbard model on a bilayer honeycomb lattice has two novel quantum phase transitions. The first is a quantum phase transition between the weakly interacting gapless Dirac fermion phase and a strongly interacting fully gapped and symmetric trivial phase, which cannot be described by the standard Gross-Neveu model. The second is a quantum critical point between a quantum spin Hall insulator with spin $S^z$ conservation and the previously mentioned strongly interacting fully gapped phase. At the latter quantum critical point the single particle excitations remain gapped, while spin and charge gap both close. We argue that the first quantum phase transition is related to the $\mathbb{Z}_{16}$ classification of the topological superconductor $^3\text{He-B}$ phase with interactions, while the second quantum phase transition is a topological phase transition described by a bosonic O(4) nonlinear sigma model field theory with a $\Theta$-term.