Quantum oscillations of the j=3/2 Fermi surface in the topological semimetal YPtBi

Abstract

The bismuth-based half-Heusler materials host a nontrivial topological band structure, unconventional superconductivity, and large spin-orbit coupling in a system with very low electron density. In particular, the inversion of $p$-orbital-derived bands with an effective angular momentum $j$ of up to 3/2 is thought to play a central role in anomalous Cooper pairing in the cubic half-Heusler semimetal YPtBi, which is thought to be the first ``high-spin'' superconductor. Here, we report an extensive study of the angular dependence of quantum oscillations (QOs) in the electrical conductivity of YPtBi, revealing an anomalous Shubnikov--de Haas effect consistent with the presence of a coherent $j=3/2$ Fermi surface. The QO signal in YPtBi manifests an extreme anisotropy upon rotation of the magnetic field from the [100] to [110] crystallographic direction, where the QO amplitude vanishes. This radical anisotropy for such a highly isotropic system cannot be explained by trivial scenarios involving changes in effective mass or impurity scattering, but rather is naturally explained by the warping feature of the $j=3/2$ Fermi surface of YPtBi, providing direct proof of active high angular momentum quasiparticles in the half-Heusler compounds.