Cavitation has long been recognized as a crucial predictor, or precursor, to the ultimate failure of various materials, ranging from ductile metals to soft and biological materials. Traditionally, cavitation in solids is defined as an unstable expansion of a void or a defect within a material. The critical applied load needed to trigger this instability -- the critical pressure -- is a lengthscale independent material property and has been predicted by numerous theoretical studies for a breadth of constitutive models. While these studies usually assume that cavitation initiates from defects in the bulk of an otherwise homogeneous medium, an alternative and potentially more ubiquitous scenario can occur if the defects are found at interfaces between two distinct media within the body. Such interfaces are becoming increasingly common in modern materials with the use of multimaterial composites and layer-by-layer additive manufacturing methods. However, a criterion to determine the threshold for interfacial failure, in analogy to the bulk cavitation limit, has yet to be reported. In this work, we fill this gap. Our theoretical model captures a lengthscale independent limit for interfacial cavitation, and is shown to agree with our observations at two distinct lengthscales, via two different experimental systems. To further understand the competition between the two cavitation modes (bulk versus interface), we expand our investigation beyond the elastic response to understand the ensuing unstable propagation of delamination at the interface. A phase diagram summarizes these results, showing regimes in which interfacial failure becomes the dominant mechanism.