Constraints play an important role in the entanglement dynamics of many quantum systems. We develop a diagrammatic formalism to exactly evaluate the entanglement spectrum of random pure states in large constrained Hilbert spaces. The resulting spectra may be classified into universal "phases" depending on their singularities. The simplest class of local constraints reveals a nontrivial phase diagram with a Marchenko-Pastur phase which terminates in a critical point with new singularities. We propose a certain quantum defect chain as a microscopic realization of the critical point. The much studied Rydberg-blockaded or Fibonacci chain lies in the Marchenko-Pastur phase with a modified Page correction to the entanglement entropy. Our results predict the entanglement of infinite temperature eigenstates in thermalizing constrained Floquet spin chains, as we confirm numerically.