We study the relative entanglement entropy (EE) among various primary excited states in two critical spin chains: the XXZ chain and the transverse field Ising chain at criticality. For the XXZ chain, which corresponds to free boson conformal field theory (CFT), we numerically calculate the relative EE by exact diagonalization and find a perfect agreement with the predictions by the CFT. For the transverse field Ising chain at criticality, which corresponds to the Ising CFT, we analytically relate its relative EE to that of the XXZ chain and confirm the relation numerically. We also calculate the ‘sandwiched’ Rényi relative EE and again the numerical results agree well with the analytical predictions. Our results are the first direct confirmation of the CFT predictions on the relative EE of the primary excited states in critical spin chains.