Abstract The ultra-high-energy (UHE) Galactic diffuse gamma-ray emission holds important information on the propagation of cosmic rays in the Galaxy. However, its measurements suffer from a contamination from unresolved sources whose contribution remains unclear. In this Letter, we propose a novel data-driven estimate of the contribution of unresolved pulsar wind nebulae and TeV halos based on the information present in the Australia Telescope National Facility and the LHAASO catalogs. We find that in the inner Galaxy, this contribution is limited to ∼38% ± 10% of the diffuse flux measured by LHAASO at ∼20 TeV in the case where all sources associated to pulsars contribute as unresolved sources, and this fraction drops to less than 21% ± 6% above 100 TeV. In the outer Galaxy, this contribution is always subdominant. In particular, it reaches at most ∼18% ± 2% at 10 TeV and is less than ∼7% ± 1% above ∼25 TeV. We conclude that the UHE Galactic diffuse gamma-ray emission cannot be dominated by unresolved pulsar sources above a few tens of TeV.