We demonstrate that astrophysical constraints on the dense-matter equation of state place an upper bound on the color-superconducting gap in dense matter above the transition from nuclear matter to quark matter. Pairing effects in the color-flavor locked quark matter phase increase the pressure at high density, and if this effect is sufficiently large then the requirements of causality and mechanical stability make it impossible to reach such a pressure in a way that is consistent with what is known at lower densities. The intermediate-density equation of state is inferred by considering extensions of chiral effective field theory to neutron star densities, and conditioning these using current astrophysical observations of neutron star radius, maximum mass, and tidal deformability (PSR J0348+0432, PSR J1624-2230, PSR J0740+6620, GW170817). At baryon number chemical potential μ=2.6 GeV we find a 95% upper limit on the color-flavor locked pairing gap Δ of 457MeV using overly conservative assumptions and 216MeV with more reasonable assumptions. This constraint may be strengthened by future astrophysical measurements as well as by future advances in high-density QCD calculations.