We demonstrate an all-optical thermometer based on an ensemble of silicon-vacancy centers (SiVs) in diamond by utilizing the sensitivity of the zero-phonon line wavelength to temperature, Δλ/ΔT=0.0124(2) nm K–1 [6.8(1) GHz K–1]. Using SiVs in bulk diamond, we achieve 70 mK precision at room temperature with a temperature uncertainty σT=360 mK/Hz. Finally, we use SiVs in 200 nm nanodiamonds as local temperature probes with 521 mK/Hz uncertainty and achieve sub-Kelvin precision. These properties deviate by less than 1% between nanodiamonds, enabling calibration-free thermometry for sensing and control of complex nanoscale systems.