We calculate the frequency-dependent dielectric function for the series of antiferromagnetic transition metal oxides (TMOs) from MnO to NiO using many-body perturbation theory. Quasiparticle, excitonic, and local-field effects are taken into account by solving the Bethe-Salpeter equation in the framework of collinear spin polarization. The optical spectra are based on electronic structures which have been obtained using density-functional theory with a hybrid functional containing screened exchange (HSE03) and a subsequent quasiparticle calculation in the $GW$ approximation to describe exchange and correlation effects adequately. These sophisticated quasiparticle band structures are mapped to electronic structures resulting from the computationally less expensive $\mathrm{GGA}+U+\ensuremath{\Delta}$ scheme that includes an on-site interaction $U$ and a scissors shift $\ensuremath{\Delta}$ and allows us to calculate the large number of electronic states that is necessary to construct the Bethe-Salpeter Hamiltonian. For an accurate description of the optical spectra, an appropriate treatment of the strong electron-hole attraction is mandatory to obtain agreement with the experimentally observed absorption-peak positions. The itinerant $s$ and $p$ states as well as the localized transition metal $3d$ states have to be considered on an equal footing. We find that a purely atomic picture is not suitable to understand the optical absorption spectra of the TMOs. Reflectivity spectra, absorption coefficients, and loss functions at vanishing momentum transfer are computed in a wide spectral range and discussed in light of the available experimental data.