Accurately described excitonic properties of transition metal dichalcogenide heterobilayers (HBLs) are crucial to comprehend the optical response and the charge carrier dynamics of them. Excitons in multilayer systems posses inter or intralayer character whose spectral positions depend on their binding energy and the band alignment of the constituent single-layers. In this study, we report the electronic structure and the absorption spectra of MoS$_2$/WS$_2$ and MoSe$_2$/WSe$_2$ HBLs from first-principles calculations. We explore the spectral positions, binding energies and the origins of inter and intralayer excitons and compare our results with experimental observations. The absorption spectra of the systems are obtained by solving the Bethe-Salpeter equation on top of a G$_0$W$_0$ calculation which corrects the independent particle eigenvalues obtained from density functional theory calculations. Our calculations reveal that the lowest energy exciton in both HBLs possesses interlayer character which is decisive regarding their possible device applications. Due to the spatially separated nature of the charge carriers, the binding energy of inter-layer excitons might be expected to be considerably smaller than that of intra-layer ones. However, according to our calculations the binding energy of lowest energy interlayer excitons is only $\sim$ 20\% lower due to the weaker screening of the Coulomb interaction between layers of the HBLs. Therefore, it can be deduced that the spectral positions of the interlayer excitons with respect to intralayer ones are mostly determined by the band offset of the constituent single-layers. By comparing oscillator strengths and thermal occupation factors, we show that in luminescence at low temperature, the interlayer exciton peak becomes dominant, while in absorption it is almost invisible.