The structure and the chemical composition of individual layers as well as of interfaces belonging to the two heterostructures M1 (BaFe12O19/YbFeO3/YSZ) and M2 (YbFeO3/BaFe12O19/YSZ) grown by pulsed laser deposition on yttria-stabilized zirconia (YSZ) substrates are deeply characterized by using a combination of methods such as high-resolution X-ray diffraction, transmission electron microscopy (TEM), and atomic-resolution scanning TEM with energy-dispersive X-ray spectroscopy. The temperature-dependent magnetic properties demonstrate two distinct heterostructures with different coercivity, anisotropy fields, and first anisotropy constants, which are related to the defect concentrations within the individual layers and to the degree of intermixing at the interface. The heterostructure with the stacking order BaFe12O19/YbFeO3, i.e., M1, exhibits a distinctive interface without any chemical intermixture, while an Fe-rich crystalline phase is observed in M2 both in atomic-resolution EDX maps and in mass density profiles. Additionally, M1 shows high c-axis orientation, which induces a higher anisotropy constant K1 as well as a larger coercivity due to a high number of phase boundaries. Despite the existence of a canted antiferromagnetic/ferromagnetic combination (T < 140 K), both heterostructures M1 and M2 do not reveal any detectable exchange bias at T = 50 K. Additionally, compressive residual strain on the BaM layer is found to be suppressing the ferromagnetism, thus reducing the Curie temperature (Tc) in the case of M1. These findings suggest that M1 (BaFe12O19/YbFeO3/YSZ) is suitable for magnetic storage applications.