Hybrid fiber-reinforced polymer (HFRP) composites are widely used in aerospace structures because of their excellent overall performance. However, it is still challenging to address the issue of high sensitivity to delamination between dissimilar material interlayers in HFRP composites. This study combines experimental and numerical simulation to analyze the low-velocity impact behavior and interlaminar damage of two types of HFRP. Based on a micromechanical model, an equivalent aramid pulp (eAP) toughening laminate model is developed. The impact behavior of the HFRP toughen by eAP only in dissimilar material interlayers is simulated based on the model. The effect of eAP areal density on the impact behavior and evolution of interlaminar damage is analyzed. Results show that the maximum force and impact stiffness of eAP-toughened HFRP increase initially with the increase in eAP areal density, and then decrease slowly. The area and extent of damage of dissimilar material interlayers in eAP toughened laminates is significantly reduced. Finally, the interlaminar toughening and failure mechanisms by eAP-toughening only in dissimilar material interlayers of the HFRP composites are systematically revealed from fiber bridging and damage transfer perspectives.