Al–Nb–B refiner has been considered as an alternative to Al–Ti–B refiner for hypoeutectic Al–Si alloys due to Si-poisoning effect. In this paper, the effects of Al–Nb–B and Al–Ti–B refiner on the microstructure and mechanical properties in indirect squeeze casting Al–8Si-0.2Mg-0.5Mn alloy were compared. Results show that the grain sizes in the edge chilling layer, segregation band, and the center region of the Al–Nb–B refined alloys were refined by 13.9%, 12.5%, and 17.6% compared to Al–Ti–B refined alloys, respectively. The yield strength, tensile strength, and elongation of Al–Nb–B inoculated alloys were superior to those of the unrefined alloys and Al–Ti–B inoculated alloys. The coexistence of NbAl3 and NbB2 particles in the Al–Nb–B refiner exhibited excellent anti Si-poisoning capability. They significantly decreased the grain size and uniformly distributed the eutectic Si particles and Fe-rich phases, resulting in an increase in the yield strength of the alloy by 16.1% compared to the Al–Ti–B inoculated alloy. The distribution of iron-rich phases in the segregation band was affected by the inoculation. The size of α-Al15(Fe, Mn)3Si2 phase in Al–Ti–B inoculated alloys was refined, and the number significantly increased, resulting in a decrease in tensile strength of 12.82 MPa. In the Al–Nb–B inoculated alloys, the size and number of α-Al15(Fe, Mn)3Si2 phase decreased, and the δ-Al4FeSi2 phase increased, which effectively improved the fracture toughness and ductility. In addition, with the increase in grain refinement (from unrefined to Al–Ti–B to Al–Nb–B inoculated alloy), the fracture behavior changed from trans-granular fracture to intergranular fracture, improving the fracture toughness.