Single-edge notch bending tests are conducted to study the influence of constituent particles on the fracture resistance of aluminum alloys 6061, 6063, and 6110 under high-constraint loading conditions. The alloys are tested in the as-cast state after homogenization and artificial aging to temper T6. Each alloy type was delivered with two different volume fractions of constituent particles to enable a quantitative assessment of its impact on the toughness of these aluminum alloys. One variant corresponds to the commercial alloy, whereas the other variant is tailor made with an increased amount of constituent particles by adding Fe and Si to the commercial alloy. All alloys exhibit a dendritic structure with particles clustered at grain boundaries and dendrite arm boundaries. The increased content of constituent particles in the tailor-made alloys is shown to be purely detrimental for the toughness and reduces relevant fracture energy parameters by more than 50% in the alloys tested herein. In the plane-strain-dominated regions of the specimens where the stress triaxiality is highest, crack propagation was found to take place on grain boundaries and dendrite arm boundaries due to void nucleation, growth, and coalescence from the constituent particles. Differences in toughness between the alloys are primarily related to variations in the content, size, and spacing of the constituent particles. A comparison between the three different alloy types, i.e. 6061, 6063, and 6110, shows that strength affects the toughness, but it does not follow the commonly reported trade-off between strength and ductility.