Fatigue life knockdown due to surface roughness induces local stress concentrations. Physical fatigue experiments are crucial to qualify materials for use but are limited in their capability to examine the mechanisms underlying the fatigue crack formation driving force due to these stress concentrations. Microstructure-sensitive computational models can aid in this regard and are employed in this work to examine the effects of surface roughness in Al 7075-T6. Digital microstructure models are subjected to crystal plasticity finite element method (CPFEM) simulations to examine the combined effects of microstructure and grain scale asperities on the driving force for fatigue crack formation. Following elastic–plastic shakedown, mesoscale volume-averaged fatigue indicator parameters (FIPs) are computed within fatigue damage process zones of grains. An increase in the intensity of realistic surface roughness profiles corresponds accordingly to larger FIPs but it is difficult to precisely interpret the mechanisms and effects of this intensification. We thus parametrically investigate surface asperities that couple with microstructure and lack of constraint on slip at the free surface to describe microstructure-sensitive surface roughness knockdown effects on fatigue resistance. The depth of a single valley (i.e., notch) due to surface roughness is found to be more detrimental to fatigue resistance than the radius (i.e., notch width, not to be confused with notch acuity). We also find that different crystallographic textures, grain morphologies, and number of sampled grains have a limited effect on the depth of influence of these notches.