Assessing the size of representative volume elements (RVEs) for fatigue-related applications is challenging. A RVE relevant to random microstructure requires a volume of material that is sufficiently large to capture the grain/phase heterogeneity that captures all statistical moments of the distribution of the driving force for fatigue crack formation at “hot spot” grains. Consequently, the large size of a microstructure RVE required to study fatigue phenomena is largely computationally intractable and difficult to explore. A more realistic objective in this work is to systematically study, as a function of the size of a statistical sample of microstructure, trends towards convergence of the simulated distribution of driving force for fatigue crack formation. The present work accordingly leverages the recently developed open-source PRISMS-Fatigue framework [Yaghoobi et al., npj Comput. Mater., 7, 38 (2021)] to examine the trends in convergence of extreme value distributions (EVD) of Fatigue Indicator Parameters (FIPs) in progressively larger polycrystalline microstructure realizations of FCC Al alloy 7075-T6 using crystal plasticity finite element method simulations. The results are compared to the traditional method in which ensembles of statistical volume elements (SVEs) are simulated to build up statistics intended to approximate those associated with a larger volume of material. The convergence of EVDs with increase of size of a SVE of microstructure is closely related to the extent of grain nearest neighbor (NN) interactions. Accordingly, the sensitivity of the local micromechanical response at hot spot grains is quantitatively investigated by systematically varying the orientations of NN grains. Results indicate that SVEs with cubic crystallographic texture tend towards convergence of the EVD of FIPs with tens of thousands of grains while the random and rolled textures require larger volumes. Simple relationships based on microstructure parameters (e.g., Schmid Factor, grain size, NN misorientation) do not completely correlate to fatigue hot spot grains. Finally, the sensitivity of the extreme value fatigue response at hot spot grains extends to the 3rd NN when a single neighborhood grain orientation is altered.