As the field of landscape genetics is progressing toward comparative empirical studies and meta-analysis, it is important to know how best to compare the strength of spatial genetic structure between studies and species. Moran’s Eigenvector Maps are a promising method that does not make an assumption of isolation-by-distance in a homogeneous environment but can discern cryptic structure that may result from multiple processes operating in heterogeneous landscapes. MEMgene uses spatial filters from Moran’s Eigenvector Maps as predictor variables to explain variation in a genetic distance matrix, and it returns adjusted R2 as a measure of the amount of genetic variation that is spatially structured. However, it is unclear whether, and under which conditions, this value can be used to compare the degree of spatial genetic structure (effect size) between studies. This study addresses the fundamental question of comparability at two levels: between independent studies (meta-analysis mode) and between species sampled at the same locations (comparative mode). We used published datasets containing 9,900 haploid, biallelic, neutral loci simulated on a quasi-continuous, square landscape under four demographic scenarios (island model, isolation-by-distance, expansion from one or two refugia). We varied the genetic resolution (number of individuals and loci) and the number of random sampling locations. We considered two measures of effect size, the MEMgene adjusted R2 and multivariate Moran’s I, which is related to Moran’s Eigenvector Maps. Both metrics were highly sensitive to the number of locations, even when using standardized effect sizes, SES, and the number of individuals sampled per location, but not to the number of loci. In comparative mode, using the same Moran Eigenvector Maps for all species, even those with missing values at some sampling locations, reduced bias due to the number of locations under isolation-by-distance (stationary process) but increased it under expansion from one or two refugia (non-stationary process). More robust measures of effect size need to be developed before the strength of spatial genetic structure can be accurately compared, either in a meta-analysis of independent empirical studies or within a comparative, multispecies landscape genetic study.