Numerous elevational gradients of phenotypic variation in plants were described, and usually treated as results of selection as a null hypothesis. However, effects of selection are difficult to disentangle from chance events and population evolutionary history. We aimed to do this on the example of Caucasian Primula vulgaris using a genome screening approach. Additionally, we wanted to understand if selection could explain the observed spatial patterns of flower color polymorphism. We analyzed 11 populations from three elevational transects from the NW Black Sea coast and four additional populations with atypical flower color variation (223 plants in all), using amplified fragment length polymorphism (AFLP) fingerprinting. We found low genetic differentiation of populations that is more likely explained by gene flow in the past. Univocal indications of elevation-driven selection that is strong enough to be detected with AFLP genome scans were absent. The observed genetic and flower color variability of P. vulgaris is most likely explained by phylogeography (history of glacial survival in Colchis refugia, deduced before). We believe that whole genome sequencing along replicated elevational transects with further experimental verification of the results would be the best approach for studies of elevational adaptation in the future.