Although floral herbivory has recently received increased attention as an important factor influencing plant reproduction, relatively little is known about how its frequency and intensity vary depending on traits of host plants. Here we report that herbivore pressure by a weevil, Zacladus geranii, is associated with a flower color polymorphism of Geranium thunbergii (Geraniaceae). Pink and white flower color morphs have been reported in G. thunbergii, and we found in a three-year field survey in multiple populations that, generally, adult weevils more preferentially visited white flowers than pink flowers. Consistently, we found more severe damage by weevil larvae in white flowers. Overall herbivore pressure for G. thunbergii varied strongly between populations, and the difference seems to be partly explained by the co-occurrence of a related plant species, Geranium yezoense, in a population, as weevils preferred it to both color morphs of G. thunbergii, thereby relaxing overall herbivore pressure for G. thunbergii. Nonetheless, despite such high variability, the preference of weevils for white morphs over pink morphs of G. thunbergii was found across multiple populations. We discuss possible mechanisms causing the association between flower color and herbivore preference as well as its evolutionary consequences.