It is well known that not every symmetry of a classical field theory is also a symmetry of its quantum version. When this occurs, we speak of quantum anomalies. The existence of anomalies imply that some classical Noether charges are no longer conserved in the quantum theory. In this paper, we discuss a new example for quantum electromagnetic fields propagating in the presence of gravity. We argue that the symmetry under electric-magnetic duality rotations of the source-free Maxwell action is anomalous in curved spacetimes. The classical Noether charge associated with these transformations accounts for the net circular polarization or the optical helicity of the electromagnetic field. Therefore, our results describe the way the spacetime curvature changes the helicity of photons and opens the possibility of extracting information from strong gravitational fields through the observation of the polarization of photons. We also argue that the physical consequences of this anomaly can be understood in terms of the asymmetric quantum creation of photons by the gravitational field.