While many in-laboratory ecotoxicological studies have shown the adverse impact of pollutants to the fitness of an individual, direct evidence from the field on the population dynamics of wildlife animals has been lacking. Here, we provide empirical support for a negative effect of pollution on Atlantic cod (Gadus morhua) population dynamics in coastal waters of Norway by combining unique time series of juvenile cod abundance, body size, environmental concentration of toxic contaminants and a spatially structured population dynamics model. The study shows that mercury concentration might have decreased the reproductive potential of cod in the region despite the general decline in the environmental concentration of mercury, cadmium and hexachlorobenzene since the implementation of national environmental laws. However, some cod populations appeared to be more resistant to mercury pollution than others, and the strength and shape of mercury effect on cod reproductive potential was fjord-specific. Additionally, cod growth rate changed at scales smaller than fjords with a gradient related to the exposure to the open ocean and offshore cod. These spatial differences in life-history traits emphasize the importance of local adaptation in shaping the dynamics of local wildlife populations. Finally, this study highlights the possibility to mitigate pollution effects on natural populations by reducing the overall pollution level, but also reveals that pollution reduction alone is not enough to rebuild local cod populations. Cod population recovery probably requires complementary efforts on fishing regulation and habitat restoration.