Quantify the impact of meteorological changes on air pollution levels is the aim of numerous recent studies. However, there is still a lack of investigations assessing the influence of land use/activities on the relationship between climate and air quality. In this study, we used a two-stage design to estimate the influence of land use types and activities on the association between weather changes and air pollution (PM2.5, NO2, SO2, O3) over 5572 municipalities in Brazil. To calculate the influence of recent weather change on air pollution concentration for each municipality, we used the “weather penalty” concept. This approach considers differences in linear trend coefficients between two generalized additive models. Then, using quantile regression, we estimated the effect of land use types and activities (8 variables related to transportation, energy generation, and land use) on weather-related increases in ambient air pollution. We found that an increase in PM2.5 was associated to recent weather changes in most municipalities (average increase of 0.07μg/m3per year) and a decrease in NO2 in most municipalities (average decrease of 0.0003 ppb per year). O3 and SO2 had more intense increases associated with weather changes in the North region. Our findings suggest the most robust positive associations between weather penalties on PM2.5 and areas with non-clean energy and oil refineries (average increase of 0.006μg/m3per year and 0.04μg/m3per year, respectively). We also found positive associations between Pasture areas, urban areas, and transportation and the weather penalties of this pollutant. In contrast, forest areas were negatively associated with PM2.5 penalties. We also found that oil refineries, urban areas, and transportation significantly positively influenced weather penalties for SO2 and O3. Overall, the study highlights the importance of considering the influence of land use types and activities on weather-related changes in ambient air pollution.