Recently, the air quality in urban areas has declined because of increasing traffic emissions. This paper aimed to determine the toxicity from exposure to pollutants among three sensitive groups of residents in urban areas. Moreover, this study also estimated the impacts of landscape and meteorological conditions on the accumulation of air pollutants in these areas. The results showed that the annual average concentration in the town exceeded the WHO air quality guidelines. Other areas that had a high traffic density also presented unacceptable levels according to the hazard quotient (HQ value ≥ 1). It was found that the air quality in the town had declined. This study also found that people living in a tropical monsoon climate should avoid exposure to air pollution in both the summer and rainy seasons; even though the pollutant concentration is lower in the rainy season, the longer exposure time causes unacceptable health risks. Humidity showed a strong impact on gas pollutant reduction (rs = −0.943). The pollutants tended to increase in areas with a high density of main roads. Additionally, building density affected the accumulation of pollutants in near-source areas and blocked pollutants in receptor areas. Therefore, this study suggests that local authorities should provide vegetation infrastructure for a sustainable air quality improvement in urban areas.