The perception of sound is a complex process that is influenced by not only the physical characteristics of the sound, but also individual characteristics of people. This study aimed to determine whether noise sensitivity and environmental sensitivity have a significant effect on people's soundscape evaluations, including sound source identification, perceived affective quality, and overall quality. Sixty participants aged 19–36 years were exposed to audiovisual stimuli derived from 10 commonly encountered urban scenes and assessed the soundscape quality. The study revealed that noise sensitivity did not significantly affect the evaluation of the soundscapes, whereas environmental sensitivity had a significant impact. Specifically, the full scale of environmental sensitivity had a significant effect on soundscape appropriateness, and the aesthetic sensitivity (AES) subscale of environmental sensitivity had a significant effect on perceived natural sound dominance, soundscape pleasantness, and overall impressions. The physical sensitivity (PHS) subscale significantly affected soundscape pleasantness, overall impressions, and perceived loudness. Moreover, an interaction effect between site and environmental sensitivity was observed in the evaluation of soundscape pleasantness and overall impressions; in sites dominated by natural environments, individuals with higher environmental sensitivity tended to perceive higher levels of soundscape pleasantness. Conversely, in sites dominated by built environments, individuals with higher environmental sensitivity tended to perceive lower levels of soundscape pleasantness. Similar patterns were observed in the overall impression evaluations. These findings can help policymakers and urban planning practitioners to recognise the diverse needs of various people and highlight the need for targeted soundscape design based on user sensitivity.