The perception of dynamic – constantly changing – pitch in speech has been extensively studied in psychoacoustics and linguistics. In psychoacoustic studies, listeners are usually presented with short stimuli such as vowels or syllables, and their ability to discriminate a pair of stimuli is assessed. On the other hand, linguistic studies concern intonation over an utterance. Intonation entails not only acoustic prominence realised by pitch, duration, and loudness, but also listeners’ knowledge about the relative prominence between syllables or between words in their language. This paper discusses a series of speech perception experiments using both psychoacoustic and linguistic tasks. Participants judged either relative pitch height or prominence between two pitch peaks or valleys in an utterance. Native English speakers in different age and dialectal groups were tested. The results showed that, first, listeners’ pitch height discrimination in the utterance context seems to be more accurate than previously reported. Second, there is a robust perceptual asymmetry between pitch peaks and valleys, the valleys posing significant challenges in perception. Third, listeners’ perception of pitch height and prominence is disassociated. The findings taken together suggest an intricate interaction between the physical properties of the stimuli and listeners’ top-down knowledge in the perception of speech intonation.