Older adults manifest difficulties in speech perception, especially when speech is accompanied by noise or when speech is rapid. Several explanations have been suggested to account for age-related changes in speech perception, such as changes in hearing sensitivity or a more general decline in cognitive functioning. The purpose of the present study was to directly examine the relative contribution of hearing sensitivity and perceptual and cognitive factors in the understanding of age-related differences in speech perception under difficult conditions. Eighty-nine healthy participants with normal hearing thresholds, age 21-82 years, were tested for speech perception under four conditions: quiet, speech noise, white noise, and time-compressed speech at 60% compression rate. As all participants had age-normal hearing, absolute thresholds were tested for click trains, 1 kHz 15-ms duration pure tone, 1 kHz 50-ms duration pure tone, and 1.8 kHz 15-ms duration pure tone, which are relatively short and discriminative for hearing ability. Cognitive ability was examined using the Wechsler Adult Intelligence Scale (Third Edition) matrices and digit span. When words were presented against a quiet background or against white noise, speech perception was not significantly affected by aging, although in the latter case, increased thresholds predicted poorer speech perception. However, when words were presented against a background of speech noise or when speech was time-compressed at a 60% rate, age significantly predicted a decline in speech perception, even after controlling for hearing thresholds and cognitive functioning. Hearing threshold for short sounds is the major factor for predicting speech perception in background noise, across age, due to changes in hearing sensitivity or in temporal resolution. For the adult and aging population with preserved cognitive ability, cognitive functioning does not predict decline in speech perception.