Perceptual narrowing typically occurs around 6 months of age, and drastically changes an infant’s perception of stimuli such as faces or spoken language according to the frequency with which the infant encounters them. It has already been well established that perceptual narrowing improves the sensitivity of infants to frequently encountered stimuli such as same-race faces and their native language while reducing their sensitivity to other-race faces and non-native languages. However, the effect of perceptual narrowing on the combined perception of face and language stimuli is not well understood. Therefore, to investigate the changes in the sensitivity of infants to matches and mismatches between faces and speech which might occur in the course of perceptual narrowing, we tested 3- and 9-month-old German infants using German faces and German spoken sentences which would be familiar to the infants, as well as completely unfamiliar Chinese faces and French spoken sentences. The infants were tested using an intermodal association paradigm, whereby each infant saw sequences of German or Chinese faces, interspersed with German or French spoken sentences. We analyzed the total looking time of infants in conditions where the faces and spoken sentences were congruent (either both familiar, or both unfamiliar), versus incongruent conditions where only the faces or only the sentences were familiar. We found that while the 9-month-olds looked for similar durations in congruent versus incongruent conditions, the 3-month-olds looked significantly longer during congruent conditions versus incongruent conditions, indicating a greater attentiveness to face-speech matches and mismatches prior to the onset of perceptual narrowing.