Infants born after fetal growth restriction (FGR)-an obstetric condition defined as the failure to achieve the genetic growth potential-are prone to neurodevelopmental delays, with language being one of the major affected areas. Yet, while verbal comprehension and expressive language impairments have been observed in FGR infants, children and even adults, specific related impairments at birth, such as in the ability to encode the sounds of speech, necessary for language acquisition, remain to be disclosed. Here, we used the frequency-following response (FFR), a brain potential correlate of the neural phase locking to complex auditory stimuli, to explore the encoding of speech sounds in FGR neonates. Fifty-three neonates born with FGR and 48 controls born with weight adequate-for-gestational age (AGA) were recruited. The FFR was recorded to the consonant-vowel stimulus (/da/) during sleep and quantified as the spectral amplitude to the fundamental frequency of the syllable and its signal-to-noise ratio (SNR). The outcome was available in 45 AGA and 51 FGR neonates, yielding no differences for spectral amplitudes. However, SNR was strongly attenuated in the FGR group compared to the AGA group at the vowel region of the stimulus. These findings suggest that FGR population present a deficit in the neural pitch tracking of speech sounds already present at birth. Our results pave the way for future research on the potential clinical use of the FFR in this population, so that if confirmed, a disrupted FFR recorded at birth may help deriving FGR neonates at risk for postnatal follow-ups.