Children with autism spectrum disorder (ASD) display difficulties with response to joint attention in natural settings but often perform comparably to typically developing (TD) children in experimental studies of gaze following. Previous work comparing infants at elevated likelihood for ASD versus TD infants has manipulated aspects of the gaze cueing stimulus (e.g. eyes only versus head and eyes together), but the role the peripheral object being attended to is not known. In this study of infants at elevated likelihood of ASD (N = 97) and TD infants (N = 29), we manipulated whether or not a target object was present in the cued area. Performance was assessed at 10, 14, and 18 months, and diagnostic assessment was conducted at age 3 years. The results showed that although infants with later ASD followed gaze to the same extent as TD infants in all conditions, they displayed faster latencies back to the model’s face when (and only when) a peripheral object was absent. These subtle atypicalities in the gaze behaviors directly after gaze following may implicate a different appreciation of the communicative situation in infants with later ASD, despite their ostensively typical gaze following ability.Lay abstractDuring the first year of life, infants start to align their attention with that of other people. This ability is called joint attention and facilitates social learning and language development. Although children with autism spectrum disorder (ASD) are known to engage less in joint attention compared to other children, several experimental studies have shown that they follow other’s gaze (a requirement for visual joint attention) to the same extent as other children. In this study, infants’ eye movements were measured at age 10, 14, and 18 months while watching another person look in a certain direction. A target object was either present or absent in the direction of the other person’s gaze. Some of the infants were at elevated likelihood of ASD, due to having an older autistic sibling. At age 3 years, infants were assessed for a diagnosis of ASD. Results showed that infants who met diagnostic criteria at 3 years followed gaze to the same extent as other infants. However, they then looked back at the model faster than typically developing infants when no target object was present. When a target object was present, there was no difference between groups. These results may be in line with the view that directly after gaze following, infants with later ASD are less influenced by other people’s gaze when processing the common attentional focus. The study adds to our understanding of both the similarities and differences in looking behaviors between infants who later receive an ASD diagnosis and other infants.