Prenatal and childhood adverse events have been shown to be related to children’s cognitive and psychological development. However, the influence of early-life adversities on child brain morphology is not well understood, and most studies are based on small samples and often examine only one adversity. Thus, the goal of our study is to examine the relationship between cumulative exposures to prenatal and childhood adversities and brain morphology in a large population-based study. Participants included 2,993 children from the Generation R Study, a cohort of children growing up in Rotterdam, the Netherlands. Recruitment was initiated between 2002 and 2006, and the study is currently performing the 17- to 19-year follow-up wave. Prenatal adversities were reported by mothers at 20–25 weeks of pregnancy, and the child’s lifetime exposure to adversities was reported by mothers when the children were 10 years old. The total brain, gray and white matter volumes, and the volume of the cerebellum, amygdala, and hippocampus were assessed with magnetic resonance imaging when children were 10 years old. In total, 36% of children had mothers who were exposed to at least one adversity during pregnancy and 35% of children were exposed to adversities in childhood. In our study sample, the cumulative number of prenatal adversities was not related to any brain outcome. In contrast, per each additional childhood adverse event, the total brain volume was 0.07 standard deviations smaller (SE = 0.02, p = 0.001), with differences in both gray and white matter volumes. Childhood adversities were not related to the amygdala or hippocampal volumes. Additionally, the link between childhood events and the preadolescent brain was not modified by prenatal events and was not explained by maternal psychopathology. Our results suggest that childhood adversities, but not prenatal adverse events, are associated with smaller global brain volumes in preadolescence. Notably, this is the first large population-based study to prospectively assess the association between the cumulative number of prenatal adversities and the preadolescent brain morphology. The study findings extend the evidence from high-risk samples, providing support for a link between cumulative childhood adverse events and brain morphology in children from the general population.