Altered structural brain development has been identified in fetuses with congenital heart disease (CHD), suggesting that the neurodevelopmental impairment observed later in life might originate in utero. There are many interacting factors that may perturb neurodevelopment during the fetal period and manifest as structural brain alterations, such as altered cerebral substrate delivery and aberrant fetal hemodynamics. We extracted structural covariance networks from the log Jacobian determinants of 435 in utero T2 weighted image magnetic resonance imaging scans, (n=67 controls, 368 with CHD) acquired during the third trimester. We fit general linear models to test whether age, sex, expected cerebral substrate delivery, and CHD diagnosis were significant predictors of structural covariance. We identified significant effects of age, sex, cerebral substrate delivery, and specific CHD diagnosis across a variety of structural covariance networks, including primary motor and sensory cortices, cerebellar regions, frontal cortex, extra-axial cerebrospinal fluid, thalamus, brainstem, and insula, consistent with widespread coordinated aberrant maturation of specific brain regions over the third trimester. Structural covariance networks offer a sensitive, data-driven approach to explore whole-brain structural changes without anatomical priors. We used them to stratify a heterogenous patient cohort with CHD, highlighting similarities and differences between diagnoses during fetal neurodevelopment. Although there was a clear effect of abnormal fetal hemodynamics on structural brain maturation, our results suggest that this alone does not explain all the variation in brain development between individuals with CHD.