Lifelong continuity of care is essential for patients with congenital heart disease (CHD) to maximize health outcomes; unfortunately, gaps in care (GIC) are common. Trends in GIC and of social determinants of health factors contributing to GIC are poorly understood. This retrospective cohort study included patients with CHD, aged 0 to 34 years, who underwent surgery between January 2003 and May 2020, followed up at a pediatric subspeciality hospital. Patients were categorized as having simple, moderate, and complex CHD based on 2018 American Heart Association and American College of Cardiology guidelines. Social determinants of health, such as race, ethnicity, language, insurance status, and Child Opportunity Index, based on home address zip code, were analyzed. Of 2012 patients with CHD, a GIC of ≥3 years was identified in 56% (n=1119). The proportion of patients with GIC per year increased by 0.51% (P<0.001). Multivariable longitudinal models showed that the odds of GIC were higher for patients who were ≥10.5 years old, had simple CHD, lived out of state, lived farther from care site, received public insurance, had less protection with additional insurance plans, and with low Child Opportunity Index. A separate model for patients with only moderate/complex CHD showed similar findings. Race and ethnicity were not associated with the odds of experiencing GIC over time. GIC have increased over time for patients with CHD. Social determinants of health, like insurance, access, and neighborhood opportunity, are key risk factors for increasing GIC. Efforts to reduce GIC in patients with CHD should focus on addressing the impact of specific social determinants of health.