The COVID-19 pandemic has had a dramatic impact on practicing physicians, with effects in clinical practice, academic pursuits, research endeavors, and personal lives. Women in medicine have been uniquely impacted. We examined the impact of the pandemic on the careers of pediatric cardiologists in the Northeast with an anonymous online survey. Participants reported demographic data, information on work hours, administrative burden, career satisfaction, academic productivity, and burnout. We approached 490 cardiologists and received 127 completed surveys (response rate 26%; 49% female). Among all respondents, 72% reported increased burnout, 43% reported decreased career satisfaction, and 57% reported decreased academic productivity. In multivariable ordinal regression analysis, when compared to male physicians, females were 2.4 times more likely to report decreased overall career satisfaction (p = 0.027), 2.6 times more likely to report decreased academic productivity (p = 0.028), and 2.6 times more likely to report increased feelings of burnout “to a large degree” (p = 0.022). Among all respondents, decreased career satisfaction was independently associated with increased household responsibility (OR = 4.4, p = 0.001). Increased administrative burden was independently associated with decreased academic productivity (OR = 2.6, p = 0.038). Open-ended responses highlighted loss of community due to remote work and blurring of the boundaries between work and home. Conversely, respondents appreciated flexibility to work remotely. In conclusion, the majority of pediatric cardiologists in the Northeast experienced negative career impacts due to the COVID-19 pandemic. Important gender differences emerged, with female physicians disproportionately reporting increased burnout, decreased career satisfaction, and decreased academic productivity.Supplementary InformationThe online version contains supplementary material available at 10.1007/s00246-022-02934-9.