During the COVID-19 pandemic, the Veterans Health Administration (VHA) expanded telehealth infrastructure. Understanding telehealth initiation and sustained engagement could inform future resource allocation for high-need populations. To describe and examine primary care use, including initiation, use, and engagement factors, of telehealth modalities (telephone, video visits, and secure messaging) from 2020 to 2022. This cohort study was conducted among 1 383 070 patients in the 75th or higher percentile for 90-day risk of hospitalization or mortality (using previously validated Care Assessment Need scores) engaged in VHA primary care from March 11, 2019, to March 10, 2022. Patient sociodemographic characteristics (age, sex, race and ethnicity, and marital and housing status), health characteristics (chronic condition count, military service disability, serious mental illness, or substance use disorder diagnoses), geographic characteristics (driving distance to clinic and rural or urban location), and Federal Communications Commission-reported broadband speed among subgroups of patients at high risk categorized by telehealth use from 2020 to 2022. Primary care utilization by modality. A total of 1 383 070 patients at high risk were engaged in VHA primary care in March 2020 (median age, 73.0 years [IQR, 65-80 years]; 92.4% male; 77.7% regular telehealth users in 2019). With the onset of the COVID-19 pandemic from March 2020 to March 2021, 92.7% of patients at high risk (1 158 804 of 1 250 438 retained in care) became regular telehealth users. The following year, most patients continued as telehealth users (83.4% [942 151 of 1 129 683 retained]), including 38.2% retention of users at high risk newly engaged in 2020. Between 2019 and 2022 among those living and engaged in VHA primary care, adjusted exploratory multinomial logit models estimated that new telehealth users in 2020 (both sustained or only transiently engaged) were more often Black non-Hispanic individuals with greater comorbidity burdens than those who never engaged in telehealth use (Black non-Hispanic with new persistent telehealth use: adjusted relative risk ratio [ARR], 1.18 [95% CI, 1.16-1.20]; Black non-Hispanic with transient telehealth use: ARR, 1.11 [95% CI, 1.08-1.13]; ≥5 chronic conditions with new persistent telehealth use: ARR, 1.92 [95% CI, 1.88-1.96]; ≥5 chronic conditions with transient telehealth use: ARR, 1.43 [95% CI, 1.40-1.46]). This cohort study suggests that primary care telehealth initiation, use and sustained engagement differed among subgroups of patients at high risk throughout the COVID-19 pandemic. Those never or only transiently engaged with telehealth had lower illness burdens and were less likely to identify as members of racial or ethnic minority groups. Variation in telehealth use among subgroups of patients at high risk during this period could inform future resource allocation.