The clinical course of coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), is largely determined by host factors, with a wide range of outcomes. Despite an extensive vaccination campaign and high rates of infection worldwide, the pandemic persists, adapting to overcome antiviral immunity acquired through prior exposure. The source of many such major adaptations is variants of concern (VOCs), novel SARS-CoV-2 variants produced by extraordinary evolutionary leaps whose origins remain mostly unknown. In this study, we tested the influence of factors on the evolutionary course of SARS-CoV-2. Electronic health records of individuals infected with SARS-CoV-2 were paired to viral whole-genome sequences to assess the effects of host clinical parameters and immunity on the intra-host evolution of SARS-CoV-2. We found slight, albeit significant, differences in SARS-CoV-2 intra-host diversity, which depended on host parameters such as vaccination status and smoking. Only one viral genome had significant alterations as a result of host parameters; it was found in an immunocompromised, chronically infected woman in her 70s. We highlight the unusual viral genome obtained from this woman, which had an accelerated mutational rate and an excess of rare mutations, including near-complete truncating of the accessory protein ORF3a. Our findings suggest that the evolutionary capacity of SARS-CoV-2 during acute infection is limited and mostly unaffected by host characteristics. Significant viral evolution is seemingly exclusive to a small subset of COVID-19 cases, which typically prolong infections in immunocompromised patients. In these rare cases, SARS-CoV-2 genomes accumulate many impactful and potentially adaptive mutations; however, the transmissibility of such viruses remains unclear.