The emotional impacts of the COVID-19 pandemic and resulting public health emergency are only beginning to be understood. We assessed the contributions of emotional and cognitive factors and age-related comorbidities to greater COVID-19 fear in a community dwelling sample of 142 younger (Mage = 19.63, SDage = 2.59) and 157 older (Mage = 72.01, SDage = 7.06) adults, between July 2020 and July 2021. We hypothesized that individuals with increased loneliness, depression, and/or decreased subjective numeracy (SN) and interpersonal trust would experience more COVID-19 fear. We also predicted that females and older adults would experience more COVID-19 fear given that age-related comorbidities are associated with increased illness severity. Results showed that the extent of loneliness in older adults was more strongly related to fear of COVID-19 than it was in younger adults (β = 0.197, p = 0.016), and poorer SN was associated with increased COVID-19 fear in both age groups (β = -0.138, p = 0.016). Further, higher interpersonal mistrust was associated with increased COVID-19 fear (β = 0.136, p = 0.039), as was identifying as female (β = 0.137, p = 0.013). Given that self-described poor numeracy was a marker for greater COVID-19 fear, investigators and policy makers might consider mitigation opportunities addressing data literacy requirements imposed by the media. Further, outreach to mitigate loneliness, particularly of the elderly, might effectively lessen the negative psychological impact of this ongoing public health crisis.