Reductions in the case fatality rate of COVID-19 in the unvaccinated have been credited to improvements in medical care. Here I test whether either of these factors predicts reductions in the case fatality rate, and whether observed reductions are better explicable by improved ascertainment of mild cases. Using weighted log-log regression, I compute the association between changes in the case fatality rate and test density between 3 July 2020 and 5 January 2021 in 162 countries; and check whether case fatality rate change is associated with either per capita medical spending (proxy for critical care access) or timing of the pandemic (proxy for COVID-specific knowledge). The median test density increased from 175 tests per thousand population to 1200, while the median case fatality rate dropped from 4.1% to 2.0%. While the case fatality rate was higher at both timepoints in Europe/North America than Africa / Asia, its association with test density was similar across countries. For each doubling in test density, the mean case fatality rate decreased by 18% (P<0.0001) with a median (interquartile rate) country-level decline of 20% (5-30) per doubling of test density. The rate of change of the case fatality rate was not associated with either medical care access or COVID-specific knowledge (all P>0.10). Declines in the case fatality rate were adequately explained by improved testing, with no effect of either medical knowledge or improvements in care. The true lethality of COVID-19 may not have changed much at the population level. Prevention should remain a priority.