Understanding the waning of vaccine-induced protection is important for both immunology and public health. Population heterogeneities in underlying (pre-vaccination) susceptibility and vaccine response can cause measured vaccine effectiveness (mVE) to change over time, even in the absence of pathogen evolution and any actual waning of immune responses. We use multi-scale agent-based models parameterized using epidemiological and immunological data, to investigate the effect of these heterogeneities on mVE as measured by the hazard ratio. Based on our previous work, we consider the waning of antibodies according to a power law and link it to protection in two ways: (1) motivated by correlates of risk data and (2) using a within-host model of stochastic viral extinction. The effect of the heterogeneities is given by concise and understandable formulas, one of which is essentially a generalization of Fisher's fundamental theorem of natural selection to include higher derivatives. Heterogeneity in underlying susceptibility accelerates apparent waning, whereas heterogeneity in vaccine response slows down apparent waning. Our models suggest that heterogeneity in underlying susceptibility is likely to dominate. However, heterogeneity in vaccine response offsets <10% to >100% (median of 29%) of this effect in our simulations. Our study suggests heterogeneity is more likely to 'bias' mVE downwards towards the faster waning of immunity but a subtle bias in the opposite direction is also plausible.