Phytoplankton biomass exhibits substantial year-to-year changes, and understanding these changes is crucial to fisheries management and projecting future climate. These annual changes result partly from low-frequency climate modes that also lead to variations in sea surface temperature (SST). Here we evaluate the contribution of small scales to annual fluctuations based on a global analysis of satellite observations of sea surface chlorophyll (SChl), an indicator of phytoplankton biomass, and of SST from 1999 to 2018. We disentangle the spatio-temporal scales of variability in the time series and find that besides the prominent seasonal cycle, SChl is dominated by high-frequency fluctuations (<three months) at small spatial scales (<50 km)—which accumulates over annual scales in contrast to SST. The results suggest that observations and models with high spatio-temporal resolutions are necessary to understand annual changes in SChl. The rapid response of SChl to small-scale physical processes, combined with intrinsic ecosystem interactions and air–sea interaction feedbacks, may explain the differences between annual variations in SST and SChl.