The question of how to measure exposure to different types of content on social media grows in importance with increased use of these platforms. Social media further complicate this task by bringing diverse content into the same space, raising the question of whether selective exposure or incidental exposure theories best explain attention patterns. We contribute to this debate in two ways. First, we test how well visual attention aligns with expressed content preferences to understand attention online. Second, we compare visual attention to diverse social media content to two types of self-reported measures of recalled attention to content – close-ended versus open-ended – to examine how best to measure attention. Using eye tracking, we demonstrate that visual attention to social, news, and political posts is not associated with interest in those topics, suggesting attention to content seen incidentally on social media is quite high. Second, we find that visual attention to social and political (but not news) posts relates to close-ended self-reported measures of recalled attention, but visual attention is associated with open-ended recalled attention only for political posts. We propose that researchers need to go beyond measures of exposure and carefully consider how best to measure attention to social media content.