Past research shows that emotion affects beauty judgments of images and music. Because it is widely supposed that our faculty of empathy facilitates aesthetic experience, we wondered whether individual levels of empathy modulate the effect of emotion on beauty. 164 participants rated the perceived beauty, happiness, and sadness of 12 art images, 12 nature photographs, and 24 songs. The stimuli were presented in two blocks, and participants took the PANAS mood questionnaire before and after each block. Between blocks, they viewed one of three mood induction videos, intended to increase their happiness, increase their sadness, or leave their mood unchanged. We also measured (trait) empathy with the Questionnaire for Cognitive and Affective Empathy. We used structural equation modeling to analyze the effect of empathy on emotion, beauty, and the relationship between them. We assessed four emotion variables: participants' felt happiness and sadness (mood questionnaire ratings) and perceived happiness and sadness (stimulus ratings). We find that higher empathy is associated with stronger positive relationships between beauty and both felt and perceived emotions, for both images and music (β ~ 0.06 per empathy point on a 10-pt. scale, p < 0.001). We also find that perceived happiness and sadness boost beauty directly for both images and music. However, sadness affects music more than images (β = 0.51 vs. 0.12, all p < 0.001), and empathy amplifies this relationship for music but not images. Thus, felt and perceived emotions produce more beauty, more so in more empathic people, and more so with music than images.