Everyday experiences often overlap, challenging our ability to maintain distinct episodic memories. One way to resolve such interference is by exaggerating subtle differences between remembered events, a phenomenon known as memory repulsion. Here, we tested whether repulsion is influenced by emotional arousal, when resolving memory interference is perhaps most needed. We adapted an existing paradigm in which participants repeatedly studied object-face associations. Participants studied two different-colored versions of each object: a to-be-tested "target" and its not-to-be-tested "competitor" pair mate. The level of interference between target and competitor pair mates was manipulated by making the object colors either highly similar or less similar, depending on the participant group. To manipulate arousal, the competitor object-face associations were preceded by either a neutral tone or an aversive and arousing burst of white noise. Memory distortion for the color of the target objects was tested after each study round to examine whether memory distortions emerge after learning. We found that participants with greater sound-induced pupil dilations, an index of physiological arousal, showed greater memory attraction of target colors towards highly similar competitor colors. Greater memory attraction was also correlated with greater memory interference in the last round of learning. Additionally, individuals who self-reported higher trait anxiety showed greater memory attraction when one of the overlapping memories was associated with something aversive. Our findings suggest that memories of similar neutral and arousing events may blur together after repeated exposures, especially in individuals who show higher arousal responses and symptoms of anxiety.