Studies in the field of emotions have yielded mixed findings regarding differences between women’s and men’s emotional reactivity. In the majority, emotional scenes, facial expressions, and movies were used as stimuli. However, music has been less frequently used, despite its capacity to evoke strong emotional responses in the listeners. In this study, we aimed to explore the role of gender in emotional reactivity to music. A sample of 110 healthy participants (60 women) listened to 42 excerpts (14 pleasant, 14 neutral, 14 unpleasant) from the Film Music Stimulus Set (FMSS) for 8 s, while their autonomic reactivity and facial expression were continuously recorded. Participants then rated each excerpt on affective dimensions (hedonic valence, tension arousal, and energy arousal), discrete emotions (happiness, anger, fear, tenderness, and sadness), musical preference, and familiarity. Some differences were found between women’s and men’s reactivity to the stimuli, such that women showed a greater deceleration of heart rate while listening to both emotional (pleasant and unpleasant) and neutral music, and rated unpleasant excerpts as less preferred. Women also scored higher on trait anxiety compared to men. These results suggest that women may have heightened sensitivity to emotional stimuli, particularly unpleasant music, which could have implications for better understanding clinical pathologies that vary in prevalence based on gender.