We investigated affective processing in aphantasia (= absent or reduced vividness of mental imagery), considering a possible overlap with alexithymia (= deficits in identifying and describing emotions), as reduced vividness of mental imagery is also reported in alexithymia. Study 1 assessed physiological reactions and self-reported sympathy in n = 30 individuals with aphantasia and n = 75 controls when confronted to visual and verbal material showing people in distress. Results demonstrated that individuals with aphantasia show reduced emotional responses, especially to verbal stimuli. This is of particular importance given the higher prevalence of alexithymic symptoms in aphantasic participants, notably in externally-oriented thinking and difficulties in describing feelings. An additional mediation analysis confirmed that vividness of visual imagery mediated the association between alexithymia and self-reported sympathy. Study 2 extended our exploration to the recognition of emotions in others using the same sample. Despite accurate recognition of emotions, individuals with aphantasia exhibited significantly slower response times, suggesting less efficient strategies that do not involve mental imagery. Our findings highlight the crucial role of mental imagery in the interplay of cognitive functions and affective processes, demonstrating how conditions such as aphantasia and alexithymia can affect sympathy and, more generally, emotions.