Empathy is defined as the capacity to resonate with others' emotions and can be subdivided into affective and cognitive components. Few studies have focused on the role of perspective-taking within this ability. Utilizing the novel Bochumer Affective and Cognitive Empathy Task (BACET), the present study aims to determine the characteristics of specific empathy components, as well as the impact of offender vs. victim perspective-taking. A total of 21 male participants (mean age = 30.6) underwent functional magnetic resonance imaging (fMRI) while watching 60 videos showing two protagonists in neutral (n = 30) or violent interactions (n = 30) thereby adopting the perspective of the (later) offender or victim. Our data show that videos showing emotional (violent) content, compared to those with neutral content, were rated more emotionally negative and induced higher affective empathic involvement, particularly when adopting the victim's perspective compared to the offender's point of view. The correct assignment of people's appropriate emotion (cognitive empathy) was found to be more accurate and faster in the emotional condition relative to the neutral one. However, no significant differences in cognitive empathy performance were observed when comparing victim vs offender conditions. On a neural level, affective empathy processing, during emotional compared to neutral videos, was related to brain areas generally involved in social information processing, particularly in occipital, parietal, insular, and frontal regions. Cognitive aspects of empathy, relative to factual reasoning questions, were located in inferior occipital areas, fusiform gyrus, temporal pole, and frontal cortex. Neural differences were found depending on the perspective, i.e., empathizing with the victim, compared to the offender, during affective empathy activated parts of the right temporal lobe, whereas empathy towards the role of the offender revealed stronger activation in the right lingual gyrus. During cognitive empathy, empathy toward the victim, relative to the offender, enhanced activity of the right supramarginal and left precentral gyri. The opposite contrast did not show any significant differences. We conclude that the BACET can be a useful tool for further studying behavioral and neurobiological underpinnings of affective and cognitive empathy, especially in forensic populations since response patterns point to a significant impact of the observer's perspective.