Altered affective state recognition is assumed to be a root cause of aggressive behavior, a hallmark of psychopathologies such as psychopathy and antisocial personality disorder. However, the two most influential models make markedly different predictions regarding the underlying mechanism. According to the integrated emotion system theory (IES), aggression reflects impaired processing of social distress cues such as fearful faces. In contrast, the hostile attribution bias (HAB) model explains aggression with a bias to interpret ambiguous expressions as angry. In a set of four experiments, we measured processing of fearful and angry facial expressions (compared to neutral and other expressions) in a sample of 65 male imprisoned violent offenders rated using the Hare Psychopathy Checklist-Revised (PCL-R, Hare, R. D. (1991). The psychopathy checklist-revised. Toronto, ON: Multi-Health Systems) and in 60 age-matched control participants. There was no evidence for a fear deficit in violent offenders or for an association of psychopathy or aggression with impaired processing of fearful faces. Similarly, there was no evidence for a perceptual bias for angry faces linked to psychopathy or aggression. However, using highly ambiguous stimuli and requiring explicit labeling of emotions, violent offenders showed a categorization bias for anger and this anger bias correlated with self-reported trait aggression (but not with psychopathy). These results add to a growing literature casting doubt on the notion that fear processing is impaired in aggressive individuals and in psychopathy and provide support for the idea that aggression is related to a hostile attribution bias that emerges from later cognitive, post-perceptual processing stages.