(1) Background: There is an urgent need for effective treatments for cocaine use disorder (CUD), and new pharmacological approaches targeting epigenetic mechanisms appear to be promising options for the treatment of this disease. Dopamine Transporter (DAT) transgenic rats recently have been proposed as a new animal model for studying susceptibility to CUD. (2) Methods: DAT transgenic rats were treated chronically with cocaine (10 mg/kg) for 8 days, and the expression of epigenetic modulators, Lysine Demethylase 6B (KDM6B) and Bromodomain-containing protein 4 (BRD4), was examined in the prefrontal cortex (PFC). (3) Results: We show that only full knockout (KO) of DAT impacts basal levels of KDM6B in females. Additionally, cocaine altered the expression of both epigenetic markers in a sex- and genotype-dependent manner. In response to chronic cocaine, KDM6B expression was decreased in male rats with partial DAT mutation (HET), while no changes were observed in wild-type (WT) or KO rats. Indeed, while HET male rats have reduced KDM6B and BRD4 expression, HET female rats showed increased KDM6B and BRD4 expression levels, highlighting the impact of sex on epigenetic mechanisms in response to cocaine. Finally, both male and female KO rats showed increased expression of BRD4, but only KO females exhibited significantly increased KDM6B expression in response to cocaine. Additionally, the magnitude of these effects was bigger in females when compared to males for both epigenetic enzymes. (4) Conclusions: This preliminary study provides additional support that targeting KDM6B and/or BRD4 may potentially be therapeutic in treating addiction-related behaviors in a sex-dependent manner.