Antiepileptic drugs (AEDs) may affect mood and behavior in people with epilepsy and intellectual disability. A high AED load, derived from AED polytherapy and/or high doses of AEDs, has been suggested to be a risk factor for behavioral side effects. Data were drawn from Wave 3 of the Intellectual Disability Supplement to the Irish Longitudinal Study on Ageing (IDS-TILDA). The Behavior Problems Inventory Short Form (BPI-S) was used to assess challenging behaviors. AED load was calculated and median AED loads obtained. Non-parametric tests and binary logistic regression were performed to determine the relationship between AED load and challenging behaviors. Of participants with a reported diagnosis of epilepsy who were taking a regular AED and had completed BPI-S (n = 142), 62.7% (n = 89) exhibited challenging behaviors. Challenging behavior was found to be more prevalent in those with more severe levels of intellectual disability (p < 0.001). Aggressive/destructive behavior and stereotyped behavior were significantly more likely in participants living in residential/campus settings. For participants with a severe/profound intellectual disability, a significantly higher median AED load was found for participants exhibiting aggressive/destructive behavior and self-injurious behavior (SIB) compared to participants not exhibiting these behaviors, indicating a high AED load may contribute to some behavioral problems in this population group. However, many factors can influence behavioral outcomes, creating difficulties in determining those that are associated and the nature of the association. Careful monitoring of AED load, together with increased vigilance for breakthrough behavioral issues is essential for dealing with these complex cases. Larger studies are needed to account for the potential confounding factors.