Prior studies showed that methamphetamine (METH) users had greater than normal age-related brain atrophy; whether having the apolipoprotein E (APOE)-ε4 allele may be a contributory factor has not been evaluated. We aimed to determine the independent and combined effects of chronic heavy METH use and having at least one copy of the APOE-ε4 allele (APOE-ε4+) on brain morphometry and cognition, especially in relation to aging. We compared brain morphometry and cognitive performance in 77 individuals with chronic heavy METH use (26 APOE-ε4+, 51 APOE-ε4-) and 226 Non-METH users (66 APOE-ε4+, 160 APOE-ε4-), using a 2 × 2 design (two-way analysis of co-variance). Vertex-wise cortical volumes, thickness and seven subcortical volumes, were automatically measured using FreeSurfer. Linear regression between regional brain measures, and cognitive scores that showed group differences were evaluated. Group differences in age-related decline in brain and cognitive measures were also explored. Regardless of APOE-ε4 genotype, METH users had lower Motor Z-scores (P = 0.005), thinner right lateral-orbitofrontal cortices (P < 0.001), smaller left pars-triangularis gyrus volumes (P = 0.004), but larger pallida, hippocampi and amygdalae (P = 0.004-0.006) than nonusers. Across groups, APOE-ε4+ METH users had the smallest volumes of superior frontal cortical gyri bilaterally, and of the smallest volume in left rostral-middle frontal gyri (all P-values <0.001). Smaller right superior-frontal gyrus predicted poorer motor function only in APOE-ε4+ participants (interaction-P < 0.001). Cortical volumes and thickness declined with age similarly across all participants; however, APOE-ε4-carriers showed thinner right inferior parietal cortices than noncarriers at younger age (interaction-P < 0.001). Chronic heavy use and having at least one copy of the APOE-ε4 allele may have synergistic effects on brain atrophy, particularly in frontal cortices, which may contribute to their poorer cognitive function. However, the enlarged subcortical volumes in METH users replicated prior studies, and are likely due to METH-mediated neuroinflammation.