The apolipoprotein E (APOE) gene ɛ4 allele is a risk factor for Alzheimer's disease and cardiovascular disease. However, its relationship with cognition and brain volume after stroke is not clear. We compared cognition and medial temporal lobe volumes in APOEɛ4 carriers and non-carriers in the first year after ischemic stroke. We sampled 20 APOEɛ4 carriers and 20 non-carriers from a larger cohort of 135 ischemic stroke participants in the longitudinal CANVAS study. Participants were matched on a range of demographic and stroke characteristics. We used linear mixed-effect models to compare cognitive domain z-scores (attention, processing speed, executive function, verbal and visual memory, language, visuospatial function) and regional medial temporal lobe volumes (hippocampal, entorhinal cortex) between groups at each time-point (3, 12-months post-stroke), and within groups across time-points. APOE gene single nucleotide polymorphisms (SNPs; rs7412, rs429358) were genotyped on venous blood. APOEɛ4 carriers and non-carriers did not differ on any demographic, clinical, or stroke variable. Carriers performed worse than non-carriers in verbal memory at 3 months post-stroke (p = 0.046), but were better in executive function at 12 months (p = 0.035). Carriers demonstrated a significant improvement in verbal memory (p = 0.012) and executive function (p = 0.015) between time-points. Non-carriers demonstrated a significant improvement in visual memory (p = 0.0005). Carriers had smaller bilateral entorhinal cortex volumes (p < 0.05), and larger right sided and contralesional hippocampal volumes, at both time-points (p < 0.05). APOE ɛ4 is associated with delayed recovery of verbal memory function and reduced entorhinal cortex volumes in the first year after ischemic stroke.