The role of apolipoprotein E4 (APOE4) in the risk of Alzheimer's disease (AD) and Lewy body disease (LBD), and their relationship with β-amyloid deposition and cognitive dysfunction, remain unclear. Using amyloid and dopamine transporter imaging, we enrolled 126 controls and 208 patients with typical AD (pure AD and Lewy body variant of AD), AD with dementia with Lewy bodies, or typical LBD (dementia with Lewy bodies with amyloid deposition and pure LBD). APOE4 was associated with an increased risk of all disease subtypes except pure LBD. APOE4 was associated with increased frontal β-amyloid burden, and typical LBD was associated with increased occipital β-amyloid levels through its interaction with APOE4. APOE4 was associated with deteriorated general cognition and memory dysfunction via its interaction with typical LBD and AD, respectively. In conclusion, the impact of APOE4 on disease risk depends on its effects on β-amyloid deposition, and APOE4 is associated with β-amyloid deposition regardless of the clinical diagnosis. However, it interacts with typical LBD to cause occipital β-amyloid deposition.