The brain-derived neurotrophic factor (BDNF) Val66Met polymorphism emerged as a risk factor for Alzheimer's disease (AD). However, little was known about its effects on the process of potential AD. To explore the effects of the Val66Met polymorphism on cognition, cerebrospinal fluid (CSF), and neuroimaging markers in non-demented elderly individuals. A total of 1,081 adults without dementia (375 healthy subjects and 706 individuals with mild cognitive impairment) were recruited from the Alzheimer's Disease Neuroimaging Initiative (ADNI) to test the influence of BDNF Val66Met polymorphism on cognitive impairment, brain structure atrophy, and change in the levels of CSF biomarkers. Moreover, we also conducted our study in abnormal amyloid-β (A+) subgroup and normal amyloid-β (A-) subgroup, as well as in APOEɛ4 carriers and non-carriers. The BDNF Val66Met polymorphism had significant association with atrophy of the entorhinal cortex and Mini-Mental State Examination (MMSE) scores in the non-demented elderly and A + subgroup, while no association was found in A-subgroup. What is more, there was a significant effect of interaction between BDNF Val66Met and amyloid-β load in MMSE. In addition, significant associations of BDNF Val66Met with the entorhinal cortex and ventricular volumes were found among APOEɛ4 non-carriers, but not APOEɛ4 carriers. The BDNF Val66Met polymorphism is associated with cognitive impairment and brain atrophy among the non-demented elderly, APOEɛ4 non-carriers and A + subgroup, implying the potential of the Val66Met polymorphism as an important genetic factor for AD-related neurodegeneration.