Abstract Background We proposed a novel classification system to distinguish among elderly subjects with Alzheimer's disease (AD), mild cognitive impairment (MCI), and normal controls (NC), based on 3D magnetic resonance imaging (MRI) scanning. Methods The method employed 3D data of 178 subjects consisting of 97 NCs, 57 MCIs, and 24 ADs. First, all these 3D MR images were preprocessed with atlas-registered normalization to form an averaged volumetric image. Then, 3D discrete wavelet transform (3D-DWT) was used to extract wavelet coefficients the volumetric image. The triplets (energy, variance, and Shannon entropy) of all subbands coefficients of 3D-DWT were obtained as feature vector. Afterwards, principle component analysis (PCA) was applied for feature reduction. On the basic of the reduced features, we proposed nine classification methods: three individual classifiers as linear SVM, kernel SVM, and kernel SVM trained by PSO with time-varying acceleration-coefficient (PSOTVAC), with three multiclass methods as Winner-Takes-All (WTA), Max-Wins-Voting, and Directed Acyclic Graph. Results The 5-fold cross validation results showed that the “WTA-KSVM + PSOTVAC” performed best over the OASIS benchmark dataset, with overall accuracy of 81.5% among all proposed nine classifiers. Moreover, the method “WTA-KSVM + PSOTVAC” exceeded significantly existing state-of-the-art methods (accuracies of which were less than or equal to 74.0%). Conclusion We validate the effectiveness of 3D-DWT. The proposed approach has the potential to assist in early diagnosis of ADs and MCIs.