ABSTRACTReadily available imaging technologies have made it possible to acquire multiple imaging modalities with complementary information for the same patient. These imaging modalities describe different properties about the organ of interest, providing an opportunity for better diagnosis, staging and treatment assessments. However, existing research in combining multi-modality imaging data has not been transformed into a clinical decision support system due to lack of flexibility, accuracy, and interpretability. This article proposes a multi-modality imaging-based diagnostic decision support system (MMI-DDS) that overcomes limitations of existing research. MMI-DDS includes three inter-connected components: (1) a modality-wise principal component analysis (PCA) that reduces data dimensionality and eliminates the need for co-registration of multi-modality images; (2) a novel constrained particle swarm optimization (cPSO) classifier that is built upon the joint set of the principal components (PCs) from all of the imaging modalities; (3) a clinical utility engine that employs inverse operations to identify contributing imaging features (a.k.a. biomarkers) in diagnosing the disease. To validate MMI-DDS, we apply it to a migraine dataset with multi-modality structural and functional magnetic resonance imaging (MRI) data. MMI-DDS shows significantly improved diagnostic accuracy than using single imaging modalities alone and also identifies biomarkers that are consistent with findings in migraine literature.