Gliomas are the most common type of primary brain tumors and one of the highest causes of mortality worldwide. Accurate grading of gliomas is of immense importance to administer proper treatment plans. In this paper, we develop a comprehensive non-invasive multimodal magnetic resonance (MR)-based computer-aided diagnostic (CAD) system to precisely differentiate between different grades of gliomas (Grades: I, II, III, and IV). A total of 99 patients with gliomas (M = 49, F = 50, age range = 1–79 years) were included after providing their informed consent to participate in this study. The proposed imaging-based glioma grading (GG-CAD) system utilizes three different MR imaging modalities, namely; contrast-enhanced T1-MR, T2-MR known as fluid-attenuated inversion-recovery (FLAIR), and diffusion-weighted (DW-MR) to extract the following imaging features: (i) morphological features based on constructing the histogram of oriented gradients (HOG) and estimating the glioma volume, (ii) first and second orders textural features by constructing histogram, gray-level run length matrix (GLRLM), and gray-level co-occurrence matrix (GLCM), (iii) functional features by estimating voxel-wise apparent diffusion coefficients (ADC) and contrast-enhancement slope. These features are then integrated together and processed using a Gini impurity-based selection approach to find the optimal set of significant features. The reduced significant features are then fed to a multi-layer perceptron artificial neural networks (MLP-ANN) classification model to obtain the final diagnosis of a glioma tumor as Grade I, II, III, or IV. The GG-CAD system was evaluated on the enrolled 99 gliomas (Grade I = 13, Grade II = 22, Grade III = 22, and Grade IV = 42) using a leave-one-subject-out (LOSO) and k-fold stratified (with k = 5 and 10) cross-validation approach. The GG-CAD achieved 0.96 ± 0.02 quadratic-weighted Cohen’s kappa and 95.8% ± 1.9% overall diagnostic accuracy at LOSO and an outstanding diagnostic performance at k = 10 and 5. Alternative classifiers, including RFs and SVM produced inferior results compared to the proposed MLP-ANN GG-CAD system. These findings demonstrate the feasibility of the proposed CAD system as a novel tool to objectively characterize gliomas using the comprehensive extracted and selected imaging features. The developed GG-CAD system holds promise to be used as a non-invasive diagnostic tool for Precise Grading of Glioma.