Magnetic Resonance (MR) images of the brain play key role in exploiting pathological changes and non-invasive investigation of many neuro-degenerative diseases. Computer Aided Diagnosis (CAD) systems assist radiologists in interpreting MR images and classifying them into “normal” and “abnormal” categories. However, reduced strength of the used magnet in the machine or involuntary motions of the patients may lead to degraded MR images, which can negatively affect the performance of CAD system compromising the classification accuracy. This work aims at modeling these types of situations via out-of-focus blur, motion blur, effect of variation in resolution, and a combination of these on brain MR images for validating the impact of image quality on classification performance. To validate this, this article mathematically models the blurs (both individually and simultaneously) by varying the strength of image quality covariates and afterwards Deep Convolutional Neural Networks (DCNN) are employed to train and classify the distorted brain MR images. Besides, a single DCNN is experimented with a good mix of image quality and characteristics to test the reliability of the model for real-life scenario. The CNN models are validated through comprehensive evaluation on both original and degraded versions of brain MR images from two benchmark datasets DS-75 and DS-160 collected by Harvard Medical School as well as a self-collected dataset NITR-DHH. This study reveals that the models are able to classify distorted MR images and hence can be used for assisting the clinicians.