The brain computer interface (BCI) aimed to offer an improved and quality life for people having disabilities. Various physiological sensors are utilized for designing the BCI application. Here, the electroencephalogram (EEG) is well-known for modeling the brain signals. However, the existing techniques based on EEG signal classification are computationally expensive and not so accurate. This paper devises Competitive Swarm Dragonfly Algorithm (CSDA) for classifying the EEG signals. In this model, the input EEG signal artifacts are discarded in pre-processing phase. The feature extraction is done to extract imperative features that include spectral-based features, like Amplitude modulation spectrogram, frequency-based features, like spectral flux, tonal power ratio, spectral centroid, spectral spread, Power Spectrum Density,logarithmic band power, and statistical features like kurtosis, entropy and skewness. Here, data augmentation is performed for making the data suitable for further processing. Deep Residual Network (DRN) is used to classify the motor imagery EEG signal. The suggested CSDA is used to train DRN, which is obtained by combining the competitive Swarm Optimizer and Dragonfly Algorithm. The performance of the adapted approach is determined using motor imagery multi-class dataset and motor imagery small training sets, in which the motor imagery multi-class dataset offers the highest specificity, accuracy, and sensitivity of 91.9% 91.6%, and 92.3%, respectively.