Coronavirus (COVID-19) is an epidemic that is rapidly spreading and causing a severe healthcare crisis resulting in more than 40 million confirmed cases across the globe. There are many intensive studies on AI-based technique, which is time consuming and troublesome by considering heavyweight models in terms of more training parameters and memory cost, which leads to higher time complexity. To improve diagnosis, this paper is aimed to design and establish a unique lightweight deep learning-based approach to perform multi-class classification (normal, COVID-19, and pneumonia) and binary class classification (normal and COVID-19) on X-ray radiographs of chest. This proposed CNN scheme includes the combination of three CBR blocks (convolutional batch normalization ReLu) with learnable parameters and one global average pooling (GP) layer and fully connected layer. The overall accuracy of the proposed model achieved 98.33% and finally compared with the pre-trained transfer learning model (DenseNet-121, ResNet-101, VGG-19, and XceptionNet) and recent state-of-the-art model. For validation of the proposed model, several parameters are considered such as learning rate, batch size, number of epochs, and different optimizers. Apart from this, several other performance measures like tenfold cross-validation, confusion matrix, evaluation metrics, sarea under the receiver operating characteristics, kappa score and Mathew’s correlation, and Grad-CAM heat map have been used to assess the efficacy of the proposed model. The outcome of this proposed model is more robust, and it may be useful for radiologists for faster diagnostics of COVID-19.