Quarantining Coronavirus (COVID-19) patients is a critical measure to prevent the spread of the disease, thus it is important to perform the correct diagnosis quickly. Among other diagnosis tools, radiograph scans exist as one of the common screening methods for detecting COVID-19. This study aims to reduce the healthcare workers’ burden by mitigating the impacts of COVID-19 pneumonia by developing an image classification system to assess radiographs in an accurate and timely manner. The proposed work also assists junior radiologists in familiarizing themselves with reading radiographs confidently using localization features through Explainable Artificial Intelligence. This project focuses on the detection of COVID-19 pneumonia in a multi-classification environment by pre-trained convolutional neural network models such as Inception-V3, VGG-16, and VGG-19. The overall system’s idea is to first perform transfer learning based on the binary classification of Normal and Pneumonia. The knowledge is then transferred to the task-specific models to classify COVID-19, bacterial, viral pneumonia, and normal. Out of all the models, VGG-19, the multiclass CNN model outperformed others by achieving a 99.27% accuracy in detecting COVID-19 in X-rays. The model also has the highest overall accuracy of 85.59% compared to multiclass Inception-V3, VGG-16, VGG-19, and the ensemble model. It was then chosen to be deployed as the Web Application where users are able to pass an input of radiograph to get an output with the label classifying whether the patient is infected with COVID-19 pneumonia, viral pneumonia, bacterial pneumonia, or normal together with highlights on the infected areas of lungs.