Toward understanding COVID-19 pneumonia: a deep-learning-based approach for severity analysis and monitoring the disease

Mohammadreza Zandehshahvar,Yashar Kiarashi,Hossein Maleki,Carlo N De Cecco,Ali Adibi,Marly Van Assen

doi:10.1038/s41598-021-90411-3

Mohammadreza Zandehshahvar, Yashar Kiarashi + Show 4 more

Open Access

https://doi.org/10.1038/s41598-021-90411-3

Copy DOI

Abstract

We report a new approach using artificial intelligence (AI) to study and classify the severity of COVID-19 using 1208 chest X-rays (CXRs) of 396 COVID-19 patients obtained through the course of the disease at Emory Healthcare affiliated hospitals (Atlanta, GA, USA). Using a two-stage transfer learning technique to train a convolutional neural network (CNN), we show that the algorithm is able to classify four classes of disease severity (normal, mild, moderate, and severe) with the average Area Under the Curve (AUC) of 0.93. In addition, we show that the outputs of different layers of the CNN under dominant filters provide valuable insight about the subtle patterns in the CXRs, which can improve the accuracy in the reading of CXRs by a radiologist. Finally, we show that our approach can be used for studying the disease progression in a single patient and its influencing factors. The results suggest that our technique can form the foundation of a more concrete clinical model to predict the evolution of COVID-19 severity and the efficacy of different treatments for each patient through using CXRs and clinical data in the early stages of the disease. This use of AI to assess the severity and possibly predicting the future stages of the disease early on, will be essential in dealing with the upcoming waves of COVID-19 and optimizing resource allocation and treatment.

Highlights

We report a new approach using artificial intelligence (AI) to study and classify the severity of COVID-19 using 1208 chest X-rays (CXRs) of 396 COVID-19 patients obtained through the course of the disease at Emory Healthcare affiliated hospitals (Atlanta, GA, USA)
We show that our model can predict the severity degree of pneumonia caused by COVID-19 from CXR images with the average area under the curve (AUC) of 0.97, 0.90, 0.90, and 0.95 for normal, mild, moderate, and severe COVID-19 classes over unseen test sets
We reduce the dimensionality of the output of the penultimate layer in the convolutional neural network (CNN) in Fig. 1 using uniform manifold approximation and projection (UMAP) to visualize the distribution of the data in a two-dimensional (2D) space, called the latent space, and compare different COVID-19 severity classes

Summary

Introduction

We report a new approach using artificial intelligence (AI) to study and classify the severity of COVID-19 using 1208 chest X-rays (CXRs) of 396 COVID-19 patients obtained through the course of the disease at Emory Healthcare affiliated hospitals (Atlanta, GA, USA). The results suggest that our technique can form the foundation of a more concrete clinical model to predict the evolution of COVID-19 severity and the efficacy of different treatments for each patient through using CXRs and clinical data in the early stages of the disease. The significant efforts and advances to foster the development of inexpensive, more accurate, faster, and easier-to-use test kits and the aforementioned reasons raise questions about using medical imaging as a tool for detecting the disease Instead, these modalities can play a crucial role in determining the severity degree of the disease and understanding the dynamics of its development from mild to severe in different patients as well as predicting its evolution and assessing the efficacy of different t reatments[2,3,8,9,10,11,12,13], which are not feasible in RT-PCR and other laboratory test kits. The increasing number of patients and the large number of CXRs burden an unprecedented workload on radiologists and calls for automatic severity prediction and monitoring systems more than any time

Methods

Results

Conclusion