A multimodal AI-based non-invasive COVID-19 grading framework powered by deep learning, manta ray, and fuzzy inference system from multimedia vital signs

Abstract

The COVID-19 pandemic has presented unprecedented challenges to healthcare systems worldwide. One of the key challenges in controlling and managing the pandemic is accurate and rapid diagnosis of COVID-19 cases. Traditional diagnostic methods such as RT-PCR tests are time-consuming and require specialized equipment and trained personnel. Computer-aided diagnosis systems and artificial intelligence (AI) have emerged as promising tools for developing cost-effective and accurate diagnostic approaches. Most studies in this area have focused on diagnosing COVID-19 based on a single modality, such as chest X-rays or cough sounds. However, relying on a single modality may not accurately detect the virus, especially in its early stages. In this research, we propose a non-invasive diagnostic framework consisting of four cascaded layers that work together to accurately detect COVID-19 in patients. The first layer of the framework performs basic diagnostics such as patient temperature, blood oxygen level, and breathing profile, providing initial insights into the patient's condition. The second layer analyzes the coughing profile, while the third layer evaluates chest imaging data such as X-ray and CT scans. Finally, the fourth layer utilizes a fuzzy logic inference system based on the previous three layers to generate a reliable and accurate diagnosis. To evaluate the effectiveness of the proposed framework, we used two datasets: the Cough Dataset and the COVID-19 Radiography Database. The experimental results demonstrate that the proposed framework is effective and trustworthy in terms of accuracy, precision, sensitivity, specificity, F1-score, and balanced accuracy. The audio-based classification achieved an accuracy of 96.55%, while the CXR-based classification achieved an accuracy of 98.55%. The proposed framework has the potential to significantly improve the accuracy and speed of COVID-19 diagnosis, allowing for more effective control and management of the pandemic. Furthermore, the framework's non-invasive nature makes it a more attractive option for patients, reducing the risk of infection and discomfort associated with traditional diagnostic methods.