Lung image quality assessment and diagnosis using generative autoencoders in unsupervised ensemble learning

Abstract

The lung is a critical organ for blood gas exchange. Early lung disease detection is often hindered by subtle symptoms. The diagnosis typically requires expert analysis of lung image scans, where both conventional methods and advanced machine learning (ML) and deep learning (DL) techniques are employed for scan segmentation and disease detection. However, it is a challenge to develop reliable computational models, due to the scarcity of high-quality data. With a major emphasis on lung disease classification and segmentation performance, this work presents a novel data quality assessment technique specifically designed for lung image datasets. The proposed pipeline combines ensemble learning, unsupervised learning, and generative autoencoders with attention mechanisms (GAME). By including attentional mechanisms in the generative autoencoders, we improve the tool’s ability to locate and prioritise areas of interest in lung images and extract the right features, which increases the accuracy of our algorithms. The pipeline automates quality checks and reduces the need for high-quality data, while reducing human oversight. Because it’s crucial to validate the robustness of the algorithms in our tool, we tested it using lung scans from the NIH-ChestXray and CheXpert datasets, and obtained IOU scores of 0.88 and 0.86, F1 scores of 0.95 and 0.95, and accuracy scores of 0.96 and 0.95, respectively, which shows they can be used as a classification tool as well as a segmentation tool. Given the challenging conditions in which the early diagnosis of lung disease is made, this is a significant step forward in the development of self-sustaining and accurate disease detection systems. The proposed model is made available to the public and the same is accessible via https://github.com/harshivchandra/LungDataQualityAssessment.