Multi-state colposcopy image fusion for cervical precancerous lesion diagnosis using BF-CNN

Abstract

Colposcopy is one of the most commonly used procedures to screen for cervical lesions in early precancerous stages, allowing effective preventive treatments for cervical cancer. Deep learning based algorithms have been proven promising for analyzing colposcopy images (i.e., cervicograms) to identify cervical precancerous lesions. However, most existing methods only use the single-state images applied with acetic acid solution (i.e., acetic acid cervicograms) as their input data, while missing the additional information provided by colposcopy images applied with Lugol's iodine solution (i.e., Lugol's iodine cervicograms). Since both acetic acid and Lugol's iodine cervicograms are available in clinic, health providers would select suspected lesion regions for further biopsy by visually inspecting both types of cervicograms. Therefore, it is essential to take these two-state images as inputs to extract features because these two types of cervicograms provide complementary information. In this work, we propose a bilinear fuse convolutional neural network (BF-CNN) that implements a feature selection module based on attention mechanisms and utilizes the factorized bilinear pooling technique to effectively fuse two-state image features for automatic diagnosis of cervical precancerous lesions. We applied BF-CNN and alternative algorithms to the real clinic cervicogram dataset consisted of 1400 patients, where both types of cervicograms were available per patient. As a result, BF-CNN obtained similar sensitivity 74.6% as well as the highest accuracy 85.5%, specificity 95.7%, and AUC 0.909, compared with known multi-modal algorithms and our improved algorithm only using one type of cervicograms. BF-CNN is expected to provide a useful tool to help physician make more precise clinic diagnosis.