Abstract
During clinical practice, radiologists often use attributes, e.g., morphological and appearance characteristics of a lesion, to aid disease diagnosis. Effectively modeling attributes as well as all relationships involving attributes could boost the generalization ability and verifiability of medical image diagnosis algorithms. In this paper, we introduce a hybrid neuro-probabilistic reasoning algorithm for verifiable attribute-based medical image diagnosis. There are two parallel branches in our hybrid algorithm, a Bayesian network branch performing probabilistic causal relationship reasoning and a graph convolutional network branch performing more generic relational modeling and reasoning using a feature representation. Tight coupling between these two branches is achieved via a cross-network attention mechanism and the fusion of their classification results. We have successfully applied our hybrid reasoning algorithm to two challenging medical image diagnosis tasks. On the LIDC-IDRI benchmark dataset for benign-malignant classification of pulmonary nodules in CT images, our method achieves a new state-of-the-art accuracy of 95.36% and an AUC of 96.54%. Our method also achieves a 3.24% accuracy improvement on an in-house chest X-ray image dataset for tuberculosis diagnosis. Our ablation study indicates that our hybrid algorithm achieves a much better generalization performance than a pure neural network architecture under very limited training data.
Highlights
D UE to their rapid progress in the past ten years, deep neural networks have achieved tremendous success in boosting the performance of image recognition [18], [19], [27], [50] and other visual computing tasks [38], [48]
A distinct global average pooling (GAP) operator is applied to the feature map at each scale of the feature pyramid network (FPN) to reduce the number of parameters
We have compared our hybrid algorithm with several stateof-the-art models for pulmonary nodule classification on the LIDC-IDRI dataset and the results are shown in Table 1, where O2 and O2∗ represent our models trained without and with standard data augmentation
Summary
D UE to their rapid progress in the past ten years, deep neural networks have achieved tremendous success in boosting the performance of image recognition [18], [19], [27], [50] and other visual computing tasks [38], [48]. A medical image diagnosis algorithm needs to learn the knowledge reasoning process followed by radiologists during clinical practice. They start from visual evidences in medical images and reach diagnostic conclusions by referring to causal relationships between diseases and the visual evidences. To exploit the complementary strengths of neural and probabilistic learning algorithms as well as overcome the limitations of both ends, we introduce a hybrid neuro-probabilistic reasoning algorithm for verifiable attribute-based medical image diagnosis. To support causal and verifiable relational modeling and reasoning, this algorithm tightly couples Bayesian networks and a graph convolutional network Such coupling is achieved via a cross-network attention mechanism and a classification result fusion scheme. The proposed algorithm achieves state-of-the-art performance on the LIDC-IDRI benchmark dataset for the first task and an in-house dataset for the second task
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IEEE transactions on pattern analysis and machine intelligence
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
