DE-Ada*: A novel model for breast mass classification using cross-modal pathological semantic mining and organic integration of multi-feature fusions

Abstract

Computer-aided breast mass classification is an effective and widely used technology to assist pathologists in formulating clinical diagnoses and improving working efficiencies. Existing studies usually use a single image feature to perform breast mass classification. Herein, we propose a simple, yet effective, model called the DE-Ada*, which is an organic integration of multi-feature fusions, for breast mass classification. Firstly, we extract a set of complementary features, namely, scale-invariant feature transform (SIFT), GIST, histogram of oriented gradient (HOG), local binary pattern (LBP), residual network (ResNet), densely connected convolutional networks (DenseNet), and visual geometry group (VGG), to characterize mammograms from diverse perspectives. We attempt to mine the cross-modal pathological semantics among these features and complete their early fusion. The dynamic weight of any feature or cross-modal pathological semantics is computed and utilized to complete mid-level feature fusion. Finally, we design two voting-based ensemble learning strategies to implement late feature fusion. Our experiments demonstrate that the DE-Ada* model outperforms baselines on two well-known mammographic datasets. Our model encourages the use of cross-modal pathological semantics to deal with the overfitting problem.