ChartFuse: a novel fusion method for chart classification using heterogeneous microstructures

Abstract

Chart images exhibit significant structural variabilities, which is called as micro-structural variabilities, which makes each image type different from others even though chart image belongs to the same class or categories. The lack of affiliation between the heterogeneous features and the structure of the chart images, make it challenging to learn these micro-variabilities features by any learning model for automatic chart recognition and interpretation. However, extracting low-level heterogeneous features from chart images remains challenging. This paper presents a novel chart image classification method by using local feature descriptor. We proposed a new heterogeneous feature extractor, namely the heterogeneity index (HI) fused with local penta pattern. Here, the microstructural features are defined on the similarity of the chroma effects, and HI is computed depending upon the basic colors and its intensity in the microstructures with similar chroma effects. HI integrates colors, textures, structural layout, and illumination details with the local features altogether for the image classification. The proposed method is tested on chart image datasets, namely FigureQA and our handcrafted chart dataset. Experimental results depict that our method classify images with an accuracy rate of 95%–97% which is an increase of 5%–10% as compared with the customary methods.