Novel framework for image attribute annotation with gene selection XGBoost algorithm and relative attribute model

Abstract

Material attribute recognition from the visual appearance of attributes is an important problem in computer vision field. However, few works model the hierarchical relationship between material attributes and their deep-level semantics that occurs in the same image. Meanwhile, single image feature is insufficient to achieve high-quality material attribute classification. In this paper, we present methods for generating a new hierarchical material attribute representation mechanism using a new-designed feature mid-fusion algorithm and the state-of-the-art relative attribute (RA) model. The novel feature mid-fusion algorithm can improve the performance of material attribute classification. The deep-level semantics of material attributes are mined by the state-of-the-art RA model. They provide considerable useful and detailed knowledge on material attributes. We call the novel feature mid-fusion algorithm gene selection eXtreme gradient boosting (GS-XGBoost). This algorithm considers the state-of-the-art boosting idea (eXtreme gradient boosting) and the popular multi-feature fusion idea (effective range-based gene selection). To comprehensively describe material attributes, we also measure the relative degree of their deep-level semantics. A new hierarchical material attribute representation mechanism is constructed on the basis of the correctly classified material attributes and their deep-level semantics. The mechanism has two forms. One is binary attribute representation mechanism, and the other is relative attribute representation mechanism. We demonstrate the effectiveness of the proposed GS-XGBoost algorithm on two different datasets. The proposed GS-XGBoost algorithm is not an end-to-end framework but is efficient and practical for fine- and coarse-grained material attribute classification problems that can be applied in different scenarios in large-scale product image retrieval, robotics, and industrial inspection. The novel hierarchical material attribute representation mechanism will help humans or robotics accurately recognize diverse materials and their deep-level semantics. Our research contributes to not only computer science but also material science and engineering.