Deep collaborative multi-task network: A human decision process inspired model for hierarchical image classification

Abstract

Hierarchical classification is significant for big data, where the original task is divided into several sub-tasks to provide multi-granularity predictions based on a tree-shape label structure. Obviously, these sub-tasks are highly correlated: results of the coarser-grained sub-tasks can reduce the candidates for the fine-grained sub-tasks, while results of the fine-grained sub-tasks provide attributes describing the coarser-grained classes. A human can integrate feedbacks from all the related sub-tasks instead of considering each sub-task independently. Therefore, we propose a deep collaborative multi-task network for hierarchical image classification. Specifically, we first extract the relationship matrix between every two sub-tasks defined by the hierarchical label structure. Then, the information of each sub-task is broadcasted to all the related sub-tasks through the relationship matrix. Finally, to combine this information, a novel fusion function based on the task evaluation and the decision uncertainty is designed. Extensive experimental results demonstrate that our model can achieve state-of-the-art performance.