Distributed image understanding with semantic dictionary and semantic expansion

Abstract

Web-scale image understanding is drawing more and more attention from the computer vision and multimedia domain. To solve the key problem of visual polysemia and concept polymorphism in the image understanding, this paper proposes a semantic dictionary to describe the images on the level of semantic. The semantic dictionary characterizes the probability distribution between visual appearances and semantic concepts, and the learning procedure of semantic dictionary is formulated into a minimization optimization problem. Mixed-norm regularization is adopted to solve the above optimization for learning the concept membership distribution of visual appearance. Furthermore, to improve the generalization ability of the semantic description, we propose the semantic expansion technology, where a concept transferring matrix is learnt to quantize the implicit relevancy among the concepts. Finally, the distributed framework on the basis of the semantic dictionary is constructed to speed up the large scale image understanding. The semantic dictionary is validated in the tasks of large scale semantic image search and image annotation.