Multimodal clothing recognition for semantic search in unconstrained surveillance imagery

Abstract

To date, surveillance based person search has focused on locating a person of interest from an image query, distinct from the law enforcement task of locating a person from a description.In this paper, we introduce a novel probabilistic framework that combines multiple traits whilst incorporating their uncertainty to tackle the emerging challenge: locating a person from a semantic query. In addressing this, we improve clothing texture recognition by leveraging Dempster-Shafer theory against an ensemble of support vector machines; achieving state-of-the-art performance for high and low resolution clothing textures.Our proposed person search framework combines information from clothing texture and colour in the torso and leg regions to produce a probabilistic match between unknown subjects and the designated target query. Results are presented on a newly created 520 subject surveillance dataset which is made available to researchers. This multi-modal person search technique achieves promising results for locating target subjects, without the requirement of pre-search target enrollment.