Canonical Correlation Analysis Feature Fusion With Patch of Interest: A Dynamic Local Feature Matching for Face Sketch Image Retrieval

Abstract

An automatic photo retrieval system based on a face sketch has very useful application as to narrow down potential suspects in criminal investigations. This is true when there is no other evident except the face sketch that is rendered based on the recollection of a victim or eyewitness. Among the noticeable difficulties in matching the sketch and photo due to its modality difference are the generated sketch has some tendency of shape exaggeration, the sketch has very less accurate details and the real-world photo may expose to lighting variation unlike the sketch. In this paper, we attempt to address these complications by matching the sketch and photos using dynamic local feature of Difference of Gaussian Oriented Gradient Histogram (DoGOGH) on some selected patches. To avoid discriminative power degradation due to a large number of gallery images, two stage matching blocks are introduced in a cascaded fashion. The front block matches the feature such that it short lists $k$ most similar photos for the second block. In this front block, Histogram of Oriented Gradient (HOG) and Gabor Wavelet (GW) features are fused by maximizing the correlation between the two using Canonical Correlation Analysis (CCA). Based on the short listed photos, the following block re-matched the sketch and photos using dynamically extracted local feature on its Patch of Interest (PoI). Eventually, the matching scores from the blocks are fused before getting rank-1 accuracy. The experimental results on two baseline datasets indicate that the proposed method outperforms the state-of-the-art methods. The extended evaluation on semi-forensic and forensic sketch datasets demonstrate its usage feasibility.