ApparelNet: Person Verification Encompassing Auxiliary Attachments Variation

Abstract

Auxiliary attachments change is more frequent than essential clothing in human beings. Usually, auxiliary attachments include coat, jumper, hat and bag etc. It is one of the hardest recognition task in machine vision. It becomes more difficult if a specific person is reappearing after a longer time period while the other influential factors are angle variation and walking speed etc. One of the key application areas for person verification is border control where auxiliary attachments variation is more common. It is usually reflection of ethnicity or fashion. In machine vision, availability of such datasets is very limited, in particular, having reappearance after longer time period i.e., more than weeks or months. To overcome limited dataset problem, transfer learning is a leading solution for improved verification. In this paper, we proposed an aggregated deep learning model called, ApparelNet, more specifically for person verification in border control environment. We used Front-View Gait (FVG) to evaluate the performance of our aggregated model. The FVG is a pedestrian dataset of people encompassing auxiliary attachments variation, having three different angles from the camera and three different walking speeds. Our ApparelNet acquires single image based detection confidence using OpenPose and later additional layers of pre-trained EfficientNetB0 are trained on custom FVG dataset, including fine tuning of the overall EfficientNetB0. The EfficientNetB0 is highly efficient and scalable transfer learning model from the family of Deep-CNN. Overall, our ApparelNet reported training and validation accuracy of 98%, while looking at border control scenario, model verification is performed by selecting random images of 12 different individuals and prediction probability is computed which accumulates to 96%. In our opinion, the model has strong candidature for person reidentification where goal is one-to-many recognition. It may become an ancillary component of any biometrics system too.