Modelling, synthesis and characterisation of occlusion in videos

Abstract

Occlusion is one of the most challenging problems in many video processing applications such as surveillance, gait recognition, activity recognition and so on. Attempts have been made to develop algorithms for handling occlusion and evaluate their performance on various datasets. However, these studies are subjective in nature and the datasets are hardly characterised in terms of the level of occlusion, thereby precluding any form of quantitative comparison of performance. This shows a compelling need to design an explicit, unambiguous and quantitative model, which should be able to objectively represent occlusion in a video. This study proposes an occlusion model based on the position and pose uncertainties of the moving subjects in a video. The proposed occlusion model is able to characterise the level of occlusion present in a video. It is also employed to synthetically generate occlusion for walking sequences, thus providing a direction for controlled dataset generation against which human identification algorithms can be tested. Given an input video with a subject moving without any occlusion, a particle swarm optimisation‐based parameter estimation methodology is presented that generates the desired level of occlusion. The proposed approaches have been tested on the TUM‐IITKGP and PETS2010 datasets. Finally, as an application, the occlusion model has been used to generate an occluded gait datasets and the performances of different gait recognition algorithms have been compared under varying levels of occlusion.