Automated multi-feature human interaction recognition in complex environment

Abstract

The recognition of people’s interactions is crucial for making the surveillance applications able to recognize unusual events in complex environments. Generally, multiple cameras are installed to capture videos from different views but these environments suffer from challenging issues: occlusions between persons and light and pose variations etc. We presented a computer vision system to recognize person-to-person interactions in public areas by considering individual actions and trajectory information under multiple camera views. We achieved our goal in two steps, namely, individual action recognition and interaction recognition. Extensive techniques have been used for individual action recognition with very good accuracy. Still, these techniques cannot handle the intricate settings in crowded areas. We have proposed Median Compound Local Binary Pattern (MDCLBP) and combined it with Histogram of Oriented Gradient (HOG). MDCLBP captures the information about the spatial organization of intensities and HOG uses histogram of oriented gradients to describe an image. MDCLBP is a modification of Compound Local Binary Pattern (CLBP). CLBP extracts texture information by using sign and magnitude information. MDCLBP is a variant of CLBP that uses sign information and instead of magnitude, difference from the median value at each 3 × 3 windows is used to get the descriptor robust to occlusions and light variations. We have combined the individual actions of two persons with trajectory information to recognize person-to-person interactions. Experiments are performed on well-known publically available IXMAS and OIXMAS datasets to demonstrate the effectiveness of our proposed technique for individual human action recognition. Person-to-person interaction recognition method is evaluated on HALLWAY dataset. Experiments carried out on varying views demonstrated that our proposed system achieved better accuracy and can meet the requirements of surveillance applications.