Hand-Held Object with Action Recognition Based On Convolutional Neural Network in Spatio Temporal Domain

Abstract

Several applications such as object recognition and face recognition are established with the progress of smart devices and computer technology, to assist human-computer interaction (HCI). In HCI, Hand-held object recognition hasamain role. This approach helps the computer to realise the user’s intentions and also meetsthe user requirements. Hand as an organ which is considered as a direct and natural way of communication for humans. The Hand-held Object Recognition (HHOR) assigns a label for the object which is heldin hand this could help machines in understanding the environment and the intention of the people. However, it has not been well studied in the community.So, in this paper, we proposed system for recognizing such activities happening between hands and faces in real time. The interaction events (e.g. eating, phoning and smoking) between hands and faces are analysed using the event analysis approach. Ratio histogram is used for obtaining the essential colour bins for detecting the desired objects via re-projection method. For object tracking and feature extraction, a code book method is used. To recognize various human-object interaction events, the dynamic and multiplicity contexts of event are modelled together. Finally, atwo stage cascaded CNN classifiers for the recognition is implemented as this technology improves the performance of object recognition.To make fair comparisons, six methods were compared in this paper based on the HMDB dataset. This system is effective and can be performed in real time because an exhaustive search process to find possible interaction pairs in the huge space of all possible event parameters is not involved. Experimental results have proved the superiority of our proposed system to analyse different human behaviours and events between hands and a face.