FPGA implementation of two multilayer perceptron neural network in cascade for efficient real time hand gestures tracking

Abstract

This paper presents implementation of a hand gestures tracking system with a fully connected multilayer perceptron and a supervised sequential learning algorithm on low cost FPGAs (Field Programmable Gate Array). Two networks are used in the study. The first network is trained to detect skins and the second identifies face and hand gestures. Also, a binary image is made from the first network, which is fed into the second network as input. Skin network has two output classes, and hand network 6. The first network recognizes skin and non-skin images and the other distinguishes hands from faces while hand gestures are tracked and classified. Both networks are trained in Matlab before each was implemented on a separate FPGA and the two FPGAs are then connected to each other. Finally, the overall result is compared. Both networks are implemented on Xilinx Spartan XC3S1000 4fg456. Two hardware-implemented networks are applied to a video sequence frame by frame. Features of both hands are extracted and the network is trained using a constructed vector to track left and right hands. Results show that the implemented system enjoys a suitable trade-off between accuracy and speed, reaching 208 fps with the least memory usage. This method has been compared to other methods, as well as methods in which only one threshold is used to detect skin color.