Systolic hand gesture recognition/detection system based on FPGA with multi-port BRAMs

Abstract

A new systolic approach of real-time vision-based hand gesture recognition is proposed. Hand gesture recognition is classified into two type, static and dynamic. The proposed system can be employed in both types. The system can detect and classify either a static hand shape in one image or a static hand shape at the start, within, and the end of gesture in a dynamic sequence in video stream. An efficient template matching (TM) based similarity measure is used to recognize gesture within the inter frame sequence of video frames. The template can be used to store the static class of different hand gesture shapes, while fast scanning approach of window over an image frame dynamically detects or recognizes the hand gesture. A suitable hardware systolic model of TM is designed. The systolic architecture is expanded to be used with multiple windows (k-templates) simultaneously. The required templates are equal to the number (k) of gestures that is to be recognized. They are stored in FPGA RAM blocks (BRAMs). To increase the bandwidth of templates BRAMs, the BRAM is multi-ported. A considerable reduction of memory access operations is achieved for the parallel systolic model in comparison to sequential/parallel software models. Consequently, the hand gesture is detected, tracked and recognized within the inter frame space of video. An acceptable recognition accuracy of up to 95% is achieved under variant observation conditions of three hand gesture classes dataset. A high frame rate video can be processed in real time.