Novel FPGA Implementation of Hand Sign Recognition System With SOM–Hebb Classifier

Abstract

This paper proposes a hardware posture recognition system with a hybrid network. The hybrid network consists of self-organizing map (SOM) and Hebbian network. Feature vectors are extracted from input posture images, which are mapped to a lower dimensional map of neurons in the SOM. The Hebbian network is a single-layer feedforward neural network trained with a Hebbian learning algorithm to identify categories. The recognition algorithm is robust to the change in location of hand signs, but it is not immune to rotation or scaling. Its robustness to rotation and scaling was improved by adding perturbation to the training data for the SOM–Hebb classifier. In addition, neuron culling is proposed to improve performance. The whole system is implemented on a field-programmable gate array employing novel video processing architecture. The system was designed to recognize 24 American sign language hand signs, and its feasibility was verified through both simulations and experiments. The experimental results revealed that the system could accomplish recognition at a speed of 60 frames/s, while achieving an accuracy of 97.1%. Due to a novel hardware implementation, the circuit size of the proposed system is very small, which is highly suitable for embedded applications.