Interpretation of sign language into English using NLP techniques

Abstract

Sign language is a way of communication for deaf & dumb. Different sign recognition techniques are there which are giving output in the form of word for recognized sign. The proposed method is focusing on interpretation of sign language in proper English sentence. Different NLP techniques are used in addition to sign recognition. Input is given as video of sign language followed by framing & segmentation on video. CamShift algorithm is used for tracking & P2DHMM for hand tracking. Haar Cascade classifier is used for sign identification. After sign recognition, the continuous words for respective sign are given as input to POS tagging module. Word net POS tagger is used which is having its own WordNet dictionary. At last LALR parser is used to frame the sentence. In this way proposed sign language interpreter model gives the output in meaningful English sentence & with a goodaccuracy.