SIGN LANGUAGE DETECTION USING ACTION RECOGNITION WITH PYTHON

Abstract

Sign language detection plays a crucial role in bridging the communication gap between individuals with hearing impairments and the rest of the population. This study presents a novel approach for sign language detection using action recognition and LSTM deep learning models implemented in Python. The proposed system aims to accurately recognize and interpret sign language gestures in real-time. The research focuses on leveraging the power of deep learning techniques, specifically the Long Short-Term Memory (LSTM) architecture, to effectively capture temporal dependencies and classify sign language gestures. A comprehensive dataset comprising a wide range of sign language gestures is collected and preprocessed to train and evaluate the LSTM model. The pipeline for sign language detection consists of several stages, including video acquisition, preprocessing, feature extraction, and model training. In the pre-processing stage, the acquired video data is segmented into individual frames, and various image processing techniques are applied to enhance the quality and remove noise. Next, robust features are extracted from the preprocessed frames using techniques like optical flow or deep learning-based feature extraction methods. The LSTM model is then trained on the extracted features to learn the temporal dynamics of sign language gestures. Transfer learning is also explored to leverage pre-trained models on large-scale action recognition datasets. The trained model is evaluated using a comprehensive set of metrics, including accuracy, precision, recall, and F1-score, to assess its performance. Experimental results demonstrate the effectiveness of the proposed approach in accurately recognizing sign language gestures. The LSTM-based deep learning model achieves a high accuracy rate, showcasing its capability to handle the temporal nature of sign language. The system exhibits real-time performance, enabling it to be deployed in various applications, such as real-time interpretation tools or assistive devices for individuals with hearing impairments