Hybrid Convolution-Based Efficientnet-Based Hand Gesture Recognition Framework with Optimized Algorithm

Abstract

The difficulties in communication and hearing are an important concern for deaf–dumb people, which stop access to their essential and basic needs. Many findings have been made to address sign languages even though this challenging problem is not still solved. Many methods aimed to propose vision-based classifiers through identical pattern investigation tasks by obtaining the difficult handcraft feature descriptions of gestures from the gathered images. However, the efficacy of all those models is less for performing with a huge signbook captured from uncontrolled and complex background conditions. So, an effective Indian Sign Language (ISL) classification method is developed by an advanced deep learning approach. At first, the hand gesture images are obtained from the data source. Only the image of the hand, even from a complicated background, is extracted from the obtained image. The features are extracted using the Scale-Invariant Feature Transform (SIFT) method and Multiscale Vision Transformer (MVT). Then, the extracted features are fed to the Hybrid Convolution-based EfficientNet (HCEN) model. The hyper-parameters in the developed HCEN model are tuned using the implemented Adaptive Political Optimizer (APO) algorithm. The recognized hand signs are obtained from the suggested HCEN model. Various experiments are conducted to determine the performance of the suggested deep learning-based hand gesture recognition model.