Arabic Sign Language Translator

Abdelmoty M Ahmed ,Reda Abo Alez,Muhammad Taha,Ahmad M.J Al Moustafa,Gamal Tharwat,Wade Ghribi,Belgacem Bouallègue

doi:10.3844/jcssp.2019.1522.1537

Abstract

Development of systems that can recognize the gestures of Arabic Sign language (ArSL) provides a method for hearing impaired to easily integrate into society. This paper aims to develop a computational structure for an intelligent translator to recognize the isolated dynamic gestures of the ArSL. In our proposed system we build a datasets for ArSL from scratch of, we used 100-sign vocabulary from ArSL, we have applied 1500 video files for these signs. These signs were divided into five types of signs, recognizing a sign language gestures from dynamic gestures could be a difficult analysis issue. This paper solves the problem using gradient based key frame extraction technique. These key frames are useful for splitting continuous language gestures into sequence of signs for removing uninformative frames. After splitting of gestures every sign has been treated as isolated gesture. Then features of pre-processed gestures are extracted using Intensity Histogram by integrating with Gray Level Co-occurrence Matrix (GLCM) features. Experiments are performed on our own ArSL dataset and the matching between the ArSL and Arabic text is tested by Euclidian distance. The evaluation of the proposed system for the automatic recognition and translation for isolated dynamic ArSL gestures has proven to be effective and highly accurate. The experimental results show that the proposed system recognizes signs with a precision of 95.8%.

Highlights

Deaf, dumb and hearing impaired cannot speak as common persons; so they have to depend upon another way of communication using vision or gestures during their life
Gray Level Co-occurrence Matrix (GLCM) texture considers the relation between two pixels at a time, called the reference and the neighbor pixels; we have proposed a set of 23 textual features extracted from the co-occurrence matrix such as Contrast, Homogeneity, Dissimilarity, Angular Second Moment, Energy and Entropy
The Graphical User Interface (GUI) for the proposed system was implemented by MATLAB, Fig. 10 Illustrate GUI of proposed system

Summary

Introduction

Dumb and hearing impaired cannot speak as common persons; so they have to depend upon another way of communication using vision or gestures during their life. Persons with hearing loss and speech are deprived of normal contact with the rest of the community. It varies by country, geographic region or even by the interests of the hearing impaired. Sign language is a combination of descriptive and non-descriptive signs as well as alphabets (fingers spelling) signs. There is no uniform format for ArSL, which make education a difficult challenge for the hearing impaired persons, so we need to have education to be bilingual for reading and writing, the shortage of skilled teachers who know ArSL makes education difficult for the hearing impaired

Objectives

Methods

Results

Conclusion