Abstract
Translation and understanding sign language may be difficult for some. Therefore, this paper proposes a solution to this problem by providing an Arabic sign language translation system using ontology and deep learning techniques. That is to interpret user’s signs to different meanings. This paper implemented ontology on the sign language domain to solve some sign language challenges. In this first version, simple static signs composed of Arabic alphabets and some Arabic words started to translate. Deep Convolution Neural Network (CNN) architecture was trained and tested on a pre-made Arabic sign language dataset and on a dataset collected in this paper to obtain better accuracy in recognition. Experimental results show that according to the pre-made Arabic sign language dataset the classification accuracy of the training set (80% of the dataset) was 98.06% and recognition accuracy of the testing set (20% of the dataset) was 88.87%. According to the collected dataset, the classification accuracy of the training set was 98.6% and Semantic recognition accuracy of the testing set was 94.31%.
Highlights
Sign language considered the only way for communication between deaf, hearing-impaired and normal people
It is hard for most people who are not interested in sign language to communicate without an interpreter
One sign may use to represent different words such as; sign in Fig. 3, represents the word ""لا, the Character ""ب and number "1" in the Arabic language. This challenge solved in Multi Sign Language Ontology (MSLO) ontology, where sign gesture interprets based on its class and domain
Summary
Sign language considered the only way for communication between deaf, hearing-impaired and normal people. Many types of research focused on sign language translation to text and spoken language and vice versa This is not easy to be done by machines since it depends on the natural language processing and image recognition. WordNet is a linguistic resource containing words of the targeted language and synsets and semantic conceptual relations between them. These relations provide semantic information about concepts and their original words. The contribution of this paper is: To enhance sign language translation using the power of semantic web technologies (i.e. ontologies) and deep learning. The proposed method applied in Arabic sign language semantic translation system.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have