Automatic translator from Portuguese (voice and text)to Portuguese sign language

Abstract

One of the major focuses of technology, engineering, and computer science is to solve problems and improvethe quality of life in health. The relationship between health and technology has made great advances in thelast decades, and many cooperations are foreseen. Some citizens still have to deal with many obstacles daily tocope with the society as it is designed. One of the obstacles citizens deal with is deaf and hearing impairment.In Portugal, there are about 100,000 to 150,000 people with some level of hearing loss, and of these, around30,000 people use Portuguese Sign Language as their mother tongue (1); National Institute for Rehabilitation, (2).The greatest difficulties are appropriate or amend if necessary. Encountered by them are poor communicationwith hearing people and the need of a translator person. Communication became even more complicated withthe mandatory use of face mask, respect for the maximum number of people per division and social distancingdue to the COVID-19 pandemic. To diminish or even solve this issue, the authors developed an automatic voiceand text translation system to Portuguese sign language with captions. Sign language will be very useful for deafand hearing impairment citizens, and captions will be useful for deaf and hearing impairment citizens who don’tunderstand sign language. For that, programming language tools were developed to create a translator with thoserequests. This article describes the developments and achievements. For the proof of concept, the translatorstarted with 16 images in the database and reached a confidence level between 70 and 90%. This is an incentiveto further developments that authors are continuing to produce more improvements in the developed tool.