Quantifying inconsistencies in the Hamburg Sign Language Notation System

Abstract

The advent of machine learning (ML) has significantly advanced the recognition and translation of sign languages, bridging communication gaps for hearing-impaired communities. At the heart of these technologies is data labeling, crucial for training ML algorithms on a huge amount of consistently labeled data to achieve models that generalize well. The adoption of language-agnostic annotations is essential to connect different sign languages, as single-language databases often provide limited lexicon examples, insufficient for training robust ML algorithms. This study critically examines the Hamburg Sign Language Notation System (HamNoSys), which describes the signer’s initial position and body movements, in contrary to the meanings of glosses. Despite HamNoSys’s utility in standardizing transcriptions across various sign languages, our investigation uncovers inconsistencies within HamNoSys that may negatively impact the development of accurate and reliable ML models. By analyzing HamNoSys labels across five sign languages, we identified a lack of standardized annotation procedures and the complexities within HamNoSys that introduce biases and errors. Our findings underscore the urgent need for unified, standardized data annotation guidelines to enhance the accuracy and efficiency of sign language recognition technologies. This research highlights the importance of addressing annotation challenges and advocates for a comprehensive, diversified database to improve the generalization of ML models.