Korean Sign Research Articles

Kurdish Sign Language Recognition Using Convolutional Neural Network (CNN)

The present study examines the obstacles encountered by the deaf population, with a specific emphasis on the increasing significance of sign language in facilitating effective communication. The main mode of communication for deaf persons is Sign Language (SL), which is a visual and expressive method of communicating meaning via facial expressions, hand motions, and body gestures. The objective of this project is to automate the identification of sign language in order to improve accessibility and decrease reliance on interpreters. Specifically, the work focused on constructing an alphabet recognition system for Kurdish Sign Language (KSL). Due to its many intricacies and resemblances to the Arabic script, KSL requires a strong recognition model. The proposed method utilizes Convolutional Neural Networks (CNN) with an actual dataset to accurately recognize both numerical values and alphabetic characters in the Korean Sign Language (KSL). Real-time operation allows for rapid recognition of hand gestures and subsequent provision of textual output. The collection comprises 132,000 hand pictures, including 33 alphabetica and (0-9) numeral signs. Significantly, the use of MediaPipe, a method for processing 3D images, greatly improves the efficiency of detection. Multiple methodologies were investigated, and the integration of Convolutional Neural Networks (CNN), TensorFlow, and MediaPipe resulted in a remarkable accuracy of 99.87% with negligible dropout rates. This study establishes a foundation for enhanced communication and independence for the deaf population, representing a notable advancement in the automation of sign language recognition.

A study of motivation for occupation words in Korean sign language and Chinese sign language

A study of motivation for occupation words in Korean sign language and Chinese sign language

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

Korean Sign Language Alphabet Recognition Through the Integration of Handcrafted and Deep Learning-Based Two-Stream Feature Extraction Approach

한국수어화자의 한국어 주격 조사 ‘이/가’ 습득 – 어순 및 주격중출 유형에 따른 주격 조사를 중심으로

한국수어화자의 한국어 주격 조사 ‘이/가’ 습득 – 어순 및 주격중출 유형에 따른 주격 조사를 중심으로

Application of Wearable Gloves for Assisted Learning of Sign Language Using Artificial Neural Networks

This study proposes the design and application of wearable gloves that can recognize sign language expressions from input images via long short-term memory (LSTM) network models and can learn sign language through finger movement generation and vibration motor feedback. It is difficult for nondisabled people who do not know sign language to express sign language accurately. Therefore, we suggest the use of wearable gloves for sign language education to help nondisabled people learn and accurately express sign language. The wearable glove consists of a direct current motor, a link (finger exoskeleton) that can generate finger movements, and a flexible sensor that recognizes the degree of finger bending. When the coordinates of the hand move in the input image, the sign language motion is fed back through the vibration motor attached to the wrist. The proposed wearable glove can learn 20 Korean sign language words, and the data used for learning are configured to represent the joint coordinates and joint angles of both the hands and body for these 20 sign language words. Prototypes were produced based on the design, and it was confirmed that the angle of each finger could be adjusted. Through experiments, a sign language recognition model was selected, and the validity of the proposed method was confirmed by comparing the generated learning results with the data sequence. Finally, we compared and verified the accuracy and learning loss using a recurrent neural network and confirmed that the test results of the LSTM model showed an accuracy of 85%.

한국수어의 영논항

The purpose of this paper is to analyze the distribution and characteristics of null arguments in Korean Sign Language (KSL). Languages such as Spanish and Italian, which have a rich system of overt morphological agreement, allow subject-drop but not object-drop. For this reason, it has been said that the null arguments are licensed by rich verbal morphology. On the other hand, languages such as Korean and Chinese do not have a rich agreement system like Spanish and Italian. Thus, the verbs are used with the same ending regardless of person, number, and gender features. In this paper, we show that KSL is different from those languages like Korean and Chinese in that arguments can be dropped with both agreeing and non-agreeing verbs. In some contexts, in addition, null arguments of non-agreeing verbs are not allowed, whereas null arguments of agreeing verbs are totally acceptable. Therefore, to solve these problems, in this paper, we suggest hybrid analysis in licensing null arguments in KSL.

Preprocessing for Keypoint-Based Sign Language Translation without Glosses

While machine translation for spoken language has advanced significantly, research on sign language translation (SLT) for deaf individuals remains limited. Obtaining annotations, such as gloss, can be expensive and time-consuming. To address these challenges, we propose a new sign language video-processing method for SLT without gloss annotations. Our approach leverages the signer's skeleton points to identify their movements and help build a robust model resilient to background noise. We also introduce a keypoint normalization process that preserves the signer's movements while accounting for variations in body length. Furthermore, we propose a stochastic frame selection technique to prioritize frames to minimize video information loss. Based on the attention-based model, our approach demonstrates effectiveness through quantitative experiments on various metrics using German and Korean sign language datasets without glosses.

Korean Text to Gloss: Self-Supervised Learning approach

Natural Language Processing (NLP) has grown tremendously in recent years. Typically, bilingual, and multilingual translation models have been deployed widely in machine translation and gained vast attention from the research community. On the contrary, few studies have focused on translating between spoken and sign languages, especially non-English languages. Prior works on Sign Language Translation (SLT) have shown that a mid-level sign gloss representation enhances translation performance. Therefore, this study presents a new large-scale Korean sign language dataset, the Museum-Commentary Korean Sign Gloss (MCKSG) dataset, including 3828 pairs of Korean sentences and their corresponding sign glosses used in Museum-Commentary contexts. In addition, we propose a translation framework based on self-supervised learning, where the pretext task is a text-to-text from a Korean sentence to its back-translation versions, then the pre-trained network will be fine-tuned on the MCKSG dataset. Using self-supervised learning help to overcome the drawback of a shortage of sign language data. Through experimental results, our proposed model outperforms a baseline BERT model by 6.22%.

Korean Sign Language Recognition Using Transformer-Based Deep Neural Network

Sign language recognition (SLR) is one of the crucial applications of the hand gesture recognition and computer vision research domain. There are many researchers who have been working to develop a hand gesture-based SLR application for English, Turkey, Arabic, and other sign languages. However, few studies have been conducted on Korean sign language classification because few KSL datasets are publicly available. In addition, the existing Korean sign language recognition work still faces challenges in being conducted efficiently because light illumination and background complexity are the major problems in this field. In the last decade, researchers successfully applied a vision-based transformer for recognizing sign language by extracting long-range dependency within the image. Moreover, there is a significant gap between the CNN and transformer in terms of the performance and efficiency of the model. In addition, we have not found a combination of CNN and transformer-based Korean sign language recognition models yet. To overcome the challenges, we proposed a convolution and transformer-based multi-branch network aiming to take advantage of the long-range dependencies computation of the transformer and local feature calculation of the CNN for sign language recognition. We extracted initial features with the grained model and then parallelly extracted features from the transformer and CNN. After concatenating the local and long-range dependencies features, a new classification module was applied for the classification. We evaluated the proposed model with a KSL benchmark dataset and our lab dataset, where our model achieved 89.00% accuracy for 77 label KSL dataset and 98.30% accuracy for the lab dataset. The higher performance proves that the proposed model can achieve a generalized property with considerably less computational cost.

Dynamic Korean Sign Language Recognition Using Pose Estimation Based and Attention-Based Neural Network

Dynamic Korean Sign Language Recognition Using Pose Estimation Based and Attention-Based Neural Network

학습수어로서의 수어교육

Objectives The purpose of this study is to present implications for deaf education of Korea by analyzing the ‘Japanese Sign Language(JSL)’ curriculum and textbooks of Meisei Gakuen in Japan, which provides bilingual and bicultural deaf education from the perspective of cognitive academic language. Methods For this purpose, the characteristics of Meisei Gakuen's kindergarten, elementary, and middle school ‘JSL’ curriculum and textbooks were analyzed from the perspective of cognitive academic language. Results First, the ‘JSL’ curriculum in kindergarten was designed to develop sign language skills as a basic communicative skills by considering the differences in sign language proficiency of deaf children. Sign language textbooks were designed to develop a sense of language for sign language through sign shapes cards and classifier textbooks. Second, the elementary school ‘JSL’ curriculum was designed to comprehensively develop the cognitive academic sign language ability in linguistic, cognitive, and socio-cultural aspects. Sign language textbooks are structured to systematically develop deaf culture and sign language’s grammar. Third, the ‘JSL’ curriculum of the middle school was structured to deepen the cognitive academic sign language ability in linguistic, cognitive, and socio-cultural aspects. Sign language textbooks were designed to deepen awareness of sign language by learning sign language linguistics. Fourth, as a limitation, there was an aspect that the sign language curriculum and sign language textbooks did not sufficiently consider the level difference by age and grade. Conclusions The direction of the development of the cognitive academic sign language curriculum for using Korean sign language as a teaching-learning language was suggested.

광주광역시 지명수어에서 나타나는 다양한 조어 방법

The purpose of this study is to examine the sign language place names of Gwangju Metropolitan City and identify some of the linguistic patterns of Korean Sign Language (KSL). First, we analyze 106 signed place names of Gwangju, and classify them into five types: (i) finger-spelling Korean place names, (ii) translating Chinese characters of Korean place names into KSL, (iii) using rebus, (iv) representing regional characteristics, and (v) mixing finger-spelling and translation of Korean place names. Through the analysis, it is shown that linguistic phenomena found in spoken languages such as folk etymology, creativity, phonological reduction and assimilation, etc., as well as those found uniquely in signed languages such as simultaneity and iconicity, can also be found in signed place names. Finally, the relations of form and meaning with respect to the position of a word and with respect to each other are discussed.

한국어, 영어, 한국수어에 나타나는 [도덕성은 청결함] 은유

Lakoff and Johnson (1999) claim that morality is embodied, meaning that the concept MORALITY is understood as PHYSICAL CLEANLINESS and morality and cleanliness are mutually influenced by each other. Many psychological experiments have been conducted to prove this mutual influence between morality and physical cleanliness. However, very few studies, if any, have focused on linguistic manifestation of the metaphor MORALITY IS PHYSICAL CLEANLINESS. In bridging the gap, this paper examines spoken and sign languages such as Korean, English, and Korean Sign Language, demonstrating the potential of the universality of the metaphor MORALITY IS PHYSICAL CLEANLINESS. The results suggest that psychological link between morality and physical cleanliness is cross-cultural.

한국수어 다의어의 의미망 분석: [기름] 수어를 중심으로

There has been abundant research on semantic networks of polysemous units but far less of such research on sign languages, despite the fact that polysemy is far more prevalent in the latter. As a case study, this paper observes a polysemous word meaning ‘oil’ in Korean Sign Language and demonstrates how the prototypical meaning ‘oil’ is extended to various meanings through conceptual metaphor and conceptual metonymy. The paper observes that the concept OIL is closely related to the concepts SMOOTHNESS, FAMILIARITY, FLUENCY, and FREQUENCY in KSL, unlike Korean. The cognitive semantics approach to the polysemous word OIL in KSL demonstrates how we, as language users, conceptualize new concepts utilizing what is used for other concepts.

Enabling Real-time Sign Language Translation on Mobile Platforms with On-board Depth Cameras

In this work we present SUGO, a depth video-based system for translating sign language to text using a smartphone's front camera. While exploiting depth-only videos offer benefits such as being less privacy-invasive compared to using RGB videos, it introduces new challenges which include dealing with low video resolutions and the sensors' sensitiveness towards user motion. We overcome these challenges by diversifying our sign language video dataset to be robust to various usage scenarios via data augmentation and design a set of schemes to emphasize human gestures from the input images for effective sign detection. The inference engine of SUGO is based on a 3-dimensional convolutional neural network (3DCNN) to classify a sequence of video frames as a pre-trained word. Furthermore, the overall operations are designed to be light-weight so that sign language translation takes place in real-time using only the resources available on a smartphone, with no help from cloud servers nor external sensing components. Specifically, to train and test SUGO, we collect sign language data from 20 individuals for 50 Korean Sign Language words, summing up to a dataset of ~5,000 sign gestures and collect additional in-the-wild data to evaluate the performance of SUGO in real-world usage scenarios with different lighting conditions and daily activities. Comprehensively, our extensive evaluations show that SUGO can properly classify sign words with an accuracy of up to 91% and also suggest that the system is suitable (in terms of resource usage, latency, and environmental robustness) to enable a fully mobile solution for sign language translation.

Development Of A First-Person View-Based Korean Sign Language Education System Using VIVE Hand Tracking

In this study, using VIVE and Tracking, we developed a ‘first-person view-based Korean sign language education system’ that guides sign language handshapes using a 3D virtual hand from the user's point of view and then evaluates the accuracy of the handshapes that learners perform. The learning effects of a Korean sign language education system that guides sign language handshapes using 3D virtual hands in a virtual space and a system that only learns with a teacher's lecture video in a virtual space were compared and analyzed. The number of times the two groups of participants tried to make the sign language correctly was measured. Then, the measurement results were compared and analyzed and interviews also conducted. As a result of the experiment, the ‘first-person view-based Korean sign language education system’ demonstrated that some sign language handshapes, which are not easy for participants to make, were implemented faster and more accurately than traditional learning methods. Therefore, if this system is used, it is expected that sign language learners will be able to learn more intuitively through the first-person perspective. Furthermore, it is expected that learning effects can be improved through the ability of being able to judge whether users are making the correct shapes with their hands. In addition, since it is possible to evaluate the accuracy of the hand shapes, it is believed that single-person learning, which was not easy in the existing sign language education paradigm, will become possible. However, for the sign language examined in the experiment, some showed significant differences while others were less meaningful. These were analyzed because the difficulty of the learning contents used in the experiment was not high. In future research, in order to more accurately verify the effectiveness of the sign language education system using hand tracking and 3D virtual hand guidance, we intend to conduct an experiment implementing learning content with more complex sentences

Classification of the Korean Sign Language Alphabet Using an Accelerometer with a Support Vector Machine

Recognition and understanding of sign language can aid communication between nondeaf and deaf people. Recently, research groups have developed sign language recognition algorithms using multiple sensors. However, in everyday life, minimizing the number of sensors would still require the use of a sign language interpreter. In this study, a sign language classification method was developed using an accelerometer to recognize the Korean sign language alphabet. The accelerometer is worn on the proximal phalanx of the index finger of the dominant hand. Triaxial accelerometer signals were used to segment the sign gesture (i.e., the time period when a user is performing a sign) and recognize the 31 Korean sign language letters (producing a chance level of 3.2%). The vector sum of the accelerometer signals was used to segment the sign gesture with 98.9% segmentation accuracy, which is comparable to that of previous multisensor systems (99.49%). The system was able to classify the Korean sign language alphabet with 92.2% accuracy. The recognition accuracy of this approach was found to be higher than that of a previous work in the same sign language alphabet classification task. The findings demonstrate that a single‐sensor accelerometer with simple features can be reliably used for Korean sign language alphabet recognition in everyday life.

A Study on the Forms and Characteristics of Korean Sign Language Translation According to Historical Changes

A Study on the Forms and Characteristics of Korean Sign Language Translation According to Historical Changes

Contrastive Linguistic Study of South and North Korean Sign Language and Japanese Sign Language at the Level of Phoneme and Lexis

Contrastive Linguistic Study of South and North Korean Sign Language and Japanese Sign Language at the Level of Phoneme and Lexis

한국수어 교육용 관용표현 연구

This study aims to: first, understand the current learning status of idiomatic expressions in Korean Sign Language(KSL) through the analysis of KSL textbooks published up till now by KSL Professional Institute, and secondly, propose a list of KSL educational idiomatic expressions. Accordingly we examined past literature and analysed KSL textbooks. As a result of the literature review, we identified that idiomatic expressions in KSL are not distinguished from modification of sign and polysemy. This is seen in the criteria of narrow sense of idiomatic expressions, that asserts more than one linguistic signs are not employed as a mere sum of meanings. This means that the modifications of signs and polysemy, both of which do not comply with the formal criteria of idiomatic expressions, are considered as idiomatic expressions. Next, as a conclusion of analysis of KSL textbooks, there were only three books which explicitly covered idiomatic expressions. Even the textbooks that introduced idiomatic expressions manifestly presented one word as idiomatic expressions, thereby violating the formal criteria for idiomatic expressions, defined as "phrases of two or more words." As per the time period when KSL idiomatic expressions were taught, they were introduced in basic level KSL textbooks, unlike the awareness that the idiomatic expressions were to be educated in advanced level; in few numbers though. We have also identified the phenomenon that the KSL idiomatic expression were few in narrow sense and more in the broad sense. Thus, we propose a list of KSL educational idiomatic expressions that includes both narrow and broad senses.