Sign Language Data Research Articles

Use of unlabeled time series data in hidden Markov models

AbstractIn this paper, the authors propose a method of assigning probabilistic class labels to unlabeled time‐series data that can be collected inexpensively to use together with labeled data as training data in order to improve the performance of hidden Markov models (HMM) as classifiers. Conventional methods of using unlabeled data according to probabilistic class labels were only for static data. They could not deal with time‐series data, which are used with HMMs. Therefore, the authors extended the conventional methods as follows. First, they formed extended tied‐mixture hidden Markov models (ETM‐HMMs) by mixing multiple HMMs that exist independently for each class. Then, they introduced the extended Baum‐Welch (EBW) algorithm as the learning algorithm from time‐series data in which labeled and unlabeled data coexist in the ETM‐HMMs. The authors then assumed a situation in which there existed insufficient labeled training data and used the proposed method to perform classification experiments for Japanese sign language data and voice data. When unlabeled data were added to the labeled training data according to the proposed method, the classification error rate for Japanese sign language signs improved from 38.7% to 30.4%. Also, by making the model more detailed in addition to adding unlabeled data, the classification error rate for phonemes improved from 69.1% to 52.9%. © 2003 Wiley Periodicals, Inc. Syst Comp Jpn, 34(13): 1–12, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10537

Fast Clustering for Time-series Data with Average-time-sequence-vector Generation Based on Dynamic Time Warping.

This paper proposes a fast clustering method for time-series data based on average time sequence vector. A clustering procedure based on an exhaustive search method is time-consuming although its result typically exhibits high quality. BIRCH, which reduces the number of examples by data squashing based on a data structure CF (Clustering Feature) tree, represents an effective solution for such a method when the data set consists of numerical attributes only. For time-series data, however, a straightforward application of BIRCH based on a Euclidean distance for a pair of sequences, miserably fails since such a distance typically differs from human's perception. A dissimilarity measure based on DTW (Dynamic Time Warping) is desirable, but to the best of our knowledge no methods have been proposed for time-series data in the context of data squashing. In order to circumvent this problem, we propose DTWS (Dynamic Time Warping Squashed) tree, which employs a dissimilarity measure based on DTW, and compresses time sequences to the average time sequence vector. An average time sequence vector is obtained by a novel procedure which estimates correct shrinkage of a result of DTW. Experiments using the Australian sign language data demonstrate the superiority of the proposed method in terms of correctness of clustering, while its degradation of time efficiency is negligible.

Development of Web-based Distributed Cooperative Development Environment of Sign-Language Animation System and its Evaluation

A web-based distributed cooperative development environment of sign-language animation system has been developed. We have extended the system from the previous animation system that was constructed as three tiered system which consists of sign-language animation interface layer, sign-language data processing layer, and sign-language animation database. Two components of a web client using VRML plug-in and web servlet are added to the previous system. The systems can support humanoid-model avatar for interoperability, and can use the stored sign language animation data shared on the database. It is noted in the evaluation of this system that the inverse kinematics function of web client improves the sign-language animation making.

Spatial and temporal coding of Nicaraguan Sign Language in MediaTagger

We recently began using the new multimedia tool MediaTagger to transcribe and code Nicaraguan Sign Language. Previously, this work had been conducted using glosses and coding spreadsheets that referred to analog video timecode. MediaTagger offers many advantages over analog videotape and glossing: it allows immediate access to tagged video segments, and enables the linking of those segments directly to the relevant transcription and coding. It does inherit some of the drawbacks of glossing, drawbacks that are shared with other tools available. In particular, the temporal order of signs takes priority over their spatial location. This bias in all of our tools may make us inadvertently attend to time more than space in our analyses. However, a new location-tagging feature may prove to overcome much of this imbalance. Overall, we find that MediaTagger provides us with a powerful and convenient tool for viewing and coding sign language data.

Spatial and temporal coding of Nicaraguan Sign Language in MediaTagger

We recently began using the new multimedia tool MediaTagger to transcribe and code Nicaraguan Sign Language. Previously, this work had been conducted using glosses and coding spreadsheets that referred to analog video timecode. MediaTagger offers many advantages over analog videotape and glossing: it allows immediate access to tagged video segments, and enables the linking of those segments directly to the relevant transcription and coding. It does inherit some of the drawbacks of glossing, drawbacks that are shared with other tools available. In particular, the temporal order of signs takes priority over their spatial location. This bias in all of our tools may make us inadvertently attend to time more than space in our analyses. However, a new location-tagging feature may prove to overcome much of this imbalance. Overall, we find that MediaTagger provides us with a powerful and convenient tool for viewing and coding sign language data.