Latent-Dynamic Conditional Random Fields Research Articles

Factored Latent-Dynamic Conditional Random Fields for single and multi-label sequence modeling

Conditional Random Fields (CRF) are frequently applied for labeling and segmenting sequence data. Morency et al. (2007) introduced hidden state variables in a labeled CRF structure in order to model the latent dynamics within class labels, thus improving the labeling performance. Such a model is known as Latent-Dynamic CRF (LDCRF). We present Factored LDCRF (FLDCRF), a structure that allows multiple latent dynamics of the class labels to interact with each other. Including such latent-dynamic interactions leads to improved labeling performance on single-label and multi-label sequence modeling experiments across two different datasets, viz., UCI gesture phase data and UCI opportunity data. FLDCRF outperforms all state-of-the-art sequence models, viz., CRF, LDCRF, LSTM, LSTM-CRF, Factorial CRF, Coupled CRF and a multi-label LSTM model across experiments in this paper. In addition, FLDCRF offers easier model selection and is more consistent across validation and test data than LSTM models. FLDCRF is also much faster to train compared to LSTM, even without a GPU. FLDCRF outshines the best LSTM model by ∼4% on a single-label task on the UCI gesture phase data and outperforms LSTM models by ∼2% on average on the multi-label sequence tagging experiment on the UCI opportunity data.

Automatic detection of COVID-19 using pruned GLCM-Based texture features and LDCRF classification

Recently, automatic computer-aided detection (CAD) of COVID-19 using radiological images has received a great deal of attention from many researchers and medical practitioners, and consequently several CAD frameworks and methods have been presented in the literature to assist the radiologist physicians in performing diagnostic COVID-19 tests quickly, reliably and accurately. This paper presents an innovative framework for the automatic detection of COVID-19 from chest X-ray (CXR) images, in which a rich and effective representation of lung tissue patterns is generated from the gray level co-occurrence matrix (GLCM) based textural features. The input CXR image is first preprocessed by spatial filtering along with median filtering and contrast limited adaptive histogram equalization to improve the CXR image's poor quality and reduce image noise. Automatic thresholding by the optimized formula of Otsu's method is applied to find a proper threshold value to best segment lung regions of interest (ROIs) out from CXR images. Then, a concise set of GLCM-based texture features is extracted to accurately represent the segmented lung ROIs of each CXR image. Finally, the normalized features are fed into a trained discriminative latent-dynamic conditional random fields (LDCRFs) model for fine-grained classification to divide the cases into two categories: COVID-19 and non-COVID-19. The presented method has been experimentally tested and validated on a relatively large dataset of frontal CXR images, achieving an average accuracy, precision, recall, and F1-score of 95.88%, 96.17%, 94.45%, and 95.79%, respectively, which compare favorably with and occasionally exceed those previously reported in similar studies in the literature.

Retrieving Semantic Image Using Shape Descriptors and Latent-Dynamic Conditional Random Fields

Abstract This paper introduces a new approach to semantic image retrieval using shape descriptors as dispersion and moment in conjunction with discriminative classifier model of latent-dynamic conditional random fields (LDCRFs). The target region is firstly localized via the background subtraction model. Then the features of dispersion and moments are employed to k-means clustering to extract object’s feature as second stage. After that, the learning process is carried out by LDCRFs. Finally, simple protocol and RDF (resource description framework) query language (i.e. SPARQL) on input text or image query is to retrieve semantic image based on sequential processes of query engine, matching module and ontology manager. Experimental findings show that our approach can be successful to retrieve images against the mammal’s benchmark with retrieving rate of 98.11%. Such outcomes are likely to compare very positively with those accessible in the literature from other researchers.

Context Model for Pedestrian Intention Prediction Using Factored Latent-Dynamic Conditional Random Fields

Smooth handling of pedestrian interactions is a key requirement for Autonomous Vehicles (AV) and Advanced Driver Assistance Systems (ADAS). Such systems call for early and accurate prediction of a pedestrian's crossing/not-crossing behaviour in front of the vehicle. Existing approaches to pedestrian behaviour prediction make use of pedestrian motion, his/her location in a scene and static context variables such as traffic lights, zebra crossings etc. We stress on the necessity of early prediction for smooth operation of such systems. We introduce the influence of vehicle interactions on pedestrian intention for this purpose. In this paper, we show a discernible advance in prediction time aided by the inclusion of such vehicle interaction context. We apply our methods to two different datasets, one in-house collected - NTU dataset and another public real-life benchmark - JAAD dataset. We also propose a generic graphical model Factored Latent-Dynamic Conditional Random Fields (FLDCRF) for single and multi-label sequence prediction as well as joint interaction modeling tasks. FLDCRF outperforms Long Short-Term Memory (LSTM) networks across the datasets ($\sim$100 sequences per dataset) over identical time-series features. While the existing best system predicts pedestrian stopping behaviour with 70\% accuracy 0.38 seconds before the actual events, our system achieves such accuracy at least 0.9 seconds on an average before the actual events across datasets.

Robot Classification of Human Interruptibility and a Study of Its Effects

As robots become increasingly prevalent in human environments, there will inevitably be times when the robot needs to interrupt a human to initiate an interaction. Our work introduces the first interruptibility-aware mobile-robot system, which uses social and contextual cues online to accurately determine when to interrupt a person. We evaluate multiple non-temporal and temporal models on the interruptibility classification task, and show that a variant of Conditional Random Fields (CRFs), the Latent-Dynamic CRF, is the most robust, accurate, and appropriate model for use on our system. Additionally, we evaluate different classification features and show that the observed demeanor of a person can help in interruptibility classification; but in the presence of detection noise, robust detection of object labels as a visual cue to the interruption context can improve interruptibility estimates. Finally, we deploy our system in a large-scale user study to understand the effects of interruptibility-awareness on human-task performance, robot-task performance, and on human interpretation of the robot’s social aptitude. Our results show that while participants are able to maintain task performance, even in the presence of interruptions, interruptibility-awareness improves the robot’s task performance and improves participant social perceptions of the robot.

Fuzzy Latent-Dynamic Conditional Neural Fields for Gesture Recognition in Video

<p>With the explosion of data on the internet led to the presence of the big data era, so it requires data processing in order to get the useful information. One of the challenges is the gesture recognition the video processing. Therefore, this study proposes Latent-Dynamic Conditional Neural Fields and compares with the other family members of Conditional Random Fields. To improve the accuracy, these methods are combined by using Fuzzy Clustering. From the result, it can be concluded that the performance of Latent-Dynamic Conditional Neural Fields are lower than Conditional Neural Fields but higher than the Conditional Random Fields and Latent-Dynamic Conditional Random Fields. Also, the combination of Latent-Dynamic Conditional Neural Fields and Fuzzy C-Means Clustering has the highest. This evaluation is tested in a temporal dataset of gesture phase segmentation.</p>

Gesture Recognition using Latent-Dynamic based Conditional Random Fields and Scalar Features

The need for segmentation and labeling of sequence data appears in several fields. The use of the conditional models such as Conditional Random Fields is widely used to solve this problem. In the pattern recognition, Conditional Random Fields specify the possibilities of a sequence label. This method constructs its full label sequence to be a probabilistic graphical model based on its observation. However, Conditional Random Fields can not capture the internal structure so that Latent-based Dynamic Conditional Random Fields is developed without leaving external dynamics of inter-label. This study proposes the use of Latent-Dynamic Conditional Random Fields for Gesture Recognition and comparison between both methods. Besides, this study also proposes the use of a scalar features to gesture recognition. The results show that performance of Latent-dynamic based Conditional Random Fields is not better than the Conditional Random Fields, and scalar features are effective for both methods are in gesture recognition. Therefore, it recommends implementing Conditional Random Fields and scalar features in gesture recognition for better performance

A Discriminative Framework for Action Recognition Using f-HOL Features

Inspired by the overwhelming success of Histogram of Oriented Gradients (HOG) features in many vision tasks, in this paper, we present an innovative compact feature descriptor called fuzzy Histogram of Oriented Lines (f-HOL) for action recognition, which is a distinct variant of the HOG feature descriptor. The intuitive idea of these features is based on the observation that the slide area of the human body skeleton can be viewed as a spatiotemporal 3D surface, when observing a certain action being performed in a video. The f-HOL descriptor possesses an immense competitive advantage, not only of being quite robust to small geometric transformations where the small translation and rotations make no large fluctuations in histogram values, but also of not being very sensitive under varying illumination conditions. The extracted features are then fed into a discriminative conditional model based on Latent-Dynamic Conditional random fields (LDCRFs) to learn to recognize actions from video frames. When tested on the benchmark Weizmann dataset, the proposed framework substantially supersedes most existing state-of-the-art approaches, achieving an overall recognition rate of 98.2%. Furthermore, due to its low computational demands, the framework is properly amenable for integration into real-time applications.

Three Dimensional Binary Edge Feature Representation for Pain Expression Analysis.

Automatic pain expression recognition is a challenging task for pain assessment and diagnosis. Conventional 2D-based approaches to automatic pain detection lack robustness to the moderate to large head pose variation and changes in illumination that are common in real-world settings and with few exceptions omit potentially informative temporal information. In this paper, we propose an innovative 3D binary edge feature (3D-BE) to represent high-resolution 3D dynamic facial expression. To exploit temporal information, we apply a latent-dynamic conditional random field approach with the 3D-BE. The resulting pain expression detection system proves that 3D-BE represents the pain facial features well, and illustrates the potential of noncontact pain detection from 3D facial expression data.

Latent-Dynamic Conditional Random Fields for recognizing activities in smart homes

As the number of elderly people in our society increases, the need of assistive technologies in home becomes urgent. Existing techniques allow elderly people to be better assisted through monitoring what goes on in smart homes and inferring their activities from sensor data via a recognition model. However, there are various cases that existing models have difficulties in accommodating relational data. In this paper, we present an application of probabilistic graphical model --Latent-Dynamic Conditional Random Field --to detect the goals of the individual subjects when observations have long range dependencies or multiple overlapping features. To validate the proposed method, we apply it to recognize activities in two different datasets which were collected in smart homes. The results demonstrate that Latent-Dynamic Conditional Random Fields favorably outperform other models, especially when there are extrinsic dynamic activities changes and intrinsic actions subactivities.

Language Recognition Using Latent Dynamic Conditional Random Field Model with Phonological Features

Spoken language recognition (SLR) has been of increasing interest in multilingual speech recognition for identifying the languages of speech utterances. Most existing SLR approaches apply statistical modeling techniques with acoustic and phonotactic features. Among the popular approaches, the acoustic approach has become of greater interest than others because it does not require any prior language-specific knowledge. Previous research on the acoustic approach has shown less interest in applying linguistic knowledge; it was only used as supplementary features, while the current state-of-the-art system assumes independency among features. This paper proposes an SLR system based on the latent-dynamic conditional random field (LDCRF) model using phonological features (PFs). We use PFs to represent acoustic characteristics and linguistic knowledge. The LDCRF model was employed to capture the dynamics of the PFs sequences for language classification. Baseline systems were conducted to evaluate the features and methods including Gaussian mixture model (GMM) based systems using PFs, GMM using cepstral features, and the CRF model using PFs. Evaluated on the NIST LRE 2007 corpus, the proposed method showed an improvement over the baseline systems. Additionally, it showed comparable result with the acoustic system based oni-vector. This research demonstrates that utilizing PFs can enhance the performance.

Latent-Dynamic Conditional Random Fields for recognizing activities in smart homes

As the number of elderly people in our society increases, the need of assistive technologies in home becomes urgent. Existing techniques allow elderly people to be better assisted through monitorin...

Human behavior recognition based on stratified fractal conditional random field

In view of real-time issue of the Hidden Conditional Random Field(HCRF) and marked deviation problem of the Latent-Dynamic Conditional Random Field(LDCRF) during behavior transforming,this article proposed a kind of behavior recognition algorithm based on Stratified Fractal Conditional Random Field(SFCRF).The proposed algorithm improved LDCRF and put forward the concept of score mark,which made the integrity and direction of human behavior specific.The experimental results show that the proposed algorithm can obtain better recognition effect than Conditional Random Field(CRF),HCRF and LDCRF.

Sketched symbol recognition using Latent-Dynamic Conditional Random Fields and distance-based clustering

In this paper we propose a two-stage method for recognizing sketched symbols that combine the use of a discriminative model, for labeling symbol strokes and a distance-based clustering algorithm, for grouping the labels belonging to the same symbol. In the first stage, we employ Latent-Dynamic Conditional Random Field (LDCRF), a discriminative model able to analyze the features of unsegmented sequences of strokes by taking into account spatio-temporal information, and to classify the symbol parts by considering contextual information. In the second stage, the labels obtained from LDCRF are grouped into symbol labels by using a distance-based clustering algorithm which takes into account the geometric relationships among strokes. The effectiveness of our method has been evaluated on the domain of electric circuit diagrams achieving accuracy values varying between 81.3% and 91.0%.

Discriminative Methods for Classification of Asynchronous Imaginary Motor Tasks From EEG Data

In this work, two methods based on statistical models that take into account the temporal changes in the electroencephalographic (EEG) signal are proposed for asynchronous brain-computer interfaces (BCI) based on imaginary motor tasks. Unlike the current approaches to asynchronous BCI systems that make use of windowed versions of the EEG data combined with static classifiers, the methods proposed here are based on discriminative models that allow sequential labeling of data. In particular, the two methods we propose for asynchronous BCI are based on conditional random fields (CRFs) and latent dynamic CRFs (LDCRFs), respectively. We describe how the asynchronous BCI problem can be posed as a classification problem based on CRFs or LDCRFs, by defining appropriate random variables and their relationships. CRF allows modeling the extrinsic dynamics of data, making it possible to model the transitions between classes, which in this context correspond to distinct tasks in an asynchronous BCI system. On the other hand, LDCRF goes beyond this approach by incorporating latent variables that permit modeling the intrinsic structure for each class and at the same time allows modeling extrinsic dynamics. We apply our proposed methods on the publicly available BCI competition III dataset V as well as a data set recorded in our laboratory. Results obtained are compared to the top algorithm in the BCI competition as well as to methods based on hierarchical hidden Markov models (HHMMs), hierarchical hidden CRF (HHCRF), neural networks based on particle swarm optimization (IPSONN) and to a recently proposed approach based on neural networks and fuzzy theory, the S-dFasArt. Our experimental analysis demonstrates the improvements provided by our proposed methods in terms of classification accuracy.

Fuzzy-based latent-dynamic conditional random fields for continuous gesture recognition

We show an original method for automatic hand gesture recognition that makes use of fuzzified latent-dynamic conditional random fields (LDCRF). In this method, fuzzy linguistic variables are used to model the features of hand gestures and then to modify the potential function in LDCRFs. By combining LDCRFs and fuzzy sets, these fuzzy-based LDCRFs (FLDCRF) have the advantages of LDCRFs in sequence labeling along with the advantage of retaining the imprecise character of gestures. The efficiency of the proposed method was tested with unsegmented gesture sequences in three different hand gesture data sets. The experimental results demonstrate that FLDCRFs compare favorably with support vector machines, hidden conditional random fields, and LDCRFs on hand gesture recognition tasks.

Continuous body and hand gesture recognition for natural human-computer interaction

Intelligent gesture recognition systems open a new era of natural human-computer interaction: Gesturing is instinctive and a skill we all have, so it requires little or no thought, leaving the focus on the task itself, as it should be, not on the interaction modality. We present a new approach to gesture recognition that attends to both body and hands, and interprets gestures continuously from an unsegmented and unbounded input stream. This article describes the whole procedure of continuous body and hand gesture recognition, from the signal acquisition to processing, to the interpretation of the processed signals. Our system takes a vision-based approach, tracking body and hands using a single stereo camera. Body postures are reconstructed in 3D space using a generative model-based approach with a particle filter, combining both static and dynamic attributes of motion as the input feature to make tracking robust to self-occlusion. The reconstructed body postures guide searching for hands. Hand shapes are classified into one of several canonical hand shapes using an appearance-based approach with a multiclass support vector machine. Finally, the extracted body and hand features are combined and used as the input feature for gesture recognition. We consider our task as an online sequence labeling and segmentation problem. A latent-dynamic conditional random field is used with a temporal sliding window to perform the task continuously. We augment this with a novel technique called multilayered filtering, which performs filtering both on the input layer and the prediction layer. Filtering on the input layer allows capturing long-range temporal dependencies and reducing input signal noise; filtering on the prediction layer allows taking weighted votes of multiple overlapping prediction results as well as reducing estimation noise. We tested our system in a scenario of real-world gestural interaction using the NATOPS dataset, an official vocabulary of aircraft handling gestures. Our experimental results show that: (1) the use of both static and dynamic attributes of motion in body tracking allows statistically significant improvement of the recognition performance over using static attributes of motion alone; and (2) the multilayered filtering statistically significantly improves recognition performance over the nonfiltering method. We also show that, on a set of twenty-four NATOPS gestures, our system achieves a recognition accuracy of 75.37%.

Chinese new word identification: a latent discriminative model with global features

Chinese new words are particularly problematic in Chinese natural language processing. With the fast development of Internet and information explosion, it is impossible to get a complete system lexicon for applications in Chinese natural language processing, as new words out of dictionaries are always being created. The procedure of Hew words identification and POS tagging are usually separated and the features of lexical information cannot be fully used. A latent discriminative model, which combines the strengths of Latent Dynamic Conditional Random Field (LDCRF) and semi-CRF, is proposed to detect new words together with their POS synchronously regardless of the types of new words from Chinese text without being pre-segmented. Unlike semi-CRF, in proposed latent discriminative model, LDCRF is applied 10 generate candidate entities, which accelerates the training speed and decreases the computational cost. The complexity of proposed hidden semi-CRF could be further adjusted by tuning the number of hidden variables and the number of candidate entities from the Nbest outputs of LDCRF model. A new-word-generating framework is proposed for model training and testing, under which the definitions and distributions of new words conform to the ones in real text. The global feature called "Global Fragment Features" for new word identification is adopted. We tested our model on the corpus from SIGHAN-6. Experimental results show that the proposed method is capable of detecting even low frequency new words together with their POS tags with satisfactory results. The proposed model performs competitively with the state-of-the-art models.

Detecting New Words from Chinese Text Using Latent Semi-CRF Models

Chinese new words and their part-of-speech (POS) are particularly problematic in Chinese natural language processing. With the fast development of internet and information technology, it is impossible to get a complete system dictionary for Chinese natural language processing, as new words out of the basic system dictionary are always being created. A latent semi-CRF model, which combines the strengths of LDCRF (Latent-Dynamic Conditional Random Field) and semi-CRF, is proposed to detect the new words together with their POS synchronously regardless of the types of the new words from the Chinese text without being pre-segmented. Unlike the original semi-CRF, the LDCRF is applied to generate the candidate entities for training and testing the latent semi-CRF, which accelerates the training speed and decreases the computation cost. The complexity of the latent semi-CRF could be further adjusted by tuning the number of hidden variables in LDCRF and the number of the candidate entities from the Nbest outputs of the LDCRF. A new-words-generating framework is proposed for model training and testing, under which the definitions and distributions of the new words conform to the ones existing in real text. Specific features called “Global Fragment Information” for new word detection and POS tagging are adopted in the model training and testing. The experimental results show that the proposed method is capable of detecting even low frequency new words together with their POS tags. The proposed model is found to be performing competitively with the state-of-the-art models presented.