Sequential Data Classification Research Articles

Classification of Sequential Data in Deep Learning Using LSTM Network

Classification of Sequential Data in Deep Learning Using LSTM Network

Sequential Data Classification under Dynamic Emission

Sequential Data Classification under Dynamic Emission

Predicting regulatory activities for socially shared regulation to optimize collaborative learning

This study utilized multimodal learning analytics and AI-based methods to examine the patterns of the socially shared regulation of collaborative learning (CL). The study involved multimodal data involving video and electrodermal activities (EDA) data collected from ninety-four secondary school students (N = 94) during five science lessons to reveal trigger events in CL. A novel concept of trigger events is introduced, which are challenging events and/or situations that may inhibit collaboration and will, therefore, require strategic adaptation in the regulation of cognition, motivation, and emotion within the group. The ANOVA results for the Skin Conductance Responses (SCRs) analysis indicated the disparity of physiological behavior activated in relation to different types of interactions for regulation. Process analysis and episode-rule mining were applied to reveal regulatory patterns in CL, while an AI approach with long short-term memory (LSTM) deep-learning networks were designed for pattern prediction. LSTM has emerged as the most widely applied artificial recurrent neural network (RNN) architecture for sequential data analysis and classification. The proposed AI network holds the potential for designing solutions for similar signal-processing problems in studying learning regulation. This study contributes to developing AI-enabled real-time support for regulation in collaborative learning.

Deep parameterizations of pairwise and triplet Markov models for unsupervised classification of sequential data

Hidden Markov models are probabilistic graphical models based on hidden and observed random variables. They are popular to address classification tasks for time series applications such as part-of-speech tagging, image segmentation, genetic sequence analysis. Direct extensions of these models, the pairwise and triplet Markov models, are considered. These models aim at relaxing the assumptions underlying the hidden Markov chain by extending the direct dependencies of the involved random variables or by considering the addition of a third latent process. While these extensions define interesting modeling capabilities that have been little explored so far, they also raise new problems such as defining the nature of their core probability distributions and their parameterization. Once the model is fixed, the unsupervised classification task (i.e. the estimation of the parameters and next of the hidden random variables) is a critical problem. These challenges are addressed, first it is shown that it is possible to embed recent deep neural networks in these models in order to exploit their full modeling power. Second, a continuous latent process in triplet Markov chains is considered. The latter aims at estimating the nature of the joint distributions of the hidden and observed random variables, in addition to their parameters. The introduction of such a continuous auxiliary latent process also offers a new way to model continuous non-stationarity in hidden Markov models. For each model that is introduced, an original unsupervised Bayesian estimation method is proposed. In particular, it takes into account the interpretability of the hidden random variables in terms of signal processing classification. Through unsupervised classification problems on synthetic and real data, it is shown that the new models outperform hidden Markov chains and their classical extensions.

Classification of long sequential data using circular dilated convolutional neural networks

Classification of long sequential data is an important Machine Learning task and appears in many application scenarios. Recurrent Neural Networks, Transformers, and Convolutional Neural Networks are three major techniques for learning from sequential data. Among these methods, Temporal Convolutional Networks (TCNs) which are scalable to very long sequences have achieved remarkable progress in time series regression. However, the performance of TCNs for sequence classification is not satisfactory because they use a skewed connection protocol and output classes at the last position. Such asymmetry restricts their performance for classification which depends on the whole sequence. In this work, we propose a symmetric multi-scale architecture called Circular Dilated Convolutional Neural Network (CDIL-CNN), where every position has an equal chance to receive information from other positions at the previous layers. Our model gives classification logits in all positions, and we can apply a simple ensemble learning to achieve a better decision. We have tested CDIL-CNN on various long sequential datasets. The experimental results show that our method has superior performance over many state-of-the-art approaches. The model and experiments are available at (https://github.com/LeiCheng-no/CDIL-CNN).

Deep learning-based networks for automated recognition and classification of awkward working postures in construction using wearable insole sensor data

Among the numerous work-related risk factors, construction workers are often exposed to awkward working postures that may lead them to develop work-related musculoskeletal disorders (WMSDs). To mitigate WMSDs among construction workers, awkward working posture recognition is the first step in proactive WMSD prevention. Several researchers have proposed wearable sensor-based systems and machine learning classifiers for awkward posture recognition. However, these wearable sensor-based systems (e.g., surface electromyography) are either intrusive or require attaching multiple sensors on workers' bodies, which may lead to workers' discomfort and systemic instability, thus, limiting their application on construction sites. In addition, machine learning classifiers are limited to human-specific shallow features which influence model performance. To address these limitations, this study proposes a novel approach by using wearable insole pressure system and recurrent neural network (RNN) models, which automate feature extraction and are widely used for sequential data classification. Therefore, the research objective is to automatically recognize and classify different types of awkward working postures in construction by using deep learning-based networks and wearable insole sensor data. The classification performance of three RNN-based deep learning models, namely: (1) long-short term memory (LSTM), (2) bidirectional LSTM (Bi-LSTM), and (3) gated recurrent units (GRU), was evaluated using plantar pressure data captured by a wearable insole system from workers on construction sites. The experimental results show that GRU model outperforms the other RNN-based deep learning models with a high accuracy of 99.01% and F1-score between 93.19% and 99.39%. These results demonstrate that GRU models can be employed to learn sequential plantar pressure patterns captured by a wearable insole system to recognize and classify different types of awkward working postures. The findings of this study contribute to wearable sensor-based posture-related recognition and classification, thus, enhancing construction workers' health and safety.

Sequence Classification Restricted Boltzmann Machines With Gated Units

For the classification of sequential data, dynamic Bayesian networks and recurrent neural networks (RNNs) are the preferred models. While the former can explicitly model the temporal dependences between the variables, and the latter have the capability of learning representations. The recurrent temporal restricted Boltzmann machine (RTRBM) is a model that combines these two features. However, learning and inference in RTRBMs can be difficult because of the exponential nature of its gradient computations when maximizing log likelihoods. In this article, first, we address this intractability by optimizing a conditional rather than a joint probability distribution when performing sequence classification. This results in the "sequence classification restricted Boltzmann machine" (SCRBM). Second, we introduce gated SCRBMs (gSCRBMs), which use an information processing gate, as an integration of SCRBMs with long short-term memory (LSTM) models. In the experiments reported in this article, we evaluate the proposed models on optical character recognition, chunking, and multiresident activity recognition in smart homes. The experimental results show that gSCRBMs achieve the performance comparable to that of the state of the art in all three tasks. gSCRBMs require far fewer parameters in comparison with other recurrent networks with memory gates, in particular, LSTMs and gated recurrent units (GRUs).

An Application of Deep Learning to Tactile Data for Object Recognition under Visual Guidance.

Drawing inspiration from haptic exploration of objects by humans, the current work proposes a novel framework for robotic tactile object recognition, where visual information in the form of a set of visually interesting points is employed to guide the process of tactile data acquisition. Neuroscience research confirms the integration of cutaneous data as a response to surface changes sensed by humans with data from joints, muscles, and bones (kinesthetic cues) for object recognition. On the other hand, psychological studies demonstrate that humans tend to follow object contours to perceive their global shape, which leads to object recognition. In compliance with these findings, a series of contours are determined around a set of 24 virtual objects from which bimodal tactile data (kinesthetic and cutaneous) are obtained sequentially and by adaptively changing the size of the sensor surface according to the object geometry for each object. A virtual Force Sensing Resistor array (FSR) is employed to capture cutaneous cues. Two different methods for sequential data classification are then implemented using Convolutional Neural Networks (CNN) and conventional classifiers, including support vector machines and k-nearest neighbors. In the case of conventional classifiers, we exploit contourlet transformation to extract features from tactile images. In the case of CNN, two networks are trained for cutaneous and kinesthetic data and a novel hybrid decision-making strategy is proposed for object recognition. The proposed framework is tested both for contours determined blindly (randomly determined contours of objects) and contours determined using a model of visual attention. Trained classifiers are tested on 4560 new sequential tactile data and the CNN trained over tactile data from object contours selected by the model of visual attention yields an accuracy of 98.97% which is the highest accuracy among other implemented approaches.

Correlation Analysis-Based Classification of Human Activity Time Series

Segmentation of sequential sensor data streams and classification of each segment are common steps in tasks dealing with the detection of events of interest in such data. In this paper, we introduce two correlation analysis-based methods for classifying time series data generated by sensors. Our first method is a lightweight supervised approach utilizing principal component analysis to jointly segment data and classify each segment into a class corresponding to an event of interest. The second method relies on unsupervised canonical correlation analysis to segment time series by clustering together consecutive data points that belong to the same event. Both methods operate without the need for prior feature extraction from the data. The theoretical model of the two methods and the solution to the resulting optimization problem are presented in detail. Classification of human activity from inertial measurement unit sensor data is used as a case study to demonstrate the applicability and effectiveness of the proposed methods.

Chaos sequence data classification mining research under network visualization analysis

ABSTRACTNetwork visualization analysis is a method of quantitative analysis of data in network services by using visualization techniques. However, it is difficult to classify the multimodal underlying sequence characteristics of the data by using the current method for the chaos sequence data classification mining; there is a problem of the large error of the classification of chaotic sequence data. In the article, this method is a method of network visualization analysis of chaotic sequence data classification mining based on multimodal subspace correlation transmission. This method uses the hidden Markov model to convert the chaotic sequence data into the likelihood space, the magnitude of the likelihood is identified by the symmetry Kullback–Leibler distance, extracting the underlying characteristics of multiple modal states, and carry out the transmission of the correlation between video data multimodal subspace. The accuracy and the mean square error were used to define the performance of the chaotic sequence data classification. The classification accuracy of this method is about 90%, and the root mean square error is less than 0.03, which greatly improves the classification accuracy. The packet loss rate of literature [Zheng B, Su H, Lin L. Application of unsupervised feature selection in time series data mining. Chin J Sci Instrum. 2014;35(4):834–840] is about 60%, the missing rate of literature [Mengli R. Algorithm of web hot data mining based on structured segmentation. Bull Sci Technol. 2015;4:115–117] is 50%, and the packet loss rate of this method is about 20%, which proves that this method has higher classification accuracy and application value, and is more suitable for other sequence data mining applications. The similarity between the time sequence data is given and reduces the dimension of the original data in order to obtain the coordinates module in the lower dimensional sequence data space, and then complete network visualization analysis to analyze chaos sequence data mining. The experimental simulation shows that the proposed method has a higher classification precision and greatly expands the application domain of the sequential data classification mining.

Reconstruct Recurrent Neural Networks via Flexible Sub-Models for Time Series Classification

Recurrent neural networks (RNNs) remain challenging, and there is still a lack of long-term memory or learning ability in sequential data classification and prediction. In this paper, we propose a flexible recurrent model, BIdirectional COnvolutional RaNdom RNNs (BICORN-RNNs), incorporating a series of sub-models: random projection, convolutional operation, and bidirectional transmission. These subcategories advance classification accuracy, which was limited by the gradient vanishing and the exploding problem. Experiments on public time series datasets demonstrate that our proposed method substantially outperforms a variety of existing models. Furthermore, the coordination of the accuracy and efficiency concerning a variety of factors, including SNR, length, data missing, and overlapping, is also discussed.

Sequential data classification by dynamic state warping

The ubiquity of sequences in many domains enhances significant recent interest in sequence learning, for which a basic problem is how to measure the distance between sequences. Dynamic time warping (DTW) aligns two sequences by nonlinear local warping and returns a distance value. DTW shows superior ability in many applications, e.g. video, image, etc. However, in DTW, two points are paired essentially based on point-to-point comparisons without considering the autocorrelation of sequences. Thus, points with different semantic meanings, e.g. peaks and valleys, may be matched providing their coordinate values are similar. As a result, DTW may be sensitive to noise and poorly interpretable. This paper proposes an improved alignment method, dynamic state warping (DSW). DSW integrates the dynamic information of sequences into DTW by converting each time point into a latent state. Alignment is performed by using the state sequences. Thus, DSW is able to yield alignment that is semantically more interpretable than that of DTW. Using one nearest neighbour classifier, DSW shows significant improvement on classification accuracy in comparison with Euclidean distance (68/85 wins), DTW (70/85 wins) and its variants. We also empirically demonstrate that DSW is more robust and scales better to long sequences than Euclidean distance and DTW.

An Event Group Based Classification Framework for Multi-variate Sequential Data

Decision tree algorithms were not traditionally considered for sequential data classification, mostly because feature generation needs to be integrated with the modelling procedure in order to avoid a localisation problem. This paper presents an Event Group Based Classification (EGBC) framework that utilises an X-of-N (XoN) decision tree algorithm to avoid the feature generation issue during the classification on sequential data. In this method, features are generated independently based on the characteristics of the sequential data. Subsequently an XoN decision tree is utilised to select and aggregate useful features from various temporal and other dimensions (as event groups) for optimised classification. This leads the EGBC framework to be adaptive to sequential data of differing dimensions, robust to missing data and accommodating to either numeric or nominal data types. The comparatively improved outcomes from applying this method are demonstrated on two distinct areas – a text based language identification task, as well as a honeybee dance behaviour classification problem. A further motivating industrial problem – hot metal temperature prediction, is further considered with the EGBC framework in order to address significant real-world demands.

Sequential Labeling With Structural SVM Under Nondecomposable Losses.

Sequential labeling addresses the classification of sequential data, which are widespread in fields as diverse as computer vision, finance, and genomics. The model traditionally used for sequential labeling is the hidden Markov model (HMM), where the sequence of class labels to be predicted is encoded as a Markov chain. In recent years, HMMs have benefited from minimum-loss training approaches, such as the structural support vector machine (SSVM), which, in many cases, has reported higher classification accuracy. However, the loss functions available for training are restricted to decomposable cases, such as the 0-1 loss and the Hamming loss. In many practical cases, other loss functions, such as those based on the $F_{1}$ measure, the precision/recall break-even point, and the average precision (AP), can describe desirable performance more effectively. For this reason, in this paper, we propose a training algorithm for SSVM that can minimize any loss based on the classification contingency table, and we present a training algorithm that minimizes an AP loss. Experimental results over a set of diverse and challenging data sets (TUM Kitchen, CMU Multimodal Activity, and Ozone Level Detection) show that the proposed training algorithms achieve significant improvements of the $F_{1}$ measure and AP compared with the conventional SSVM, and their performance is in line with or above that of other state-of-the-art sequential labeling approaches.

AdOn HDP-HMM: An Adaptive Online Model for Segmentation and Classification of Sequential Data.

Recent years have witnessed an increasing need for the automated classification of sequential data, such as activities of daily living, social media interactions, financial series, and others. With the continuous flow of new data, it is critical to classify the observations on-the-fly and without being limited by a predetermined number of classes. In addition, a model should be able to update its parameters in response to a possible evolution in the distributions of the classes. This compelling problem, however, does not seem to have been adequately addressed in the literature, since most studies focus on offline classification over predefined class sets. In this paper, we present a principled solution for this problem based on an adaptive online system leveraging Markov switching models and hierarchical Dirichlet process priors. This adaptive online approach is capable of classifying the sequential data over an unlimited number of classes while meeting the memory and delay constraints typical of streaming contexts. In this paper, we introduce an adaptive "learning rate" that is responsible for balancing the extent to which the model retains its previous parameters or adapts to new observations. Experimental results on stationary and evolving synthetic data and two video data sets, TUM Assistive Kitchen and collated Weizmann, show a remarkable performance in terms of segmentation and classification, particularly for sequences from evolutionary distributions and/or those containing previously unseen classes.

Sequential Classification of Palm Gestures Based on A* Algorithm and MLP Neural Network for Quadrocopter Control

Abstract This paper presents an alternative approach to the sequential data classification, based on traditional machine learning algorithms (neural networks, principal component analysis, multivariate Gaussian anomaly detector) and finding the shortest path in a directed acyclic graph, using A* algorithm with a regression-based heuristic. Palm gestures were used as an example of the sequential data and a quadrocopter was the controlled object. The study includes creation of a conceptual model and practical construction of a system using the GPU to ensure the realtime operation. The results present the classification accuracy of chosen gestures and comparison of the computation time between the CPU- and GPU-based solutions.

Multi-label methods for prediction with sequential data

The number of methods available for classification of multi-label data has increased rapidly over recent years, yet relatively few links have been made with the related task of classification of sequential data. If labels indices are considered as time indices, the problems can often be seen as equivalent. In this paper we detect and elaborate on connections between multi-label methods and Markovian models, and study the suitability of multi-label methods for prediction in sequential data. From this study we draw upon the most suitable techniques from the area and develop two novel competitive approaches which can be applied to either kind of data. We carry out an empirical evaluation investigating performance on real-world sequential-prediction tasks: electricity demand, and route prediction. As well as showing that several popular multi-label algorithms are in fact easily applicable to sequencing tasks, our novel approaches, which benefit from a unified view of these areas, prove very competitive against established methods.

Glove-Based Continuous Arabic Sign Language Recognition in User-Dependent Mode

In this paper, we propose a glove-based Arabic sign language recognition system using a novel technique for sequential data classification. We compile a sensor-based dataset of 40 sentences using an 80-word lexicon. In the dataset, hand movements are captured using two DG5-VHand data gloves. Data labeling is performed using a camera to synchronize hand movements with their corresponding sign language words. Low-complexity preprocessing and feature extraction techniques are applied to capture and emphasize the temporal dependence of the data. Subsequently, a Modified k-Nearest Neighbor (MKNN) approach is used for classification. The proposed MKNN makes use of the context of feature vectors for the purpose of accurate classification. The proposed solution achieved a sentence recognition rate of 98.9%. The results are compared against an existing vision-based approach that uses the same set of sentences. The proposed solution is superior in terms of classification rates while eliminating restrictions of vision-based systems.

Echo-State Conditional Restricted Boltzmann Machines

Restricted Boltzmann machines (RBMs) are a powerful generative modeling technique, based on a complex graphical model of hidden (latent) variables. Conditional RBMs (CRBMs) are an extension of RBMs tailored to modeling temporal data. A drawback of CRBMs is their consideration of linear temporal dependencies, which limits their capability to capture complex temporal structure. They also require many variables to model long temporal dependencies, a fact that might provoke overfitting proneness. To resolve these issues, in this paper we propose the echo-state CRBM (ES-CRBM): our model uses an echo-state network reservoir in the context of CRBMs to efficiently capture long and complex temporal dynamics, with much fewer trainable parameters compared to conventional CRBMs. In addition, we introduce an (implicit) mixture of ES-CRBM experts (im-ES-CRBM) to enhance even further the capabilities of our ES-CRBM model. The introduced im-ES-CRBM allows for better modeling temporal observations which might comprise a number of latent or observable subpatterns that alternate in a dynamic fashion. It also allows for performing sequence segmentation using our framework. We apply our methods to sequential data modeling and classification experiments using public datasets. As we show, our approach outperforms both existing RBM-based approaches as well as related state-of-the-art methods, such as conditional random fields.

Joint Action Segmentation and Classification by an Extended Hidden Markov Model

Hidden Markov models (HMMs) provide joint segmentation and classification of sequential data by efficient inference algorithms and have therefore been employed in fields as diverse as speech recognition, document processing, and genomics. However, conventional HMMs do not suit action segmentation in video due to the nature of the measurements which are often irregular in space and time, high dimensional and affected by outliers. For this reason, in this paper we present a joint action segmentation and classification approach based on an extended model: the hidden Markov model for multiple, irregular observations (HMM-MIO). Experiments performed over a concatenated version of the popular KTH action dataset and the challenging CMU multi-modal activity dataset (CMU-MMAC) report accuracies comparable to or higher than those of a bag-of-features approach, showing the usefulness of improved sequential models for joint action segmentation and classification tasks.