Classification Of Video Sequences Research Articles

Recognizing human actions with multiple Fourier transforms

Abstract In this paper we present an approach for action recognition that uses various applications of Fourier transform. The main idea is to classify video sequences based on action representations obtained using shape descriptors. For shape representation we use the Two-Dimensional Fourier Descriptor, Generic Fourier Descriptor and UNL-Fourier Descriptor. For each sequence of binary silhouettes we derive a set of shape descriptors and match the descriptor of the first frame with the rest of descriptors to obtain a vector of similarities (correlation coefficients or C1 correlations) or dissimilarities (Euclidean distances). Then normalized vectors are transformed into action representations using discrete Fourier transform, power spectral density estimate or a combination of both. Classification is performed using leave-one-out cross-validation and various matching measures. Additionally, we incorporate a coarse classification step that distinguish between actions performed in place and actions with changing location of a silhouette. Extensive experiments are carried out to investigate all possible combinations of selected processing steps, and experimental results are promising.

The choice of methods for the construction of PCA-based features and the selection of SVM parameters for person identification by gait

The paper presents the results of the continued research of the principal component analysis (PCA) and support vector machine (SVM) techniques for person identification by gait. The experimental studies performed using the CASIA GAIT dataset allowed us to compare two methods of work with input data for PCA. According to the results of experiments, the optimal method of forming the feature space and the most effective parameters of the SVM-classifier were selected. The classification of video sequences recorded by video cameras located frontally, at an angle and orthogonal to the direction of objects movement was carried out.

QoE Multi-Stage Machine Learning for Dynamic Video Streaming

The rapid growth of video traffic in cellular networks is a crucial issue to be addressed by mobile operators. An emerging and promising trend in this regard is the development of solutions that aim at maximizing the quality of experience (QoE) of the end users. However, predicting the QoE perceived by the users in different conditions remains a major challenge. In this paper, we propose a machine learning approach to support QoE-based video admission control and resource management algorithms. More specifically, we develop a multi-stage learning system that combines the unsupervised learning of video features from the size of H.264-encoded video frames with a supervised classifier trained to automatically extract the quality-rate characteristics of unknown video sequences. This QoE characterization is then used to manage simultaneous video transmissions through a shared channel in order to guarantee a minimum quality level delivered to the final users. Simulation results show that the proposed video admission control and resource management algorithms, which are based on learning-based QoE classification of video sequences, outperform standard content-agnostic strategies.

Combined curvelets and hidden Markov models for human fall detection

Fall events detection is one of the most crucial issues in the health care of elderly people. This paper proposes an innovative approach for reliably detecting fall incidents based on human silhouette shape variation in vision monitoring. This mission is achieved by: (i) introducing the curvelet transform and area ratios for identifying human postures in images; (ii) reducing the feature vector dimension using differential evolution technique; (iii) identifying postures by a support vector machine, and (iv) adapting a hidden Markov model for classifying video sequences into non-fall and fall events. Experimental results are obtained on several "Fall Detection" datasets. For evaluation, several assessment measures are computed. These evaluation measures demonstrate the effectiveness of the proposed methodology when compared to some state-of-the-art approaches.

A Tensor-Driven Temporal Correlation Model for Video Sequence Classification

The task of video sequence classification plays a critical role in the development of computer vision. Considering this fact, this letter proposes a novel tensor decomposition method called tensor-driven temporal correlation in which general tensors are used as input for video sequence classification. Because distortion and redundancy may exist in the tensor representations of video sequences, we project the original tensor into subspaces spanned by spatial basis matrices in the proposed formulation. Moreover, to better preserve the temporal smoothness between consecutive slices of the tensor, the basis matrices are jointly learned by introducing an autoregressive model. An experiment on the commonly used Cambridge hand-gesture database demonstrates that our proposed method reaches convergence within a small number of iterations during the training stage and achieves promising results compared with state-of-the-art methods.

Realtime Classification of Fish in Underwater Sonar Videos

SummaryDual-frequency identification sonar delivers video-like underwater images which allow the investigation of fish behaviour even in cloudy and muddy water. Generally, images are recorded in a resolution of up to 10 pictures per second, so that practically one obtains a video of underwater movements. These videos allow ecologists to observe, count or investigate fish behaviour. We focus on automatic classification of fish based on such sonar videos. After appropriate preprocessing of the videos, we show how we can count and classify fish into different species on the basis of their shape and movement. The procedures developed work in realtime, i.e. data processing and classification of video sequences are faster than the length of the video sequences themselves.

Compressive Sequential Learning for Action Similarity Labeling.

Human action recognition in videos has been extensively studied in recent years due to its wide range of applications. Instead of classifying video sequences into a number of action categories, in this paper, we focus on a particular problem of action similarity labeling (ASLAN), which aims at verifying whether a pair of videos contain the same type of action or not. To address this challenge, a novel approach called compressive sequential learning (CSL) is proposed by leveraging the compressive sensing theory and sequential learning. We first project data points to a low-dimensional space by effectively exploring an important property in compressive sensing: the restricted isometry property. In particular, a very sparse measurement matrix is adopted to reduce the dimensionality efficiently. We then learn an ensemble classifier for measuring similarities between pairwise videos by iteratively minimizing its empirical risk with the AdaBoost strategy on the training set. Unlike conventional AdaBoost, the weak learner for each iteration is not explicitly defined and its parameters are learned through greedy optimization. Furthermore, an alternative of CSL named compressive sequential encoding is developed as an encoding technique and followed by a linear classifier to address the similarity-labeling problem. Our method has been systematically evaluated on four action data sets: ASLAN, KTH, HMDB51, and Hollywood2, and the results show the effectiveness and superiority of our method for ASLAN.

A spatial-temporal iterative tensor decomposition technique for action and gesture recognition

Classification of video sequences is an important task with many applications in video search and action recognition. As opposed to some traditional approaches that transform original video sequences into forms of visual feature vectors, tensor-based methods have been proposed for classifying video sequences with natural representation of original data. However, one obvious limitation of tensor-based methods is that the input video sequences are often required to be preprocessed with a unified length of time. In this paper, we propose a technique for handling classification of video sequences in unequal length of time, namely Spatial-Temporal Iterative Tensor Decomposition (S-TITD) for uniform length. The proposed framework contains two primary steps. We first represent original video sequences as a third-order tensor and perform Tucker-2 decomposition to obtain the reduced-dimension core tensor. Then we encode the third order of core tensor to a uniform length by adaptively selecting the most informative slices. Notably, the above two steps are embedded into a dynamic learning framework to guarantee the proposed method has the ability of updating results over time. We conduct a series of experiments on three public datasets in gesture and action recognition, and the experimental results show that the proposed S-TITD approach achieves better performances than the state-of-the-art algorithms.

The MAHNOB Mimicry Database: A database of naturalistic human interactions

People mimic verbal and nonverbal expressions and behaviour of their counterparts in various social interactions. Research in psychology and social sciences has shown that mimicry has the power to influence social judgment and various social behaviours, including negotiation and debating, courtship, empathy and helping behaviour. Hence, automatic recognition of mimicry behaviour would be a valuable tool in various domains, and especially in negotiation skills enhancement and medical help provision training. In this work, we present the MAHNOB mimicry database, a set of fully synchronised, multi-sensory, audiovisual recordings of naturalistic dyadic interactions, suitable for investigation of mimicry and negotiation behaviour. The database contains 11 h of recordings, split over 54 sessions of dyadic interactions between 12 confederates and their 48 counterparts, being engaged either in a socio-political discussion or negotiating a tenancy agreement. To provide a benchmark for efforts in machine understanding of mimicry behaviour, we report a number of baseline experiments based on visual data only. Specifically, we consider face and head movements, and report on binary classification of video sequences into mimicry and non-mimicry categories based on the following widely-used methodologies: two similarity-based methods (cross correlation and time warping), and a state-of-the-art temporal classifier (Long Short Term Memory Recurrent Neural Network). The best reported results are session-dependent, and affected by the sparsity of positive examples in the data. This suggests that there is much room for improvement upon the reported baseline experiments.

Classification of video sequences into chosen generalized use classes of target size and lighting level.

The VQiPS (Video Quality in Public Safety) Working Group, supported by the U.S. Department of Homeland Security, has been developing a user guide for public safety video applications. According to VQiPS, five parameters have particular importance influencing the ability to achieve a recognition task. They are: usage time-frame, discrimination level, target size, lighting level, and level of motion. These parameters form what are referred to as Generalized Use Classes (GUCs). The aim of our research was to develop algorithms that would automatically assist classification of input sequences into one of the GUCs. Target size and lighting level parameters were approached. The experiment described reveals the experts′ ambiguity and hesitation during the manual target size determination process. However, the automatic methods developed for target size classification make it possible to determine GUC parameters with 70 % compliance to the end-users’ opinion. Lighting levels of the entire sequence can be classified with an efficiency reaching 93 %. To make the algorithms available for use, a test application has been developed. It is able to process video files and display classification results, the user interface being very simple and requiring only minimal user interaction.

Automated Classification of Two Persons' Interactive Activities

This paper presents a novel classification method for interactive activities, which is represented by R transform descriptor and classified by Hidden Markov Model s (HMM s ). It solves the problem that trades off activity recognition rate and computational complexity, rather than highlight the former exclusively. We extract binary silhouette images after the background model is created. Then the low-level features are described by R transform and principal vectors are determined by Principal Component Analysis (PCA). We utilize HMMs to train and classify video sequences, and demonstrate the usability with many sequences. Compared with others, our method is applicable to intelligent surveillance, as its advantage of R transform descriptor lying in robustness, computational complexity, geometric invariance and classification performance, and HMM in medium computational cost. So the video surveillance based on these is practicable in (but not limited to) many scenarios where the background is known.

Computing the Discrete Compactness of Orthogonal Pseudo-Polytopes via Their D-EVM Representation

This work is devoted to present a methodology for the computation of Discrete Compactness in -dimensional orthogonal pseudo-polytopes. The proposed procedures take in account compactness' definitions originally presented for the 2D and 3D cases and extend them directly for considering the D case. There are introduced efficient algorithms for computing discrete compactness which are based on an orthogonal polytopes representation scheme known as the Extreme Vertices Model in the -Dimensional Space (D-EVM). It will be shown the potential of the application of Discrete Compactness in higher-dimensional contexts by applying it, through EVM-based algorithms, in the classification of video sequences, associated to the monitoring of a volcano's activity, which are expressed as 4D orthogonal polytopes in the space-color-time geometry.

Using PFA in Feature Analysis and Selection for H.264 Adaptation

Classification of video sequences based on their contents is a vital process for adaptation techniques. It helps decide which adaptation technique best fits the resource reduction requested by the client. In this paper we used the principal feature analysis algorithm to select a reduced subset of video features. The main idea is to select only one feature from each class based on the similarities between the features within that class. Our results showed that using this feature reduction technique the source video features can be completely omitted from future classification of video sequences. Keywords—Adaptation, feature selection, H.264, Principal Feature Analysis (PFA)

ABNORMAL SECTIONS DETECTION BASED ON INTER-FRAMES INFORMATION IN VIDEO SEQUENCE

An approach is proposed for abnormal sections detection and rank classification in video sequences. In this approach, we give the definition of the abnormality of the video sections and create an abnormal factor to evaluate the abnormality. Firstly, the histogram is selected to describe the color change in the section, and then the histograms of the frames selected from the section compose the histogram matrix. In order to improve the process efficiency, the principal components analysis is used to reduce dimensions of the histogram matrix. Secondly, the information of frames differences is used to describe the motion change information, and the information twice frame difference is used to describe the change rate. Lastly, the results from the features are fused to get the final result. This method is experimented with some samples. The results show that this method can realize the abnormal detection and rank classification of video sequences, and the results are better than the other related techniques.

Finding and identifying unknown commercials using repeated video sequence detection

Automated commercial detection can be performed by matching features extracted from commercials or by detecting embedded codes that are hidden within the commercial. In both cases, it is necessary to create a database of known commercials that contain the information necessary for detection. In this paper, we present an automated technique for locating previously unknown commercials by continuously monitoring broadcast television signals. Our system has two components: repeated video sequence detection, and feature-based classification of video sequences as commercials or non-commercials. Our system utilizes customized temporal video segmentation techniques to automatically partition the digital video signal into semantically sensible shots and scenes. As each frame of the video source is processed, we extract auxiliary information to facilitate repeated sequence detection. When the video transition marking the end of the shot/scene is detected, we are able to rapidly locate all previous occurrences of the video clip. In order to classify video sequences as commercials or non-commercials, we extract a number of features from each video sequence that characterize the temporal and chromatic variations within the clip. We have evaluated three classification approaches using this information and have consistently achieved over 93% accuracy identifying new commercials and non-commercials as they are broadcast.