Dynamic Texture Research Articles

Joint Dynamic Pose Image and Space Time Reversal for Human Action Recognition from Videos

Human action recognition aims to classify a given video according to which type of action it contains. Disturbance brought by clutter background and unrelated motions makes the task challenging for video frame-based methods. To solve this problem, this paper takes advantage of pose estimation to enhance the performances of video frame features. First, we present a pose feature called dynamic pose image (DPI), which describes human action as the aggregation of a sequence of joint estimation maps. Different from traditional pose features using sole joints, DPI suffers less from disturbance and provides richer information about human body shape and movements. Second, we present attention-based dynamic texture images (att-DTIs) as pose-guided video frame feature. Specifically, a video is treated as a space-time volume, and DTIs are obtained by observing the volume from different views. To alleviate the effect of disturbance on DTIs, we accumulate joint estimation maps as attention map, and extend DTIs to attention-based DTIs (att-DTIs). Finally, we fuse DPI and att-DTIs with multi-stream deep neural networks and late fusion scheme for action recognition. Experiments on NTU RGB+D, UTD-MHAD, and Penn-Action datasets show the effectiveness of DPI and att-DTIs, as well as the complementary property between them.

Dynamic Texture Classification Using Unsupervised 3D Filter Learning and Local Binary Encoding

Local binary descriptors, such as local binary pattern (LBP) and its various variants, have been studied extensively in texture and dynamic texture analysis due to their outstanding characteristics, such as grayscale invariance, low computational complexity and good discriminability. Most existing local binary feature extraction methods extract spatio-temporal features from three orthogonal planes of a spatio-temporal volume by viewing a dynamic texture in 3D space. For a given pixel in a video, only a proportion of its surrounding pixels is incorporated in the local binary feature extraction process. We argue that the ignored pixels contain discriminative information that should be explored. To fully utilize the information conveyed by all the pixels in a local neighborhood, we propose extracting local binary features from the spatio-temporal domain with 3D filters that are learned in an unsupervised manner so that the discriminative features along both the spatial and temporal dimensions are captured simultaneously. The proposed approach consists of three components: 1) 3D filtering; 2) binary hashing; and 3) joint histogramming. Densely sampled 3D blocks of a dynamic texture are first normalized to have zero mean and are then filtered by 3D filters that are learned in advance. To preserve more of the structure information, the filter response vectors are decomposed into two complementary components, namely, the signs and the magnitudes, which are further encoded separately into binary codes. The local mean pixels of the 3D blocks are also converted into binary codes. Finally, three types of binary codes are combined via joint or hybrid histograms for the final feature representation. Extensive experiments are conducted on three commonly used dynamic texture databases: 1) UCLA; 2) DynTex; and 3) YUVL. The proposed method provides comparable results to, and even outperforms, many state-of-the-art methods.

High Order Volumetric Directional Pattern for Video-Based Face Recognition

Describing the dynamic textures has attracted growing attention in the field of computer vision and pattern recognition. In this paper, a novel approach for recognizing dynamic textures, namely, high order volumetric directional pattern (HOVDP), is proposed. It is an extension of the volumetric directional pattern (VDP) which extracts and fuses the temporal information (dynamic features) from three consecutive frames. HOVDP combines the movement and appearance features together considering the nth order volumetric directional variation patterns of all neighboring pixels from three consecutive frames. In experiments with two challenging video face databases, YouTube Celebrities and Honda/UCSD, HOVDP clearly outperformed a set of state-of-the-art approaches.

Early Fire Detection on Video Using LBP and Spread Ascending of Smoke

This paper proposes a methodology for early fire detection based on visual smoke characteristics such as movement, color, gray tones and dynamic texture, i.e., diverse but representative and discriminant characteristics, as well as its ascending expansion, which is sequentially processed to find the candidate smoke regions. Thus, once a region with movement is detected, the pixels inside it that are smoke color are estimated to obtain a more detailed description of the smoke candidate region. Next, to increase the system efficiency and reduce false alarms, each region is characterized using the local binary pattern, which analyzes its texture and classifies it by means of a multi-layer perceptron. Finally, the ascending expansion of the candidate region is analyzed and those smoke regions that maintain or increase their ascending growth over a time span are considered as a smoke regions, and an alarm is triggered. Evaluations were performed using two different classifiers, namely multi-Layer perceptron and the support vector machine, with a standard database smoke video. Evaluation results show that the proposed system provides fire detection accuracy of between 97.85% and 99.83%.

Capturing the impact of oral processing behaviour on consumption time and dynamic sensory perception of ice creams differing in hardness

Little is known about the oral processing behaviour of ice creams for which the quantification of oral manipulations remains challenging. The impact of oral processing behaviour on dynamic sensory perception of ice creams has not been reported previously, although ice creams are anecdotally known to be consumed following different oral processing strategies. The aims of the study were (1) to compare different methodologies to characterise oral behaviours applied during consumption of ice cream, and (2) to understand how oral processing behaviour influences dynamic sensory perception of ice creams. Oral processing behaviour of ice creams was characterized by self-reporting and video recording of n = 103 consumers. Most consumers applied either tongue movements or combined tongue and jaw oral behaviours. The video recording was clearly better than self-reporting in capturing the actually displayed eating behaviour of ice creams. Consumption time was prolonged considerably when ice cream hardness increased or when oral behaviour was changed during Temporal Dominance of Sensations (TDS) evaluations. When ice creams were consumed by letting them melt in mouth, dominance of sweetness and firmness were significantly prolonged. When ice creams were consumed by chewing, dominance of fruity aroma and coldness were significantly prolonged. We conclude that (1) eating behaviour of ice creams is captured better by video recordings than self-reporting and that (2) oral processing behaviour considerably changes dynamic texture and flavour perception of ice creams. This highlights the importance of controlling oral processing behaviour when quantifying sensory properties of ice creams.

Randomized neural network based signature for dynamic texture classification

Dynamic texture analysis has been the focus of intensive research in recent years. Thus, this paper presents an innovative and highly discriminative dynamic texture analysis method, whose signature is composed of the weights of the output layer of a randomized neural network after a training procedure. This training is performed by using the pixels of slices of each orthogonal plane of the video (XY, YT, and XT) as input feature vectors and corresponding output labels. The obtained video signature provided an accuracy of 97.05%, 98.54%, 97.74% and 96.51% on the UCLA-50 classes, UCLA-9 classes, UCLA-8 classes and Dyntex++, respectively. These results, when compared to other dynamic texture analysis methods, demonstrate that our descriptors are very effective and that our proposed approach can contribute significantly to the field of dynamic texture analysis.

Dynamic texture analysis with diffusion in networks

Dynamic texture is a field of research that has gained considerable interest from computer vision community due to the explosive growth of multimedia databases. In addition, dynamic texture is present in a wide range of videos, which makes it very important in expert systems based on videos such as medical systems, traffic monitoring systems, forest fire detection system, among others. In this paper, a new method for dynamic texture characterization based on diffusion in directed networks is proposed. The dynamic texture is modeled as a directed network. The method consists in the analysis of the dynamic of this network after a series of graph cut transformations based on the edge weights. For each network transformation, the activity for each vertex is estimated. The activity is the relative frequency that one vertex is visited by random walks in balance. Then, texture descriptor is constructed by concatenating the activity histograms. The main contributions of this paper are the use of directed network modeling and diffusion in network to dynamic texture characterization. These tend to provide better performance in dynamic textures classification. Experiments with rotation and interference of the motion pattern were conducted in order to demonstrate the robustness of the method. The proposed approach is compared to other dynamic texture methods on two very well known dynamic texture database and on traffic condition classification, and outperform in most of the cases.

Face Anti-Spoofing Using Texture-Based Techniques and Filtering Methods

User authentication for an accurate biometric system is the demand of the hour in today’s world. When somebody attempts to take on the appearance of another person by introducing a phony face or video before the face detection camera and gets illegitimate access, a face presentation attack usually happens. To effectively protect the privacy of a person, it is very critical to build a face authentication and anti-spoofing system. This paper introduces a novel and appealing face spoof detection technique, which is primarily based on the study of contrast and dynamic texture features of both seized and spoofed photos. Valid identification of photo spoofing is anticipated here. A modified version of the DoG filtering method, and local binary pattern variance (LBPV) based technique, which is invariant to rotation, are designated to be used in this paper. Support vector machine (SVM) is used when feature vectors are extracted for further analysis. The publicly available NUAA photo-imposter database is adapted to test the system, which includes facial images with different illumination and area. The accuracy of the method can be assessed using the false acceptance rate (FAR) and false rejection rate (FRR). The results express that our method performs better on key indices compared to other state-of-the-art techniques following the provided evaluation protocols tested on a similar dataset.

Glioma grading using structural magnetic resonance imaging and molecular data.

A glioma grading method using conventional structural magnetic resonance image (MRI) and molecular data from patients is proposed. The noninvasive grading of glioma tumors is obtained using multiple radiomic texture features including dynamic texture analysis, multifractal detrended fluctuation analysis, and multiresolution fractal Brownian motion in structural MRI. The proposed method is evaluated using two multicenter MRI datasets: (1)the brain tumor segmentation (BRATS-2017) challenge for high-grade versus low-grade (LG) and (2)the cancer imaging archive (TCIA) repository for glioblastoma (GBM) versus LG glioma grading. The grading performance using MRI is compared with that of digital pathology (DP) images in the cancer genome atlas (TCGA) data repository. The results show that the mean area under the receiver operating characteristic curve (AUC) is 0.88 for the BRATS dataset. The classification of tumor grades using MRI and DP images in TCIA/TCGA yields mean AUC of 0.90 and 0.93, respectively. This work further proposes and compares tumor grading performance using molecular alterations (IDH1/2 mutations) along with MRI and DP data, following the most recent World Health Organization grading criteria, respectively. The overall grading performance demonstrates the efficacy of the proposed noninvasive glioma grading approach using structural MRI.

Squirrel-Cage Local Binary Pattern and Its Application in Video Anomaly Detection

Local binary pattern (LBP) is one of the most successful feature descriptors. However, LBP and its variants have not been as successful as other feature descriptors in video anomaly detection (VAD). This is because LBP and its variants are mainly designed for spatial texture analysis. Although the volume LBP (VLBP) and the LBP-three orthogonal planes (LPB-TOP) have the capability of describing dynamic texture, they are seldom used as descriptors for VAD because 1) both VLBP and LBP-TOP are more suitable for natural scenes with rich dynamic textures, but sensitive to noise in the scenes with less dynamic textures, 2) the combination of motion and appearance not only limits their capability of motion characterizing but also brings the irrelevant appearance information such as background, and 3) high dimensionality is another drawback. In this paper, a new variant of the LBP called the squirrel-cage LBP (SCLBP) is proposed for VAD. By imitating the structure of squirrel cage rotor, the proposed SCLBP can be regarded as a stretched LBP in temporal direction and has two distinct features: 1) it is computed at vector-wise, rather than at pixel-wise (i.e., for the central vector with respect to its surrounding parallel vectors), and 2) the sign between two vectors is determined by the angle-based thresholding scheme. The SCLBP can effectively encode the motion information and is insensitive to noise and irrelevant disturbances caused by dynamic background and illumination change. To the best of our knowledge, The SCLBP is the first variant of the LBP specially designed for motion characterizing. The SCLBP has great flexibility, extendibility, and low dimensionality (only one-third of the LBP-TOP descriptor). The effectiveness of the proposed SCLBP descriptor is demonstrated on different public data sets and compared with the other dominant descriptors and state-of-the-art approaches in VAD.

Joint Discriminative Learning of Deep Dynamic Textures for 3D Mask Face Anti-Spoofing

Three-dimensional mask spoofing attacks have been one of the main challenges in face recognition. Compared with a 3D mask, a real face displays different facial motion patterns that are reflected by different facial dynamic textures. However, a large portion of these facial motion differences is subtle. We find that the subtle facial motion can be fully captured by multiple deep dynamic textures from a convolutional layer of a convolutional neural network, but not all deep dynamic textures from different spatial regions and different channels of a convolutional layer are useful for differentiation of subtle motions between real faces and 3D masks. In this paper, we propose a novel feature learning model to learn discriminative deep dynamic textures for 3D mask face anti-spoofing. A novel joint discriminative learning strategy is further incorporated in the learning model to jointly learn the spatial- and channel-discriminability of the deep dynamic textures. The proposed joint discriminative learning strategy can be used to adaptively weight the discriminability of the learned feature from different spatial regions or channels, which ensures that more discriminative deep dynamic textures play more important roles in face/mask classification. Experiments on several publicly available data sets validate that the proposed method achieves promising results in intra- and cross-data set scenarios.

An Investigation on Microstructure, Texture and Mechanical Properties of AZ80 Mg Alloy Processed by Annular Channel Angular Extrusion.

Annular channel angular extrusion has been recently developed as a new single-pass severe plastic deformation method suitable for producing large size cup-shaped parts from cylindrical billets. In this study, the novel technology was successfully applied to commercial AZ80 Mg alloy at 300 °C, and microstructure, texture evolution, and mechanical properties were investigated. Due to severe shear deformation, the initial microstructure, including the coarse grains and large eutectic β-phases, was greatly refined. The strong basal texture formed during the initial deformation stage was modified into a weak tilted dynamic texture. During the deformation process, fine β-particles separated from eutectic phases effectively hindered the grain boundary migration and rotation, enhancing the grain refinement and texture weakening. More than 63% of the microhardness increase was achieved in this extruded part. Also, tensile tests showed the yield strength and elongation in both directions (transverse and longitudinal) of extruded part were improved more than 2.5 times, and the ultimate tensile strength was improved more than 2 times, compared to the initial material state. The improved material properties were mainly attributed to microstructure (grain and phase) refinement and texture weakening. It was demonstrated that the annular channel angular extrusion process can be considered as a novel and effective single-pass severe plastic deformation method.

Dynamic texture recognition using local tetra pattern—three orthogonal planes (LTrP-TOP)

Dynamic texture (DT) is the annexation of an altering series of images where temporal regularity is present. Characterization, classification and identification of DTs have been investigated by many researchers in the past 2 decades. Especially, several image texture descriptors based on local binary pattern (LBP) have been enhanced for DT classification in the spatiotemporal domain. Local tetra pattern (LTrP) is an extension of LBP in which the calculation of the feature code depends on the referenced central pixel as well as its neighbor’s directions. In this work, we have proposed an LTrP-based novel spatiotemporal descriptor for characterization and subsequent identification of textures in videos. We term this descriptor as local tetra patterns on three orthogonal planes (LTrP-TOP). In our proposed work, the video frames are initially divided into three orthogonal planes on the basis of the XY, YT and XT directions. The direction of the center and neighborhood pixels are then detected in vertical and horizontal orientations using first-order derivatives. This sequence of operations computes the LTrP for each center pixel. Subsequently, we create the histogram for a particular plane by considering the LTrP codes for all the pixels in that plane. Finally, the histograms from the three orthogonal planes are concatenated for forming the final texture descriptor of the video. Comprehensive experimental assessments on benchmark DT databases (Dyntex++ and UCLA) demonstrate that the LTrP-TOP descriptor exhibits better performance than most of the other state-of-the-art DT descriptors (in terms of classification accuracy %).

Spatio-Temporal Feature Extraction/Recognition in Videos Based on Energy Optimization.

Videos are spatio-temporally rich in static to dynamic objects/scenes, sparse to dense, and periodic to non-periodic motions. Particularly, the dynamic texture (DT) exhibits complex appearance and motion changes that remain a challenge to deal with. This paper presents an energy optimization method for feature extraction and recognition in videos. For noise and background jitter, the Tikhonov regularization with eigen-vector and Frenet-Serret formula-based energy constraints is also proposed. The different periodicity of DT can be adapted by the time-varying number of learning temporal frames. The optimal duration of an image sequence is determined from the temporal property of its eigen-values. Unlike the state-of-the-art recognition methods, i.e., sparse coding and slow feature analysis, the proposed method can capture the physical property of objects and scenes: velocity, acceleration, and orientation. Also, the static and dynamic image regions can be locally classified. Owing to these spatio-temporal features, stability, robustness, and accuracy of feature extraction and recognition are enhanced. Using DT videos, the superiority of the proposed method compared to the state-of-the-art recognition methods is experimentally shown.

Improved background modeling of video sequences using spatio-temporal extension of fuzzy local binary pattern

Background subtraction is a method of motion detection in video sequences captured by static camera based on construction of a background model and its progressive comparison with each frame of the video. Sometimes the changes in the background objects are not permanent and appear at a rate faster than that of the background update, and this leads to emergence of dynamic textures in the background. As a result, high-quality background modeling plays a major role in motion detection performance, especially in videos with dynamic backgrounds and adverse environmental conditions such as noise. Although Local Binary Pattern (LBP) is a successful methodology for background subtraction, but it cannot properly extract textures from uniform areas of the foreground. In recent years, Fuzzy Local Binary Pattern (FLBP) has been developed to improve the performance of LBP operator in texture extraction from images with additive noise. The use of FLBP texture descriptor for background subtraction has led to improved robustness against noise and low sensitivity of the background model to slight changes in the texture gray scale values, and therefore better texture extraction from uniform areas, even in the presence of dynamic backgrounds and adverse environmental conditions. But despite this improvement, this operator is still sensitive to time-variations of pixels in dynamic backgrounds. To avoid this issue and incorporate correlation of pixel values over successive frames, this study proposes the use of spatio-temporal extension of FLBP with symmetry about central pixel in combination with the Local Color Histogram (LCH) in the Improved Hue Luminance and Saturation (IHLS) color space for describing the pixel color features. The results of tests conducted with standard databases show that for dynamic backgrounds, the use of proposed Spatio-Temporal Fuzzy Center Symmetric Local Binary Pattern (STFCS-LBP) operator with the spatio-temporal neighborhood texture patterns and the local color histogram yields better results than the existing methods.

Dynamic texture classification using Gumbel mixtures in the complex wavelet domain

Dynamic texture (DT) classification has attracted extensive attention in the field of image sequence analysis. The probability distribution model, which has been used to analysis DT, can describe well the distribution property of signals. Here, the authors introduce the finite mixtures of Gumbel distributions (MoGD) and the corresponding parameter estimation method based on expectation-maximisation algorithm. Then, the authors propose the DT features based on MoGD model for DT classification. Specifically, after decomposing DTs with the dual-tree complex wavelet transform (DT-CWT), the median values of complex wavelet coefficient magnitudes of non-overlapping blocks in detail subbands are modelled with MoGDs. The model parameters are accumulated into a feature vector to describe DT. During the classification, a variational approximation version of the Kullback-Leibler divergence is used to measure the similarity between different DTs. The experimental evaluations on two popular benchmark DT data sets (UCLA and DynTex++) demonstrate the effectiveness of the proposed approach.

Unified dynamic texture segmentation system based on local and global spatiotemporal techniques

Dynamic texture (DT) is temporal extension of static texture. There are two broad types of dynamic textures: natural and manmade. The examples of dynamic textures are waving tree, sea water, fountain, traffic, moving crowd, etc. Dynamic texture segmentation is a technique used to separate the moving objects from stationary content in the video. Majority of the techniques give good results either for natural or for manmade dynamic textures. The proposed approach for DT Segmentation is combination of local spatiotemporal technique, i.e., local binary pattern-Weber local descriptor and global spatiotemporal technique, i.e., contourlet transform. The local spatiotemporal technique considers the appearance and motion of the object for segmentation. The technique is computationally less complex than optical flow and gives good results for manmade dynamic textures. The global spatiotemporal technique is based on Laplacian pyramid and directional filters. It gives good results for natural dynamic textures. The proposed technique discussed in this paper is unified approach for any type of dynamic texture. The regions of images commonly segmented by both the techniques are considered as the final segmented output. The proposed system works equally well on any kind of dynamic texture.

Unified dynamic texture segmentation system based on local and global spatiotemporal techniques

Dynamic texture (DT) is temporal extension of static texture. There are two broad types of dynamic textures: natural and manmade. The examples of dynamic textures are waving tree, sea water, fountain, traffic, moving crowd, etc. Dynamic texture segmentation is a technique used to separate the moving objects from stationary content in the video. Majority of the techniques give good results either for natural or for manmade dynamic textures. The proposed approach for DT Segmentation is combination of local spatiotemporal technique, i.e., local binary pattern-Weber local descriptor and global spatiotemporal technique, i.e., contourlet transform. The local spatiotemporal technique considers the appearance and motion of the object for segmentation. The technique is computationally less complex than optical flow and gives good results for manmade dynamic textures. The global spatiotemporal technique is based on Laplacian pyramid and directional filters. It gives good results for natural dynamic textures. The proposed technique discussed in this paper is unified approach for any type of dynamic texture. The regions of images commonly segmented by both the techniques are considered as the final segmented output. The proposed system works equally well on any kind of dynamic texture.

Dynamic texture recognition under adverse lighting and weather conditions for outdoor environments

Recognizing dynamic patterns based on visual processing is significant for many applications. In this paper dynamic texture recognition focuses on outdoor scenarios where a crisis event might occur (i.e. fire in a forest, floods/flooding etc.) Real outdoor scenes may include the objects with dynamic behaviour due to illumination, blurring, or weather conditions effects. Under bad weather conditions the imaging systems is degraded and produce low visibility images. In this work precipitation artefacts and lightning effects for dynamic texture analysis were studied. Experimental results show that the proposed method of weather and adverse lighting effects compensation is feasible and effective for videobased dynamic texture analysis under bad weather conditions.

Point Primitives Based Virtual Surgery System

In order to achieve a high degree of visual realism in surgery simulation, we present a virtual surgery system framework, which is based on point primitives for the virtual surgery scene rendering and the biomechanical calculation of the soft tissue. To embody the superiority of this framework, two virtual surgery systems based on point primitives we developed are exhibited in this paper. Six critical functional modules were selected as representative of basic and advanced virtual surgery skill. These modules were: 1) point-based texture mapping; 2) deformation simulation; 3) cutting simulation; 4) tearing simulation; 5) dynamic texture mapping; and 6) 3-D display. These modules were elaborated by including working principle, execution process, and the performance of the algorithm. The experimental results have shown that point primitives-based virtual surgery systems obtained higher performance in terms of computational efficiency and rendering effect than traditional meshes-based virtual surgery system.