Motion Templates Research Articles

Recognition and detection of unusual activities in ATM using dual-channel capsule generative adversarial network

Real-time detection of human activity has become essential for monitoring and security of public spaces such as bank ATMs and workplaces, due to daily rise in criminal activities. Currently, monocular CCTV cameras that only record RGB video are used for monitoring such restricted areas. In addition to RGB data, the RGB + D sensor also offers depth information about the scene. To overcome the issue of online recognition of anomalous activities in Bank ATMs, a supervised deep learning method utilizing Dual-Channel Capsule Generative Adversarial Network (DCCGAN) and RGB + D sensor is proposed. The input RGB + D Dataset is given to super pixel motion detection method to find the region of interest (ROI). Motion detection is a significant stage examination of wide scene for background subtraction and foreground detection. After arranging the motion detection, its region is positioned frame by frame. Then, the detected ROI is given to fast discrete curvelet transform with the wrapping (FDCT-WRP) method. FDCT-WRP feature extraction method. These extracting features are supplied to Deep Convolutional Spiking neural network (DCSNN), which realize the object. RGB and depth video segments are used to create motion templates from the RGB + D data online video stream. These templates are trained on DCCGANs to identify distrustful events in current activity and categorized as normal and abnormal. Additionally, a unique RGB + D dataset is employed because there was no existing dataset available for analyzing human activity in ATMs. The proposed DUA-DCCGAN-ATM approach is assessed on qualitative with quantitative statistical evaluation parameters and identify suspect occurrence with 0.932 precision and 98.2 % accuracy. The detailed statistical analysis exemplifies that the proposed technique can identify distrustful events in a real-time online manner prior completing the anomalous activity.

An algorithm for extracting similar segments of moving target trajectories based on shape matching

Trajectory similarity analysis of moving target is the foundation for mining high-value and regular behavioral information such as motion preferences, activity hotspots and frequent paths. Unlike most trajectory similarity analysis methods aimed at discovering correlations of target activities in time, space or spatio-temporal domains, this paper focuses on the shape matching of target trajectories. If some specific shapes frequently appear in historical trajectories, extracting these local shapes would be beneficial for analyzing the target motion templates and behavior modes. Trajectory segments with similar shapes may not have spatio-temporal correlation, and the shapes also have geometric transformation characteristics such as rotation, scaling and translation. Since the existing trajectory similarity analysis methods cannot be directly applied, an algorithm for extracting similar segments based on shape matching is proposed. First, a new shape descriptor based on signed barycenter distance (SBD) is established. It describes a trajectory as a one-dimensional shape feature sequence, which has the advantage of low computational complexity. Then, the distributed nearest neighbor search strategy is used in the particle swarm optimization (PSO) method, which aims to accelerate the retrieval of trajectory segments with similar shapes and improve the matching accuracy. Experiments on MPEG-7, handwritten character and maneuvering target simulation trajectory data sets show that compared with the existing typical shape descriptors, SBD shape descriptor has advantages in accuracy and noise insensitivity, and the improved PSO method can efficiently and accurately obtain the local shape matching results.

Extraction of Lumbar Spine Motion Using a 3-IMU Wearable Cluster.

Spine movement is a daily activity that can indicate health status changes, including low back pain (LBP) problems. Repetitious and continuous movement on the spine and incorrect postures during daily functional activities may lead to the potential development and persistence of LBP problems. Therefore, monitoring of posture and movement is essential when designing LBP interventions. Typically, LBP diagnosis is facilitated by monitoring upper body posture and movement impairments, particularly during functional activities using body motion sensors. This study presents a fully wireless multi-sensor cluster system to monitor spine movements. The study suggests an attempt to develop a new method to monitor the lumbopelvic movements of interest selectively. In addition, the research employs a custom-designed robotic lumbar spine simulator to generate the ideal lumbopelvic posture and movements for reference sensor data. The mechanical motion templates provide an automated sensor pattern recognition mechanism for diagnosing the LBP.

Meta Reinforcement Learning for Optimal Design of Legged Robots

The process of robot design is a complex task and the majority of design decisions are still based on human intuition or tedious manual tuning. A more informed way of facing this task is computational design methods where design parameters are concurrently optimized with corresponding controllers. Existing approaches, however, are strongly influenced by predefined control rules or motion templates and cannot provide end-to-end solutions. In this paper, we present a design optimization framework using model-free meta reinforcement learning, and its application to the optimizing kinematics and actuator parameters of quadrupedal robots. We use meta reinforcement learning to train a locomotion policy that can quickly adapt to different designs. This policy is used to evaluate each design instance during the design optimization. We demonstrate that the policy can control robots of different designs to track random velocity commands over various rough terrains. With controlled experiments, we show that the meta policy achieves close-to-optimal performance for each design instance after adaptation. Lastly, we compare our results against a model-based baseline and show that our approach allows higher performance while not being constrained by predefined motions or gait patterns.

Two-stream fusion model using 3D-CNN and 2D-CNN via video-frames and optical flow motion templates for hand gesture recognition.

Hand gestures are useful tools for many applications in the human-computer interaction community. Here, the objective is to track the movement of the hand irrespective of the shape, size and color of the hand. And, for this, a motion template guided by optical flow (OFMT) is proposed. OFMT is a compact representation of the motion information of a gesture encoded into a single image. Recently, deep networks have shown impressive improvements as compared to conventional hand-crafted feature-based techniques. Moreover, it is seen that the use of different streams with informative input data helps to increase the recognition performance. This work basically proposes a two-stream fusion model for hand gesture recognition. The two-stream network consists of two layers—a 3D convolutional neural network (C3D) that takes gesture videos as input and a 2D-CNN that takes OFMT images as input. C3D has shown its efficiency in capturing spatiotemporal information of a video, whereas OFMT helps to eliminate irrelevant gestures providing additional motion information. Though each stream can work independently, they are combined with a fusion scheme to boost the recognition results. We have shown the efficiency of the proposed two-stream network on two databases.

Fingertip interactive tracking registration method for AR assembly system

Aiming at the problems of single input mode and lack of naturalness in the assembly process of existing AR systems, a tracking registration method of mobile AR assembly system is proposed based on multi-quantity and multi-degree of freedom natural fingertip interaction. Firstly, the real-time and stable tracking of hand area in complex environment is realized based on the hand region tracking; secondly, the fingertip detection and recognition based on K-COS and parallel vector is used to improve the precision and stability of fingertip recognition; thirdly, the special movement track of fingertip is recognized based on improved DTW algorithm, which has strong compatibility and feature gradient transformation for complex fingertip trajectory recognition; finally, through the real-time transformation of projection relationship between fingertip and virtual object, the interaction between fingertip and virtual object is made more natural and realistic. The experimental results show that in the complex environment of background, illumination, scale and rotation, the precision of fingertip detection and recognition is about 93%, and the precision of fingertip motion template matching is about 91%. The translation error of the registration method based on visual feature recognition is reduced by about 100pix compared with fingertip tracking registration method, and the efficiency of mobile AR-guided assembly method is improved by about 24.77% compared with the traditional manual assisted assembly method. These data verifies the strong interaction and practicability of the fingertips based on the user's multi-quantity and multi-degree of freedom features in the process of mobile AR guided assembly.

Path Tracking Strategy for Quadruped Robots Using a Hierarchical Framework

We present a general path tracking strategy, with continuous gaits, for quadruped robots on flat and rigid terrain. We use a hierarchical framework to decompose the task into unique motion templates (actions) such as translation and turning. A Linear Inverted Pendulum Model (LIPM) based algorithm is applied to create physically correct actions while minimizing the sum of squared accelerations of the CoM. These actions are stored in an online action library and are combined using an autonomous steering system that decides when to use which action. The steering system also computes the input parameters for these actions. We also develop transition maneuvers to execute a smooth switching between these actions. This strategy can help reduce the repeated need to perform comprehensive online computation for motion planning. Moreover, one can enrich the action library to execute more complicated tasks involving different gaits and non-flat terrain. We use an 18 DoF quadruped robot model, and simulations are carried out in MATLAB to demonstrate the capabilities of this approach.

Sequential convolutional network for behavioral pattern extraction in gait recognition

As a unique and promising biometric, video-based gait recognition has broad applications. The key step of this methodology is to learn the walking pattern of individuals, which, however, often suffers challenges to extract the behavioral feature from a sequence directly. Most existing methods just focus on either the appearance or the motion pattern. To overcome these limitations, we propose a sequential convolutional network (SCN) from a novel perspective, where spatiotemporal features can be learned by a basic convolutional backbone. In SCN, behavioral information extractors (BIE) are constructed to comprehend intermediate feature maps in time series through motion templates where the relation between frames can be analyzed, thereby distilling the information of the walking pattern. Furthermore, a multi-frame aggregator in SCN performs feature integration on a sequence whose length is uncertain, via a mobile 3D convolutional layer. To demonstrate the effectiveness, experiments have been conducted on two popular public benchmarks, CASIA-B and OU-MVLP, and our approach is demonstrated excellent performance, comparing with the state-of-art methods.

Legged Robot Running Using a Physics-Data Hybrid Motion Template

We report on the methodology of developing a hybrid model and utilizing it as a template to initiate running behavior in a legged robot. The hybrid model is comprised of a physics-based, rolling, spring-loaded, inverted pendulum (R-SLIP) model, and a data-driven model that compensates for unmodeled dynamics using Gaussian process (GP) regression. The hybrid R-SLIP-GP model retains the R-SLIP's intrinsic running dynamics, as well as improves the template's accuracy without intensive data training and iteration efforts. The proposed hybrid model was evaluated via simulation and in empirical robot running experiments. The results confirm that the added GP-based model greatly improved the model's accuracy, especially discrepancies resulting from the originally difficult-to-model complex leg-ground interactions. In addition, the R-SLIP-GP model was utilized as a complete motion template to design and control the robot's running motion. The experimental results demonstrate that the hybrid template can initiate more stable and power-efficient running motion in the robot than using the R-SLIP template.

Deep temporal motion descriptor (DTMD) for human action recognition

Spatiotemporal features have significant importance in human action recognition, as they provide the actor's shape and motion characteristics specific to each action class. This paper presents a new deep spatiotemporal human action representation, the deep temporal motion descriptor (DTMD), which shares the attributes of holistic and deep learned features. To generate the DTMD descriptor, the actor?s silhouettes are gathered into single motion templates by applying motion history images. These motion templates capture the spatiotemporal movements of the actor and compactly represent the human actions using a single 2D template. Then deep convolutional neural networks are used to compute discriminative deep features from motion history templates to produce the DTMD. Later, DTMD is used for learning a model to recognize human actions using a softmax classifier. The advantage of DTMD are that DTMD is automatically learned from videos and contains higher-dimensional discriminative spatiotemporal representations as compared to handcrafted features; DTMD reduces the computational complexity of human activity recognition as all the video frames are compactly represented as a single motion template; and DTMD works effectively for single and multiview action recognition. We conducted experiments on three challenging datasets: MuHAVI-Uncut, iXMAS, and IAVID-1. The experimental findings reveal that DTMD outperforms previous methods and achieves the highest action prediction rate on the MuHAVI-Uncut dataset.

Combined penalized weights based GM-PHD for point target tracking in starry-sky background

It is well known that the point targets observed by the space-based platform are extremely difficult to track due to the interference of a large number of stellar targets and background noise. The traditional GM-PHD, due to its probability accumulation, is easy to mistakenly identify the star as a derived new target when the real target passes closely to the star. In order to achieve precise tracking of multiple moving point targets for space-based observations in a complex background environment such as a starry sky background, this paper proposes a simple and effective combined penalized weights based GM-PHD for filtering. First, a unique label is given for each target to determine which real target the estimate came from. Next, the moving region of target is discretized into 10 irregular intervals to construct ten different motion templates, numbered one by one, for discretization of the direction of the target motion. Next, the true target at time k − 1 and all the estimated values having the same label at time k are used to calculate the direction penalty factor and the speed penalty factor to obtain the penalty strength matrix and penalized weight matrix, respectively. Finally, the penalized weight matrix is used to output the target with the highest weight in each label as the real target at time k. And a given penalty threshold is used to remove some fuzzy weights with greater penalty in the penalty strength matrix, so as to selectively filter out some less relevant targets to further avoid the stars participating in the next iteration. In order to verify the effectiveness of the proposed algorithm, we used the Tycho-2 catalog to simulate four kinds of starry-sky data sets with different complexity and conducted comparative experiments on each data set. The experimental results show that the proposed algorithm can effectively track multiple space-based infrared weak point targets in complex starry-sky scenarios.

Recovering Human Motion Patterns from Passive Infrared Sensors: A Geometric-Algebra Based Generation-Template-Matching Approach

The low-cost, indoor-feasibility, and non-intrusive characteristic of passive infrared sensors (PIR sensors) makes it widely used in human motion detection, but the limitation of its object identification ability makes it difficult to further analyze in the field of Geographic Information System (GIS). We present a template matching approach based on geometric algebra (GA) that can recover the semantics of different human motion patterns through the binary activation data of PIR sensor networks. A 5-neighborhood model was first designed to represent the azimuth of the sensor network and establish the motion template generation method based on GA coding. Full sets of 36 human motion templates were generated and then classified into eight categories. According to human behavior characteristics, we combined the sub-sequences of activation data to generate all possible semantic sequences by using a matrix-free searching strategy with a spatiotemporal constraint window. The sub-sequences were used to perform the matching operation with the generation-templates. Experiments were conducted using Mitsubishi Electric Research Laboratories (MERL) motion datasets. The results suggest that the sequences of human motion patterns could be efficiently extracted in different observation periods. The extracted sequences of human motion patterns agreed well with the event logs under various circumstances. The verification based on the environment and architectural space shows that the accuracy of the result of our method was up to 96.75%.

Deep motion templates and extreme learning machine for sign language recognition

Sign language is a visual language used by persons with hearing and speech impairment to communicate through fingerspellings and body gestures. This paper proposes a framework for automatic sign language recognition without the need of hand segmentation. The proposed method first generates three different types of motion templates: motion history image, dynamic image and our proposed RGB motion image. These motion templates are used to fine-tune three ConvNets trained on ImageNet dataset. Fine-tuning avoids learning all the parameters from scratch, leading to faster network convergence even with a small number of training samples. For combining the output of three ConvNets, we propose a fusion technique based on Kernel-based extreme learning machine (KELM). The features extracted from the last fully connected layer of trained ConvNets are used to train three KELMs, and the final class label is predicted by averaging their scores. The proposed approach is validated on a number of publicly available sign language as well as human action recognition datasets, and state-of-the-art results are achieved. Finally, an Indian sign language dataset is also collected using a thermal camera. The experimental results obtained show that our ConvNet-based deep features along with proposed KELM-based fusion are robust for any type of human motion recognition.

Developing new robust motion templates of martial art techniques using R-GDL approach: a case study of SSCM

Martial art (MA) is considered a conserved heritage primarily for promoting certain level of identities and cultures. With the technology advancement, motion capture has been widely used in MA to capture and evaluate human performance. However, methods to create motion templates (templates) of MA techniques from scratch are rarely exposed because there is no complete evaluation system framework suggested for MA. This paper presents in detaile how to create the MA templates using framework of extrinsic feedback-based evaluation system generally and R-GDL specifically. To create more robust templates, GDL has been used in validation tests. SSCM becomes the case study in this research. The main contribution in this paper is introduction of new datasets of templates for SSCM techniques. Evaluation of trainees' performance will be done using the datasets for future works. The datasets are believed to promise MAs' authenticity and originality and to preserve this valuable cultural heritage.

Developing new robust motion templates of martial art techniques using R-GDL approach: a case study of SSCM

Martial art (MA) is considered a conserved heritage primarily for promoting certain level of identities and cultures. With the technology advancement, motion capture has been widely used in MA to capture and evaluate human performance. However, methods to create motion templates (templates) of MA techniques from scratch are rarely exposed because there is no complete evaluation system framework suggested for MA. This paper presents in detaile how to create the MA templates using framework of extrinsic feedback-based evaluation system generally and R-GDL specifically. To create more robust templates, GDL has been used in validation tests. SSCM becomes the case study in this research. The main contribution in this paper is introduction of new datasets of templates for SSCM techniques. Evaluation of trainees' performance will be done using the datasets for future works. The datasets are believed to promise MAs' authenticity and originality and to preserve this valuable cultural heritage.

Is infantile esotropia subcortical in origin?

Essential infantile esotropia (EIE) is often attributed to a primary disturbance within the visual cortex based upon the findings of monocular horizontal optokinetic asymmetry and correlative horizontal motion detection asymmetry. However, these physiologic aberrations conform to what would be observed if the visual cortex secondarily reconfigured itself to the preexisting subcortical optokinetic motion template. This analysis examines the perspective that the measured cortical aberrations can be explained by prolonged subcortical neuroplasticity, leading to a secondary rewiring of cortical motion pathways. Evolutionary evidence indicates that EIE is generated by subcortical ocular motor centers that subserve nasalward optokinesis. These phylogenetically older subcortical visuo-vestibular pathways include the nucleus of the optic tract, accessory optic system, inferior olive, cerebellar flocculus, and vestibular nucleus. In normal humans, the subcortical visual system becomes inactivated after the first few months of infancy. Mutations or other perturbations that prolong subcortical neuroplasticity may create a persistent simultaneous nasalward optokinetic bias in both eyes to generate infantile esotropia.

改进运动模板与直方图匹配的视频典型目标检测方法

改进运动模板与直方图匹配的视频典型目标检测方法

Averaging of motion capture recordings for movements\u2019 templates generation

In this paper we propose, describe and evaluate the novel motion capture (MoCap) data averaging framework. It incorporates hierarchical kinematic model, angle coordinates’ preprocessing methods, that recalculate the original MoCap recording making it applicable for further averaging algorithms, and finally signals averaging processing. We have tested two signal averaging methods namely Kalman Filter (KF) and Dynamic Time Warping barycenter averaging (DBA). The propose methods have been tested on MoCap recordings of elite Karate athlete, multiple champion of Oyama karate knockdown kumite who performed 28 different karate techniques repeated 10 times each. The proposed methods proved to have not only high effectiveness measured with root-mean-square deviation (4.04 ± 5.03 degrees for KF and 5.57 ± 6.27 for DBA) and normalized Dynamic Time Warping distance (0.90 ± 1.58 degrees for KF and 0.93 ± 1.23 for DBA), but also the reconstruction and visualization of those recordings persists all crucial aspects of those complicated actions. The proposed methodology has many important applications in classification, clustering, kinematic analysis and coaching. Our approach generates an averaged full body motion template that can be practically used for example for human actions recognition. In order to prove it we have evaluated templates generated by our method in human action classification tasks using DTW classifier. We have made two experiments. In first leave - one - out cross - validation we have obtained 100% correct recognitions. In second experiment when we classified recordings of one person using templates of another recognition rate 94.2% was obtained.

Human Actions Analysis: Templates Generation, Matching and Visualization Applied to Motion Capture of Highly-Skilled Karate Athletes.

The aim of this paper is to propose and evaluate the novel method of template generation, matching, comparing and visualization applied to motion capture (kinematic) analysis. To evaluate our approach, we have used motion capture recordings (MoCap) of two highly-skilled black belt karate athletes consisting of 560 recordings of various karate techniques acquired with wearable sensors. We have evaluated the quality of generated templates; we have validated the matching algorithm that calculates similarities and differences between various MoCap data; and we have examined visualizations of important differences and similarities between MoCap data. We have concluded that our algorithms works the best when we are dealing with relatively short (2–4 s) actions that might be averaged and aligned with the dynamic time warping framework. In practice, the methodology is designed to optimize the performance of some full body techniques performed in various sport disciplines, for example combat sports and martial arts. We can also use this approach to generate templates or to compare the correct performance of techniques between various top sportsmen in order to generate a knowledge base of reference MoCap videos. The motion template generated by our method can be used for action recognition purposes. We have used the DTW classifier with angle-based features to classify various karate kicks. We have performed leave-one-out action recognition for the Shorin-ryu and Oyama karate master separately. In this case, actions were correctly classified. In another experiment, we used templates generated from Oyama master recordings to classify Shorin-ryu master recordings and vice versa. In this experiment, the overall recognition rate was , which is a very good result for this type of complex action.

Anomaly detection based on two global grid motion templates

In this paper, we propose a novel method to detect abnormal events from videos based on two global grid motion templates (GGMTs) which are able to capture the motion distribution, space and scale information. The GGMTs contain the maximum and minimum grid motion templates which can effectively distinguish the anomalies from the normal motion distribution. One GGMT is composed of several non-overlap local grid motion templates with each one corresponding to a special location. Each local grid motion template is represented by a motion histogram obtained by computing the maximum/minimum motion distribution from the training samples. Experiments on the public datasets show that our method can effectively detect abnormal events in complex scenes.