Motion Mapping Research Articles

Research on optimization of dust measurement pipeline based on Bernoulli Effect

According to the principle of Bernoulli Effect in fluid mechanics, a dust measurement pipeline with Bernoulli Effect is designed. According to the previous research of the projec, this paper simulates by CFD software, obtains the particle motion velocity and electrostatic induction under different pipe diameter ratio, and finds out the maximum electrostatic induction under the better tube diameter ratio. The annular electrostatic sensor is used to find a better diameter ratio for Bernoulli Effect tube. The improved device model with different pipe diameter ratio was established in Gambit2.4, and 2D stereo model was used for experimental simulation. The velocity cloud map of particle motion is obtained by using FLUENT6.3 to solve the problem, and the corresponding velocity value is obtained according to the cloud image. The electrostatic induction calculation by MATLAB shows that when the diameter ratio d / D is 0.4 and the velocity is 9.76m/s, the electrostatic induced charge of dust particles is the largest at this time. It has certain guiding significance for the design of dust measurement pipeline.

Traffic extreme situations detection in video sequences based on integral optical flow

Road traffic analysis is an important task in many applications and it can be used in video surveillance systems to prevent many undesirable events. In this paper, we propose a new method based on integral optical flow to analyze cars movement in video and detect flow extreme situations in real-world videos. Firstly, integral optical flow is calculated for video sequences based on optical flow, thus random background motion is eliminated; secondly, pixel-level motion maps which describe cars movement from different perspectives are created based on integral optical flow; thirdly, region-level indicators are defined and calculated; finally, threshold segmentation is used to identify different cars movements. We also define and calculate several parameters of moving car flow including direction, speed, density, and intensity without detecting and counting cars. Experimental results show that our method can identify cars directional movement, cars divergence and cars accumulation effectively.

Storm-time convection dynamics viewed from optical auroras

A series of statistical and event studies have demonstrated that the motion of patches in regions of Patchy Pulsating Aurora (PPA) is very close to, if not exactly, E×B convection. Therefore, 2D maps of PPA motion provide us the opportunity to remotely sense magnetospheric convection with relatively high space and time resolution, subject to uncertainties associated with the mapping between the ionosphere and magnetosphere. In this study, we use THEMIS ASI (All Sky Imager) aurora observations combined with RBSP electric field and magnetic field measurements to explore convection dynamics during storm time. From 0500 UT to 0600 UT on March 19 2015, auroral observations across ~4 h of magnetic local time (MLT) show that increases in the westward velocities of patches are closely related to earthward flow bursts in the inner plasma sheet. Together with the meridian scanning photometer (MSP) data, this suggests that the increase in the westward velocities of PPA patches is caused by earthward-moving ion injection structures carried by the fast earthward flows.

Wound Healing Monitoring by Video Sequence Using Integral Optical Flow

A method for monitoring dynamic changes of wound video sequence based on integral optical flow is proposed. In this method, frame-by-frame sequential analysis is performed first, and then integral optical flow and motion maps are constructed for the video sequence. Motion maps allow one to determine the rate of epithelialization of wounds in critical regions that are characterized by impaired tissue growth. The stages of tissue healing are also determined from motion maps. Dynamic characteristics of wound tissue changes are introduced and calculated.

Repetition Estimation

Visual repetition is ubiquitous in our world. It appears in human activity (sports, cooking), animal behavior (a bee’s waggle dance), natural phenomena (leaves in the wind) and in urban environments (flashing lights). Estimating visual repetition from realistic video is challenging as periodic motion is rarely perfectly static and stationary. To better deal with realistic video, we elevate the static and stationary assumptions often made by existing work. Our spatiotemporal filtering approach, established on the theory of periodic motion, effectively handles a wide variety of appearances and requires no learning. Starting from motion in 3D we derive three periodic motion types by decomposition of the motion field into its fundamental components. In addition, three temporal motion continuities emerge from the field’s temporal dynamics. For the 2D perception of 3D motion we consider the viewpoint relative to the motion; what follows are 18 cases of recurrent motion perception. To estimate repetition under all circumstances, our theory implies constructing a mixture of differential motion maps: mathbf {F}, {varvec{nabla }}mathbf {F}, {varvec{nabla }}{varvec{cdot }} mathbf {F} and {varvec{nabla }}{varvec{times }} mathbf {F}. We temporally convolve the motion maps with wavelet filters to estimate repetitive dynamics. Our method is able to spatially segment repetitive motion directly from the temporal filter responses densely computed over the motion maps. For experimental verification of our claims, we use our novel dataset for repetition estimation, better-reflecting reality with non-static and non-stationary repetitive motion. On the task of repetition counting, we obtain favorable results compared to a deep learning alternative.

Sparse Depth Enhanced Direct Thermal-Infrared SLAM Beyond the Visible Spectrum

In this letter, we propose a thermal-infrared simultaneous localization and mapping (SLAM) system enhanced by sparse depth measurements from light detection ranging (LiDAR). Thermal-infrared cameras are relatively robust against fog, smoke, and dynamic lighting conditions compared to RGB cameras operating under the visible spectrum. Exploiting thermal-infrared cameras for motion estimation and mapping is highly appealing. However, utilizing a thermal-infrared camera directly in existing vision-based methods is difficult because of the modality difference. This letter proposes a method to use sparse depth measurement for 6-DOF motion estimation via direct tracking under 14-bit raw measurement from the thermal camera. We also refine the local accuracy and include a loop closure to maintain global consistency. The experimental results demonstrate that the system is not only robust under various lighting conditions such as day and night, but it also overcomes the scale problem of monocular cameras.

AMiCUS-A Head Motion-Based Interface for Control of an Assistive Robot.

Within this work we present AMiCUS, a Human-Robot Interface that enables tetraplegics to control a multi-degree of freedom robot arm in real-time using solely head motion, empowering them to perform simple manipulation tasks independently. The article describes the hardware, software and signal processing of AMiCUS and presents the results of a volunteer study with 13 able-bodied subjects and 6 tetraplegics with severe head motion limitations. As part of the study, the subjects performed two different pick-and-place tasks. The usability was assessed with a questionnaire. The overall performance and the main control elements were evaluated with objective measures such as completion rate and interaction time. The results show that the mapping of head motion onto robot motion is intuitive and the given feedback is useful, enabling smooth, precise and efficient robot control and resulting in high user-acceptance. Furthermore, it could be demonstrated that the robot did not move unintendedly, giving a positive prognosis for safety requirements in the framework of a certification of a product prototype. On top of that, AMiCUS enabled every subject to control the robot arm, independent of prior experience and degree of head motion limitation, making the system available for a wide range of motion impaired users.

Dynamic 3D Hand Gesture Recognition by Learning Weighted Depth Motion Maps

Hand gesture recognition (HGR) from sequences of depth maps is a challenging computer vision task because of the low inter-class and high intra-class variability, different execution rates of each gesture, and the high articulated nature of the human hand. In this paper, a multilevel temporal sampling (MTS) method is first proposed that is based on the motion energy of keyframes of depth sequences. As a result, long, middle, and short sequences are generated that contain the relevant gesture information. The MTS results in increasing the intra-class similarity while raising the inter-class dissimilarities. The weighted depth motion map (WDMM) is then proposed to extract the spatiotemporal information from generated summarized sequences by an accumulated weighted absolute difference of consecutive frames. The histogram of gradient and local binary pattern are exploited to extract features from WDMM. The obtained results define the current state-of-the-art on three public benchmark datasets of: MSR Gesture 3D, SKIG, and MSR Action 3D, for 3D HGR. We also achieve competitive results on NTU action dataset.

Gesturing the source domain

Abstract Gesture is aptly described as a “backdoor” to cognition (Sweetser, 2007, p. 203). Co-speech gesture has been shown to aid in the representation of abstract concepts (Parrill & Sweetser, 2004) and, specifically, encode metaphorical source domains (Cienki, 1998). This paper examines how co-speech gesture aligns with spoken and written narrative to support a spatially based representation of gender identity. Repeated gestural patterns include inward facing palms used to mime fictive category boundaries, gestural mapping of motion across metaphorical gender regions, manual deictic reference to interior and exterior self, and distancing from past gender assignment signaled through emblematic scare quotes. The data examined in this paper confirm the important role gesture plays in supplementing the instantiation of the metaphorical models that organize transgender speakers’ experience with and discussion of gender and transition.

ACTION RECOGNITION USING UNDECIMATED DUAL TREE COMPLEX WAVELET TRANSFORM FROM DEPTH MOTION MAPS / DEPTH SEQUENCES

Abstract. Accumulating the motion information from a video sequence is one of the highly challenging and significant phase in Human Action Recognition. To achieve this, several classical and compact representations are proposed by the research community with proven applicability. In this paper, we propose a compact Depth Motion Map based representation methodology with hastey striding, consisely accumulating the motion information. We extract Undecimated Dual Tree Complex Wavelet Transform features from the proposed DMM, to form an efficient feature descriptor. We designate a Sequential Extreme Learning Machine for classifying the human action secquences on benchmark datasets, MSR Action 3D dataset and DHA Dataset. We empirically prove the feasability of our method under standard protocols, achieving proven results.

Numerical study of the material transport in the viscous vortex dipole flow

This paper presents a numerical study of the material transport of Lamb dipole(s) in the two-dimensional viscous flow. We focus on the properties of the rate of strain tensor, which has received less attention in the literature. It is noted that the eigenpairs of the tensor explicitly indicate the strength and direction of material stretching and compressing. The tensor provides a clear map of the material motion regardless of the complexity of the vortical flow. The strain rate field displays a rich structure as it contains five elliptic points and six hyperbolic points. It is interesting to observe that the left elliptic point of the strain rate field bifurcates into two at t > 0. Two kinds of material curves, circular and vertical, are used to illustrate the flow transport. The transport mechanism discussed here can be employed to explore the transport in more complex vortex flows.

A very deep two-stream network for crowd type recognition

Crowd type identification is a crucial task in the emergency alert. In this paper, to solve accurate identification of crowd type, the crowd type description triad C-BMO < Behavior, Mood, Organized > and a novel crowd type recognition network (CTRN): very deep two-stream network architecture are proposed, respectively. The very deep two-stream network architecture is based on the static map and motion map in the video. To early warn the emergency, the reasoning rules of the emergency alert are proposed based on joining the crowd type and the crowd characteristics. To verify the proposed method, the crowd type dataset is collected, and we experiment with the proposed plan on the crowd type dataset. The experimental results demonstrate that the proposed model is competitive compared with the state-of-the-art techniques.

Setup strategies and uncertainties in esophageal radiotherapy based on detailed intra- and interfractional tumor motion mapping

Background and purposeDetailed knowledge of target motion is important for improved accuracy and decreased toxicity of esophageal cancer radiotherapy. This study uses the 3D trajectories of implanted markers during setup CBCT scans to investigate the intra- and interfractional tumor motion in esophageal cancer radiotherapy. Material and methodsFor 21 esophageal cancer patients with implanted fiducial markers, 60-s 3D marker trajectories were estimated from the 2D marker positions in the projections of daily setup CBCT scans by a probability-based method. The motion was separated into respiratory and cardiac components by frequency analysis and motion magnitude (2nd–98th percentile) was extracted for each marker. The mean motion was calculated over all markers. The daily mean setup interfraction error for bony-anatomy and soft-tissue setup was used to estimate the margin accounting for interfractional motion. ResultsA total of 1036 marker trajectories were extracted using 427 CBCT scans and 63 markers. The mean motion magnitude over all markers was 2.9 mm (left–right (LR)), 8.8 mm (cranio-caudal (CC)) and 4.1 mm (anterior–posterior (AP)) for the full motion during CBCT acquisition with mean magnitudes of 2.7 mm (LR), 8.4 mm (CC) and 3.5 mm (AP) for respiratory motion and 1.0 mm (LR), 1.5 mm (CC) and 1.4 mm (AP) for cardiac motion. Substantial daily marker shifts relative to bones resulted in margins of 8.9 mm (LR), 9.5 mm (CC), and 7.3 mm (AP). Soft-tissue based setup in and near the CTV combined with rescanning of patients with anatomical changes reduced the margins to 6.9 mm (LR), 6.8 mm (CC), and 5.6 mm (AP). ConclusionsEsophageal tumor motion was mapped with unprecedented detail throughout the radiotherapy course. Respiratory motion dominated and was largest in the CC direction. Soft-tissue matching and an adaptive strategy reduced interfractional margins by 2–3 mm compared to bony-anatomy matching.

Multi-target tracking by non-linear motion patterns based on hierarchical network flows

We propose a multi-target tracking method for pedestrians, through locating them and maintaining their identities to make their individual trajectories. In most existing data association-based tracking approaches, the motion information in the affinity model is always linear, but the motion of target is non-linear in real-world surveillance video, especially when there is occlusion. This paper proposes a global optimization method based on hierarchical network flows with a non-linear motion model to track multiple targets. In our method, each node in network represents a tracklet, and each edge represents the likelihood of neighboring tracklets belonging to the same trajectory measured by affinity score that we proposed, it differs from other network flow formulation whose nodes represent detection responses. A non-linear motion map is used for explaining non-linear motion pattern between neighboring tracklets and getting motion affinity when there are time gaps. We evaluate our approach on 2DMOT2015, MOT16 database, and several real surveillance videos. Experimental results demonstrate that the proposed method can achieve continuous tracking trajectories under the case of motion direction changes and complete occlusions.

Resource-efficient and Automated Image-based Indoor Localization

Image-based indoor localization has aroused much interest recently because it requires no infrastructure support. Previous approaches on image-based localization, due to their computation and storage requirements, often process queries at servers. This does not scale well, incurs round-trip delay, and requires constant network connectivity. Many also require users to manually confirm the shortlisted matched landmarks, which is inconvenient, slow, and prone to selection error. To overcome these limitations, we propose a h ighly a utomated (in terms of image confirmation after taking images) i mage-based l ocalization algorithm (HAIL), distributed in mobile devices. HAIL achieves resource efficiency (in terms of storage and processing) by keeping only distinguishing visual features for each landmark, and employing the efficient k-d tree to search for features. It further utilizes motion sensors and map constraints to enhance the localization accuracy without user operation. We have implemented HAIL on Android platforms and conducted extensive experiments in a food plaza and a premium shopping mall. Experimental results show that it achieves much higher localization accuracy (reducing the localization error by more than 20%) and computation efficiency (by more than 40% in time) as compared with the state-of-the-art approaches.

3D Behavior Recognition Based on Multi-Modal Deep Space-Time Learning

This paper proposes a dual-stream 3D space-time convolutional neural network action recognition framework. The original depth map sequence data is set as the input in order to study the global space-time characteristics of each action category. The high correlation within the human action itself is considered in the time domain, and then the deep motion map sequence is introduced as the input to another stream of the 3D space-time convolutional network. Furthermore, the corresponding 3D skeleton sequence data is set as the third input of the whole recognition framework. Although the skeleton sequence data has the advantage of including 3D information, it is also confronted with the problems of the existence of rate change, temporal mismatch and noise. Thus, specially designed space-time features are applied to cope with these problems. The proposed methods allow the whole recognition system to fully exploit and utilize the discriminatory space-time features from different perspectives, and ultimately improve the classification accuracy of the system. Experimental results on different public 3D data sets illustrate the effectiveness of the proposed method.

Action recognition from depth sequence using depth motion maps-based local ternary patterns and CNN

This paper presents a method for human action recognition from depth sequences captured by the depth camera. The main idea of the method is the action mapping image classification via convolutional neural network (CNN) based approach. Firstly, we project the raw frames onto three orthogonal Cartesian planes and stack the results into three still images (corresponding to the front, side, and top views) to form the Depth Motion Maps (DMMs). Secondly, Local Ternary Pattern (LTP) is introduced as an image filter for DMMs, thus to improve the distinguishability of similar actions. Finally, we apply CNN to action recognition by classifying corresponding LTP-encoded images. Experimental results on the popular and challenging benchmark MSR-Action 3D and MSR-Gesture dataset show the effectiveness of the presented method and meet real-time action recognition task requirements.

ChaboNet : Design of a deep CNN for prediction of visual saliency in natural video

Prediction of visual saliency in images and video is needed for video understanding, search and retrieval, coding, watermarking and other applications. The majority of prediction models are founded only on “bottom-up” features. Nevertheless, the “top-down” component of human visual attention becomes prevalent as human observers explore the visual scene. Visual saliency which is always a mix of bottom-up and top-down cues can be predicted on the basis of seen data. In this paper, a model of prediction of visual saliency in video on the basis of Deep convolutional neural networks (CNNs) is proposed. A Deep CNN architecture is designed. Various input channels for a CNN architecture are studied: using the known sensitivity of human visual system to residual motion, pixel colour values are completed with residual motion map. The latter is a normalized energy of residual motion in video frames with regard to the estimated global affine motion model. The experiments show that the choice of the input features for the Deep CNN depends on visual task: for highly dynamic content, the proposed model with residual motion is more efficient and gives decent results with relatively shallow Deep architecture.

3D-CNN-Based Fused Feature Maps with LSTM Applied to Action Recognition

Human activity recognition is an active field of research in computer vision with numerous applications. Recently, deep convolutional networks and recurrent neural networks (RNN) have received increasing attention in multimedia studies, and have yielded state-of-the-art results. In this research work, we propose a new framework which intelligently combines 3D-CNN and LSTM networks. First, we integrate discriminative information from a video into a map called a ‘motion map’ by using a deep 3-dimensional convolutional network (C3D). A motion map and the next video frame can be integrated into a new motion map, and this technique can be trained by increasing the training video length iteratively; then, the final acquired network can be used for generating the motion map of the whole video. Next, a linear weighted fusion scheme is used to fuse the network feature maps into spatio-temporal features. Finally, we use a Long-Short-Term-Memory (LSTM) encoder-decoder for final predictions. This method is simple to implement and retains discriminative and dynamic information. The improved results on benchmark public datasets prove the effectiveness and practicability of the proposed method.

A Real-Time Upper-Body Robot Imitation System

An upper-body robot imitation (UBRI) system is proposed and developed to enable the human upper body imitation by a humanoid robot in real time. To achieve the imitation of arm motions, a geometry-based analytical method is presented and applied to extracting the joint angles of the human and mapping to the robot. Comparing to the traditional numerical methods of inverse kinematic computations, the geometrical analysis method generates a lower computational cost and maintains good imitation similarity. To map the human head motions to the head of the humanoid robot, a face tracking algorithm is employed to recognize the human face and track the human head poses in real time. A hand extraction and hand state recognition algorithm is proposed to achieve the hand motion mapping. At last, the completion rate and similarity evaluation experiments are conducted to verify the effectiveness of the proposed UBRI system.