Biological Motion Patterns Research Articles

A PCA-Based Active Appearance Model for Characterising Modes of Spatiotemporal Variation in Dynamic Facial Behaviours.

Faces carry key personal information about individuals, including cues to their identity, social traits, and emotional state. Much research to date has employed static images of faces taken under tightly controlled conditions yet faces in the real world are dynamic and experienced under ambient conditions. A common approach to studying key dimensions of facial variation is the use of facial caricatures. However, such techniques have again typically relied on static images, and the few examples of dynamic caricatures have relied on animating graphical head models. Here, we present a principal component analysis (PCA)-based active appearance model for capturing patterns of spatiotemporal variation in videos of natural dynamic facial behaviours. We demonstrate how this technique can be applied to generate dynamic anti-caricatures of biological motion patterns in facial behaviours. This technique could be extended to caricaturing other facial dimensions, or to more general analyses of spatiotemporal variations in dynamic faces.

Increasing the Sampling Efficiency of Protein Conformational Change by Combining a Modified Replica Exchange Molecular Dynamics and Normal Mode Analysis.

Understanding conformational change at an atomic level is significant when determining a protein functional mechanism. Replica exchange molecular dynamics (REMD) is a widely used enhanced sampling method to explore protein conformational space. However, REMD with an explicit solvent model requires huge computational resources, immensely limiting its application. In this study, a variation of parallel tempering metadynamics (PTMetaD) with the omission of solvent-solvent interactions in exchange attempts and the use of low-frequency modes calculated by normal-mode analysis (NMA) as collective variables (CVs), namely ossPTMetaD, is proposed with the aim to accelerate MD simulations simultaneously in temperature and geometrical spaces. For testing the performance of ossPTMetaD, five protein systems with diverse biological functions and motion patterns were selected, including large-scale domain motion (AdK), flap movement (HIV-1 protease and BACE1), and DFG-motif flip in kinases (p38α and c-Abl). The simulation results showed that ossPTMetaD requires much fewer numbers of replicas than temperature REMD (T-REMD) with a reduction of ∼70% to achieve a similar exchange ratio. Although it does not obey the detailed balance condition, ossPTMetaD provides consistent results with T-REMD and experimental data. The high accessibility of the large conformational change of protein systems by ossPTMetaD, especially in simulating the very challenging DFG-motif flip of protein kinases, demonstrated its high efficiency and robustness in the characterization of the large-scale protein conformational change pathway and associated free energy profile.

Points and Stripes: A Novel Technique for Masking Biological Motion Point-Light Stimuli.

Human articulated motion can be readily recognized robustly even from impoverished so-called point-light displays. Such sequence information is processed by separate visual processing channels recruiting different stages at low and intermediate levels of the cortical visual processing hierarchy. The different contributions that motion and form information make to form articulated, or biological, motion perception are still under investigation. Here we investigate experimentally whether and how specific spatio-temporal features, such as extrema in the motion energy or maximum limb expansion, indicated by the lateral and longitudinal extension, constrain the formation of the representations of articulated body motion. In order to isolate the relevant stimulus properties we suggest a novel masking technique, which allows to selectively impair the ankle information of the body configuration while keeping the motion of the point-light locations intact. Our results provide evidence that maxima in feature channel representations, e.g., the lateral or longitudinal extension, define elemental features to specify key poses of biological motion patterns. These findings provide support for models which aim at automatically building visual representations for the cortical processing of articulated motion by identifying temporally localized events in a continuous input stream.

Discrimination of movement and visual transfer abilities in cichlids (Pseudotropheus zebra)

Fish rival birds and mammals in many of their cognitive skills, and have been shown to successfully discriminate between a range of stationary and moving objects. The present study tested the ability of Pseudotropheus zebra to recognize unique movement patterns shown in the form of a single moving dot, point displays (PDs), point-light displays (PLDs), and videos of moving organisms in two alternative forced-choice experiments. Cichlids successfully distinguished between (1) different directions of movement, (2) a biological vs. a random movement, (3) a biological (human walking) vs. a scrambled motion pattern, and (4) two biological motion patterns (human walking to the left vs. to the right). Following training in (3), it was tested if the walking human was also correctly identified when either the positive, the alternative, or both stimuli were altered or presented inverted; following training in (4), stimuli were presented inverted or moving backwards. With the exception of the presentation of inverted and backwards-moving stimuli, fish excelled at these tasks. Furthermore, cichlids successfully discriminated between videos of different organisms such as eel vs. trout, human vs. dog, eagle vs. bat, and dolphin vs. shark. Following each training, a series of transfer tests elucidated whether P. zebra could also recognize these organisms when shown in transfer test trials (a) from a different perspective (front or sideways), (b) enlarged or downsized, or (c) as PDs. With few exceptions, all individuals learned all tasks and significantly often chose the previously reinforced (but altered) training stimulus over the alternative one during transfer tests. This indicates that cichlids have the ability to recognize a familiar organism under new conditions, for example, based on its movement alone, which may be helpful in recognizing approaching predators early on. Fish can solve a variety of learning and memory tasks including visual discrimination of objects. Here it was tested if cichlids (Pseudotropheus zebra) can distinguish between pairs of videos featuring individually moving dots, dot patterns and moving organisms. Overall, fish were very successful and solved most of the presented tasks. They differentiated between movement directions and movement types, recognized stimuli in transfer tests under altered conditions, and identified familiar organisms when shown resized, from new perspectives or in form of point displays. To be able to recognize movement and to identify organisms based on their specific movement patterns (alone) in the wild is essential to individual survival, as predators, conspecifics and/or prey are usually mobile. Additionally, predators are often encountered at night or when light levels are low, making it harder to successfully distinguish general body features such as coloration and adding significance to being able to quickly recognize motion cues. It should be equally advantageous to be able to recognize organisms such as predators from unfamiliar perspectives. Stimuli used in this study were not of any ecological relevance; they were chosen purely for their characteristic type of movement and just served as vectors to determine transfer abilities of gained knowledge onto new situations. Nonetheless, having excelled at tasks using these stimuli, it seems quite likely that cichlids would perform even better under ecologically significant conditions.

Visual identification of biological motion for underwater human–robot interaction

We present an algorithm for underwater robots to visually detect and track human motion. Our objective is to enable human–robot interaction by allowing a robot to follow behind a human moving in (up to) six degrees of freedom. In particular, we have developed a system to allow a robot to detect, track and follow a scuba diver by using frequency-domain detection of biological motion patterns. The motion of biological entities is characterized by combinations of periodic motions which are inherently distinctive. This is especially true of human swimmers. By using the frequency-space response of spatial signals over a number of video frames, we attempt to identify signatures pertaining to biological motion. This technique is applied to track scuba divers in underwater domains, typically with the robot swimming behind the diver. The algorithm is able to detect a range of motions, which includes motion directly away from or towards the camera. Once detected, the motion of the diver relative to the vehicle is then tracked using an Unscented Kalman Filter, an approach for non-linear estimation. The efficiency of our approach makes it attractive for real-time applications on-board our underwater vehicle, and in future applications we intend to track scuba divers in real-time with the robot. The paper presents an algorithmic overview of our approach, together with experimental evaluation based on underwater video footage.

Recognition of the semantics and kinematics of gestures: Neural responses to “what” and “how”?

The extensive use of gestures for human-human communication, independently of culture and language, suggests an underlying universal neural mechanism for gesture recognition. The mirror neuron system (MNS) is known to respond to observed human actions, and overlaps with self-action. The minimal cues needed for activation of the MNS for gesture recognition, facial expressions and bodily dynamics, is not yet defined. Using LED-point representations of gestures, we compared two types of brain activations: 1) in response to human recognizable vs non-recognizable motion and 2) in response to human vs non-human motion. Our preliminary results show that parts of the MNS respond only to human kinematics, and not to nonhuman kinematics, suggesting that the brain has a mechanism of discriminating human from nonhuman motion, even if the pattern of motion is meaningless, but still follows biological motion patterns. This implies that mechanisms of learning-mimicking, empathy and emotional communication, are possibly constrained by biological motion patterns. We then suggest a two-tier-model of human-bodily-communication: (1) recognition of human biological kinematics; (2) recognition of meaning. Implications are both theoretical (understanding the underlying mechanism for action recognition) and applicative (in digital graphical social representations, motion should be reasonably biological to generate the same emotional and mimicking automatic mechanisms as in face-to-face social interactions).

Spatiotemporal Deformable Prototypes for Motion Anomaly Detection

This paper presents an approach for motion-based anomaly detection, where a prototype pattern is detected and elastically registered against a test sample to detect anomalies in the test sample. The prototype model is learned from multiple sequences to define accepted variations. "Supertrajectories" based on hierarchical clustering of dense point trajectories serve as an efficient and robust representation of motion patterns. An efficient hashing approach provides transformation hypotheses that are refined by a spatiotemporal elastic registration. We propose a new method for elastic registration of 3D+time trajectory patterns that induces spatial elasticity from trajectory affinities. The method is evaluated on a new motion anomaly dataset of juggling patterns and performs well in detecting subtle anomalies. Moreover, we demonstrate the applicability to biological motion patterns.

The effect of oxytocin on biological motion perception in dogs (Canis familiaris).

Recent studies have shown that the neuropeptide oxytocin is involved in the regulation of several complex human social behaviours. There is, however, little research on the effect of oxytocin on basic mechanisms underlying human sociality, such as the perception of biological motion. In the present study, we investigated the effect of oxytocin on biological motion perception in dogs (Canis familiaris), a species adapted to the human social environment and thus widely used to model many aspects of human social behaviour. In a within-subjects design, dogs (N = 39), after having received either oxytocin or placebo treatment, were presented with 2D projection of a moving point-light human figure and the inverted and scrambled version of the same movie. Heart rate (HR) and heart rate variability (HRV) were measured as physiological responses, and behavioural response was evaluated by observing dogs' looking time. Subjects were also rated on the personality traits of Neuroticism and Agreeableness by their owners. As expected, placebo-pretreated (control) dogs showed a spontaneous preference for the biological motion pattern; however, there was no such preference after oxytocin pretreatment. Furthermore, following the oxytocin pretreatment female subjects looked more at the moving point-light figure than males. The individual variations along the dimensions of Agreeableness and Neuroticism also modulated dogs' behaviour. Furthermore, HR and HRV measures were affected by oxytocin treatment and in turn played a role in subjects' looking behaviour. We discuss how these findings contribute to our understanding of the neurohormonal regulatory mechanisms of human (and non-human) social skills.

Electroluminescent Materials Can Improve Nighttime Pedestrian Conspicuity When Added to Retroreflective Garments

Retroreflective materials make nighttime pedestrians more conspicuous to drivers, especially when configured in a biological motion pattern. Because retroreflectors are only effective when they are illuminated by a light source near the driver, they are not useful for pedestrians who are positioned outside the beam of a vehicle’s headlamps. Electroluminescent materials have the potential to enhance conspicuity in these conditions. We investigated the conspicuity benefits of adding electroluminescent material to clothing containing retroreflectors. We compared, using an open-road course at night, the distances at which 120 observers responded to a test pedestrian wearing one of two garments at one of three roadside locations. Response distances to the garment that contained both electroluminescent and retroreflective materials were significantly longer, particularly when the pedestrian was more poorly illuminated as a result of being positioned on the left shoulder. These findings suggest electroluminescent materials have the potential to enhance the conspicuity of pedestrians.

The development of gaze behaviors in response to biological motion displays

Although the relationship between biological motion perception as depicted by point-light displays and social cognition has been investigated in recent decades, the developmental course of the integration of social cognition and the perception of biological motion is not well understood. To better understand this development, we investigated the ability of 9- and 12-month-old infants to shift their gaze toward a point-light upright human figure using a paradigm similar to that used by Yoon and Johnson (2009). We found that 12-month-old, but not 9-month-old, infants were able to follow the direction of attention of the upright point-light figure (Experiments 1 and 2). However, both the younger and older infants were able to follow the attentional shift of others under the full-view condition (Experimental 3). These results suggest that the ability to process the higher-level information provided by biological motion patterns, such as the attentional direction of others, develops by 12 months, but not by 9 months, of age. The relationship between the development of social cognition and that of biological motion perception is discussed.

Automatic video tracking of Chinese mitten crabs based on the particle filter algorithm using a biologically constrained probe and resampling

The behavioral patterns of crabs affect the quality and quantity of their production. Currently there are no methods to track behavioral patterns using imaging-based algorithms. This study provides a precise pathway tracking method for the research of the relationship between behavioral patterns of crabs and their living environment. The Particle Filter algorithm is an appropriate framework to handle non-Gaussian movement, the biological constraints are used to decrease the computational complexity and improve the accuracy of tracking the crabs’ pathways and a biological probe is utilized to determine the area of movement. Then, this newly determined area replaces the entire image frame as the search space to reduce the complexity of the computation. In the resampling step, a traditional Gaussian particle distribution is substituted by a fusiform particle distribution, which better matches the crab’s biological motion patterns, to represent the probability of the crab movement. This strategy allows the crab positions to be covered using fewer particles, which is more accurate for analyzing abrupt motion or long-term stationary situations than traditional particle distributions. To determine the robustness and accuracy of the results, 3000 and 12,000 frames were used, respectively. The coverage ratio and accuracy increased by 28.79% and 5.75%, respectively, compared with the color histogram-based particle filter (CHPF) and by 69.57% and 37.66% compared with the fission bootstrap particle filter (FBPF). The experimental results show that the proposed tracking method is feasible and can be used as an efficient tool to get the pathway of crabs under water.

View dependencies in the visual recognition of social interactions.

Recognizing social interactions, e.g., two people shaking hands, is important for obtaining information about other people and the surrounding social environment. Despite the visual complexity of social interactions, humans have often little difficulties to visually recognize social interactions. What is the visual representation of social interactions and the bodily visual cues that promote this remarkable human ability? Viewpoint dependent representations are considered to be at the heart of the visual recognition of many visual stimuli including objects (Bülthoff and Edelman, 1992), and biological motion patterns (Verfaillie, 1993). Here we addressed the question whether complex social actions acted out between pairs of people, e.g., hugging, are also represented in a similar manner. To this end, we created 3-D models from motion captured actions acted out by two people, e.g., hugging. These 3-D models allowed to present the same action from different viewpoints. Participants' task was to discriminate a target action from distractor actions using a one-interval-forced-choice (1IFC) task. We measured participants' recognition performance in terms of reaction times (RT) and d-prime (d'). For each tested action we found one view that led to superior recognition performance compared to other views. This finding demonstrates view-dependent effects of visual recognition, which are in line with the idea of a view-dependent representation of social interactions. Subsequently, we examined the degree to which velocities of joints are able to predict the recognition performance of social interactions in order to determine candidate visual cues underlying the recognition of social interactions. We found that the velocities of the arms, both feet, and hips correlated with recognition performance.

Healthy Older Observers Cannot Use Biological-Motion Point-Light Information Efficiently within 4 m of Themselves

Healthy aging is associated with a number of perceptual changes, but measures of biological-motion perception have yielded conflicting results. Biological motion provides information about a walker, from gender and identity to speed, direction, and distance. In our natural environment, as someone approaches us (closer distances), the walker spans larger areas of our field of view, the extent of which can be underutilized with age. Yet, the effect of age on biological-motion perception in such real-world scenarios remains unknown. We assessed the effect of age on discriminating walking direction in upright and inverted biological-motion patterns, positioned at various distances in virtual space. Findings indicate that discrimination is worse at closer distances, an effect exacerbated by age. Older adults’ performance decreases at distances as far away as 4 m, whereas younger adults maintain their performance as close as 1 m (worse at 0.5 m). This suggests that older observers are limited in their capacity to integrate information over larger areas of the visual field and supports the notion that age-related effects are more apparent when larger neural networks are required to process simultaneous information. This has further implications for social contexts where information from biological motion is critical.

Percepção de velocidade do movimento biológico: mais resistente ao fenômeno de interferência?

O sistema visual humano é frequentemente referido como altamente preparado para extrair informação relevante de padrões de movimento biológico. Nesse sentido, este estudo analisa o efeito de contraste na percepção de velocidade. Os participantes realizaram o julgamento de velocidade numa situação na qual dois point-light walkers simultâneos foram apresentados com diferentes contrastes relativamente ao fundo e com diferentes velocidades de translação. Na Experiência 1, o movimento de translação biológico canônico foi comparado com o movimento de translação rígido, enquanto na Experiência 2 ele foi comparado com o movimento de translação biológico invertido. O padrão biológico canônico apresenta maior taxa de erro, tempos de reação mais elevados e maior vulnerabilidade ao efeito de contraste na percepção da velocidade do que o padrão rígido. No entanto, não foram encontradas diferenças significativas entre o estímulo canônico e o invertido. A Experiência 3 foi implementada com o objetivo de se controlar o papel das pistas posicionais na tarefa de julgamento de velocidade. Os pontos iniciais e finais da trajetória foram combinados de modo a que os point-light walkers mais rápidos e os mais lentos pudessem terminar o ensaio numa posição relativamente mais avançada ou atrasada. Apesar desta variação, o padrão de resultados foi congruente com as observações das Experiências 1 e 2. Aparentemente, os participantes realizaram julgamentos de velocidade factuais, ao invés do uso de pistas espaciais como uma espécie de referência ou comparação de posicionamento. Dado que a percepção dos padrões biológicos foi mais vulnerável aos efeitos de contraste, mas não foi afetada pela familiaridade, este estudo sugere que a percepção de movimento biológico e rígido poderá obedecer às mesmas regras computacionais, pelo menos em tarefas que envolvam padrões em translação e julgamentos de velocidade.

Perceptual and computational analysis of critical features for biological motion

Among the most common events in our daily lives is seeing people in action. Scientists have accumulated evidence suggesting humans may have developed specialized mechanisms for recognizing these visual events. In the current experiments, we apply the "bubbles" technique to construct space-time classification movies that reveal the key features human observers use to discriminate biological motion stimuli (point-light and stick figure walkers). We find that observers rely on similar features for both types of stimuli, namely, form information in the upper body and dynamic information in the relative motion of the limbs. To measure the contributions of motion and form analyses in this task, we computed classification movies from the responses of a biologically plausible model that can discriminate biological motion patterns (M. A. Giese & T. Poggio, 2003). The model classification movies reveal similar key features to observers, with the model's motion and form pathways each capturing unique aspects of human performance. In a second experiment, we computed classification movies derived from trials of varying exposure times (67-267 ms) and demonstrate the transition to form-based strategies as motion information becomes less available. Overall, these results highlight the relative contributions of motion and form computations to biological motion perception.

Perception of biological motion in common marmosets (Callithrix jacchus): by females only

The ability to perceive biological motion (BM) has been demonstrated in a number of species including humans but the few studies of non-human primates have been relatively inconclusive. We investigated whether common marmosets (Callithrix jacchus) are able to perceive biological motion, using a novel method to test non-human primates. Marmosets (7 male and 7 female) were trained to remove a cover from a container and look inside it, revealing a computer screen. Then they were presented with images on this computer screen consisting of a novel BM pattern (a walking hen) and 4 manipulations of that pattern (a static frame of this pattern and inverted, scrambled, and rotating versions of the pattern). The behavioural responses of the marmosets were recorded and used to assess discrimination between stimuli. BM was attended to by females but not males, as shown by active inspection behaviour, mainly movement of the head towards the stimulus. Females paid significantly less attention to all of the other stimuli. This indicates the females' ability to attend to biological motion. Females showed slightly more attention to the inverted BM than to the static, scrambled, and rotating patterns. The males were less attentive to all of the stimuli than were the females and, unlike the females, responded to all stimuli in a similar manner. This sex difference could be due to an inability of males to recognise BM altogether or to a lesser amount of curiosity. Considered together with the findings of previous studies on chicks and humans, the results of the present study support the notion of a common mechanism across species for the detection of BM.

Metrics of the perception of body movement

Body movements are recognized with speed and precision, even from strongly impoverished stimuli. While cortical structures involved in biological motion recognition have been identified, the nature of the underlying perceptual representation remains largely unknown. We show that visual representations of complex body movements are characterized by perceptual spaces with well-defined metric properties. By multidimensional scaling, we reconstructed from similarity judgments the perceptual space configurations of stimulus sets generated by motion morphing. These configurations resemble the true stimulus configurations in the space of morphing weights. In addition, we found an even higher similarity between the perceptual metrics and the metrics of a physical space that was defined by distance measures between joint trajectories, which compute spatial trajectory differences after time alignment using a robust error norm. These outcomes were independent of the experimental paradigm for the assessment of perceived similarity (pairs-comparison vs. delayed match-to-sample) and of the method of stimulus presentation (point-light stimuli vs. stick figures). Our findings suggest that the visual perception of body motion is veridical and closely reflects physical similarities between joint trajectories. This implies that representations of form and motion share fundamental properties and places constraints on the computational mechanisms that support the recognition of biological motion patterns.

Static and dynamic body image in bulimia nervosa: Mental representation of body dimensions and biological motion patterns

The aim of the present study was to find out whether in bulimia nervosa the perceptual component of a disturbed body image is restricted to the overestimation of one's own body dimensions (static body image) or can be extended to a misperception of one's own motion patterns (dynamic body image). Participants with bulimia nervosa (n = 30) and normal controls (n = 55) estimated their body dimensions by means of a photo distortion technique and their walking patterns using a biological motion distortion device. Not only did participants with bulimia nervosa overestimate their own body dimensions, but also they perceived their own motion patterns corresponding to a higher BMI than did controls. Static body image was correlated with shape/weight concerns and drive for thinness, whereas dynamic body image was associated with social insecurity and body image avoidance. In bulimia nervosa, body image disturbances can be extended to a dynamic component.

Patterns of fMRI Activity Dissociate Overlapping Functional Brain Areas that Respond to Biological Motion

Accurate perception of the actions and intentions of other people is essential for successful interactions in a social environment. Several cortical areas that support this process respond selectively in fMRI to static and dynamic displays of human bodies and faces. Here we apply pattern-analysis techniques to arrive at a new understanding of the neural response to biological motion. Functionally defined body-, face-, and motion-selective visual areas all responded significantly to "point-light" human motion. Strikingly, however, only body selectivity was correlated, on a voxel-by-voxel basis, with biological motion selectivity. We conclude that (1) biological motion, through the process of structure-from-motion, engages areas involved in the analysis of the static human form; (2) body-selective regions in posterior fusiform gyrus and posterior inferior temporal sulcus overlap with, but are distinct from, face- and motion-selective regions; (3) the interpretation of region-of-interest findings may be substantially altered when multiple patterns of selectivity are considered.

Gait perception of part‐reverse biological motion pattern produced by eight point‐lights attached to the back of a walker1

Abstract: The Johansson side‐view point‐light walker (PLW) gives a vivid impression of an invisible human, and the Cutting Cm (center of moment), determined by the relative widths of the shoulders and hips of the walker, is a well‐known important parameter for sex identification of perceived gait. The back‐view PLWs produced by the eight point‐lights attached to the back of a walker also give a clear image of human gait (Sumi, 2000). In the present investigation, the part‐reverse (P‐R) PLWs were used as stimuli. In the P‐R PLWs, the lower‐half elements (knees and heels) of the back‐view PLWs were arranged to be the reverse of their normal PLWs and the upper half elements (shoulders and hips) were arranged exactly the same as their normal PLWs. Results revealed that the P‐R PLWs were identified correctly as often as the normal PLWs, indicating that the Cutting Cm is valid for the back‐view PLWs. It was also found that, when the sex of the P‐R PLW was identified correctly, the age was estimated to be younger, whereas when the sex was not identified correctly, the age was estimated to be older. This finding was discussed in relation to the difference between the data profiles obtained from the P‐R PLWs and the normal PLWs.