Encoding Of Motion Research Articles

Encoding of global visual motion in the avian pretectum shifts from a bias for temporal-to-nasal selectivity to omnidirectional excitation across speeds.

The pretectum of vertebrates contains neurons responsive to global visual motion. These signals are sent to the cerebellum, forming a subcortical pathway for processing optic flow. Global motion neurons exhibit selectivity for both direction and speed, but this is usually assessed by first determining direction preference at intermediate velocity (16-32 deg/sec), and then assessing speed tuning at the preferred direction. A consequence of this approach is that it is unknown if and how direction preference changes with speed. We measured directional selectivity in 114 pretectal neurons from 44 zebra finches (Taeniopygia guttata) across spatial and temporal frequencies, corresponding to a speed range of 0.062 to 1024°/s. Pretectal neurons were most responsive at 32-64°/s with lower activity as speed increased or decreased. At each speed, we determined if cells were directionally-selective, bidirectionally-selective, omnidirectionally responsive, or unmodulated. Notably, at 32°/s, 60% of the cells were directionally selective and 28% were omnidirectionally responsive. In contrast, at 1024°/s, 20% of the cells were directionally selective and nearly half of the population was omnidirectionally responsive. Only 15% of the cells were omnidirectionally excited across most speeds. The remaining 85% of the cells had direction tuning that changed with speed. Collectively, these results indicate a shift from a bias for directional tuning at intermediate speeds of global visual motion to a bias for omnidirectional responses at faster speeds. These results suggest a potential role for the pretectum during flight by detecting unexpected drift or potentials collisions, depending on the speed of the optic flow signal.Significance Statement During locomotion, images of edges and surfaces in the environment move across the retina, a signal of global visual motion called optic flow. Retinal recipient areas in the accessory optic system and the pretectum are the earliest sites to encode this signal, and the neurons are selective for direction and speed. Previous work suggested that directional selectivity may change across speeds but this has never been systematically studied. We measured direction preferences from 0.062 to 1024°/s in the avian pretectum. We found that pretectal global motion neurons are biased for temporal-to-nasal motion at intermediate speeds but biased for omnidirectional responses at faster speeds. These results suggest the pretectum could function to detect both unexpected drift and potential collisions during locomotion.

Development of neural circuits for social motion perception in schooling fish

The collective behavior of animal groups emerges from the interactions among individuals. These social interactions produce the coordinated movements of bird flocks and fish schools, but little is known about their developmental emergence and neurobiological foundations. By characterizing the visually based schooling behavior of the micro glassfish Danionella cerebrum, we found that social development progresses sequentially, with animals first acquiring the ability to aggregate, followed by postural alignment with social partners. This social maturation was accompanied by the development of neural populations in the midbrain that were preferentially driven by visual stimuli that resemble the shape and movements of schooling fish. Furthermore, social isolation over the course of development impaired both schooling behavior and the neural encoding of social motion in adults. This work demonstrates that neural populations selective for the form and motion of conspecifics emerge with the experience-dependent development of collective movement.

Schematicity vs. lexicality: typological differences between Danish and Spanish

ABSTRACT The aim of this article is to demonstrate how Danish and Spanish from a typological, cross-linguistic perspective differ systematically from each other both within the domain of verbal lexicalization patterns related to the encoding of the semantic components Motion, Manner and Path and in terms of the predicate formation strategies dominating the two languages. First, on the basis of a contrastive analysis between the Spanish Motion-Path conflating verb pairs meter ‘insert’/sacar ‘extract’ and entrar ‘enter’/salir ‘exit’ and the Danish activity/Manner-of-motion verbs sprøjte ‘spray’ and stikke ‘stick’, it is argued that each language disposes of sets of abstract lexemes that do not exist in the other language. They lexicalize on different levels, one could say. Second, and strongly related to the first point, the article suggests that the preference of Danish to create complex schematic expression structures, where Spanish tends to concentrate information in monomorphemic, semantically saturated lexemes, can be explained by applying a template for valence variation that distinguishes between two basic and two extended sentence structures. Specifically, it is the (im)possibility of realizing the so-called A relation, the ‘third’ argument, that constitutes the differentiating factor between Danish and Spanish.

Multimodal encoding of motion events in speech, gesture and cognition

Abstract How people communicate about motion events and how this is shaped by language typology are mostly studied with a focus on linguistic encoding in speech. Yet, human communication typically involves an interactional exchange of multimodal signals, such as hand gestures that have different affordances for representing event components. Here, we review recent empirical evidence on multimodal encoding of motion in speech and gesture to gain a deeper understanding of whether and how language typology shapes linguistic expressions in different modalities, and how this changes across different sensory modalities of input and interacts with other aspects of cognition. Empirical evidence strongly suggests that Talmy’s typology of event integration predicts multimodal event descriptions in speech and gesture and visual attention to event components prior to producing these descriptions. Furthermore, variability within the event itself, such as type and modality of stimuli, may override the influence of language typology, especially for expression of manner.

Distributed and hierarchical neural encoding of multidimensional biological motion attributes in the human brain.

The human visual system can efficiently extract distinct physical, biological, and social attributes (e.g. facing direction, gender, and emotional state) from biological motion (BM), but how these attributes are encoded in the brain remains largely unknown. In the current study, we used functional magnetic resonance imaging to investigate this issue when participants viewed multidimensional BM stimuli. Using multiple regression representational similarity analysis, we identified distributed brain areas, respectively, related to the processing of facing direction, gender, and emotional state conveyed by BM. These brain areas are governed by a hierarchical structure in which the respective neural encoding of facing direction, gender, and emotional state is modulated by each other in descending order. We further revealed that a portion of the brain areas identified in representational similarity analysis was specific to the neural encoding of each attribute and correlated with the corresponding behavioral results. These findings unravel the brain networks for encoding BM attributes in consideration of their interactions, and highlight that the processing of multidimensional BM attributes is recurrently interactive.

Dissociation in neuronal encoding of object versus surface motion in the primate brain

A paradox exists in our understanding of motion processing in the primate visual system: neurons in the dorsal motion processing stream often strikingly fail to encode long-range and perceptually salient jumps of a moving stimulus. Psychophysical studies suggest that such long-range motion, which requires integration over more distant parts of the visual field, may be based on higher-order motion processing mechanisms that rely on feature or object tracking. Here, we demonstrate that ventral visual area V4, long recognized as critical for processing static scenes, includes neurons that maintain direction selectivity for long-range motion, even when conflicting local motion is present. These V4 neurons exhibit specific selectivity for the motion of objects, i.e., targets with defined boundaries, rather than the motion of surfaces behind apertures, and are selective for direction of motion over a broad range of spatial displacements and defined by a variety of features. Motion direction at a range of speeds can be accurately decoded on single trials from the activity of just a few V4 neurons. Thus, our results identify a novel motion computation in the ventral stream that is strikingly different from, and complementary to, the well-established system in the dorsal stream, and they support the hypothesis that the ventral stream system interacts with the dorsal stream to achieve the higher level of abstraction critical for tracking dynamic objects.

A two-stage framework for neural processing of biological motion

It remains to be understood how biological motion is hierarchically computed, from discrimination of local biological motion animacy to global dynamic body perception. Here, we addressed this functional separation of the correlates of the perception of local biological motion from perception of global motion of a body. We hypothesized that local biological motion processing can be isolated, by using a single dot motion perceptual decision paradigm featuring the biomechanical details of local realistic motion of a single joint. To ensure that we were indeed tackling processing of biological motion properties we used discrimination instead of detection task. We discovered using representational similarity analysis that two key early dorsal and two ventral stream regions (visual motion selective hMT+ and V3A, extrastriate body area EBA and a region within fusiform gyrus FFG) showed robust and separable signals related to encoding of local biological motion and global motion-mediated shape. These signals reflected two independent processing stages, as revealed by representational similarity analysis and deconvolution of fMRI responses to each motion pattern. This study showed that higher level pSTS encodes both classes of biological motion in a similar way, revealing a higher-level integrative stage, reflecting scale independent biological motion perception. Our results reveal a two-stage framework for neural computation of biological motion, with an independent contribution of dorsal and ventral regions for the initial stage.

Asymmetrical cross-modal influence on neural encoding of auditory and visual features in natural scenes

Natural scenes contain multi-modal information, which is integrated to form a coherent perception. Previous studies have demonstrated that cross-modal information can modulate neural encoding of low-level sensory features. These studies, however, mostly focus on the processing of single sensory events or rhythmic sensory sequences. Here, we investigate how the neural encoding of basic auditory and visual features is modulated by cross-modal information when the participants watch movie clips primarily composed of non-rhythmic events. We presented audiovisual congruent and audiovisual incongruent movie clips, and since attention can modulate cross-modal interactions, we separately analyzed high- and low-arousal movie clips. We recorded neural responses using electroencephalography (EEG), and employed the temporal response function (TRF) to quantify the neural encoding of auditory and visual features. The neural encoding of sound envelope is enhanced in the audiovisual congruent condition than the incongruent condition, but this effect is only significant for high-arousal movie clips. In contrast, audiovisual congruency does not significantly modulate the neural encoding of visual features, e.g., luminance or visual motion. In summary, our findings demonstrate asymmetrical cross-modal interactions during the processing of natural scenes that lack rhythmicity: Congruent visual information enhances low-level auditory processing, while congruent auditory information does not significantly modulate low-level visual processing.

Gaussian information bottleneck and the non-perturbative renormalization group

The renormalization group (RG) is a class of theoretical techniques used to explain the collective physics of interacting, many-body systems. It has been suggested that the RG formalism may be useful in finding and interpreting emergent low-dimensional structure in complex systems outside of the traditional physics context, such as in biology or computer science. In such contexts, one common dimensionality-reduction framework already in use is information bottleneck (IB), in which the goal is to compress an ‘input’ signal X while maximizing its mutual information with some stochastic ‘relevance’ variable Y. IB has been applied in the vertebrate and invertebrate processing systems to characterize optimal encoding of the future motion of the external world. Other recent work has shown that the RG scheme for the dimer model could be ‘discovered’ by a neural network attempting to solve an IB-like problem. This manuscript explores whether IB and any existing formulation of RG are formally equivalent. A class of soft-cutoff non-perturbative RG techniques are defined by families of non-deterministic coarsening maps, and hence can be formally mapped onto IB, and vice versa. For concreteness, this discussion is limited entirely to Gaussian statistics (GIB), for which IB has exact, closed-form solutions. Under this constraint, GIB has a semigroup structure, in which successive transformations remain IB-optimal. Further, the RG cutoff scheme associated with GIB can be identified. Our results suggest that IB can be used to impose a notion of ‘large scale’ structure, such as biological function, on an RG procedure.

The development of the encoding of deictic motion in the Bantu language Rangi: grammaticalisation and change

AbstractThe close cross-linguistic relation between the domains of space and time has been well described. The frequent emergence of Tense-Aspect-Mood (TAM) markers from deictic motion verbs in particular, has also been extensively detailed in the literature. This paper focusses on the less well-known link between associated motion, a category of functional morphemes expressing (deictic) motion events, and TAM, in a language contact situation. Specifically, it provides a synchronic and diachronic description of three associated motion prefixes,joo-,tóó-andkoo-, found in the Tanzanian Bantu language Rangi, spoken in an area of high linguistic diversity. It proposes that the prefixjoo-encodes movement towards a deictic centre,tóó-encodes movement towards a goal which is not the deictic centre, andkoo-encodes movement away from a deictic centre. It further contends that whiletóó-andkoo-have maintained a purely deictic function,joo-has grammaticalised to assume an additional function whereby it encodes future tense, possibly aided by the absence of a dedicated future tense marker in the language. This three-way morphological encoding of spatial relations on the verb form is not a common characteristic of East African Bantu languages. However, this paper proposes that the system in Rangi can be accounted for on the basis of cross-linguistically widely attested pathways of grammatical change.

Faster Post-Earthquake Damage Assessment Based on 1D Convolutional Neural Networks

Contemporary deep learning approaches for post-earthquake damage assessments based on 2D convolutional neural networks (CNNs) require encoding of ground motion records to transform their inherent 1D time series to 2D images, thus requiring high computing time and resources. This study develops a 1D CNN model to avoid the costly 2D image encoding. The 1D CNN model is compared with a 2D CNN model with wavelet transform encoding and a feedforward neural network (FNN) model to evaluate prediction performance and computational efficiency. A case study of a benchmark reinforced concrete (r/c) building indicated that the 1D CNN model achieved a prediction accuracy of 81.0%, which was very close to the 81.6% prediction accuracy of the 2D CNN model and much higher than the 70.8% prediction accuracy of the FNN model. At the same time, the 1D CNN model reduced computing time by more than 90% and reduced resources used by more than 69%, as compared to the 2D CNN model. Therefore, the developed 1D CNN model is recommended for rapid and accurate resultant damage assessment after earthquakes.

Predictive encoding of motion begins in the primate retina.

Predictive motion encoding is an important aspect of visually guided behavior that allows animals to estimate the trajectory of moving objects. Motion prediction is understood primarily in the context of translational motion, but the environment contains other types of behaviorally salient motion correlation such as those produced by approaching or receding objects. However, the neural mechanisms that detect and predictively encode these correlations remain unclear. We report here that four of the parallel output pathways in the primate retina encode predictive motion information, and this encoding occurs for several classes of spatiotemporal correlation that are found in natural vision. Such predictive coding can be explained by known nonlinear circuit mechanisms that produce a nearly optimal encoding, with transmitted information approaching the theoretical limit imposed by the stimulus itself. Thus, these neural circuit mechanisms efficiently separate predictive information from nonpredictive information during the encoding process.

Spatially displaced excitation contributes to the encoding of interrupted motion by a retinal direction-selective circuit.

Spatially distributed excitation and inhibition collectively shape a visual neuron's receptive field (RF) properties. In the direction-selective circuit of the mammalian retina, the role of strong null-direction inhibition of On-Off direction-selective ganglion cells (On-Off DSGCs) on their direction selectivity is well-studied. However, how excitatory inputs influence the On-Off DSGC's visual response is underexplored. Here, we report that On-Off DSGCs have a spatially displaced glutamatergic receptive field along their horizontal preferred-null motion axes. This displaced receptive field contributes to DSGC null-direction spiking during interrupted motion trajectories. Theoretical analyses indicate that population responses during interrupted motion may help populations of On-Off DSGCs signal the spatial location of moving objects in complex, naturalistic visual environments. Our study highlights that the direction-selective circuit exploits separate sets of mechanisms under different stimulus conditions, and these mechanisms may help encode multiple visual features.

Roll up / venez: an invitation to corpus-based research in motion typology

This paper highlights some facets of motion typology, applied here to mainly English and French. These two languages are not perfect examples of satellite-framed and verb-framed languages, in Leonard Talmy’s well-known typology, but they can nonetheless be shown to differ in a number of related respects: compared to English (and other Germanic languages), French (like other Romance languages) is quite constrained in its use of Manner-of-motion verbs. French also lacks true particles – Path satellites without a Ground that can be syntactically detached from the verb. Drawing on some of my previous research, I briefly discuss two simple but apparently sufficiently efficient corpus -based translation studies that reveal that these differences show up when we compare English texts originally written in English with English texts translated from French vs. English texts translated from German (or other Germanic languages). A third, more recent, study contrasts a single English novel with its French and Dutch translations, focusing on expressions of visual motion. Here, too, some of the basic encoding preferences (satellite-framed vs. verb-framed) that these languages exhibit for actual motion appear to apply, by and large, for visual motion. This paper also lists some precursors of Talmy, one of whom is famously linked with the linguistic relative hypothesis. It is suggested that French, because of its typological nature, may not urge its speakers to convey much detail (neither of Manner nor of Path) in the encoding of motion. It remains an open question, one that goes beyond the purview of corpus linguistics, whether this stylistic difference is matched with a deeper cognitive one.

Cartesian Control of Sit-to-Stand Motion Using Head Position Feedback.

Sit-to-stand (STS) motion is an indicator of an individual's physical independence and well-being. Determination of various variables that contribute to the execution and control of STS motion is an active area of research. In this study, we evaluate the clinical hypothesis that besides numerous other factors, the central nervous system (CNS) controls STS motion by tracking a prelearned head position trajectory. Motivated by the evidence for a task-oriented encoding of motion by the CNS, we adopt a robotic approach for the synthesis of STS motion and propose this scheme as a solution to this hypothesis. We propose an analytical biomechanical human CNS modeling framework where the head position trajectory defines the high-level task control variable. The motion control is divided into low-level task generation and motor execution phases. We model CNS as STS controller and its Estimator subsystem plans joint trajectories to perform the low-level task. The motor execution is done through the Cartesian controller subsystem that generates torque commands to the joints. We do extensive motion and force capture experiments on human subjects to validate our analytical modeling scheme. We first scale our biomechanical model to match the anthropometry of the subjects. We do dynamic motion reconstruction through the control of simulated custom human CNS models to follow the captured head position trajectories in real time. We perform kinematic and kinetic analyses and comparison of experimental and simulated motions. For head position trajectories, root mean square (RMS) errors are 0.0118 m in horizontal and 0.0315 m in vertical directions. Errors in angle estimates are 0.55 rad, 0.93 rad, 0.59 rad, and 0.0442 rad for ankle, knee, hip, and head orientation, respectively. RMS error of ground reaction force (GRF) is 50.26 N, and the correlation between ground reaction torque and the support moment is 0.72. Low errors in our results validate (1) the reliability of motion/force capture methods and anthropometric technique for customization of human models and (2) high-level task control framework and human CNS modeling as a solution to the hypothesis. Accurate modeling and detailed understanding of human motion can have significant scope in the fields of rehabilitation, humanoid robotics, and virtual characters' motion planning based on high-level task control schemes.

Neuro-fuzzy control of sit-to-stand motion using head position tracking

Based on the clinical evidence that head position measured by the multisensory system contributes to motion control, this study suggests a biomechanical human-central nervous system modeling and control framework for sit-to-stand motion synthesis. Motivated by the evidence for a task-oriented encoding of motion by the central nervous system, we propose a framework to synthesize and control sit-to-stand motion using only head position trajectory in the high-level-task-control environment. First, we design a generalized analytical framework comprising a human biomechanical model and an adaptive neuro-fuzzy inference system to emulate central nervous system. We introduce task-space training algorithm for adaptive neuro-fuzzy inference system training. The adaptive neuro-fuzzy inference system controller is optimized in the number of membership functions and training cycles to avoid over-fitting. Next, we develop custom human models based on anthropometric data of real subjects. Using the weighting coefficient method, we estimate body segment parameter. The subject-specific body segment parameter values are used (1) to scale human model for real subjects and (2) in task-space training to train custom adaptive neuro-fuzzy inference system controllers. To validate our modeling and control scheme, we perform extensive motion capture experiments of sit-to-stand transfer by real subjects. We compare the synthesized and experimental motions using kinematic analyses. Our analytical modeling-control scheme proves to be scalable to real subjects’ body segment parameter and the task-space training algorithm provides a means to customize adaptive neuro-fuzzy inference system efficiently. The customized adaptive neuro-fuzzy inference system gives 68%–98% improvement over general adaptive neuro-fuzzy inference system. This study has a broader scope in the fields of rehabilitation, humanoid robotics, and virtual characters’ motion planning based on high-level-task-control scheme.

"Motion or Emotion? Recognition of Emotional Bodily Expressions in Children With Autism Spectrum Disorder With and Without Intellectual Disability".

The recognition of emotional body movement (BM) is impaired in individuals with Autistic Spectrum Disorder ASD, yet it is not clear whether the difficulty is related to the encoding of body motion, emotions, or both. Besides, BM recognition has been traditionally studied using point-light displays stimuli (PLDs) and is still underexplored in individuals with ASD and intellectual disability (ID). In the present study, we investigated the recognition of happy, fearful, and neutral BM in children with ASD with and without ID. In a non-verbal recognition task, participants were asked to recognize pure-body-motion and visible-body-form stimuli (by means of point-light displays-PLDs and full-light displays-FLDs, respectively). We found that the children with ASD were less accurate than TD children in recognizing both the emotional and neutral BM, either when presented as FLDs or PLDs. These results suggest that the difficulty in understanding the observed BM may rely on atypical processing of BM information rather than emotion. Moreover, we found that the accuracy improved with age and IQ only in children with ASD without ID, suggesting that high level of cognitive resources can mediate the acquisition of compensatory mechanisms which develop with age.

The lateral prefrontal cortex of primates encodes stimulus colors and their behavioral relevance during a match-to-sample task

The lateral prefrontal cortex of primates (lPFC) plays a central role in complex cognitive behavior, in decision-making as well as in guiding top-down attention. However, how and where in lPFC such behaviorally relevant signals are computed is poorly understood. We analyzed neural recordings from chronic microelectrode arrays implanted in lPFC region 8Av/45 of two rhesus macaques. The animals performed a feature match-to-sample task requiring them to match both motion and color information in a test stimulus. This task allowed to separate the encoding of stimulus motion and color from their current behavioral relevance on a trial-by-trial basis. We found that upcoming motor behavior can be robustly predicted from lPFC activity. In addition, we show that 8Av/45 encodes the color of a visual stimulus, regardless of its behavioral relevance. Most notably, whether a color matches the searched-for color can be decoded independent of a trial’s motor outcome and while subjects detect unique feature conjunctions of color and motion. Thus, macaque area 8Av/45 computes, among other task-relevant information, the behavioral relevance of visual color features. Such a signal is most critical for both the selection of responses as well as the deployment of top-down modulatory signals, like feature-based attention.

Constructional Meaning and Knowledge-Driven Interpretation of Motion Events

SummaryCovert encoding is one of the strategies available to languages for the encoding of motion, in which, in accordance with the laws of Gestalt, the meaning of an expression encoding motion is not coincident with the mere sum of the meanings of each of its constitutive units, relying on the mediation of grammatical and co(n)text­established knowledge for its interpretability. Moving on from a data set gathered for a previous study and adopting a holistic, constructional approach, several strategies were found in a diachronic corpus of Italian, French and Spanish for the covert encoding of motion in such languages, based on whether the motion­interpretation attributed to the covert construction is mediated by either linguistic or extralinguistic knowledge. The diachronic investigation also showed that the use of such patterns is diachronically consistent in frequency, thus proving that such patterns of interaction between language and cognition may be regarded as functional for speakers of all languages and time stages, also confirming that a holistic, constructional approach to language study can help shed light on linguistic–cognitive phenomena on the basis of language variation and change.

Event-based encoding of biological motion and location in visual working memory.

We make use of discrete yet meaningful events to orient ourselves to the dynamic environment. Among these events, biological motion, referring to the movements of animate entities, is one of the most biologically salient. We usually encounter biological motions of multiple human beings taking place simultaneously at distinct locations. How we encode biological motions into visual working memory (VWM) to form a coherent experience of the external world and guide our social behaviour remains unclear. This study for the first time addressed the VWM encoding mechanism of biological motions and their corresponding locations. We tested an event-based encoding hypothesis for biological motion and location: When one element of an event is required to be memorised, the irrelevant element of an event will also be extracted into VWM. We presented participants with three biological motions at different locations and required them to memorise only the biological motions or their locations while ignoring the other dimension. We examined the event-based encoding by probing a distracting effect: If the event-based encoding took place, the change of irrelevant dimension in the probe would lead to a significant distraction and impair the performance of detecting target dimension. We found significant distracting effects, which lasted for 3 s but vanished at 6 s, regardless of the target dimension (biological motions vs. locations, Experiment 1) and the exposure time of memory array (1 s vs. 3 s, Experiment 2). These results together support an event-based encoding mechanism during VWM encoding of biological motions and their corresponding locations.