High-level Motion Processing Research Articles

Interpretation of Swedish Sign Language Using Convolutional Neural Networks and Transfer Learning

The automatic interpretation of sign languages is a challenging task, as it requires the usage of high-level vision and high-level motion processing systems for providing accurate image perception. In this paper, we use Convolutional Neural Networks (CNNs) and transfer learning to make computers able to interpret signs of the Swedish Sign Language (SSL) hand alphabet. Our model consists of the implementation of a pre-trained InceptionV3 network, and the usage of the mini-batch gradient descent optimization algorithm. We rely on transfer learning during the pre-training of the model and its data. The final accuracy of the model, based on 8 study subjects and 9400 images, is 85%. Our results indicate that the usage of CNNs is a promising approach to interpret sign languages, and transfer learning can be used to achieve high testing accuracy despite using a small training dataset. Furthermore, we describe the implementation details of our model to interpret signs as a user-friendly web application.

The effects of attention and adaptation duration on the motion aftereffect.

The motion aftereffect (MAE) is the perception of illusory motion following extended exposure to a moving stimulus. The MAE has been used to probe the role of attention in motion processing. Many studies have reported that MAEs are reduced if attention is diverted from the adaptation stimulus, but others have argued that motion adaptation is independent of attention. We explored several factors that might modulate the attention-adaptation relationship and therefore explain apparent inconsistencies, namely (a) adaptation duration, (b) motion type: translating versus complex, and (c) response bias. Participants viewed translating (Experiments 1a and 2) or rotating (Experiment 1b) random dot patterns while fixating a central letter stream. During adaptation, participants reported brief changes in the adaptor speed (attention-focused) or the presence of white vowels within the letter stream (attention-diverted). Trials consisted of multiple adaptation-test cycles, and the MAE was measured after each adaptation period. Across experiments, focused attention produced significantly larger MAEs than did diverted attention (15% change, Cohen's d = .41). Attention affected the MAE asymptote, rather than its accumulation rate, and had larger effects for translational than for complex motion. The effect of attention remained evident after controlling for response bias. Our results suggest that attention affects multiple levels of the motion-processing hierarchy: not only higher level motion processing, as seen with apparent motion, but also low-level motion processing, as evidenced by the MAE. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Time dilation in a perceptually jittering dot pattern.

Although it is known that a moving stimulus appears to dilate in duration compared to a stationary stimulus, whether subjective motion devoid of stimulus motion is sufficient remains unknown. To elucidate this, we used a motion illusion in which an actually static stimulus clearly appears to move, a useful dissociation between actual and subjective motions. We used the jitter aftereffect resulting from adaptation to dynamic noise as such a tool and measured subjective durations of a static random-dot pattern in which illusory jitter was seen, an actually oscillating pattern mimicking the illusory jitter, and a static pattern without illusory jitter. Pattern oscillation as tiny as fixational eye movements robustly evoked time dilation, and time dilation to a similar extent was also induced by an actually static but subjectively jittering pattern. Taken together with the previous knowledge that this subjective jitter is related to a visually based compensation of spurious retinal image motions due to fixational eye movements, these findings demonstrate that visual duration computation is influenced by a representation at a high-level motion processing stage at which a stable visual world despite jittery retinal inputs has been established.

Limited attention facilitates coherent motion processing.

It has been suggested that sensitivities for visual motion are typically impaired by poor attention. Here, we show that limited attention paradoxically improves performance on a global motion detection task. Psychophysical experiments revealed that deliberately attending to an irrelevant stimulus enhanced sensitivity for detecting coherent motion in random-dot kinematograms but did not affect contrast and velocity sensitivity for local luminance motion. Subsequent experiments further demonstrated that the dual task reduced sensitivity for detecting spatial modulations in local motion direction and induced illusory motion assimilation. Additional measurements confirmed that the secondary task had no effect when attentional load was extremely high or when motion stimuli were presented peripherally. These results may be explained by the idea that limited attention dynamically expands the spatial extent of motion integration by reducing center-surround interactions at high-level motion processing.

On the effect of scene motion on color constancy

A series of experiments with human subjects have shown that color constancy improves when an object moves. It has been hypothesized that this effect is due to some kind of influence of high-level motion processing. We have built a computational model for color perception which replicates the results qualitatively which have been obtained with human subjects. We show that input from high-level motion processing is not required. In our model, the dependence is an effect of eye movement in combination with neural processing. Depending on the type of stimulus used, the eye either tracks the object or the background. When the object moves but is tracked by the observer, the background appears to move when considering the stimulus with respect to eye coordinates. Hence, the retinal input is different for the two conditions leading to a difference in color constancy performance.

A cortical locus for high-level motion processing?

A cortical locus for high-level motion processing?

Low- and high-level motion perception deficits in anisometropic and strabismic amblyopia: Evidence from fMRI

Maximum motion displacement (Dmax) is the largest dot displacement in a random-dot kinematogram (RDK) at which direction of motion can be correctly discriminated [Braddick, O. (1974). A short-range process in apparent motion. Vision Research, 14, 519–527]. For first-order RDKs, Dmax gets larger as dot size increases and/or dot density decreases. It has been suggested that this increase in Dmax reflects greater involvement of high-level feature-matching motion mechanisms and less dependence on low-level motion detectors [Sato, T. (1998). Dmax: Relations to low- and high-level motion processes. In T. Watanabe (Ed.), High-level motion processing, computational, neurobiological, and psychophysical perspectives (pp. 115–151). Boston: MIT Press]. Recent psychophysical findings [Ho, C. S., & Giaschi, D. E. (2006). Deficient maximum motion displacement in amblyopia. Vision Research, 46, 4595–4603; Ho, C. S., & Giaschi, D. E. (2007). Stereopsis-dependent deficits in maximum motion displacement. Vision Research, 47, 2778–2785] suggest that this “switch” from low-level to high-level motion processing is also observed in children with anisometropic and strabismic amblyopia as RDK dot size is increased and/or dot density is decreased. However, both high- and low-level Dmax were reduced relative to controls. In this study, we used functional MRI to determine the motion-sensitive areas that may account for the reduced Dmax in amblyopia In the control group, low-level RDKs elicited stronger responses in low-level (posterior occipital) areas and high-level RDKs elicited a greater response in high-level (extra-striate occipital–parietal) areas when activation for high-level RDKs was compared to that for low-level RDKs. Participants with anisometropic amblyopia showed the same pattern of cortical activation although extent of activation differences was less than in controls. For those with strabismic amblyopia, there was almost no difference in the cortical activity for low-level and high-level RDKs, and activation was reduced relative to the other groups. Differences in the extent of cortical activation may be related to amblyogenic subtype.

Two Distinct Visual Motion Mechanisms for Smooth Pursuit: Evidence from Individual Differences

Smooth-pursuit eye velocity to a moving target is more accurate after an initial catch-up saccade than before, an enhancement that is poorly understood. We present an individual-differences-based method for identifying mechanisms underlying a physiological response and use it to test whether visual motion signals driving pursuit differ pre- and postsaccade. Correlating moment-to-moment measurements of pursuit over time with two psychophysical measures of speed estimation during fixation, we find two independent associations across individuals. Presaccadic pursuit acceleration is predicted by the precision of low-level (motion-energy-based) speed estimation, and postsaccadic pursuit precision is predicted by the precision of high-level (position-tracking) speed estimation. These results provide evidence that a low-level motion signal influences presaccadic acceleration and an independent high-level motion signal influences postsaccadic precision, thus presenting a plausible mechanism for postsaccadic enhancement of pursuit.

Ocular tracking as a measure of auditory motion perception

Motion is a potent sub-modality of vision. Motion cues alone can be used to segment images into figure and ground and break camouflage. Specific patterns of motion support vivid percepts of form, guide locomotion by specifying directional heading and the passage of objects, and in case of an impending collision, the time to impact. Visual motion also drives smooth pursuit eye movements (SPEMs) that serve to stabilize the retinal image of objects in motion. In contrast, the auditory system does not appear to be particularly sensitive to motion. We review the ambiguous status of auditory motion processing from the psychophysical and electrophysiological perspectives. We then report the results of two experiments that use ocular tracking performance as an objective measure of the perception of auditory motion in humans. We examine ocular tracking of auditory motion, visual motion, combined auditory + visual motion and imagined motion in both the frontal plane and in depth. The results demonstrate that ocular tracking of auditory motion is no better than ocular tracking of imagined motion. These results are consistent with the suggestion that, unlike the visual system, the human auditory system is not endowed with low-level motion sensitive elements. We hypothesize however, that auditory information may gain access to a recently described high-level motion processing system that is heavily dependent on `top-down' influences, including attention.

Shifts in perceived position of flashed stimuli by illusory object motion

Moving stimuli cause the position of flashed stimuli to appear shifted in the direction of motion (position capture). To examine whether position capture depends on low-level motion interactions or perception of integrated object motion, we employed a slit-view display. Two line-drawn diamonds translated horizontally in opposite directions, one above and one below the fixation cross, either behind an occluding surface with a narrow slit or without occluding surface. When the diamonds were in vertical alignment, two vertical bars were flashed, one in the center of each diamond. In the slit-view condition, the diamonds were visible through a 4-, 2-, or 1-pixel vertical slit; the width of the flashed bars always matched the width of the slit. Even though the horizontal component of physical motion was greatly reduced or absent in the slit-view conditions, observers perceived diamonds moving behind the occluding surface. Furthermore, the position of the flashed bar was captured by the moving diamonds such that each bar appeared shifted in the direction of perceived motion. We conclude that the position capture reported here has a component based on high-level motion processing that is responsible for dynamically integrating object motion and shape.

Chinese and Americans see opposite apparent motions in a Chinese character

The perceived direction of apparent motion can be influenced by both “top-down” factors, such as expectation, and by “bottom-up” or stimulus-driven factors, such as grouping (Tse, P., Cavanagh, P. & Nakayama, K. (1998). The role of parsing in high-level motion processing. In T. Watanabe, High-level motion processing – computational, neurobiological and psychophysical perspectives. Cambridge, MA: MIT Press). Here we report the results of a single experiment that pitted top-down cues against bottom-up cues in an apparent motion sequence over the successive strokes of a Chinese character. Although each stroke was in fact presented all at once, subjects raised in China tended to see apparent motion over a single stroke in the direction it would have taken when drawn by hand, even though bottom-up cues drive a percept of apparent motion in the opposite direction for observers unfamiliar with the Chinese language. There is therefore a learned component to motion perception arising from top-down expectations capable of overriding bottom-up cues to motion.

A neural model of high-level motion processing: Line motion and formotion dynamics

The percepts known variously as the line motion illusion, motion induction, and transformational apparent motion have attracted a great deal of experimental interest, since they sensitively probe interactions between preattentive and attentive vision processes. The present article develops a neural model that qualitatively explains essentially all the data reported thus far, and quantitatively simulates key illustrative percepts. The model suggests how these data arise from neural mechanisms of preattentive boundary and surface formation, long-range apparent motion, formotion interactions, and spatial attention. The boundary and surface formation processes model aspects of the interblob V1-->interstripe V2-->V4 and blob V1-->thin stripe V2-->V4 cortical processing streams, respectively. The long-range apparent motion process models aspects of the V1-->MT-->MST processing stream. An interstream V2-->MT form-motion interaction is proposed to allow the motion processing stream to track transient properties of emergent boundaries and filled-in surface colors from the form processing stream. It does so by generating motion waves using the long-range apparent motion process. This interstream interaction controls the formation of form-motion percepts, which are herein called formotion percepts. Other transients directly cause motion waves within the motion processing stream. All the data are attributed to properties of such motion waves. It is also suggested how bottom-up motion mechanisms can engage top-down attention as part of the motion capture process that solves the aperture problem. This interaction is proposed to occur between areas MT and MST. The model hereby explains how attention can be engaged even in percepts whose explanation can be derived from preattentive mechanisms.

Motion aftereffect with flickering test patterns reveals higher stages of motion processing

A series of experiments was conducted to clarify the distinction between motion aftereffects (MAEs) with static and counterphasing test patterns (static and flicker MAEs). It was found that while the motion of higher-order structure, such as areas defined by texture, flicker, or stereoscopic depth, induces little static MAE, such motion reliably generates flicker MAE. It was also found that static and flicker MAEs were induced in opposite directions for stimuli in which first- and second-order structures moved in opposite directions (compound graftings of 2 f + 3 f or 2 f + 3 f + 4 f, shifting a half cycle of 2 f). When the test was static, MAE was induced in the direction opposite to the first-order motion; but when the test was counterphasing, MAE was induced in the direction opposite to the second-order motion. This means that static MAE is predominantly induced by first-order motion, but that flicker MAE is affected strongly by second-order motion, along with first-order motion. The present results suggest that static MAE primarily reflects adaptation of a low-level motion mechanism, where first-order motion is processed, while flicker MAE reveals a high-level motion processing, where both first- and second-order motion signals are available.