Target Reappearance Research Articles

Saccadic “adaptation” at late target reappearance

Saccadic “adaptation” at late target reappearance

A Visual Tracker Offering More Solutions.

Most trackers focus solely on robustness and accuracy. Visual tracking, however, is a long-term problem with a high time limitation. A tracker that is robust, accurate, with long-term sustainability and real-time processing, is of high research value and practical significance. In this paper, we comprehensively consider these requirements in order to propose a new, state-of-the-art tracker with an excellent performance. EfficientNet-B0 is adopted for the first time via neural architecture search technology as the backbone network for the tracking task. This improves the network feature extraction ability and significantly reduces the number of parameters required for the tracker backbone network. In addition, maximal Distance Intersection-over-Union is set as the target estimation method, enhancing network stability and increasing the offline training convergence rate. Channel and spatial dual attention mechanisms are employed in the target classification module to improve the discrimination of the trackers. Furthermore, the conjugate gradient optimization strategy increases the speed of the online learning target classification module. A two-stage search method combined with a screening module is proposed to enable the tracker to cope with sudden target movement and reappearance following a brief disappearance. Our proposed method has an obvious speed advantage compared with pure global searching and achieves an optimal performance on OTB2015, VOT2016, VOT2018-LT, UAV-123 and LaSOT while running at over 50 FPS.

Frontotemporal dementia patients exhibit deficits in predictive saccades.

Prediction and time estimation are all but required for motor function in everyday life. In the context of eye movements, for instance, they allow predictive saccades and eye re-acceleration in anticipation of a target re-appearance. While the neural pathways involved are not fully understood, it is known that the frontal lobe plays an important role. As such, neurological disorders that affect it, such as frontotemporal (FTD) dementia, are likely to induce deficits in such movements. In this work, we study the performances of frontotemporal dementia patients in an oculomotor task designed to elicit predictive saccades at different rates, and compare them to young and older adults. Clear deficits in the production of predictive saccades were found in patients, in particular when the time between saccades was short (~500ms). Furthermore, one asymptomatic C9ORF72 mutation bearer showed patterns of oculomotor behavior similar to FTD patients. He exhibited FTD symptoms within 3years post-measure, suggesting that an impairment of oculomotor function could be an early clinical sign. Taken together, these results argue in favor of a role of the frontal lobe in predictive movements timing over short timescales, and suggest that predictive saccades in FTD patients warrant further investigation to fully assess their potential as a diagnostic aid.

Spatial and temporal adaptation of predictive saccades based on motion inference

Moving objects are often occluded behind larger, stationary objects, but we can easily predict when and where they reappear. Here, we show that the prediction of object reappearance is subject to adaptive learning. When monkeys generated predictive saccades to the location of target reappearance, systematic changes in the location or timing of target reappearance independently altered the endpoint or latency of the saccades. Furthermore, spatial adaptation of predictive saccades did not alter visually triggered reactive saccades, whereas adaptation of reactive saccades altered the metrics of predictive saccades. Our results suggest that the extrapolation of motion trajectory may be subject to spatial and temporal recalibration mechanisms located upstream from the site of reactive saccade adaptation. Repetitive exposure of visual error for saccades induces qualitatively different adaptation, which might be attributable to different regions in the cerebellum that regulate learning of trajectory prediction and saccades.

Ocular tracking of occluded ballistic trajectories: Effects of visual context and of target law of motion.

In tracking a moving target, the visual context may provide cues for an observer to interpret the causal nature of the target motion and extract features to which the visual system is weakly sensitive, such as target acceleration. This information could be critical when vision of the target is temporarily impeded, requiring visual motion extrapolation processes. Here we investigated how visual context influences ocular tracking of motion either congruent or not with natural gravity. To this end, 28 subjects tracked computer-simulated ballistic trajectories either perturbed in the descending segment with altered gravity effects (0g/2g) or retaining natural-like motion (1g). Shortly after the perturbation (550 ms), targets disappeared for either 450 or 650 ms and became visible again until landing. Target motion occurred with either quasi-realistic pictorial cues or a uniform background, presented in counterbalanced order. We analyzed saccadic and pursuit movements after 0g and 2g target-motion perturbations and for corresponding intervals of unperturbed 1g trajectories, as well as after corresponding occlusions. Moreover, we considered the eye-to-target distance at target reappearance. Tracking parameters differed significantly between scenarios: With a neutral background, eye movements did not depend consistently on target motion, whereas with pictorial background they showed significant dependence, denoting better tracking of accelerated targets. These results suggest that oculomotor control is tuned to realistic properties of the visual scene.

Behavioral characterization of prediction and internal models in adolescents with autistic spectrum disorders

Autism has been considered as a deficit in prediction of the upcoming event or of the sensory consequences of our own movements. To test this hypothesis, we recorded eye movements from high-functioning autistic adolescents and from age-matched controls during a blanking paradigm. In this paradigm, adolescents were instructed to follow a moving target with their eyes even during its transient disappearance. Given the absence of visual information during the blanking period, eye movements during this period are solely controlled on the basis of the prediction of the ongoing target motion. Typical markers of predictive eye movements such as the number and accuracy of predictive saccades and the predictive reacceleration before target reappearance were identical in the two populations. In addition, the synergy of predictive saccades and smooth pursuit observed during the blanking periods, which is a marker for the quality of internal models about target/eye motions, was comparable between these two populations. These results suggest that, in our large population of high-functioning autistic adolescent, both predictive abilities and internal models are left intact in Autism, at least for low-level sensorimotor transformations.

Asymmetric visual interactions across the boundary of awareness.

A salient visual object can disappear from conscious perception when surrounded by a moving texture, a phenomenon known as MIB, Motion-Induced Blindness (Bonneh, Cooperman, & Sagi, 2001). Here we tested the information available in the brain from such stimuli that do not access awareness by examining interactions across the boundary of awareness between stimuli that reach awareness and those that do not. Observers performed the MIB task in which a "Cue" was presented next to the "Target" after observers reported the perceptual disappearance of the target (Kawabe, Yamada, & Miura, 2007). Oriented Gabor patches were used as targets and cues; observers reported the target's reappearance. The results indicated an interaction between the target and the cue, depending on the orientation difference (∼30° bandwidth) and distance (∼1° range), indicating preserved properties of features in the absence of awareness. Object-based representation (binding) of unseen stimuli was tested by examining the interaction between a compound stimulus and its composing features. Here we used vertical and horizontal Gabor patches and their combinations (plaids) as targets and cues. Results indicated asymmetric relations between aware and unaware object representations; a plaid cue was not effective with a component target, but a plaid target efficiently reappeared by its component cues. This result suggests that the unseen, but not the seen plaid, is decomposed into its features. Plaid targets also reappeared with plaid cues, supporting binding without awareness. Our findings suggest preconscious representations of objects and their features, with conscious perception confined to object representations.

The temporal advantage for reloading vs. uploading conscious representations decays over time.

In motion-induced blindness (MIB), a static target superimposed on a global moving pattern frequently disappears and reappears into consciousness. We previously reported an intriguing illusory temporal reversal whereby a new stimulus onset (e.g. a dot flash) presented during MIB triggers an early reappearance of the target, yet is systematically perceived as occurring after the target reappearance. This illusion implies that the unconscious target representation can be quickly reactivated, with a temporal advantage for its conscious reloading as compared to the conscious uploading of a newly presented visual stimulus. However, it remains unclear whether the temporal advantage for conscious representation reloading strengthens, decays, or remains constant over time after we lose the initial conscious access to the stimulus. To address this question, we examined the relation between the duration of MIB and the percentage of illusory temporal reversals, and we found a negative correlation between the two measures, both between and within observers. The results suggest that although the unconscious target representation retains a certain level of activation during MIB, the temporal advantage for reloading its preexisting representation into consciousness decays over time.

Development of internal models and predictive abilities for visual tracking during childhood.

The prediction of the consequences of our own actions through internal models is an essential component of motor control. Previous studies showed improvement of anticipatory behaviors with age for grasping, drawing, and postural control. Since these actions require visual and proprioceptive feedback, these improvements might reflect both the development of internal models and the feedback control. In contrast, visual tracking of a temporarily invisible target gives specific markers of prediction and internal models for eye movements. Therefore, we recorded eye movements in 50 children (aged 5-19 yr) and in 10 adults, who were asked to pursue a visual target that is temporarily blanked. Results show that the youngest children (5-7 yr) have a general oculomotor behavior in this task, qualitatively similar to the one observed in adults. However, the overall performance of older subjects in terms of accuracy at target reappearance and variability in their behavior was much better than the youngest children. This late maturation of predictive mechanisms with age was reflected into the development of the accuracy of the internal models governing the synergy between the saccadic and pursuit systems with age. Altogether, we hypothesize that the maturation of the interaction between smooth pursuit and saccades that relies on internal models of the eye and target displacement is related to the continuous maturation of the cerebellum.

Traveling waves of dominance in motion-induced blindness

There is accumulated evidence supporting the idea that motion-induced blindness (MIB) and binocular rivalry (BR) may be mediated by a common mechanism(s). To test this hypothesis, we examined the spatiotemporal dynamics of perceptual switches in MIB. It is well known that perceptual switches occur in a traveling-wave-like fashion during binocular rivalry, meaning that a suppressed stimulus during BR regains its dominance locally and progressively expands its dominance over space and time. If MIB and BR share a common mechanism, their spatiotemporal dynamics should resemble each other. In each trial, observers viewed a MIB display consisting of a rotating grid of black crosses on a grey background and a static, segmented, white arc presented in the periphery within protection zones. Observers pressed and held a key once the entire arc completely disappeared from awareness due to MIB. Target reappearance was induced by briefly flickering a portion of the end of the arc right after it completely disappeared. Observers released the response key when a segment of the arc reappeared at a designated location. The target reappearance location was systematically varied in distance from the flicker pulse along the arc across trials. We found that the reappearance of the arc reached locations further from the flicker pulse with greater latencies, resembling the traveling-wave-like propagation of dominance observed in BR. Participants indeed reported that reappearance of the targets appeared to propagate in a wave-like manner along the arcs. Our results indicate that the suppressed targets in MIB regain visual awareness in a wave-like fashion similar to BR. The consistent spatiotemporal dynamics between BR and MIB indicates that both visual phenomena may be mediated by a common mechanism(s). Meeting abstract presented at VSS 2015.

Dynamic interaction between retinal and extraretinal signals in motion integration for smooth pursuit

Due to the aperture problem, the initial direction of tracking responses to a translating bar is biased towards the direction orthogonal to the bar. This observation offers a powerful way to explore the interactions between retinal and extraretinal signals in controlling our actions. We conducted two experiments to probe these interactions by briefly (200 and 400 ms) blanking the moving target (45° or 135° tilted bar) during steady state (Experiment 1) and at different moments during the early phase of pursuit (Experiment 2). In Experiment 1, we found a marginal but statistically significant directional bias on target reappearance for all subjects in at least one blank condition (200 or 400 ms). In Experiment 2, no systematic significant directional bias was observed at target reappearance after a blank. These results suggest that the weighting of retinal and extraretinal signals is dynamically modulated during the different phases of pursuit. Based on our previous theoretical work on motion integration, we propose a new closed-loop two-stage recurrent Bayesian model where retinal and extraretinal signals are dynamically weighted based on their respective reliabilities and combined to compute the visuomotor drive. With a single free parameter, the model reproduces many aspects of smooth pursuit observed across subjects during and immediately after target blanking. It provides a new theoretical framework to understand how different signals are dynamically combined based on their relative reliability to adaptively control our actions. Overall, the model and behavioral results suggest that human subjects rely more strongly on prediction during the early phase than in the steady state phase of pursuit.

Dramatic impairment of prediction due to frontal lobe degeneration

Prediction is essential for motor function in everyday life. For instance, predictive mechanisms improve the perception of a moving target by increasing eye speed anticipatively, thus reducing motion blur on the retina. Subregions of the frontal lobes play a key role in eye movements in general and in smooth pursuit in particular, but their precise function is not firmly established. Here, the role of frontal lobes in the timing of predictive action is demonstrated by studying predictive smooth pursuit during transient blanking of a moving target in mild frontotemporal lobar degeneration (FTLD) and Alzheimer's disease (AD) patients. While control subjects and AD patients predictively reaccelerated their eyes before the predicted time of target reappearance, FTLD patients did not. The difference was so dramatic (classification accuracy >90%) that it could even lead to the definition of a new biomarker. In contrast, anticipatory eye movements triggered by the disappearance of the fixation point were still present before target motion onset in FTLD patients and visually guided pursuit was normal in both patient groups compared with controls. Therefore, FTLD patients were only impaired when the predicted timing of an external event was required to elicit an action. These results argue in favor of a role of the frontal lobes in predictive movement timing.

Oscillatory modulations in human fusiform cortex during motion-induced blindness: Intracranial recording

ObjectiveMotion-induced blindness (MIB) is an illusory phenomenon, in which a static target surrounded by moving distracters is perceived to disappear. We determined the electrocorticographic (ECoG) correlates of MIB. MethodsWhile undergoing intracranial ECoG recording, a patient with focal epilepsy was instructed to report the transitions of a visual target, which was designed to illusorily or physically disappear and reappear. We then determined the neural modulations associated with illusory and physical transitions of the target. We also tested whether the phase of local delta activity could predict exclusively illusory transitions. ResultsHigh-gamma activity at 80–150Hz was attenuated in the fusiform region prior to the reports of illusory and real visual target disappearance. Conversely, such high-gamma activity was augmented prior to the report of real target reappearance. Exclusively around illusory disappearance but not around real one, the delta phases in the fusiform region showed a highly skewed distribution with preference of the negative peak. ConclusionsNeuronal modulations in the fusiform region may be involved in visual awareness, while spontaneous fluctuations of neural states entrained on delta rhythm may be involved in generation of MIB. SignificanceOur study increases our understanding of the mechanisms of visual awareness.

Alpha Oscillations Related to Anticipatory Attention Follow Temporal Expectations

Temporal expectations have been shown to enhance visual analysis of task-relevant events, especially when these are coupled with spatial expectations. Oscillatory brain activity, particularly in the alpha band, has been implicated in regulating excitability in visual areas as a function of anticipatory spatial attention. Here we asked whether temporal expectations derived from regular, rhythmic events can modulate ongoing oscillatory alpha-band activity, so that the changes in cortical excitability are focused over the time intervals at which target events are expected. The task we used involved making a perceptual discrimination about a small target stimulus that reappeared from "behind" a peripheral occluding band. Temporal expectations were manipulated by the regular, rhythmic versus irregular, arrhythmic approach of the stimulus toward the occluding band. Alpha-band activity was measured during the occlusion period, in which no stimulus was presented, but target reappearance was anticipated in conditions of high versus low temporal expectation. Time-frequency analysis showed that the amplitude of alpha-desynchronization followed the time course of temporal expectations. Alpha desynchronization increased rhythmically, peaking just before the expected reappearance of target times. Analysis of the event-related potentials evoked by the subsequent target stimuli showed enhancement of processing at both visual and motor stages. Our findings support a role for oscillations in regulating cortical excitability and suggest a plausible mechanism for biasing perception and action by temporal expectations.

Extraction of visual motion information for the control of eye and head movement during head-free pursuit

We investigated how effectively briefly presented visual motion could be assimilated and used to track future target motion with head and eyes during target disappearance. Without vision, continuation of eye and head movement is controlled by internal (extra-retinal) mechanisms, but head movement stimulates compensatory vestibulo-ocular reflex (VOR) responses that must be countermanded for gaze to remain in the direction of target motion. We used target exposures of 50–200 ms at the start of randomised step-ramp stimuli, followed by >400 ms of target disappearance, to investigate the ability to sample target velocity and subsequently generate internally controlled responses. Subjects could appropriately grade gaze velocity to different target velocities without visual feedback, but responses were fully developed only when exposure was >100 ms. Gaze velocities were sustained or even increased during target disappearance, especially when there was expectation of target reappearance, but they were always less than for controls, where the target was continuously visible. Gaze velocity remained in the direction of target motion throughout target extinction, implying that compensatory (VOR) responses were suppressed by internal drive mechanisms. Regression analysis revealed that the underlying compensatory response remained active, but with gain slightly less than unity (0.85), resulting in head-free gaze responses that were very similar to, but slightly greater than, head-fixed. The sampled velocity information was also used to grade head velocity, but in contrast to gaze, head velocity was similar whether the target was briefly or continuously presented, suggesting that head motion was controlled by internal mechanisms alone, without direct influence of visual feedback.

The Temporal Interplay between Conscious and Unconscious Perceptual Streams

An optimal correspondence of temporal information between the physical world and our perceptual world is important for survival. In the current study, we demonstrate a novel temporal illusion in which the cause of a perceptual event is perceived after the event itself. We used a paradigm referred to as motion-induced blindness (MIB), in which a static visual target presented on a constantly rotating background disappears and reappears from awareness periodically, with the dynamic characteristics of bistable perception. A sudden stimulus onset (e.g., a flash) presented during a period of perceptual suppression (i.e., during MIB) is known to trigger the almost instantaneous reappearance of the suppressed target. Surprisingly, however, we report here that although the sudden flash is the cause of the static target's reappearance (the corresponding effect), it is systematically perceived as occurring after this reappearance. Further investigation revealed that this illusory temporal reversal is caused by an approximately 100 ms advantage for the unconscious representation of the perceptually suppressed target to access consciousness, as compared to the newly presented flash. This new temporal illusion therefore reveals the normally hidden delays in bringing new visual events to awareness.

Orientation-tuned suppression in binocular rivalry reveals general and specific components of rivalry suppression

Visually guided catch-up saccades during the pursuit of a moving target are highly influenced by smooth pursuit performance. For example, the decision to execute a saccade and its amplitude is driven by the difference in velocity between the eye and the target. In previous studies, we have demonstrated that the predictive saccades that occur during the blanking of the moving target compensate for the variability of the smooth pursuit response. Therefore, we wondered whether the predictive smooth pursuit response during target blanking influenced the occurrence of predictive saccades, which is the case for visually guided catch-up saccades. To answer this question, we asked subjects to track visually a target moving along a circular path. From time to time, the target was unexpectedly blanked for some randomized durations and disappeared from the screen. Surprisingly, we did not find any differences in smooth pursuit performance between the blanks that did and those that did not contain predictive saccades. In addition, during the blanks, the differences in smooth pursuit performance across the sessions or across the subjects did not correlate with the differences in the number of predictive saccades. Therefore, this study demonstrates that smooth pursuit performance does not influence the occurrence of predictive saccades. We interpret these results in light of the possible minimization of position error at target reappearance, which heavily depends on the saccadic amplitudes but not on their timing.

Neural Modulation by Regularity and Passage of Time

The current study tested whether multiple rhythms could flexibly induce temporal expectations (temporal orienting) and whether these expectations interact with temporal expectations associated with the passage of time (foreperiod effects). A visual stimulus that moved following a regular rhythm was temporarily occluded for a variable duration (occlusion foreperiod). The task involved making a speeded perceptual discrimination about the target stimulus that reappeared after the occlusion. Temporal-orienting effects were measured by comparing performance and event-related potentials on conditions in which the timing for target reappearance was predictable (valid) versus unpredictable (invalid) according to the rhythm. Foreperiod effects were measured by comparing conditions in which the target was occluded for progressively longer periods of time (short, medium, and long foreperiods) and hence were increasingly predictable. The results showed strong interactions between temporal orienting and foreperiod effects during the facilitation of behavior and neural activity associated with late perceptual and response selection processes. Temporal orienting attenuated the N2 amplitude and decreased the P3 latency only at short foreperiods. Temporal preparation related to foreperiod effects abolished temporal orienting effects at medium and long foreperiods. Likewise, foreperiod effects attenuated the N1 and N2 amplitudes and decreased the P3 latency only in the invalid orienting condition as preparation related to temporal orienting abolished foreperiod effects in the valid condition. This high degree of neural overlap between the effects of temporal orienting driven by rhythms and foreperiod effects associated with the passage of time suggests the involvement of a common mechanism for temporal preparation.

Evidence for a Link Between the Extra-Retinal Component of Random-Onset Pursuit and the Anticipatory Pursuit of Predictable Object Motion

During pursuit of moving targets that temporarily disappear, residual smooth eye movements represent the internal (extra-retinal) component of pursuit. However, this response is dependent on expectation of target reappearance. By comparing responses with and without such expectation during early random-onset pursuit, we examined the temporal development of the extra-retinal component and compared it with anticipatory pursuit, another form of internally driven response. In an initial task (mid-ramp extinction), a moving, random-velocity target was initially visible for 100 or 150 ms but then extinguished for 600 ms before reappearing and continuing to move. Responses comprised an initial visually driven rapid rise in eye velocity, followed by a secondary slower increase during extinction. In a second task (short ramp), with identical initial target presentation but no expectation of target reappearance, the initial rapid rise in eye velocity was followed by decay toward zero. The expectation-dependent difference between responses to these tasks increased in velocity during extinction much more slowly than the initial, visually driven component. In a third task (initial extinction), the moving target was extinguished at motion onset but reappeared 600 ms later. Repetition of identical stimuli evoked anticipatory pursuit triggered by initial target offset. Temporal development and scaling of this anticipatory response, which was based on remembered velocity from prior stimuli, was remarkably similar to and covaried with the difference between mid-ramp extinction and short ramp tasks. Results suggest a common mechanism is responsible for anticipatory pursuit and the extra-retinal component of random-onset pursuit, a finding that is consistent with a previously developed model of pursuit.

The Influence of Briefly Presented Randomized Target Motion on the Extraretinal Component of Ocular Pursuit

We assessed the ability to extract velocity information from brief exposure of a moving target and sought evidence that this information could be used to modulate the extraretinal component of ocular pursuit. A step-ramp target motion was initially visible for a brief randomized period of 50, 100, 150, or 200 ms, but then extinguished for a randomized period of 400 or 600 ms before reappearing and continuing along its trajectory. Target speed (5-20 degrees /s), direction (left/right), and intertrial interval (2.7-3.7 s) were also randomized. Smooth eye movements were initiated after about 130 ms and comprised an initial visually dependent component, which reached a peak velocity that increased with target velocity and initial exposure duration, followed by a sustained secondary component that actually increased throughout extinction for 50- and 100-ms initial exposures. End-extinction eye velocity, reflecting extraretinal drive, increased with initial exposure from 50 to 100 ms but remained similar for longer exposures; it was significantly scaled to target velocity for 150- and 200-ms exposures. The results suggest that extraretinal drive is based on a sample of target velocity, mostly acquired during the first 150 ms, that is stored and forms a goal for generating appropriately scaled eye movements during absence of visual input. End-extinction eye velocity was significantly higher when target reappearance was expected than when it was not, confirming the importance of expectation in generating sustained smooth movement. However, end-extinction eye displacement remained similar irrespective of expectation, suggesting that the ability to use sampled velocity information to predict future target displacement operates independently of the control of smooth eye movement.