Non-visual Signals Research Articles

Preference for animate domain sounds in the fusiform gyrus of blind individuals is modulated by shape–action mapping

In high-level visual areas in the human brain, preference for inanimate objects is observed regardless of stimulation modality (visual/auditory/tactile) and individual's visual experience (sighted/blind) whereas preference for animate entities seems robust mainly in the visual modality. Here, we test a hypothesis explaining this domain difference: Object representations can be activated through nonvisual stimulation when their shapes are systematically related to action system representations, a quality typical of most inanimate objects but of only specific animate entities. We studied functional magnetic resonance imaging activations in congenitally blind and sighted individuals listening to animal, object, and human sounds. In blind individuals, the typical location of the fusiform face area preferentially responded to human facial expression sounds clearly related to specific facial actions and resulting face shapes but not to speech or animal sounds. No univariate preference for any sound category was observed in the fusiform gyrus in sighted individuals, but the expected multivoxel effects were present. We conclude that nonvisual signals can activate shape representations of those stimuli-inanimate or animate-for which shape and action computations are transparently related. However, absence of potentially competing visual inputs seems necessary for this effect to be clearly detectable in the case of animate representation.

Watching the Effects of Gravity. Vestibular Cortex and the Neural Representation of "Visual" Gravity.

Gravity is a physical constraint all terrestrial species have adapted to through evolution. Indeed, gravity effects are taken into account in many forms of interaction with the environment, from the seemingly simple task of maintaining balance to the complex motor skills performed by athletes and dancers. Graviceptors, primarily located in the vestibular otolith organs, feed the Central Nervous System with information related to the gravity acceleration vector. This information is integrated with signals from semicircular canals, vision, and proprioception in an ensemble of interconnected brain areas, including the vestibular nuclei, cerebellum, thalamus, insula, retroinsula, parietal operculum, and temporo-parietal junction, in the so-called vestibular network. Classical views consider this stage of multisensory integration as instrumental to sort out conflicting and/or ambiguous information from the incoming sensory signals. However, there is compelling evidence that it also contributes to an internal representation of gravity effects based on prior experience with the environment. This a priori knowledge could be engaged by various types of information, including sensory signals like the visual ones, which lack a direct correspondence with physical gravity. Indeed, the retinal accelerations elicited by gravitational motion in a visual scene are not invariant, but scale with viewing distance. Moreover, the “visual” gravity vector may not be aligned with physical gravity, as when we watch a scene on a tilted monitor or in weightlessness. This review will discuss experimental evidence from behavioral, neuroimaging (connectomics, fMRI, TMS), and patients’ studies, supporting the idea that the internal model estimating the effects of gravity on visual objects is constructed by transforming the vestibular estimates of physical gravity, which are computed in the brainstem and cerebellum, into internalized estimates of virtual gravity, stored in the vestibular cortex. The integration of the internal model of gravity with visual and non-visual signals would take place at multiple levels in the cortex and might involve recurrent connections between early visual areas engaged in the analysis of spatio-temporal features of the visual stimuli and higher visual areas in temporo-parietal-insular regions.

Population calcium responses of Purkinje cells in the oculomotor cerebellum driven by nonvisual input.

The climbing fiber input to the cerebellum conveys instructive signals that can induce synaptic plasticity and learning by triggering complex spikes accompanied by large calcium transients in Purkinje cells. In the cerebellar flocculus, which supports oculomotor learning, complex spikes are driven by image motion on the retina, which could indicate an oculomotor error. In the same neurons, complex spikes also can be driven by nonvisual signals. It has been shown that the calcium transients accompanying each complex spike can vary in amplitude, even within a given cell, therefore, we compared the calcium responses associated with the visual and nonvisual inputs to floccular Purkinje cells. The calcium indicator GCaMP6f was selectively expressed in Purkinje cells, and fiber photometry was used to record the calcium responses from a population of Purkinje cells in the flocculus of awake behaving mice. During visual (optokinetic) stimuli and pairing of vestibular and visual stimuli, the calcium level increased during contraversive retinal image motion. During performance of the vestibulo-ocular reflex in the dark, calcium increased during contraversive head rotation and the associated ipsiverse eye movements. The amplitude of this nonvisual calcium response was comparable to that during conditions with retinal image motion present that induce oculomotor learning. Thus, population calcium responses of Purkinje cells in the cerebellar flocculus to visual and nonvisual input are similar to what has been reported previously for complex spikes, suggesting that multimodal instructive signals control the synaptic plasticity supporting oculomotor learning.NEW & NOTEWORTHY It was long known that the climbing fiber input to Purkinje cells in the cerebellar flocculus conveys visual feedback about the accuracy of image-stabilizing oculomotor reflexes. More recently, the same climbing fibers were reported to carry nonvisual signals. Here, we report that both visual and nonvisual inputs can elicit robust calcium responses in the Purkinje cells, suggesting that the instructive signals guiding oculomotor plasticity are multimodal.

Multiple factors affect the evolution of repertoire size across birds

Abstract Changes in signaling repertoires across species allow for insight into the macroevolutionary forces that control signaling systems. Signaling systems are theorized to be affected by both the social and ecological environments of species. With respect to social variables, increased social complexity is thought to lead to increased vocal complexity. Although ecology can affect signaling systems in numerous ways, one potential effect of ecology is that more cluttered habitats should lead to greater reliance on nonvisual (e.g., vocal) signals. To test these concepts on a macroevolutionary scale, we compiled a large dataset of avian vocal repertoires. We amassed vocal repertoires for 821 species of birds and for many of these species categorized their vocalizations into usage categories (e.g., alarm, contact). To analyze the social and ecological forces that act on repertoire evolution, we incorporated datasets with several social variables (e.g., cooperative breeding and length of social bond), and included data on the habitat and foraging behaviors of species within the dataset. We used Bayesian phylogenetic analysis to test for potential relationships within the data. We found that cooperative breeding was a significant predictor of larger repertoire size in birds; we also find several, more targeted effects. For instance, foraging strata affected repertoire size and repertoire composition. In sum, we find considerable evidence that social features affect repertoire size while certain ecological variables have more targeted effects on vocal repertoires.

Mouse Visual Cortex Is Modulated by Distance Traveled and by Theta Oscillations

SummaryThe visual responses of neurons in the primary visual cortex (V1) are influenced by the animal’s position in the environment [1, 2, 3, 4, 5]. V1 responses encode positions that co-fluctuate with those encoded by place cells in hippocampal area CA1 [2, 5]. This correlation might reflect a common influence of non-visual spatial signals on both areas. Place cells in CA1, indeed, do not rely only on vision; their place preference depends on the physical distance traveled [6, 7, 8, 9, 10, 11] and on the phase of the 6–9 Hz theta oscillation [12, 13]. Are V1 responses similarly influenced by these non-visual factors? We recorded V1 and CA1 neurons simultaneously while mice performed a spatial task in a virtual corridor by running on a wheel and licking at a reward location. By changing the gain that couples the wheel movement to the virtual environment, we found that ∼20% of V1 neurons were influenced by the physical distance traveled, as were ∼40% of CA1 place cells. Moreover, the firing rate of ∼24% of V1 neurons was modulated by the phase of theta oscillations recorded in CA1 and the response profiles of ∼7% of V1 neurons shifted spatially across the theta cycle, analogous to the phase precession observed in ∼37% of CA1 place cells. The influence of theta oscillations on V1 responses was more prominent in putative layer 6. These results reveal that, in a familiar environment, sensory processing in V1 is modulated by the key non-visual signals that influence spatial coding in the hippocampus.

Image captions: global-local and joint signals attention model (GL-JSAM)

For automated visual captioning, existing neural encoder-decoder methods commonly use a simple sequence-to-sequence or an attention-based mechanism. The attention-based models pay attention to specific visual areas or objects; using a single heat map that indicates which portion of the image is most important rather than treating the objects (within the image) equally. These models are usually a mixture of Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) architectures. CNN’s generally extract global visual signals that only provide global information of main objects, attributes, and their relationship, but fail to provide local (regional) information within objects, such as lines, corners, curve and edges. On one hand, missing some of the information and details of local visual signals may lead to misprediction, misidentification of objects or completely missing the main object(s). On the other hand, additional visual signals information produces meaningless and irrelevant description, which may be coming from objects in foreground or background. To address these concerns, we created a new joint signals attention image captioning model for global and local signals that is adaptive by nature. Primarily, proposed model extracts global visual signals at image-level and local visual signals at object-level. The joint signal attention model (JSAM) plays a dual role in visual signal extraction and non-visual signal prediction. Initially, JSAM selects meaningful global and regional visual signals to discard irrelevant visual signals and integrates selected visual signals smartly. Subsequently, in a language model, smart JSAM decides at each time-step (level) on how to attend visual or non-visual signals to generate accurate, descriptive, and elegant sentences. Lastly, we examine the efficiency and superiority of the projected model over recent similar image captioning models by conducting essential experimentations on the MS-COCO dataset.

Multisensory Cues for Encoding Urgency of System Hazards: Effect of Operator Experience on Perceived Urgency

ABSTRACTObjective: This study evaluated sonification and tactification for encoding urgency of system health status presented in the ground control station (GCS) visual interface of an unmanned aircraft system (UAS), and the observer’s perception of urgency.Background: The barrage of data in the GCS visual interface has the potential to isolate the operator from detecting system hazards that threatens the ability of the operator to operate the UAS effectively.Method: The pitch of the UAS’s engine revolutions per minute was mapped to a sonification, and excessive attitude upset of the UAS was mapped to a tactification in order to present a continuous awareness of the system’s health without being invasive and obtrusive. Participants with and without flying experience were required to monitor system health, while carrying out a secondary task.Results: Regardless of flying experience, sonification enhanced hazard detection compared to a visual-only GCS interface, but tactification did not aid performance.Conclusion: While multimodal displays have been studied in remotely piloted vehicles, this is the initial effort to demonstrate that sonification can influence perceived urgency leading to greater warning compliance. Further research is warranted to develop guidelines to ensure that non-visual signals can convey different levels of urgency for a continuous awareness of a system’s health, and thereby permit the operator to establish the appropriate level of priority to address the alarmed condition.

Body orientation contributes to modelling the effects of gravity for target interception in humans.

It is known that interception of targets accelerated by gravity involves internal models coupled with visual signals. Non-visual signals related to head and body orientation relative to gravity may also contribute, although their role is poorly understood. In a novel experiment, we asked pitched observers to hit a virtual target approaching with an acceleration that was either coherent or incoherent with their pitch-tilt. Initially, the timing errors were large and independent of the coherence between target acceleration and observer's pitch. With practice, however, the timing errors became substantially smaller in the coherent conditions. The results show that information about head and body orientation can contribute to modelling the effects of gravity on a moving target. Orientation cues from vestibular and somatosensory signals might be integrated with visual signals in the vestibular cortex, where the internal model of gravity is assumed to be encoded. Interception of moving targets relies on visual signals and internal models. Less is known about the additional contribution of non-visual cues about head and body orientation relative to gravity. We took advantage of Galileo's law of motion along an incline to demonstrate the effects of vestibular and somatosensory cues about head and body orientation on interception timing. Participants were asked to hit a ball rolling in a gutter towards the eyes, resulting in image expansion. The scene was presented in a head-mounted display, without any visual information about gravity direction. In separate blocks of trials participants were pitched backwards by 20° or 60°, whereas ball acceleration was randomized across trials so as to be compatible with rolling down a slope of 20° or 60°. Initially, the timing errors were large, independently of the coherence between ball acceleration and pitch angle, consistent with responses based exclusively on visual information because visual stimuli were identical at both tilts. At the end of the experiment, however, the timing errors were systematically smaller in the coherent conditions than the incoherent ones. Moreover, the responses were significantly (P=0.007) earlier when participants were pitched by 60° than when they were pitched by 20°. Therefore, practice with the task led to incorporation of information about head and body orientation relative to gravity for response timing. Instead, posture did not affect response timing in a control experiment in which participants hit a static target in synchrony with the last of a predictable series of stationary audiovisual stimuli.

Processing of object motion and self-motion in the lateral subdivision of the medial superior temporal area in macaques.

Multiple areas of macaque cortex are involved in visual motion processing, but their relative functional roles remain unclear. The medial superior temporal (MST) area is typically divided into lateral (MSTl) and dorsal (MSTd) subdivisions that are thought to be involved in processing object motion and self-motion, respectively. Whereas MSTd has been studied extensively with regard to processing visual and nonvisual self-motion cues, little is known about self-motion signals in MSTl, especially nonvisual signals. Moreover, little is known about how self-motion and object motion signals interact in MSTl and how this differs from interactions in MSTd. We compared the visual and vestibular heading tuning of neurons in MSTl and MSTd using identical stimuli. Our findings reveal that both visual and vestibular heading signals are weaker in MSTl than in MSTd, suggesting that MSTl is less well suited to participate in self-motion perception than MSTd. We also tested neurons in both areas with a variety of combinations of object motion and self-motion. Our findings reveal that vestibular signals improve the separability of coding of heading and object direction in both areas, albeit more strongly in MSTd due to the greater strength of vestibular signals. Based on a marginalization technique, population decoding reveals that heading and object direction can be more effectively dissociated from MSTd responses than MSTl responses. Our findings help to clarify the respective contributions that MSTl and MSTd make to processing of object motion and self-motion, although our conclusions may be somewhat specific to the multipart moving objects that we employed. NEW & NOTEWORTHY Retinal image motion reflects contributions from both the observer's self-motion and the movement of objects in the environment. The neural mechanisms by which the brain dissociates self-motion and object motion remain unclear. This study provides the first systematic examination of how the lateral subdivision of area MST (MSTl) contributes to dissociating object motion and self-motion. We also examine, for the first time, how MSTl neurons represent translational self-motion based on both vestibular and visual cues.

Touch engages visual spatial contextual processing

The spatial context in which we view a visual stimulus strongly determines how we perceive the stimulus. In the visual tilt illusion, the perceived orientation of a visual grating is affected by the orientation signals in its surrounding context. Conceivably, the spatial context in which a visual grating is perceived can be defined by interactive multisensory information rather than visual signals alone. Here, we tested the hypothesis that tactile signals engage the neural mechanisms supporting visual contextual modulation. Because tactile signals also convey orientation information and touch can selectively interact with visual orientation perception, we predicted that tactile signals would modulate the visual tilt illusion. We applied a bias-free method to measure the tilt illusion while testing visual-only, tactile-only or visuo-tactile contextual surrounds. We found that a tactile context can influence visual tilt perception. Moreover, combining visual and tactile orientation information in the surround results in a larger tilt illusion relative to the illusion achieved with the visual-only surround. These results demonstrate that the visual tilt illusion is subject to multisensory influences and imply that non-visual signals access the neural circuits whose computations underlie the contextual modulation of vision.

Physiological responses of anti-predation in prey fish to the threat of piscivorous fish in different underwater visibility conditions

ABSTRACT Considering that the anti-predation behaviour of prey fishes may vary with predator hunting strategy, we experimentally investigated the physiological responses of Astyanax bimaculatus (Linnaeus, 1758) shoals to the presence of two piscivorous fish species with different hunting modes: sit-and-wait and active pursuit. In addition, we evaluated the influence of underwater visibility conditions on the interaction between predator and prey. We tested the hypotheses that: (i) prey plasma cortisol and glucose levels vary according to piscivore hunting strategy, and (ii) no reduction in plasma glucose and cortisol levels takes place in turbid environments due to the inability of prey to accurately recognise predators by non-visual signals. The results revealed that the presence of piscivorous fish increased plasma cortisol levels in prey, with no significant difference recorded between the two hunting strategy treatments. We also observed no significant change in plasma glucose levels in different water transparency conditions. Thus, we conclude that physiological changes in the selected prey fish do not vary with predator hunting mode, and it is therefore necessary to consider the ability of the prey to recognise and evaluate danger, regardless of piscivore hunting strategy.

The respective contributions of visual and proprioceptive afferents to the mirror illusion in virtual reality.

The reflection of passive arm displacement in a mirror is a powerful means of inducing a kinaesthetic illusion in the static arm hidden behind the mirror. Our recent research findings suggest that this illusion is not solely visual in origin but results from the combination of visual and proprioceptive signals from the two arms. To determine the respective contributions of visual and proprioceptive signals to this illusion, we reproduced the mirror paradigm in virtual reality. As in the physical version of the mirror paradigm, one of the participant’s arms (the left arm, in our study) could be flexed or extended passively. This movement was combined with displacements of the avatar’s left and right forearms, as viewed in a first-person perspective through a virtual reality headset. In order to distinguish between visual and proprioceptive contributions, two unimodal conditions were applied separately: displacement of the avatar’s forearms in the absence of physical displacement of the left arm (the visual condition), and displacement of the left forearm while the avatar’s forearms were masked (the proprioceptive condition). Of the 34 female participants included in the study, 28 experienced a kinaesthetic mirror illusion in their static (right) arm. The strength of the illusion (expressed in terms of speed and duration) evoked by the bimodal condition was much higher than that observed in either of the two unimodal conditions. Our present results confirm that the involvement of visual signals in the mirror illusion—often considered as a prototypic visual illusion—has been overstated. The mirror illusion also involves non-visual signals (bilateral proprioceptive-somaesthetic signals, in fact) that interact with the visual signals and strengthen the kinaesthetic effect.

Saccade-based termination responses in macaque V1 and visual perception.

Neurons in visual areas of the brain are generally characterized by the increase in firing rate that occurs when a stimulus is flashed on in the receptive field (RF). However, neurons also increase their firing rate when a stimulus is turned off. These "termination responses" or "after-discharges" that occur with flashed stimuli have been observed in area V1 and they may be important for vision as stimulus terminations have been shown to influence visual perception. The goal of the present study was to determine the strength of termination responses in the more natural situation in which eye movements move a stimulus out of an RF. We find that termination responses do occur in macaque V1 when termination results from a saccadic eye movement, but they are smaller in amplitude compared to flashed-off stimuli. Furthermore, there are termination responses even in the absence of visual stimulation. These findings demonstrate that termination responses are a component of naturalistic vision. They appear to be based on both visual and nonvisual signals in visual cortex. We speculate that the weakening of termination responses might be a neural correlate of saccadic suppression, the loss of perceptual sensitivity around the time of saccades.

Role of visual and non-visual cues in damselfly mate recognition

In many species of damselflies, sexual conflict in the form of male harassment is thought to explain the widespread existence of female-limited color polymorphisms. With a few exceptions, the majority of investigations into these mating systems have assumed that male damselflies primarily use visual cues to detect and recognize their mates. Recently, laboratory studies have demonstrated that damselflies orient to olfactory signals from prey and that males orient to chemical cues from conspecific females. However, to date there are no field experiments which explicitly test the role that chemical cues play in sex recognition. Here we used a field experiment on Enallagma civile damselflies to test if free flying males detect and recognize females in the absence of visual cues through the use of non-visual signals. In the absence of visual cues males did not exhibit positive responses toward female conspecifics, whereas when both visual and non-visual cues were present, males readily detected females and oft...

Signaling in a Polluted World: Oxidative Stress as an Overlooked Mechanism Linking Contaminants to Animal Communication

The capacity to communicate effectively with other individuals plays a critical role in the daily life of an individual and can have important fitness consequences. Animals rely on a number of visual and non-visual signals, whose production brings costs to the individual. The theory of honest signaling states that these costs are higher for low than for high-quality individuals, which prevents cheating and makes signals, such as skin and plumage colouration, indicators of individual’s quality or condition. The condition-dependent nature of signals makes them ideally suited as indicators of environmental quality, implying that signal production might be affected by contaminants. In this mini-review article, we have made the point that oxidative stress (OS) is one overlooked mechanism linking exposure to contaminants to signaling because (i) many contaminants can influence the individual’s oxidative balance, and (ii) generation of both visual and non-visual signals is sensitive to oxidative stress. To this end, we have provided the first comprehensive review on the way both non-organic (heavy metals, especially mercury) and organic (persistent organic pollutants) contaminants may influence either OS or sexual signaling. We have also paid special attention to emerging classes of pollutants like brominated flame-retardants and perfluoroalkoxy alkanes in order to stimulate research in this area. We have finally provided suggestions and warnings for future work on the links among OS, sexual signaling and contaminant exposure.

Direction-dependent activation of the insular cortex during vertical and horizontal hand movements

The planning of any motor action requires a complex multisensory processing by the brain. Gravity – immutable on Earth – has been shown to be a key input to these mechanisms. Seminal fMRI studies performed during visual perception of falling objects and self-motion demonstrated that humans represent the action of gravity in parts of the cortical vestibular system; in particular, the insular cortex and the cerebellum. However, little is known as to whether a specific neural network is engaged when processing non-visual signals relevant to gravity. We asked participants to perform vertical and horizontal hand movements without visual control, while lying in a 3T-MRI scanner. We highlighted brain regions activated in the processing of vertical movements, for which the effects of gravity changed during execution. Precisely, the left insula was activated in vertical movements and not in horizontal movements. Moreover, the network identified by contrasting vertical and horizontal movements overlapped with neural correlates previously associated to the processing of simulated self-motion and visual perception of the vertical direction. Interestingly, we found that the insular cortex activity is direction-dependent which suggests that this brain region processes the effects of gravity on the moving limbs through non-visual signals.

Distinct Computational Principles Govern Multisensory Integration in Primary Sensory and Association Cortices

Human observers typically integrate sensory signals in a statistically optimal fashion into a coherent percept by weighting them in proportion to their reliabilities. An emerging debate in neuroscience is to which extent multisensory integration emerges already in primary sensory areas or is deferred to higher-order association areas. This fMRI study used multivariate pattern decoding to characterize the computational principles that define how auditory and visual signals are integrated into spatial representations across the cortical hierarchy. Our results reveal small multisensory influences that were limited to a spatial window of integration in primary sensory areas. By contrast, parietal cortices integrated signals weighted by their sensory reliabilities and task relevance in line with behavioral performance and principles of statistical optimality. Intriguingly, audiovisual integration in parietal cortices was attenuated for large spatial disparities when signals were unlikely to originate from a common source. Our results demonstrate that multisensory interactions in primary and association cortices are governed by distinct computational principles. In primary visual cortices, spatial disparity controlled the influence of non-visual signals on the formation of spatial representations, whereas in parietal cortices, it determined the influence of task-irrelevant signals. Critically, only parietal cortices integrated signals weighted by their bottom-up reliabilities and top-down task relevance into multisensory spatial priority maps to guide spatial orienting.

Odor signals in wild western lowland gorillas: an involuntary and extra-group communication hypothesis

Odors constitute one of the most ancient ways of communication among animals. Whereas the key role of olfactory communication is well established in insects or rodents, its contribution to primate behavior remains very speculative. In a recent report, Klailova and Lee [1] studied the variations of intensity of the typical steroid musk odor produced by wild adult male gorilla silverbacks during encounters with opponents from other social units (lone males or other groups). The odor intensity increased in situations of potential conflicts, when the silverback encountered another male gorilla, which constitutes a potential threat for his infants and mate competitor for his females. Importantly, the odor intensity was greater when the silverback reacted with a demonstrative threat display, and milder when he decided to react quietly and avoid the conflict by sneaking away. According to the authors, this indicates that silverback gorillas can flexibly adjust the odor signal to the social context, and the mild signal associated with the quiet response is used within the silverback's group to promote cohesion. Rather, based on both physiological and behavioral data, we propose that (1) the odor intensity varies monotonically with arousal, across all situations, and (2) that the function of the odor signal is most relevant for the interactions between males of different groups. In such framework, the odor stimulus might also constitute a non-visual signal indicator of the emitter's identity, and therefore play a role in the regulation of the interactions with other gorillas. Establishing the balance among those potential functions is critical to understand the role of odors in the regulation of social and ecological interactions in primates.

Nonvisual Complex Spike Signals in the Rabbit Cerebellar Flocculus

In addition to the well-known signals of retinal image slip, floccular complex spikes (CSs) also convey nonvisual signals. We recorded eye movement and CS activity from Purkinje cells in awake rabbits sinusoidally oscillated in the dark on a vestibular turntable. The stimulus frequency ranged from 0.2 to 1.2 Hz, and the velocity amplitude ranged from 6.3 to 50°/s. The average CS modulation was evaluated at each combination of stimulus frequency and amplitude. More than 75% of the Purkinje cells carried nonvisual CS signals. The amplitude of this modulation remained relatively constant over the entire stimulus range. The phase response of the CS modulation in the dark was opposite to that during the vestibulo-ocular reflex (VOR) in the light. With increased frequency, the phase response systematically shifted from being aligned with contraversive head velocity toward peak contralateral head position. At fixed frequency, the phase response was dependent on peak head velocity, indicating a system nonlinearity. The nonvisual CS modulation apparently reflects a competition between eye movement and vestibular signals, resulting in an eye movement error signal inferred from nonvisual sources. The combination of this error signal with the retinal slip signal in the inferior olive results in a net error signal reporting the discrepancy between the actual visually measured eye movement error and the inferred eye movement error derived from measures of the internal state. The presence of two error signals requires that the role of CSs in models of the floccular control of VOR adaption be expanded beyond retinal slip.

Perceived Size Change Induced by Nonvisual Signals in Darkness: The Relative Contribution of Vergence and Proprioception

Most of the time, the human visual system computes perceived size by scaling the size of an object on the retina with its perceived distance. There are instances, however, in which size-distance scaling is not based on visual inputs but on extraretinal cues. In the Taylor illusion, the perceived afterimage that is projected on an observer's hand will change in size depending on how far the limb is positioned from the eyes-even in complete darkness. In the dark, distance cues might derive from hand position signals either by an efference copy of the motor command to the moving hand or by proprioceptive input. Alternatively, there have been reports that vergence signals from the eyes might also be important. We performed a series of behavioral and eye-tracking experiments to tease apart how these different sources of distance information contribute to the Taylor illusion. We demonstrate that, with no visual information, perceived size changes mainly as a function of the vergence angle of the eyes, underscoring its importance in size-distance scaling. Interestingly, the strength of this relationship decreased when a mismatch between vergence and proprioception was introduced, indicating that proprioceptive feedback from the arm also affected size perception. By using afterimages, we provide strong evidence that the human visual system can benefit from sensory signals that originate from the hand when visual information about distance is unavailable.