Neural Summation Research Articles

Plaid masking explained with input-dependent dendritic nonlinearities

A serious obstacle for understanding early spatial vision comes from the failure of the so-called standard model (SM) to predict the perception of plaid masking. But the SM originated from a major oversimplification of single neuron computations, ignoring fundamental properties of dendrites. Here we show that a spatial vision model including computations mimicking the input-dependent nature of dendritic nonlinearities, i.e. including nonlinear neural summation, has the potential to explain plaid masking data.

A simple non-linear neural summation model predicts basic and complex motion perception phenomena

A simple non-linear neural summation model predicts basic and complex motion perception phenomena

State-of-the-art image and video quality assessment with a metric based on an intrinsically non-linear neural summation model.

The development of automatic methods for image and video quality assessment that correlate well with the perception of human observers is a very challenging open problem in vision science, with numerous practical applications in disciplines such as image processing and computer vision, as well as in the media industry. In the past two decades, the goal of image quality research has been to improve upon classical metrics by developing models that emulate some aspects of the visual system, and while the progress has been considerable, state-of-the-art quality assessment methods still share a number of shortcomings, like their performance dropping considerably when they are tested on a database that is quite different from the one used to train them, or their significant limitations in predicting observer scores for high framerate videos. In this work we propose a novel objective method for image and video quality assessment that is based on the recently introduced Intrinsically Non-linear Receptive Field (INRF) formulation, a neural summation model that has been shown to be better at predicting neural activity and visual perception phenomena than the classical linear receptive field. Here we start by optimizing, on a classic image quality database, the four parameters of a very simple INRF-based metric, and proceed to test this metric on three other databases, showing that its performance equals or surpasses that of the state-of-the-art methods, some of them having millions of parameters. Next, we extend to the temporal domain this INRF image quality metric, and test it on several popular video quality datasets; again, the results of our proposed INRF-based video quality metric are shown to be very competitive.

Binocular summation revisited: Beyond √2.

Our ability to detect faint images is better with two eyes than with one, but how great is this improvement? A meta-analysis of 65 studies published across more than 5 decades shows definitively that psychophysical binocular summation (the ratio of binocular to monocular contrast sensitivity) is significantly greater than the canonical value of √2. Several methodological factors were also found to affect summation estimates. Binocular summation was significantly affected by both the spatial and temporal frequency of the stimulus, and stimulus speed (the ratio of temporal to spatial frequency) systematically predicts summation levels, with slow speeds (high spatial and low temporal frequencies) producing the strongest summation. We furthermore show that empirical summation estimates are affected by the ratio of monocular sensitivities, which varies across individuals, and is abnormal in visual disorders such as amblyopia. A simple modeling framework is presented to interpret the results of summation experiments. In combination with the empirical results, this model suggests that there is no single value for binocular summation, but instead that summation ratios depend on methodological factors that influence the strength of a nonlinearity occurring early in the visual pathway, before binocular combination of signals. Best practice methodological guidelines are proposed for obtaining accurate estimates of neural summation in future studies, including those involving patient groups with impaired binocular vision.

Higher-order neural processing tunes motion neurons to visual ecology in three species of hawkmoths.

To sample information optimally, sensory systems must adapt to the ecological demands of each animal species. These adaptations can occur peripherally, in the anatomical structures of sensory organs and their receptors; and centrally, as higher-order neural processing in the brain. While a rich body of investigations has focused on peripheral adaptations, our understanding is sparse when it comes to central mechanisms. We quantified how peripheral adaptations in the eyes, and central adaptations in the wide-field motion vision system, set the trade-off between resolution and sensitivity in three species of hawkmoths active at very different light levels: nocturnal Deilephila elpenor, crepuscular Manduca sexta, and diurnal Macroglossum stellatarum. Using optical measurements and physiological recordings from the photoreceptors and wide-field motion neurons in the lobula complex, we demonstrate that all three species use spatial and temporal summation to improve visual performance in dim light. The diurnal Macroglossum relies least on summation, but can only see at brighter intensities. Manduca, with large sensitive eyes, relies less on neural summation than the smaller eyed Deilephila, but both species attain similar visual performance at nocturnal light levels. Our results reveal how the visual systems of these three hawkmoth species are intimately matched to their visual ecologies.

Deafferentation of the Superior Colliculus Abolishes Spatial Summation of Redundant Visual Signals.

Two visual signals appearing simultaneously are detected more rapidly than either signal appearing alone. Part of this redundant target effect (RTE) can be attributed to neural summation that has been proposed to occur in the superior colliculus (SC). We report direct evidence in two neurological patients for neural summation in the SC, and that it is mediated by afferent visual information transmitted through its brachium. The RTE was abolished in one patient with a hemorrhage involving the right posterior thalamus that damaged part of the SC and that disrupted its brachium; and in another patient in whom the SC appeared intact but deafferented due to traumatic avulsion of its brachium. In addition reaction time for unilateral targets in the contralesional field was slowed in both patients, providing the first evidence that visual afferents to the SC contribute to the efficiency of target detection.

Neural Summation in the Hawkmoth Visual System Extends the Limits of Vision in Dim Light

Most of the world's animals are active in dim light and depend on good vision for the tasks of daily life. Many have evolved visual adaptations that permit a performance superior to that of manmade imaging devices [1]. In insects, a major model visual system, nocturnal species show impressive visual abilities ranging from flight control [2, 3], to color discrimination [4, 5], to navigation using visual landmarks [6-8] or dim celestial compass cues [9, 10]. In addition to optical adaptations that improve their sensitivity in dim light [11], neural summation of light in space and time-which enhances the coarser and slower features of the scene at the expense of noisier finer and faster features-has been suggested to improve sensitivity in theoretical [12-14], anatomical [15-17], and behavioral [18-20] studies. How these summation strategies function neurally is, however, presently unknown. Here, we quantified spatial and temporal summation in the motion vision pathway of a nocturnal hawkmoth. We show that spatial and temporal summation combine supralinearly to substantially increase contrast sensitivity and visual information rate over four decades of light intensity, enabling hawkmoths to see at light levels 100 times dimmer than without summation. Our results reveal how visual motion is calculated neurally in dim light and how spatial and temporal summation improve sensitivity while simultaneously maximizing spatial and temporal resolution, thus extending models of insect motion vision derived predominantly from diurnal flies. Moreover, the summation strategies we have revealed may benefit manmade vision systems optimized for variable light levels [21].

Neural summation in human motor cortex by subthreshold transcranial magnetic stimulations.

Integration of diverse synaptic inputs is a basic neuronal operation that relies on many neurocomputational principles, one of which is neural summation. However, we lack empirical understanding of neuronal summation in the human brains in vivo. Here, we explored the effect of neural summation on the motor cortex using two subthreshold pulses of transcranial magnetic stimulation (TMS), each with intensities ranging from 60 to 95% of the resting motor threshold (RMT) and interstimulus interval (ISI) varying from 1 to 25 ms. We found that two subthreshold TMS pulses can produce suprathreshold motor response when ISIs were less than 10 ms, most prominent at 1, 1.5 and 3 ms. This facilitatory, above-threshold response was evident when the intensity of the subthreshold pulses was above 80% of RMT but was absent as the intensity was 70% or below. Modeling of the summation data across intensity suggested that they followed an exponential function with excellent model fitting. Understanding the constraints for inducing summation of subthreshold stimulations to generate above-threshold response may have implications in modeling neural operations and potential clinical applications.

The Remarkable Visual Abilities of Nocturnal Insects: Neural Principles and Bioinspired Night-Vision Algorithms

Despite their tiny eyes and brains, nocturnal insects have remarkable visual abilities. Recent work - particularly on fast-flying moths and bees and on ball-rolling dung beetles - has shown that nocturnal insects are able to distinguish colors, to detect faint movements, to learn visual landmarks, to orient to the faint pattern of polarized light produced by the moon, and to navigate using the stars. These impressive visual abilities are the result of exquisitely adapted eyes and visual systems, the product of millions of years of evolution. Even though we are only at the threshold of understanding the neural mechanisms responsible for reliable nocturnal vision, growing evidence suggests that the neural summation of photons in space and time is critically important: even though vision in dim light becomes necessarily coarser and slower, those details that are preserved are seen clearly. These benefits of spatio-temporal summation have obvious implications for dim-light video technologies. In addition to reviewing the visual adaptations of nocturnal insects, we here describe an algorithm inspired by nocturnal visual processing strategies - from amplification of primary image signals to optimized spatio-temporal summation to reduce noise - that dramatically increases the reliability of video collected in dim light, including the preservation of color.

Binocular summation of second-order global motion signals in human vision

Although many studies have examined the principles governing first-order global motion perception, the mechanisms that mediate second-order global motion perception remain unresolved. This study investigated the existence, nature and extent of the binocular advantage for encoding second-order (contrast-defined) global motion. Motion coherence thresholds (79.4% correct) were assessed for determining the direction of radial, rotational and translational second-order motion trajectories as a function of local element modulation depth (contrast) under monocular and binocular viewing conditions. We found a binocular advantage for second-order global motion processing for all motion types. This advantage was mainly one of enhanced modulation sensitivity, rather than of motion-integration. However, compared to findings for first-order motion where the binocular advantage was in the region of a factor of around 1.7 (Hess et al., 2007), the binocular advantage for second-order global motion was marginal, being in the region of around 1.2. This weak enhancement in sensitivity with binocular viewing is considerably less than would be predicted by conventional models of either probability summation or neural summation.

Visual Cortex Responses to Single- and Simultaneous Multiple-Electrode Stimulation of the Retina: Implications for Retinal Prostheses

The aim of this study was to compare simultaneous stimulation of multiple electrodes to single-electrode stimulation in a retinal prosthesis. A platinum electrode array was implanted into the suprachoroidal space in six normally sighted anesthetized cats. Multiunit activity from the primary visual cortex in response to retinal stimulation was recorded. Cortical thresholds, yield of responses, dynamic ranges, and the spread of retinal activation were measured for three modes of stimulation: single electrode, half-row (six-electrode horizontal line), and column (seven-electrode vertical line). Stimulation of the best half-rows and columns was found to elicit activity with higher yield and lower charge thresholds per electrode compared to the best single electrodes. Dynamic ranges between the three modes were similar. As expected, peak voltages measured for columns and half-rows were lower than those measured for single electrodes. Spread of retinal activation, determined by the increase in threshold with distance in the retina from the best site, was found to be similar between single- and multiple-electrode stimulation but dependent on orientation. The lower thresholds, higher yield, equivalent dynamic ranges, and equivalent spread of retinal activation observed from simultaneous stimulation of multiple electrodes may be due to current and/or neural summation within the retina. Such stimulation techniques could be useful for the presentation of lines and edges of objects using a suprachoroidal retinal stimulator with low voltage compliance. Furthermore, the results suggest that more complex visual processing strategies in addition to sequential stimulation of individual electrodes should be considered for retinal prostheses.

Bilateral redundancy gain and callosal integrity in a man with callosal lipoma: A diffusion-tensor imaging study

We investigated whether abnormalities in the structural organization of the corpus callosum in the presence of curvilinear lipoma are associated with increased facilitation of response time to bilateral stimuli, an effect known as the redundancy gain (RG). A patient (A.J.) with a curvilinear lipoma of the corpus callosum, his genetically-identical twin, and age-matched control participants made speeded responses to luminant stimuli. Structural organization of callosal regions was assessed with diffusion-tensor imaging. A.J. was found to have reduced structural integrity in the splenium of the corpus callosum and produced a large RG suggestive of neural summation.

Influence of target size and eccentricity on binocular summation of reaction time in kinetic perimetry

To assess how target size and eccentricity affect binocular summation (BS) of reaction time (RT) at suprathreshold level, we measured RT using targets of 0.108° and 0.216° at four eccentricities (0°, 5°, 15°, 25°) in six normal volunteers. The difference between the monocular/binocular RT differentials for both sizes significantly increased in the periphery (P<0.05). The smaller target required significantly longer monocular RT at 25° (P<0.01) and generated greater neural summation than the larger target (P<0.01). This suggests that when monocular function has reached its limit in visual processing in the periphery, BS increases, facilitates visual processing, and shortens binocular RT.

Spatial Contrast Sensitivity at Twilight: Luminance, Monocularity, and Oxygenation

Visual performance in dim light is compromised by lack of oxygen (hypoxia). The possible influence of altered oxygenation on foveal contrast sensitivity under mesopic (twilight) viewing conditions is relevant to aircrew flying at night, including when using night vision devices, but is poorly documented. Foveal contrast sensitivity was measured binocularly and monocularly in 12 subjects at 7 spatial frequencies, ranging from 0.5 to approximately 16 cycles per degree, using sinusoidal Gabor patch gratings. Hypoxic performance breathing 14.1% oxygen, equivalent to altitude exposure at 3048 m (10,000 ft), was compared with breathing air at sea level (normoxia) at low photopic (28 cd x m(-2)), borderline upper mesopic (approximately 2.1 cd x m(-2)) and midmesopic (approximately 0.26 cd x m(-2)) luminance. Mesopic performance was also assessed breathing 100% oxygen (hyperoxia). Typical 'inverted U' log/log plots of the contrast sensitivity function were obtained, with elevated thresholds (reduced sensitivity) at lower luminance. Binocular viewing enhanced sensitivity by a factor approximating square root of 2 for most conditions, supporting neural summation of the contrast signal, but had greater influence at the lowest light level and highest spatial frequencies (8.26 and 16.51 cpd). Respiratory challenges had no effect. Contrast sensitivity is poorer when viewing monocularly and especially at midmesopic luminance, with relevance to night flying. The foveal contrast sensitivity function is unaffected by respiratory disturbance when twilight conditions favor cone vision, despite known effects on retinal illumination (pupil size). The resilience of the contrast sensitivity function belies the vulnerability of foveal low contrast acuity to mild hypoxia at mesopic luminance.

Spatial and Temporal Effects of Interleaved Masking in Cochlear Implants

Modern cochlear implants utilize interleaved presentation of pulses on different electrodes to avoid physical interference among multiple current fields, yet neural interaction still exists. In the present study, masking was examined with four Nucleus24 users with the banded electrode array in an interleaved masking paradigm, where a probe stimulus was interleaved with a masker stimulus. Spatial and temporal aspects of masking were addressed by fixing the masker at the middle of the electrode array and changing the location of the probe and by testing various stimulation rates: 125, 500, 2,000, and 6,410 Hz. In addition, growth of masking (GOM) was assessed by changing the masker level in six steps. Results indicated that masking patterns were generally much wider, regardless of stimulation rate, than those in acoustic hearing. The amount of masking decreased from the peak at the rate of approximately 0.5 dB/mm even at the highest masker level. The pattern of GOM with the rates higher than 500 Hz was different from that observed in previous masking studies, characterized by markedly shallow growth at low masker levels or overall shallow growth. A facilitating effect of the masker (lowering the threshold) was suspected, except for the 125-Hz condition, due to the fibers that were subliminally excited, but not discharged, by the masker with local perturbations of membrane potentials, and were subsequently discharged easily by a lower level probe when the temporal gap between masker and probe was sufficiently short. These results suggest that both refractory characteristics of neurons and neural summation be considered in interleaved stimulation of pulses at high, but clinically relevant, stimulation rates. Overall, the present masking study might provide a basis for models in psychophysics and speech understanding in current cochlear implant systems utilizing high-rate interleaved stimulation.

Summation characteristics of the detection of compound gratings

Many classical experiments have shown that two superimposed gratings are more easily detected than a single grating, in keeping with probability theory. Here we test the rules for the detection of 2-component compound gratings by extending the range of parameters used in previous experiments. Two complementary methods of deriving summation indices are described. Data are presented so that the conditions for the transition from probability to neural summation are easily identified. True probability summation occurs only when grating contrasts are carefully perceptually equalised and spatial frequency differs by more than a factor of 2. A wide range of contrast ratios of the component gratings were explored such that gratings were at different contrasts, relative to respective thresholds. We find clear evidence of suppressive interactions when the compound gratings are composed of a close to threshold low frequency component and a below-threshold higher spatial frequency component.

Visual ecology of Indian carpenter bees II: adaptations of eyes and ocelli to nocturnal and diurnal lifestyles

Most bees are diurnal, with behaviour that is largely visually mediated, but several groups have made evolutionary shifts to nocturnality, despite having apposition compound eyes unsuited to vision in dim light. We compared the anatomy and optics of the apposition eyes and the ocelli of the nocturnal carpenter bee, Xylocopa tranquebarica, with two sympatric species, the strictly diurnal X. leucothorax and the occasionally crepuscular X. tenuiscapa. The ocelli of the nocturnal X. tranquebarica are unusually large (diameter ca. 1 mm) and poorly focussed. Moreover, their apposition eyes show specific visual adaptations for vision in dim light, including large size, large facets and very wide rhabdoms, which together make these eyes 9 times more sensitive than those of X. tenuiscapa and 27 times more sensitive than those of X. leucothorax. These differences in optical sensitivity are surprisingly small considering that X. tranquebarica can fly on moonless nights when background luminance is as low as 10(-5) cd m(-2), implying that this bee must employ additional visual strategies to forage and find its way back to the nest. These strategies may include photoreceptors with longer integration times and higher contrast gains as well as higher neural summation mechanisms for increasing visual reliability in dim light.

Optimum spatiotemporal receptive fields for vision in dim light

Many nocturnal insects depend on vision for daily life and have evolved different strategies to improve their visual capabilities in dim light. Neural summation of visual signals is one strategy to improve visual performance, and this is likely to be especially important for insects with apposition compound eyes. Here we develop a model to determine the optimum spatiotemporal sampling of natural scenes at gradually decreasing light levels. Image anisotropy has a strong influence on the receptive field properties predicted to be optimal at low light intensities. Spatial summation between visual channels is predicted to extend more strongly in the direction with higher correlations between the input signals. Increased spatiotemporal summation increases signal-to-noise ratio at low frequencies but sacrifices signal-to-noise ratio at higher frequencies. These results, while obtained from a model of the insect visual system, are likely to apply to visual systems in general.

Visual Reliability and Information Rate in the Retina of a Nocturnal Bee

Nocturnal animals relying on vision typically have eyes that are optically and morphologically adapted for both increased sensitivity and greater information capacity in dim light. Here, we investigate whether adaptations for increased sensitivity also are found in their photoreceptors by using closely related and fast-flying nocturnal and diurnal bees as model animals. The nocturnal bee Megalopta genalis is capable of foraging and homing by using visually discriminated landmarks at starlight intensities. Megalopta's near relative, Lasioglossum leucozonium, performs these tasks only in bright sunshine. By recording intracellular responses to Gaussian white-noise stimuli, we show that photoreceptors in Megalopta actually code less information at most light levels than those in Lasioglossum. However, as in several other nocturnal arthropods, Megalopta's photoreceptors possess a much greater gain of transduction, indicating that nocturnal photoreceptors trade information capacity for sensitivity. By sacrificing photoreceptor signal-to-noise ratio and information capacity in dim light for an increased gain and, thus, an increased sensitivity, this strategy can benefit nocturnal insects that use neural summation to improve visual reliability at night.

Multisensory integration for orienting responses in humans requires the activation of the superior colliculus

Animal studies have shown that the superior colliculus (SC) is important for synthesising information from multiple senses into a unified map of space. Here, we tested whether the SC is a critical neural substrate for multisensory spatial integration in humans. To do so, we took advantage of neurophysiological findings revealing that the SC does not receive direct projections from short-wavelength-sensitive S cones. In a simple reaction-time task, participants responded more quickly to concurrent peripheral (extra-foveal) audiovisual (AV) stimuli than to an auditory or visual stimulus alone, a phenomenon known as the redundant target effect (RTE). We show that the nature of this RTE was dependent on the colour of the visual stimulus. When using purple short-wavelength stimuli, to which the SC is blind, RTE was simply explained by probability summation, indicating that the redundant auditory and visual channels are independent. Conversely, with red long-wavelength stimuli, visible to the SC, the RTE was related to nonlinear neural summation, which constitutes evidence of integration of different sensory information. We also demonstrate that when AV stimuli were presented at fixation, so that the spatial orienting component of the task was reduced, neural summation was possible regardless of stimulus colour. Together, these findings provide support for a pivotal role of the SC in mediating multisensory spatial integration in humans, when behaviour involves spatial orienting responses.