Visual Hierarchy Research Articles

Threat assessment using visual hierarchy and conceptual firearms ontology

The work that established and explored the links between visual hierarchy and conceptual ontology of firearms for the purpose of threat assessment is continued. The previous study used geometrical information to find a target in the visual hierarchy and through the links with the conceptual ontology to derive high-level information that was used to assess a potential threat. Multiple improvements and new contributions are reported. The theoretical basis of the geometric feature extraction method was improved in terms of accuracy. The sample space used for validations is expanded from 31 to 153 firearms. Thus, a new larger and more accurate sequence of visual hierarchies was generated using a modified Gonzalez’ clustering algorithm. The conceptual ontology is elaborated as well and more links were created between the two kinds of hierarchies (visual and conceptual). The threat assessment equation is refined around ammunition-related properties and uses high-level information from the conceptual hierarchy. The experiments performed on weapons identification and threat assessment showed that our system recognized 100% of the cases if a weapon already belongs to the ontology and in 90.8% of the cases, determined the correct third ancestor (level concept) if the weapon is unknown to the ontology. To validate the accuracy of identification for a very large data set, we calculated the intervals of confidence for our system.

Predictions of Visual Content across Eye Movements and Their Modulation by Inferred Information.

The brain is proposed to operate through probabilistic inference, testing and refining predictions about the world. Here, we search for neural activity compatible with the violation of active predictions, learned from the contingencies between actions and the consequent changes in sensory input. We focused on vision, where eye movements produce stimuli shifts that could, in principle, be predicted. We compared, in humans, error signals to saccade-contingent changes of veridical and inferred inputs by contrasting the electroencephalographic activity after saccades to a stimulus presented inside or outside the blind spot. We observed early (<250 ms) and late (>250 ms) error signals after stimulus change, indicating the violation of sensory and associative predictions, respectively. Remarkably, the late response was diminished for blind-spot trials. These results indicate that predictive signals occur across multiple levels of the visual hierarchy, based on generative models that differentiate between signals that originate from the outside world and those that are inferred.

Arousal transitions in sleep, wakefulness, and anesthesia are characterized by an orderly sequence of cortical events

Many aspects of brain function are influenced by modulatory processes, including arousal. The most abrupt changes in arousal occur at the wake–sleep transition and at the induction of anesthetic conditions. They are accompanied by major electrophysiological changes, including an emergence of low-frequency (sleep-like) activity and a loss of mid-frequency (wake-like) activity that has been linked to feedback processes of the brain. Nevertheless, the causal relationship between these two types of electrophysiological changes, as well as the cortical mechanisms underlying changes in arousal and consciousness, remain poorly understood. To address this, we studied spontaneous electro-cortical activity during arousal changes in macaques. During sleep and at loss of consciousness induced by propofol anesthesia, we identified a prototypical sequence of cortical events in which the loss of mid-frequency activity preceded, by seconds, the increases in low-frequency activity. Furthermore, in visual areas, an influence of mid-frequency change onto high-frequency activity was observed across visual hierarchy. These results are consistent with the notion that drops in arousal and consciousness are facilitated by a release of feedback cortical inhibition.

Brightness masking is modulated by disparity structure

The luminance contrast at the borders of a surface strongly influences surface’s apparent brightness, as demonstrated by a number of classic visual illusions. Such phenomena are compatible with a propagation mechanism believed to spread contrast information from borders to the interior. This process is disrupted by masking, where the perceived brightness of a target is reduced by the brief presentation of a mask (Paradiso & Nakayama, 1991), but the exact visual stage that this happens remains unclear. In the present study, we examined whether brightness masking occurs at a monocular-, or a binocular-level of the visual hierarchy. We used backward masking, whereby a briefly presented target stimulus is disrupted by a mask coming soon afterwards, to show that brightness masking is affected by binocular stages of the visual processing. We manipulated the 3-D configurations (slant direction) of the target and mask and measured the differential disruption that masking causes on brightness estimation. We found that the masking effect was weaker when stimuli had a different slant. We suggest that brightness masking is partly mediated by mid-level neuronal mechanisms, at a stage where binocular disparity edge structure has been extracted.

Effectiveness of jatropha barriers as a soil and water conservation technology to rehabilitate gullies in northern Ethiopia

The dorsal and ventral visual pathways represent both visual and conceptual object properties. Yet the relative contribution of these two factors in the representational content of visual areas is unclear. Indeed, research investigating brain category representations rarely dissociate visual and semantic properties of objects. We present a human event-related fMRI study with a two-factorial stimulus set with 54 images that explicitly dissociates shape from category to investigate their independent contribution as well as their interactions through representational similarity analyses. Results reveal a contribution from each dimension in both streams, with a transition from shape to category along the posterior-to-anterior anatomical axis. The nature of category representations differs in the two pathways: ventral areas represent object animacy and dorsal areas represent object action properties. Furthermore, information about shape evolved from low-level pixel-based to high-level perceived shape following a posterior-to-anterior gradient similar to the shape-to-category emergence. To conclude, results show that representations of shape and category independently coexist, but at the same time they are closely related throughout the visual hierarchy. <b>SIGNIFICANCE STATEMENT</b> Research investigating visual cortex conceptual category representations rarely takes into account visual properties of objects. In this report, we explicitly dissociate shape from category and investigate independent contributions and interactions of these two highly correlated dimensions.

Multisensory causal inference in the brain.

At any given moment, our brain processes multiple inputs from its different sensory modalities (vision, hearing, touch, etc.). In deciphering this array of sensory information, the brain has to solve two problems: (1) which of the inputs originate from the same object and should be integrated and (2) for the sensations originating from the same object, how best to integrate them. Recent behavioural studies suggest that the human brain solves these problems using optimal probabilistic inference, known as Bayesian causal inference. However, how and where the underlying computations are carried out in the brain have remained unknown. By combining neuroimaging-based decoding techniques and computational modelling of behavioural data, a new study now sheds light on how multisensory causal inference maps onto specific brain areas. The results suggest that the complexity of neural computations increases along the visual hierarchy and link specific components of the causal inference process with specific visual and parietal regions.

Identifying relevant feature-action associations for grasping unmodelled objects

AbstractAction affordance learning based on visual sensory information is a crucial problem within the development of cognitive agents. In this paper, we present a method for learning action affordances based on basic visual features, which can vary in their granularity, order of combination and semantic content. The method is provided with a large and structured set of visual features, motivated by the visual hierarchy in primates and finds relevant feature action associations automatically. We apply our method in a simulated environment on three different object sets for the case of grasp affordance learning. For box objects,we achieve a 0.90 success probability, 0.80 for round objects and up to 0.75 for open objects, when presented with novel objects. In thiswork,we demonstrate, in particular, the effect of choosing appropriate feature representations. We demonstrate a significant performance improvement by increasing the complexity of the perceptual representation. By that, we present important insights in how the design of the feature space influences the actual learning problem.

Area summation of first- and second-order modulations of luminance.

To extend our understanding of the early visual hierarchy, we investigated the long-range integration of first- and second-order signals in spatial vision. In our first experiment we performed a conventional area summation experiment where we varied the diameter of (a) luminance-modulated (LM) noise and (b) contrast-modulated (CM) noise. Results from the LM condition replicated previous findings with sine-wave gratings in the absence of noise, consistent with long-range integration of signal contrast over space. For CM, the summation function was much shallower than for LM suggesting, at first glance, that the signal integration process was spatially less extensive than for LM. However, an alternative possibility was that the high spatial frequency noise carrier for the CM signal was attenuated by peripheral retina (or cortex), thereby impeding our ability to observe area summation of CM in the conventional way. To test this, we developed the "Swiss cheese" stimulus of Meese and Summers (2007) in which signal area can be varied without changing the stimulus diameter, providing some protection against inhomogeneity of the retinal field. Using this technique and a two-component subthreshold summation paradigm we found that (a) CM is spatially integrated over at least five stimulus cycles (possibly more), (b) spatial integration follows square-law signal transduction for both LM and CM and (c) the summing device integrates over spatially-interdigitated LM and CM signals when they are co-oriented, but not when cross-oriented. The spatial pooling mechanism that we have identified would be a good candidate component for a module involved in representing visual textures, including their spatial extent.

Reinforcing Visual Grouping Cues to Communicate Complex Informational Structure.

In his book Multimedia Learning [7], Richard Mayer asserts that viewers learn best from imagery that provides them with cues to help them organize new information into the correct knowledge structures. Designers have long been exploiting the Gestalt laws of visual grouping to deliver viewers those cues using visual hierarchy, often communicating structures much more complex than the simple organizations studied in psychological research. Unfortunately, designers are largely practical in their work, and have not paused to build a complex theory of structural communication. If we are to build a tool to help novices create effective and well structured visuals, we need a better understanding of how to create them. Our work takes a first step toward addressing this lack, studying how five of the many grouping cues (proximity, color similarity, common region, connectivity, and alignment) can be effectively combined to communicate structured text and imagery from real world examples. To measure the effectiveness of this structural communication, we applied a digital version of card sorting, a method widely used in anthropology and cognitive science to extract cognitive structures. We then used tree edit distance to measure the difference between perceived and communicated structures. Our most significant findings are: 1) with careful design, complex structure can be communicated clearly; 2) communicating complex structure is best done with multiple reinforcing grouping cues; 3) common region (use of containers such as boxes) is particularly effective at communicating structure; and 4) alignment is a weak structural communicator.

Learning Person-Specific Representations From Faces in the Wild

Humans are natural face recognition experts, far out-performing current automated face recognition algorithms, especially in naturalistic, “in the wild” settings. However, a striking feature of human face recognition is that we are dramatically better at recognizing highly familiar faces, presumably because we can leverage large amounts of past experience with the appearance of an individual to aid future recognition. Meanwhile, the analogous situation in automated face recognition, where a large number of training examples of an individual are available, has been largely underexplored, in spite of the increasing relevance of this setting in the age of social media. Inspired by these observations, we propose to explicitly learn enhanced face representations on a per-individual basis, and we present two methods enabling this approach. By learning and operating within person-specific representations, we are able to significantly outperform the previous state-of-the-art on PubFig83, a challenging benchmark for familiar face recognition in the wild, using a novel method for learning representations in deep visual hierarchies. We suggest that such person-specific representations aid recognition by introducing an intermediate form of regularization to the problem.

Attraction of Position Preference by Spatial Attention throughout Human Visual Cortex

Voluntary spatial attention concentrates neural resources at the attended location. Here, we examined the effects of spatial attention on spatial position selectivity in humans. We measured population receptive fields (pRFs) using high-field functional MRI (fMRI) (7T) while subjects performed an attention-demanding task at different locations. We show that spatial attention attracts pRF preferred positions across the entire visual field, not just at the attended location. This global change in pRF preferred positions systematically increases up the visual hierarchy. We model these pRF preferred position changes as an interaction between two components: an attention field and a pRF without the influence of attention. This computational model suggests that increasing effects of attention up the hierarchy result primarily from differences in pRF size and that the attention field is similar across the visual hierarchy. A similar attention field suggests that spatial attention transforms different neural response selectivities throughout the visual hierarchy in a similar manner.

The fractal nature of maps and mapping

A fractal can be simply understood as a set or pattern in which there are far more small things than large ones, for example, far more small geographic features than large ones on the earth surface, or far more large-scale maps than small-scale maps for a geographic region. This article attempts to argue and provide evidence for the fractal nature of maps and mapping. It is the underlying fractal structure of geographic features, either natural or man-made, that make reality mappable, large-scale maps generalizable, and cities imageable. The fractal nature is also what underlies the beauty of maps. After introducing some key fractal concepts such as recursion, self-similarity, scaling ratio, and scaling exponent, this article demonstrates that fractal thought is rooted in long-standing map-making practices such as series maps subdivision, visual hierarchy, and Töpfer’s radical law. Drawing on previous studies on head/tail breaks, mapping can be considered a head/tail breaks process; that is to divide things around an average, according to their geometric, topological and/or semantic properties, into the head (for those above the average) and the tail (for those below the average), and recursively continue the dividing process for the head for map generalization, statistical mapping, and cognitive mapping. Given the fractal nature of maps and mapping, cartography should be considered a perfect combination of science and art, and scaling must be formulated as a law of cartography or that of geography in general.

Limits to the usability of iconic memory

Human vision briefly retains a trace of a stimulus after it disappears. This trace—iconic memory—is often believed to be a surrogate for the original stimulus, a representational structure that can be used as if the original stimulus were still present. To investigate its nature, a flicker-search paradigm was developed that relied upon a full scan (rather than partial report) of its contents. Results show that for visual search it can indeed act as a surrogate, with little cost for alternating between visible and iconic representations. However, the duration over which it can be used depends on the type of task: some tasks can use iconic memory for at least 240 ms, others for only about 190 ms, while others for no more than about 120 ms. The existence of these different limits suggests that iconic memory may have multiple layers, each corresponding to a particular level of the visual hierarchy. In this view, the inability to use a layer of iconic memory may reflect an inability to maintain feedback connections to the corresponding representation.

Learning and comparison of visual feature representations

Recent developments on the encoding and decoding of visual stimuli have relied on different feature representations such as pixel-level, Gabor wavelet or semantic representations. In previous work, we showed that high-quality reconstructions of images can be obtained via the analytical inversion of regularized linear models operating on individual pixels. However, such simple models do not account for the complex nonlinear transformations of sensory input that take place in the visual hierarchy. I will argue that these nonlinear transformations can be estimated independent of brain data using statistical approaches. Decoding based on the resulting feature space is shown to yield better results than those obtained using a hand-designed feature space based on Gabor wavelets. I will discuss how alternative feature spaces that are either learned or hand-designed can be compared with one another, thereby providing insight into what visual information is represented where in the brain. Finally, I will present some recent encoding and decoding results obtained using ultra-high field MRI. Meeting abstract presented at VSS 2014

Spatial resolution in visual memory.

Representations in visual short-term memory are considered to contain relatively elaborated information on object structure. Conversely, representations in earlier stages of the visual hierarchy are thought to be dominated by a sensory-based, feed-forward buildup of information. In four experiments, we compared the spatial resolution of different object properties between two points in time along the processing hierarchy in visual short-term memory. Subjects were asked either to estimate the distance between objects or to estimate the size of one of the objects' features under two experimental conditions, of either a short or a long delay period between the presentation of the target stimulus and the probe. When different objects were referred to, similar spatial resolution was found for the two delay periods, suggesting that initial processing stages are sensitive to object-based properties. Conversely, superior resolution was found for the short, as compared with the long, delay when features were referred to. These findings suggest that initial representations in visual memory are hybrid in that they allow fine-grained resolution for object features alongside normal visual sensitivity to the segregation between objects. The findings are also discussed in reference to the distinction made in earlier studies between visual short-term memory and iconic memory.

Feature integration and object representations along the dorsal stream visual hierarchy

The visual system is split into two processing streams: a ventral stream that receives color and form information and a dorsal stream that receives motion information. Each stream processes that information hierarchically, with each stage building upon the previous. In the ventral stream this leads to the formation of object representations that ultimately allow for object recognition regardless of changes in the surrounding environment. In the dorsal stream, this hierarchical processing has classically been thought to lead to the computation of complex motion in three dimensions. However, there is evidence to suggest that there is integration of both dorsal and ventral stream information into motion computation processes, giving rise to intermediate object representations, which facilitate object selection and decision making mechanisms in the dorsal stream. First we review the hierarchical processing of motion along the dorsal stream and the building up of object representations along the ventral stream. Then we discuss recent work on the integration of ventral and dorsal stream features that lead to intermediate object representations in the dorsal stream. Finally we propose a framework describing how and at what stage different features are integrated into dorsal visual stream object representations. Determining the integration of features along the dorsal stream is necessary to understand not only how the dorsal stream builds up an object representation but also which computations are performed on object representations instead of local features.

On the Role of Suppression in Spatial Attention: Evidence from Negative BOLD in Human Subcortical and Cortical Structures

There is clear evidence that spatial attention increases neural responses to attended stimuli in extrastriate visual areas and, to a lesser degree, in earlier visual areas. Other evidence shows that neurons representing unattended locations can also be suppressed. However, the extent to which enhancement and suppression is observed, their stimulus dependence, and the stages of the visual system at which they are expressed remains poorly understood. Using fMRI we set out to characterize both the task and stimulus dependence of neural responses in the lateral geniculate nucleus (LGN), primary visual cortex (V1), and visual motion area (V5) in humans to determine where suppressive and facilitatory effects of spatial attention are expressed. Subjects viewed a lateralized drifting grating stimulus, presented at multiple stimulus contrasts, and performed one of three tasks designed to alter the spatial location of their attention. In retinotopic representations of the stimulus location, we observed increasing attention-dependent facilitation and decreasing dependence on stimulus contrast moving up the visual hierarchy from the LGN to V5. However, in the representations of unattended locations of the LGN and V1, we observed suppression, which was not significantly dependent on the attended stimulus contrast. These suppressive effects were also found in the pulvinar, which has been frequently associated with attention. We provide evidence, therefore, for a spatially selective suppressive mechanism that acts at a subcortical level.

The functional anatomy of schizophrenia: A dynamic causal modeling study of predictive coding

This paper tests the hypothesis that patients with schizophrenia have a deficit in selectively attending to predictable events. We used dynamic causal modeling (DCM) of electrophysiological responses – to predictable and unpredictable visual targets – to quantify the effective connectivity within and between cortical sources in the visual hierarchy in 25 schizophrenia patients and 25 age-matched controls. We found evidence for marked differences between normal subjects and schizophrenia patients in the strength of extrinsic backward connections from higher hierarchical levels to lower levels within the visual system. In addition, we show that not only do schizophrenia subjects have abnormal connectivity but also that they fail to adjust or optimize this connectivity when events can be predicted. Thus, the differential intrinsic recurrent connectivity observed during processing of predictable versus unpredictable targets was markedly attenuated in schizophrenia patients compared with controls, suggesting a failure to modulate the sensitivity of neurons responsible for passing sensory information of prediction errors up the visual cortical hierarchy. The findings support the proposed role of abnormal connectivity in the neuropathology and pathophysiology of schizophrenia.

Multistable network dynamics through lateral inhibition: an efficient mechanism for selective information routing

A remarkable feature of the brain is its ability to flexibly process information in a context-dependent manner. A hallmark example is selective visual attention. It allows to focus on relevant information, and to ignore distracting features of a visual scene, effectively routing information through the visual hierarchy [1]. A seminal hypothesis about how this routing is accomplished is 'Communication through Coherence' (CTC) [2]: A neuron with oscillating membrane potential can be driven to fire by a relatively weak input when it is close to the threshold, while if it is in a phase of low excitability, even stronger input might fail to elicit an action potential. Thus, shifting relative phases of oscillating sending and receiving populations could be utilized to gate information. Here, we investigate neural mechanisms of information routing and their effectiveness in a generic, two-layered cortical network model with lateral recurrent interactions. Building on the CTC hypothesis, we construct a biophysically plausible network model mimicking the fan-in structure of the visual processing stream. It comprises four populations A, B, C and D of recurrently coupled networks of conductance-based spiking neurons. A and B correspond to sending populations in a lower visual area and have non-overlapping receptive fields. They project to receiving populations C and D in a higher visual area with larger receptive fields. Upon activation, all populations engage in oscillatory activity. We show that introducing distance-dependent lateral inhibition brings the receiving populations in phase, and the sending populations in anti-phase, thus establishing a phase competition between the two sending populations: Only one population can be in a favorable in-phase relation with the receiving populations, since the other is automatically forced into anti-phase, thus hindering information transfer. This 'self-organization' of phase relations mediated by lateral inhibition distinguishes our approach from previous modeling efforts (e.g. [3-5]). When increasing the strength of the lateral inhibition between A and B, the system transitions from a regime of mixed representations to a bistable one, where one sending population is in a favorable in-phase relation for an extended period of time. In this regime, a brief current pulse to one sending population is sufficient to set a desired phase relation and thus 'switch' information routing in the network. This mechanism bears two interesting consequences: First, it allows neural circuits to control routing using energy-efficient, short bursts of neural activity instead of requiring sustained elevation of firing rates. Second, such pulses delivered by electrical stimulation could be employed to externally direct the focus of attention, a possible application in the field of brain prosthetics.

Fractal image perception provides novel insights into hierarchical cognition

Hierarchical structures play a central role in many aspects of human cognition, prominently including both language and music. In this study we addressed hierarchy in the visual domain, using a novel paradigm based on fractal images. Fractals are self-similar patterns generated by repeating the same simple rule at multiple hierarchical levels. Our hypothesis was that the brain uses different resources for processing hierarchies depending on whether it applies a “fractal” or a “non-fractal” cognitive strategy. We analyzed the neural circuits activated by these complex hierarchical patterns in an event-related fMRI study of 40 healthy subjects.Brain activation was compared across three different tasks: a similarity task, and two hierarchical tasks in which subjects were asked to recognize the repetition of a rule operating transformations either within an existing hierarchical level, or generating new hierarchical levels. Similar hierarchical images were generated by both rules and target images were identical.We found that when processing visual hierarchies, engagement in both hierarchical tasks activated the visual dorsal stream (occipito-parietal cortex, intraparietal sulcus and dorsolateral prefrontal cortex). In addition, the level-generating task specifically activated circuits related to the integration of spatial and categorical information, and with the integration of items in contexts (posterior cingulate cortex, retrosplenial cortex, and medial, ventral and anterior regions of temporal cortex). These findings provide interesting new clues about the cognitive mechanisms involved in the generation of new hierarchical levels as required for fractals.