Object Recognition In Clutter Research Articles

Persistent Homology Meets Object Unity: Object Recognition in Clutter

Persistent Homology Meets Object Unity: Object Recognition in Clutter

Critical resolution: A superior measure of crowding

Visual object recognition is essential for adaptive interactions with the environment. It is fundamentally limited by crowding, a breakdown of object recognition in clutter. The spatial extent over which crowding occurs is proportional to the eccentricity of the target object, but nevertheless varies substantially depending on various stimulus factors (e.g. viewing time, contrast). However, a lack of studies jointly manipulating such factors precludes predictions of crowding in more heterogeneous scenes, such as the majority of real life situations.To establish how such co-occurring variations affect crowding, we manipulated combinations of 1) flanker contrast and backward masking, 2) flanker contrast and presentation duration, and 3) flanker preview and pop-out while measuring participants’ ability to correctly report the orientation of a target stimulus. In all three experiments, combining two manipulations consistently modulated the spatial extent of crowding in a way that could not be predicted from an additive combination. However, a simple transformation of the measurement scale completely abolished these interactions and all effects became additive. Precise quantitative predictions of the magnitude of crowding when combining multiple manipulations are thus possible when it is expressed in terms of what we label the ‘critical resolution’. Critical resolution is proportional to the inverse of the smallest flanker free area surrounding the target object necessary for its unimpaired identification. It offers a more parsimonious description of crowding than the traditionally used critical spacing and may thus constitute a measure of fundamental importance for understanding object recognition.

Crowding for faces is determined by visual (not holistic) similarity: Evidence from judgements of eye position

Crowding (the disruption of object recognition in clutter) presents the fundamental limitation on peripheral vision. For simple objects, crowding is strong when target/flanker elements are similar and weak when they differ – a selectivity for target-flanker similarity. In contrast, the identification of upright holistically-processed face stimuli is more strongly impaired by upright than inverted flankers, whereas inverted face-targets are impaired by both – a pattern attributed to an additional stage of crowding selective for “holistic similarity” between faces. We propose instead that crowding is selective for target-flanker similarity in all stimuli, but that this selectivity is obscured by task difficulty with inverted face-targets. Using judgements of horizontal eye-position that are minimally affected by inversion, we find that crowding is strong when target-flanker orientations match and weak when they differ for both upright and inverted face-targets. By increasing task difficulty, we show that this selectivity for target-flanker similarity is obscured even for upright face-targets. We further demonstrate that this selectivity follows differences in the spatial order of facial features, rather than “holistic similarity” per se. There is consequently no need to invoke a distinct stage of holistic crowding for faces – crowding is selective for target-flanker similarity, even with complex stimuli such as faces.

Variations in crowding, saccadic precision, and spatial localization reveal the shared topology of spatial vision

Visual sensitivity varies across the visual field in several characteristic ways. For example, sensitivity declines sharply in peripheral (vs. foveal) vision and is typically worse in the upper (vs. lower) visual field. These variations can affect processes ranging from acuity and crowding (the deleterious effect of clutter on object recognition) to the precision of saccadic eye movements. Here we examine whether these variations can be attributed to a common source within the visual system. We first compared the size of crowding zones with the precision of saccades using an oriented clock target and two adjacent flanker elements. We report that both saccade precision and crowded-target reports vary idiosyncratically across the visual field with a strong correlation across tasks for all participants. Nevertheless, both group-level and trial-by-trial analyses reveal dissociations that exclude a common representation for the two processes. We therefore compared crowding with two measures of spatial localization: Landolt-C gap resolution and three-dot bisection. Here we observe similar idiosyncratic variations with strong interparticipant correlations across tasks despite considerably finer precision. Hierarchical regression analyses further show that variations in spatial precision account for much of the variation in crowding, including the correlation between crowding and saccades. Altogether, we demonstrate that crowding, spatial localization, and saccadic precision show clear dissociations, indicative of independent spatial representations, whilst nonetheless sharing idiosyncratic variations in spatial topology. We propose that these topological idiosyncrasies are established early in the visual system and inherited throughout later stages to affect a range of higher-level representations.

There Is a "U" in Clutter: Evidence for Robust Sparse Codes Underlying Clutter Tolerance in Human Vision.

The neural mechanisms underlying object recognition in cluttered scenes (i.e., containing more than one object) remain poorly understood. Studies have suggested that neural responses to multiple objects correspond to an average of the responses to the constituent objects. Yet, it is unclear how the identities of an unknown number of objects could be disentangled from a confounded average response. Here, we use a popular computational biological vision model to show that averaging-like responses can result from responses of clutter-tolerant neurons to suboptimal stimuli. The model also provides a novel prediction, that human detection ability should show a U-shaped dependency on target-clutter similarity, which is confirmed experimentally, supporting a simple, unifying account of how the brain performs object recognition in clutter.

A Global Hypothesis Verification Framework for 3D Object Recognition in Clutter.

Pipelines to recognize 3D objects despite clutter and occlusions usually end up with a final verification stage whereby recognition hypotheses are validated or dismissed based on how well they explain sensor measurements. Unlike previous work, we propose a Global Hypothesis Verification (GHV) approach which regards all hypotheses jointly so as to account for mutual interactions. GHV provides a principled framework to tackle the complexity of our visual world by leveraging on a plurality of recognition paradigms and cues. Accordingly, we present a 3D object recognition pipeline deploying both global and local 3D features as well as shape and color. Thereby, and facilitated by the robustness of the verification process, diverse object hypotheses can be gathered and weak hypotheses need not be suppressed too early to trade sensitivity for specificity. Experiments demonstrate the effectiveness of our proposal, which significantly improves over the state-of-art and attains ideal performance (no false negatives, no false positives) on three out of the six most relevant and challenging benchmark datasets.

Uncovering foveal crowding?

Visual crowding, as context modulation, reduce the ability to recognize objects in clutter, sets a fundamental limit on visual perception and object recognition. It's considered that crowding does not exist in the fovea and extensive efforts explored crowding in the periphery revealed various models that consider several aspects of spatial processing. Studies showed that spatial and temporal crowding are correlated, suggesting a tradeoff between spatial and temporal processing of crowding. We hypothesized that limiting stimulus availability should decrease object recognition in clutter. Here we show, for the first time, that robust contour interactions exist in the fovea for much larger target-flanker spacing than reported previously: participants overcome crowded conditions for long presentations times but exhibit contour interaction effects for short presentation times. Thus, by enabling enough processing time in the fovea, contour interactions can be overcome, enabling object recognition. Our results suggest that contemporary models of context modulation should include both time and spatial processing.

Object recognition in clutter: cortical responses depend on the type of learning

Theoretical studies suggest that the visual system uses prior knowledge of visual objects to recognize them in visual clutter, and posit that the strategies for recognizing objects in clutter may differ depending on whether or not the object was learned in clutter to begin with. We tested this hypothesis using functional magnetic resonance imaging (fMRI) of human subjects. We trained subjects to recognize naturalistic, yet novel objects in strong or weak clutter. We then tested subjects' recognition performance for both sets of objects in strong clutter. We found many brain regions that were differentially responsive to objects during object recognition depending on whether they were learned in strong or weak clutter. In particular, the responses of the left fusiform gyrus (FG) reliably reflected, on a trial-to-trial basis, subjects' object recognition performance for objects learned in the presence of strong clutter. These results indicate that the visual system does not use a single, general-purpose mechanism to cope with clutter. Instead, there are two distinct spatial patterns of activation whose responses are attributable not to the visual context in which the objects were seen, but to the context in which the objects were learned.

Robust Object Recognition with Cortex-Like Mechanisms

We introduce a new general framework for the recognition of complex visual scenes, which is motivated by biology: We describe a hierarchical system that closely follows the organization of visual cortex and builds an increasingly complex and invariant feature representation by alternating between a template matching and a maximum pooling operation. We demonstrate the strength of the approach on a range of recognition tasks: From invariant single object recognition in clutter to multiclass categorization problems and complex scene understanding tasks that rely on the recognition of both shape-based as well as texture-based objects. Given the biological constraints that the system had to satisfy, the approach performs surprisingly well: It has the capability of learning from only a few training examples and competes with state-of-the-art systems. We also discuss the existence of a universal, redundant dictionary of features that could handle the recognition of most object categories. In addition to its relevance for computer vision, the success of this approach suggests a plausibility proof for a class of feedforward models of object recognition in cortex.

Intracellular measurements of spatial integration and the MAX operation in complex cells of the cat primary visual cortex.

We have examined the spatial integration properties of complex cells to determine whether some of their responses can be described by a maximum operation (MAX)-like computation, as suggested by Riesenhuber and Poggio's model of object recognition. Membrane potential was recorded from anesthetized cats while optimally oriented bars were presented, either alone or in pairs, in different parts of the cells' receptive field. In most cells, the membrane potential response to two bars presented simultaneously could not be predicted by the sum of the responses to individual bars. In many cells, however, the responses closely approximated a MAX-like model. That is, the response of the cell to two bars was similar to the larger of the two individual responses ("soft-MAX"). The degree of nonlinear summation varied from cell to cell and varied within single cells from one stimulus configuration to another but on average fit most closely to the MAX model. The firing response of the cells was also well predicted by the MAX-like model. The MAX-like behavior was independent of the distance between the bars (orthogonal to the preferred orientation), independent of the relative amplitude of the responses, and slightly less pronounced at low levels of contrast. This MAX-like behavior of a subset of complex cells may play an important role in invariant object recognition in clutter.