Visual Object Search Research Articles

Simple actions modulate context-dependent visual size perception at late processing stages

A simple button press towards a prime stimulus enhances subsequent visual search for objects that match the prime. The present study investigated whether this action effect is a general phenomenon across different task domains, and the underlying neural mechanisms. The action effect was measured in an unspeeded size-matching task, with the presentation of the central target and the surrounding inducers of the Ebbinghaus illusion together to one eye or separately to each eye, and when repetitive TMS was applied over right primary motor cortex (M1). The results showed that a prior key-press significantly reduced the illusion effect compared to passive viewing. Notably, the action effect persisted with dichoptic presentation of the Ebbinghaus configuration, but disappeared with the right M1 disruption. These results suggest that action guides visual perception to influence human behavior, which mainly affects the late visual processing stage and probably relies on feedback projections from the motor cortex.

Just-in-time: Gaze guidance in natural behavior.

Natural eye movements have primarily been studied for over-learned activities such as tea-making, sandwich-making, and hand-washing, which have a fixed sequence of associated actions. These studies demonstrate a sequential activation of low-level cognitive schemas facilitating task completion. However, whether these action schemas are activated in the same pattern when a task is novel and a sequence of actions must be planned in the moment is unclear. Here, we recorded gaze and body movements in a naturalistic task to study action-oriented gaze behavior. In a virtual environment, subjects moved objects on a life-size shelf to achieve a given order. To compel cognitive planning, we added complexity to the sorting tasks. Fixations aligned with the action onset showed gaze as tightly coupled with the action sequence, and task complexity moderately affected the proportion of fixations on the task-relevant regions. Our analysis revealed that gaze fixations were allocated to action-relevant targets just in time. Planning behavior predominantly corresponded to a greater visual search for task-relevant objects before the action onset. The results support the idea that natural behavior relies on the frugal use of working memory, and humans refrain from encoding objects in the environment to plan long-term actions. Instead, they prefer just-in-time planning by searching for action-relevant items at the moment, directing their body and hand to it, monitoring the action until it is terminated, and moving on to the following action.

Gravity's impact on visual search asymmetries: Is visual gravitational motion a distinct visual feature or a familiar dynamic event?

A wealth of converging research lines has led support to the notion that specialized neural processes output a priori information about the expected effects of gravity to fine-tune motor and perceptual responses to dynamic events. Arguably, these putative internal models of gravity might modulate the efficiency in visual search for objects conforming or not to gravitationally coherent dynamics. In the present work, we explored this possibility with a visual search task involving arrays of two to eight objects moving periodically back and forth. The target could be an accelerating/decelerating ball (as if bouncing on earth's surface-1g) with distractors moving at a constant speed (0g) or the reverse. Moreover, the direction of the gravitational pull, as implied by the 1g motion patterns, could be aligned or misaligned with Earth's gravity. Overall, searches for 1g targets were more efficient than 0g targets except, notably, when stimuli displays were congruent with Earth's gravitational pull, in which case the visual search asymmetry is significantly reduced. Outcomes are interpreted as reflecting the joint and mutually cancelling contribution of low-level detection of acceleration patterns and higher level detection of unexpected violations of gravitational motion. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Hybrid visual and memory search for scenes and objects with variable viewpoints.

In hybrid search, observers search visual arrays for any of several target types held in memory. The key finding in hybrid search is that response times (RTs) increase as a linear function of the number of items in a display (visual set size), but RTs increase linearly with the log of the memory set size. Previous experiments have shown this result for specific targets (find exactly this picture of a boot on a blank background) and for broad categorical targets (find any animal). Arguably, these are rather unnatural situations. In the real world, objects are parts of scenes and are seen from multiple viewpoints. The present experiments generalize the hybrid search findings to scenes (Experiment 1) and multiple viewpoints (Experiment 2). The results replicated the basic pattern of hybrid search results: RTs increased logarithmically with the number of scene photos/categories held in memory. Experiment 3 controls the experiment for which viewpoints were seen in an initial learning phase. The results replicate the findings of Experiment 2. Experiment 4 compares hybrid search for specific viewpoints, variable viewpoints, and categorical targets. Search difficulty increases from specific viewpoints to variable viewpoints and then to categorical targets. The results of the four experiments show the generality of logarithmic search through memory in hybrid search.

Labeling Out-of-View Objects in Immersive Analytics to Support Situated Visual Searching.

Augmented Reality (AR) embeds digital information into objects of the physical world. Data can be shown in-situ, thereby enabling real-time visual comparisons and object search in real-life user tasks, such as comparing products and looking up scores in a sports game. While there have been studies on designing AR interfaces for situated information retrieval, there has only been limited research on AR object labeling for visual search tasks in the spatial environment. In this article, we identify and categorize different design aspects in AR label design and report on a formal user study on labels for out-of-view objects to support visual search tasks in AR. We design three visualization techniques for out-of-view object labeling in AR, which respectively encode the relative physical position (height-encoded), the rotational direction (angle-encoded), and the label values (value-encoded) of the objects. We further implement two traditional in-view object labeling techniques, where labels are placed either next to the respective objects (situated) or at the edge of the AR FoV (boundary). We evaluate these five different label conditions in three visual search tasks for static objects. Our study shows that out-of-view object labels are beneficial when searching for objects outside the FoV, spatial orientation, and when comparing multiple spatially sparse objects. Angle-encoded labels with directional cues of the surrounding objects have the overall best performance with the highest user satisfaction. We discuss the implications of our findings for future immersive AR interface design.

Do chimpanzees see a face on Mars? A search for face pareidolia in chimpanzees.

We sometimes perceive meaningful patterns or images in random arrangements of colors and shapes. This phenomenon is called pareidolia and has recently been studied intensively, especially face pareidolia. In contrast, there are few comparative-cognitive studies on face pareidolia with nonhuman primates. This study explored behavioral evidence for face pareidolia in chimpanzees using visual search and matching tasks. Faces are processed in a configural manner, and their perception and recognition are hampered by inversion and misalignment of top and bottom parts. We investigated whether the same effect occurs in a visual search for face-like objects. The results showed an effect of misalignment. On the other hand, consistent results were not obtained with the photographs of fruits. When only the top or bottom half of the face-like object was presented, chimpanzees showed better performance for the top-half condition, suggesting the importance of the eye area in face pareidolia. In the positive-control experiments, chimpanzees received the same experiment using human faces and human participants with face-like objects and fruits. As a result, chimpanzees showed an inefficient search for inverted and misaligned faces and humans for manipulated face-like objects. Finally, to examine the role of face awareness, we tested matching a human face to a face-like object in chimpanzees but obtained no substantial evidence that they saw the face-like object as a "face." Based on these results, we discussed the extents and limits of face pareidolia in chimpanzees.

The time course of categorical and perceptual similarity effects in visual search.

During visual search for objects (e.g., an apple), the surrounding distractor objects may share perceptual (tennis ball), categorical (banana), or both (peach) properties with the target. Previous studies showed that the perceptual similarity between target and distractor objects influences visual search. Here, we tested whether categorical target-distractor similarity also influences visual search, and how this influence depends on perceptual similarity. By orthogonally manipulating categorical and perceptual target-distractor similarity, we could investigate how and when the two similarities interactively affect search performance and neural correlates of spatial attention (N2pc) using electroencephalography (EEG). Behavioral results showed that categorical target-distractor similarity interacted with perceptual target-distractor similarity, such that the effect of categorical similarity was strongest when target and distractor objects were perceptually similar. EEG results showed that perceptual similarity influenced the early part of the N2pc (200-250 ms after stimulus onset), while categorical similarity influenced the later part (250-300 ms). Mirroring the behavioral results, categorical similarity interacted with perceptual similarity during this later time window, with categorical effects only observed for perceptually similar target-distractor pairs. Together, these results provide evidence for hierarchical processing in visual search: categorical properties influence spatial attention only when perceptual properties are insufficient to guide attention to the target. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Higher susceptibility to central crowding in glaucoma

ABSTRACT Clinical relevance Crowding limits many daily life activities, such as reading and the visual search for objects in cluttered environments. Excessive sensitivity to crowding, especially in central vision, may amplify the difficulties of patients with ocular pathologies. It is thus important to investigate what limits visual activities and how to improve it. Background Numerous studies have reported reduced contrast sensitivity in central vision in patients with glaucoma. However, deficits have also been observed for letter recognition at high contrast, suggesting that contrast alone cannot completely account for impaired central perception. Method Seventeen patients and fifteen age-matched controls were randomly presented with letters in central or parafoveal vision at 5° eccentricity for 200 ms. They were asked to decide whether the central T was upright or inverted. The T was either presented in isolation (uncrowded) or flanked by two Hs (crowded) at various spacings. Contrast was manipulated: 60% and 5%. Results Compared to controls, patients exhibited a significant effect of crowding in central vision, with higher accuracy for the isolated T than for HTH only at low contrast. In parafoveal vision, an effect of crowding was also observed only in patients. The spacing to escape crowding varied as a function of contrast. Larger spacing was required at low contrast than at high contrast. Susceptibility to crowding was related to central visual field defect for central presentations and to contrast sensitivity for parafoveal presentations, only at low contrast. Controls were at ceiling level both for central and parafoveal presentations. Conclusion Crowding limits visual perception, impeding reading and object recognition in cluttered environments. Visual field defects and lower contrast sensitivity in glaucoma can increase susceptibility to central and parafoveal crowding, the deleterious effect of which can be improved by manipulating contrast and spacing between elements.

The impact of simulated hemianopia on visual search for faces, words, and cars.

Tests of visual search can index the effects of perceptual load and compare the processing efficiency for different object types, particularly when one examines the set-size effect, the increase in search time for each additional stimulus in an array. Previous studies have shown that the set-size effect is increased by manoeuvres that impede object processing, and in patients with object processing impairments. In this study, we examine how the low-level visual impairment of hemianopia affects visual search for complex objects, using a virtual paradigm. Forty-two healthy subjects performed visual search for faces, words, or cars with full-viewing as well as gaze-contingent simulations of complete left or right hemianopia. Simulated hemianopia lowered accuracy and discriminative power and increased response times and set-size effects, similarly for faces, words and cars. A comparison of set-size effects between target absent and target present trials did not show a difference between full-view and simulated hemianopic conditions, and a model of decision-making suggested that simulated hemianopia reduced the rate of accumulation of perceptual data, but did not change decision thresholds. We conclude that simulated hemianopia reduces the efficiency of visual search for complex objects, and that such impairment should be considered when interpreting results from high-level object processing deficits.

Task-Dependent Warping of Semantic Representations during Search for Visual Action Categories.

Object and action perception in cluttered dynamic natural scenes relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. It has been suggested that during visual search for objects, distributed semantic representation of hundreds of object categories is warped to expand the representation of targets. Yet, little is known about whether and where in the brain visual search for action categories modulates semantic representations. To address this fundamental question, we studied brain activity recorded from five subjects (one female) via functional magnetic resonance imaging while they viewed natural movies and searched for either communication or locomotion actions. We find that attention directed to action categories elicits tuning shifts that warp semantic representations broadly across neocortex and that these shifts interact with intrinsic selectivity of cortical voxels for target actions. These results suggest that attention serves to facilitate task performance during social interactions by dynamically shifting semantic selectivity toward target actions and that tuning shifts are a general feature of conceptual representations in the brain.SIGNIFICANCE STATEMENT The ability to swiftly perceive the actions and intentions of others is a crucial skill for humans that relies on efficient allocation of limited brain resources to prioritize the attended targets over distractors. However, little is known about the nature of high-level semantic representations during natural visual search for action categories. Here, we provide the first evidence showing that attention significantly warps semantic representations by inducing tuning shifts in single cortical voxels, broadly spread across occipitotemporal, parietal, prefrontal, and cingulate cortices. This dynamic attentional mechanism can facilitate action perception by efficiently allocating neural resources to accentuate the representation of task-relevant action categories.

Meaning and expected surfaces combine to guide attention during visual search in scenes.

How do spatial constraints and meaningful scene regions interact to control overt attention during visual search for objects in real-world scenes? To answer this question, we combined novel surface maps of the likely locations of target objects with maps of the spatial distribution of scene semantic content. The surface maps captured likely target surfaces as continuous probabilities. Meaning was represented by meaning maps highlighting the distribution of semantic content in local scene regions. Attention was indexed by eye movements during the search for target objects that varied in the likelihood they would appear on specific surfaces. The interaction between surface maps and meaning maps was analyzed to test whether fixations were directed to meaningful scene regions on target-related surfaces. Overall, meaningful scene regions were more likely to be fixated if they appeared on target-related surfaces than if they appeared on target-unrelated surfaces. These findings suggest that the visual system prioritizes meaningful scene regions on target-related surfaces during visual search in scenes.

Bilingual Non-Selective Activation Modulates Attention During Visual Object Search

Bilingual Non-Selective Activation Modulates Attention During Visual Object Search

3D sketching for 3D object retrieval

Sketching provides the most natural way to provide a visual search query for visual object search. However, how to draw 3D sketches in a three-dimensional space and how to use a hand-drawn 3D sketch to search similar 3D models are not only interesting and novel, but also challenging research topics. In this paper, we try to answer them by initiating a novel study on 3D sketching and build a 3D sketching system which allows users to freely draw 3D sketches in the air and demonstrate its promising potentials in related applications such as collecting 3D sketch data and conducting 3D sketch-based 3D model retrieval. By utilizing the 3D sketching system, we collect a 3D sketch dataset, build a 3D sketch-based 3D model retrieval benchmark, and organize a Eurographics Shape Retrieval Contest (SHREC) track on 3D sketch-based shape retrieval based on the benchmark. We investigate 3D sketch and model matching problems and propose a novel 3D sketch-based model retrieval algorithm CNN-SBR based on Convolutional Neural Networks (CNNs) and achieve the best performance in the SHREC track. We wish that the 3D sketching system, the 3D sketch-based model retrieval benchmark, and the proposed 3D sketch-based model retrieval algorithm CNN-SBR will further promote sketch-based shape retrieval and its applications. We have made all of these publicly available on the project homepage: http://orca.st.usm.edu/~bli/SBR16/project.html .

Guest Editorial Introduction to the Special Section on Representation Learning for Visual Content Understanding

Representation learning methods allow a system to automatically learn robust and discriminative features from raw data for given goals, which play an important role in various visual content understanding applications, such as visual object segmentation, detection, tracking, recognition, and search. The performance of visual content understanding tasks is heavily dependent on the choice of data representation (or features) on which they are applied. Conventional feature representation methods usually employ transformations of data that make it easier to extract useful information, such as scale-invariant feature transform (SIFT), local binary patterns (LBP), and histogram of oriented gradients (HOG). In recent years, deep learning techniques have been widely applied to learn data-driven representations with supervised annotations and achieved great success in different visual content understanding tasks. Representative methods include the ResNet method for image classification, the DeepFace method for face recognition, and the feature pyramid networks (FPNs) method for object detection. Despite recent progresses on deep representation learning with a great amount of annotated data, how to effectively learn visual representation with limited data annotations still requires many efforts. This special section focuses on data-effective representation learning methods for visual content understanding.

Visual Search for Complex Objects in Healthy Subjects and Individuals with Developmental Prosopagnosia (1894)

Visual Search for Complex Objects in Healthy Subjects and Individuals with Developmental Prosopagnosia (1894)

Visual Object Search by Learning Spatial Context

We present a visual navigation approach that uses context information to navigate an agent to find and reach a target object. To learn context from the objects present in the scene, we transform visual information into an intermediate representation called context grid which essentially represents how much the object at the location is semantically similar to the target object. As this representation can encode the target object and other objects together, it allows us to navigate an agent in a human-inspired way: the agent will go to the likely place by seeing surrounding context objects in the beginning when the target is not visible and, once the target object comes into sight, it will reach the target quickly. Since context grid does not directly contain visual or semantic feature values that change according to introductions of new objects, such as new instances of the same object with different appearance or an object from a slightly different class, our navigation model generalizes well to unseen scenes/objects. Experimental results show that our approach outperforms previous approaches in navigating in unseen scenes, especially for broad scenes. We also evaluated human performances in the target-driven navigation task and compared with machine learning based navigation approaches including this work.

Activation of manipulation and function knowledge during visual search for objects.

This study aimed at comparing the time course of the activation of function and manipulation knowledge during object identification. The influence of visual similarity and context information was also assessed. In 3 eye-tracking experiments, conducted with the Visual-World-Paradigm, participants heard the name of an object and had to identify it among four pictures. The target object (e.g., shopping cart) could be presented along with objects related by (a) function (e.g., basket), (b) manipulation (e.g., lawnmower), (c) context (e.g., cash register), (d) visual similarity (e.g., toaster), and (e) completely unrelated objects. Growth curve analyses were used to assess competition effects among semantically (a, b, and c), visually related (d), and unrelated competitors (e). Results showed that manipulation- and function-related, but not context-related objects received more fixations than the unrelated ones, with a temporal advantage for the manipulation-related objects (Experiment 1). However, the visually similar objects faded the semantic competition effects, especially for function-related objects (Experiment 2). Finally, no temporal differences appeared when manipulation- and function-related objects were shown within the same visual array (Experiment 3). These results support the idea that both function and manipulation are relevant features of object semantic representations, but in the absence of other semantic competitors the activation of manipulation features appears prioritized during object identification. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Characteristic Sounds Facilitate Object Search in Real-Life Scenes.

Real-world events do not only provide temporally and spatially correlated information across the senses, but also semantic correspondences about object identity. Prior research has shown that object sounds can enhance detection, identification, and search performance of semantically consistent visual targets. However, these effects are always demonstrated in simple and stereotyped displays that lack ecological validity. In order to address identity-based cross-modal relationships in real-world scenarios, we designed a visual search task using complex, dynamic scenes. Participants searched for objects in video clips recorded from real-life scenes. Auditory cues, embedded in the background sounds, could be target-consistent, distracter-consistent, neutral, or just absent. We found that, in these naturalistic scenes, characteristic sounds improve visual search for task-relevant objects but fail to increase the salience of irrelevant distracters. Our findings generalize previous results on object-based cross-modal interactions with simple stimuli and shed light upon how audio–visual semantically congruent relationships play out in real-life contexts.

The integration of visual and target signals in V4 and IT during visual object search

Searching for a specific visual object requires our brain to compare the items in view with a remembered representation of the sought target to determine whether a target match is present. This comparison is thought to be implemented, in part, via the combination of top-down modulations reflecting target identity with feed-forward visual representations. However, it remains unclear whether top-down signals are integrated at a single locus within the ventral visual pathway (e.g., V4) or at multiple stages [e.g., both V4 and inferotemporal cortex (IT)]. To investigate, we recorded neural responses in V4 and IT as rhesus monkeys performed a task that required them to identify when a target object appeared across variation in position, size, and background context. We found nonvisual, task-specific signals in both V4 and IT. To evaluate whether V4 was the only locus for the integration of top-down signals, we evaluated several feed-forward accounts of processing from V4 to IT, including a model in which IT preferentially sampled from the best V4 units and a model that allowed for nonlinear IT computation. IT task-specific modulation was not accounted for by any of these feed-forward descriptions, suggesting that during object search, top-down signals are integrated directly within IT.NEW & NOTEWORTHY To find specific objects, the brain must integrate top-down, target-specific signals with visual information about objects in view. However, the exact route of this integration in the ventral visual pathway is unclear. In the first study to systematically compare V4 and inferotemporal cortex (IT) during an invariant object search task, we demonstrate that top-down signals found in IT cannot be described as being inherited from V4 but rather must be integrated directly within IT itself.

Incidental memory following rapid object processing: The role of attention allocation strategies.

When observers search for multiple (rather than singular) targets, they are slower and less accurate, yet have better incidental memory for nontarget items encountered during the task (Hout & Goldinger, 2010). One explanation for this may be that observers titrate their attention allocation based on the expected difficulty suggested by search cues. Difficult search cues may implicitly encourage observers to narrow their attention, simultaneously enhancing distractor encoding and hindering peripheral processing. Across three experiments, we manipulated the difficulty of search cues preceding passive visual search for real-world objects, using a Rapid Serial Visual Presentation (RSVP) task to equate item exposure durations. In all experiments, incidental memory was enhanced for distractors encountered while participants monitored for difficult targets. Moreover, in key trials, peripheral shapes appeared at varying eccentricities off center, allowing us to infer the spread and precision of participants' attentional windows. Peripheral item detection and identification decreased when search cues were difficult, even when the peripheral items appeared before targets. These results were not an artifact of sustained vigilance in miss trials, but instead reflect top-down modulation of attention allocation based on task demands. Implications for individual differences are discussed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).