Object Search Task Research Articles

Path planning for robot search task using belief criteria decision-making

PurposeThis study aims to realize efficient, fast and safe robot search task, the belief criteria decision-making approach is proposed to solve the object search task with an uncertain location.Design/methodology/approachThe study formulates the robot search task as a partially observable Markov decision process, uses the semantic information to evaluate the belief state and designs the belief criteria decision-making approach. A cost function considering a trade-off among belief state, path length and movement effort is modelled to select the next best location in path planning.FindingsThe semantic information is successfully modelled and propagated, which can represent the belief of finding object. The belief criteria decision-making (BCDM) approach is evaluated in both Gazebo simulation platform and physical experiments. Compared to greedy, uniform and random methods, the performance index of path length and execution time is superior by BCDM approach.Originality/valueThe prior knowledge of robot working environment, especially semantic information, can be used for path planning to achieve efficient task execution in path length and execution time. The modelling and updating of environment information can lead a promising research topic to realize a more intelligent decision-making method for object search task.

A neural code for time and space in the human brain

Time and space are primary dimensions of human experience. Separate lines of investigation have identified neural correlates of time and space, yet little is known about how these representations converge during self-guided experience. Here, 10 subjects with intracranially implanted microelectrodes play a timed, virtual navigation game featuring object search and retrieval tasks separated by fixed delays. Time cells and place cells activate in parallel during timed navigation intervals, whereas a separate time cell sequence spans inter-task delays. The prevalence, firing rates, and behavioral coding strengths of time cells and place cells are indistinguishable-yet time cells selectively remap between search and retrieval tasks, while place cell responses remain stable. Thus, the brain can represent time and space as overlapping but dissociable dimensions. Time cells and place cells may constitute a biological basis for the cognitive map of spatiotemporal context onto which memories are written.

Semantic Grounding for Long-Term Autonomy of Mobile Robots Toward Dynamic Object Search in Home Environments

In this article, we presents an effective semantic grounding scheme to long-term mobile robots for dynamic object search in open/dynamic home environments. The challenge of this strategy lies in dealing with situations where the robot suffers the home environment with an unknown layout of the object and the target object is moved without the robot’s knowledge. Therefore, a general probabilistic model and a semantic model are established by investigating typical spatial location relations between object and room type, in order to guide the robot to prioritize the search effort for spaces that are most promising to find the target object. Furthermore, a semantic grounding solution is proposed to address the mutual independence between objects, endowing the robot with inference capability for dynamic object search. Also, our proposal can be shared with other robots that have not explored the environment in order to perform object search tasks. The presented scheme is fully evaluated by extensive experiments in the real-world environment. The results demonstrate the validity of our approach by comparing it with the other three schemes in terms of search time and trajectory length, and show that the proposed method allows the mobile robot to efficiently and robustly find the dynamic object while achieving the human-like performance.

GO-Finder: A Registration-free Wearable System for Assisting Users in Finding Lost Hand-held Objects

People spend an enormous amount of time and effort looking for lost objects. To help remind people of the location of lost objects, various computational systems that provide information on their locations have been developed. However, prior systems for assisting people in finding objects require users to register the target objects in advance. This requirement imposes a cumbersome burden on the users, and the system cannot help remind them of unexpectedly lost objects. We propose GO-Finder (“Generic Object Finder”), a registration-free wearable camera-based system for assisting people in finding an arbitrary number of objects based on two key features: automatic discovery of hand-held objects and image-based candidate selection. Given a video taken from a wearable camera, GO-Finder automatically detects and groups hand-held objects to form a visual timeline of the objects. Users can retrieve the last appearance of the object by browsing the timeline through a smartphone app. We conducted user studies to investigate how users benefit from using GO-Finder. In the first study, we asked participants to perform an object retrieval task and confirmed improved accuracy and reduced mental load in the object search task by providing clear visual cues on object locations. In the second study, the system’s usability on a longer and more realistic scenario was verified, accompanied by an additional feature of context-based candidate filtering. Participant feedback suggested the usefulness of GO-Finder also in realistic scenarios where more than one hundred objects appear.

Searching for meaning: Local scene semantics guide attention during natural visual search in scenes.

Models of visual search in scenes include image salience as a source of attentional guidance. However, because scene meaning is correlated with image salience, it could be that the salience predictor in these models is driven by meaning. To test this proposal, we generated meaning maps that represented the spatial distribution of semantic informativeness in scenes, and salience maps which represented the spatial distribution of conspicuous image features and tested their influence on fixation densities from two object search tasks in real-world scenes. The results showed that meaning accounted for significantly greater variance in fixation densities than image salience, both overall and in early attention across both studies. Here, meaning explained 58% and 63% of the theoretical ceiling of variance in attention across both studies, respectively. Furthermore, both studies demonstrated that fast initial saccades were not more likely to be directed to higher salience regions than slower initial saccades, and initial saccades of all latencies were directed to regions containing higher meaning than salience. Together, these results demonstrated that even though meaning was task-neutral, the visual system still selected meaningful over salient scene regions for attention during search.

Probabilistic 3D Reconstruction Using Two Sonar Devices.

Three-dimensional reconstruction is a crucial technique for mapping and object-search tasks, but it is challenging in sonar imaging because of the nature of acoustics. In underwater sensing, many advanced studies have introduced approaches that have included feature-based methods and multiple imaging at different locations. However, most existing methods are prone to environmental conditions, and they are not adequate for continuous data acquisition on moving autonomous underwater vehicles (AUVs). This paper proposes a sensor fusion method for 3D reconstruction using acoustic sonar data with two sonar devices that provide complementary features. The forward-looking multibeam sonar (FLS) is an imaging sonar capable of short-range scanning with a high horizontal resolution, and the profiling sonar (PS) is capable of middle-range scanning with high reliability in vertical information. Using both sonars, which have different data acquisition planes and times, we propose a probabilistic sensor fusion method. First, we extract the region of interest from the background and develop a sonar measurement model. Thereafter, we utilize the likelihood field generated by the PS and estimate the elevation ambiguity using importance sampling. We also present the evaluation of our method in a ray-tracing-based sonar simulation environment and the generation of the pointclouds. The experimental results indicate that the proposed method can provide a better accuracy than that of the conventional method. Because of the improved accuracy of the generated pointclouds, this method can be expanded for pointcloud-based mapping and classification methods.

Object segmentation in cluttered environment based on gaze tracing and gaze blinking

People with disabilities, such as patients with motor paralysis conditions, lack independence and cannot move most parts of their bodies except for their eyes. Supportive robot technology is highly beneficial in supporting these types of patients. We propose a gaze-informed location-based (or gaze-based) object segmentation, which is a core module of successful patient-robot interaction in an object-search task (i.e., a situation when a robot has to search for and deliver a target object to the patient). We have introduced the concepts of gaze tracing (GT) and gaze blinking (GB), which are integrated into our proposed object segmentation technique, to yield the benefit of an accurate visual segmentation of unknown objects in a complex scene. Gaze tracing information can be used as a clue as to where the target object is located in a scene. Then, gaze blinking can be used to confirm the position of the target object. The effectiveness of our proposed method has been demonstrated using a humanoid robot in experiments with different types of highly cluttered scenes. Based on the limited gaze guidance from the user, we achieved an 85% F-score of unknown object segmentation in an unknown environment.

A Gaussian mixture-hidden Markov model of human visual behavior

Vision is an important way for human beings to interact with the outside world and obtain information. In order to research human visual behavior under different conditions, this paper uses a Gaussian mixture-hidden Markov model (GMM-HMM) to model the scanpath, and proposes a new model optimization method, time-shifting segmentation (TSS). The TSS method can highlight the characteristics of the time dimension in the scanpath, improve the pattern recognition results, and enhance the stability of the model. In this paper, a linear discriminant analysis (LDA) method is used for multi-dimensional feature pattern recognition to evaluates the rationality and the accuracy of the proposed model. Four sets of comparative trials were carried out for the model evaluation. The first group applied the GMM-HMM to model the scanpath, and the average accuracy of the classification could reach 0.507, which is greater than the opportunity probability of three classification (0.333). The second set of trial applied TSS method, and the mean accuracy of classification was raised to 0.610. The third group combined GMM-HMM with TSS method, and the mean accuracy of classification reached 0.602, which was more stable than the second model. Finally, comparing the model analysis results with the saccade amplitude (SA) characteristics analysis results, the modeling analysis method is much better than the basic information analysis method. Via analyzing the characteristics of three types of tasks, the results show that the free viewing task have higher specificity value and a higher sensitivity to the cued object search task. In summary, the application of GMM-HMM model has a good performance in scanpath pattern recognition, and the introduction of TSS method can enhance the difference of scanpath characteristics. Especially for the recognition of the scanpath of search-type tasks, the model has better advantages. And it also provides a new solution for a single state eye movement sequence.

A COMPARATIVE RESEARCH ON G-HMM AND TSS TECHNOLOGIES FOR EYE MOVEMENT TRACKING ANALYSIS

Eye movement analysis provides a new way for disease screening, quantification and assessment. In order to track and analyze eye movement scanpaths under different conditions, this paper proposed the Gaussian mixture-Hidden Markov Model (G-HMM) modeling the eye movement scanpath during saccade, combing with the Time-Shifting Segmentation (TSS) method for model optimization, and also the Linear Discriminant Analysis (LDA) method was utilized to perform the recognition and evaluation tasks based on the multi-dimensional features. In the experiments, 800 real scene images of eye-movement sequences datasets were used, and the experimental results show that the G-HMM method has high specificity for free searching tasks and high sensitivity for prompt object search tasks, while TSS can strengthen the difference of eye movement characteristics, which is conducive to eye movement pattern recognition, especially for search tasks.

Efficient Robotic Object Search Via HIEM: Hierarchical Policy Learning With Intrinsic-Extrinsic Modeling

Despite the significant success at enabling robots with autonomous behaviors makes deep reinforcement learning a promising approach for robotic object search task, the deep reinforcement learning approach severely suffers from the nature sparse reward setting of the task. To tackle this challenge, we present a novel policy learning paradigm for the object search task, based on hierarchical and interpretable modeling with an intrinsic-extrinsic reward setting. More specifically, we explore the environment efficiently through a proxy low-level policy which is driven by the intrinsic rewarding sub-goals. We further learn our hierarchical policy from the efficient exploration experience where we optimize both of our high-level and low-level policies towards the extrinsic rewarding goal to perform the object search task well. Experiments conducted on the House3D environment validate and show that the robot, trained with our model, can perform the object search task in a more optimal and interpretable way.

Object search in neovascular age‐related macular degeneration: the crowding effect

BackgroundVisual search, an activity that relies on central vision, is frequent in daily life. This study investigates the effect of spacing between items in an object search task in participants with central vision loss.MethodsPatients with neovascular age‐related macular degeneration (AMD), age‐matched controls, and young controls were included. The stimuli were displays of four, six and nine objects randomly presented in a ‘crowded’ (spacing 1.5°) or ‘uncrowded’ (spacing 6°) condition. For each of 96 trials, participants were asked to search for a predefined target that remained on the screen until the response was recorded. Accuracy, search time, and eye movements (number of fixations and scan path ratio) were recorded.ResultsCompared to older controls, accuracy decreased by 31 per cent and search time increased by 61 per cent in AMD participants. Ageing also affected performance with a lower accuracy by 13.5 per cent and longer search times by 46 per cent in older compared to younger controls. Increasing the spacing between elements increased accuracy by 21 per cent in AMD participants but it had no effect in older and younger controls. Performance was not related to visual acuity or to duration of neovascular AMD, but search time was correlated to the lesion size in the ‘crowded’ condition.ConclusionsObject search is ubiquitous in daily life activities. When visual acuity is irrevocably reduced, increasing the spacing between elements can reliably improve object search performance in patients.

Do Helpful Mothers Help? Effects of Maternal Scaffolding and Infant Engagement on Cognitive Performance.

Infants are highly social and much early learning takes place in a social context during interactions with caregivers. Previous research shows that social scaffolding – responsive parenting and joint attention – can confer benefits for infants’ long-term development and learning. However, little previous research has examined whether dynamic (moment-to-moment) adaptations in adults’ social scaffolding are able to produce immediate effects on infants’ performance. Here we ask whether infants’ success on an object search task is more strongly influenced by maternal behavior, including dynamic changes in response behavior, or by fluctuations in infants’ own engagement levels. Thirty-five mother-infant dyads (infants aged 10.8 months, on average) participated in an object search task that was delivered in a naturalistic manner by the child’s mother. Measures of maternal responsiveness (teaching duration; sensitivity) and infant engagement (engagement score; visual attention) were assessed. Mothers varied their task delivery trial by trial, but neither measure of maternal responsiveness significantly predicted infants’ success in performing the search task. Rather, infants’ own level of engagement was the sole significant predictor of accuracy. These results indicate that while parental scaffolding is offered spontaneously (and is undoubtedly crucial for development), in this context children’s endogenous engagement proved to be a more powerful determinant of task success. Future work should explore this interplay between parental and child-internal factors in other learning and social contexts.

The integration of visual and target signals in V4 and IT during visual object search

Searching for a specific visual object requires our brain to compare the items in view with a remembered representation of the sought target to determine whether a target match is present. This comparison is thought to be implemented, in part, via the combination of top-down modulations reflecting target identity with feed-forward visual representations. However, it remains unclear whether top-down signals are integrated at a single locus within the ventral visual pathway (e.g., V4) or at multiple stages [e.g., both V4 and inferotemporal cortex (IT)]. To investigate, we recorded neural responses in V4 and IT as rhesus monkeys performed a task that required them to identify when a target object appeared across variation in position, size, and background context. We found nonvisual, task-specific signals in both V4 and IT. To evaluate whether V4 was the only locus for the integration of top-down signals, we evaluated several feed-forward accounts of processing from V4 to IT, including a model in which IT preferentially sampled from the best V4 units and a model that allowed for nonlinear IT computation. IT task-specific modulation was not accounted for by any of these feed-forward descriptions, suggesting that during object search, top-down signals are integrated directly within IT.NEW & NOTEWORTHY To find specific objects, the brain must integrate top-down, target-specific signals with visual information about objects in view. However, the exact route of this integration in the ventral visual pathway is unclear. In the first study to systematically compare V4 and inferotemporal cortex (IT) during an invariant object search task, we demonstrate that top-down signals found in IT cannot be described as being inherited from V4 but rather must be integrated directly within IT itself.

Inferotemporal cortex multiplexes behaviorally-relevant target match signals and visual representations in a manner that minimizes their interference.

Finding a sought visual target object requires combining visual information about a scene with a remembered representation of the target to create a “target match” signal that indicates when a target is in view. Target match signals have been reported to exist within high-level visual brain areas including inferotemporal cortex (IT), where they are mixed with representations of image and object identity. However, these signals are not well understood, particularly in the context of the real-world challenge that the objects we search for typically appear at different positions, sizes, and within different background contexts. To investigate these signals, we recorded neural responses in IT as two rhesus monkeys performed a delayed-match-to-sample object search task in which target objects could appear at a variety of identity-preserving transformations. Consistent with the existence of behaviorally-relevant target match signals in IT, we found that IT contained a linearly separable target match representation that reflected behavioral confusions on trials in which the monkeys made errors. Additionally, target match signals were highly distributed across the IT population, and while a small fraction of units reflected target match signals as target match suppression, most units reflected target match signals as target match enhancement. Finally, we found that the potentially detrimental impact of target match signals on visual representations was mitigated by target match modulation that was approximately (albeit imperfectly) multiplicative. Together, these results support the existence of a robust, behaviorally-relevant target match representation in IT that is configured to minimally interfere with IT visual representations.

Familiarity with images affects how dogs ( Canis familiaris) process life-size video projections of humans.

A central problem of behavioural studies providing artificial visual stimuli for non-human animals is to determine how subjects perceive and process these stimuli. Especially in the case of videos, it is important to ascertain that animals perceive the actual content of the images and are not just reacting to the motion cues in the presentation. In this study, we set out to investigate how dogs process life-sized videos. We aimed to find out whether dogs perceive the actual content of video images or whether they only react to the videos as a set of dynamic visual elements. For this purpose, dogs were presented with an object search task where a life-sized projected human was hiding a target object. The videos were either normally oriented or displayed upside down, and we analysed dogs' reactions towards the projector screen after the video presentations, and their performance in the search task. Results indicated that in the case of the normally oriented videos, dogs spontaneously perceived the actual content of the images. However, the 'Inverted' videos were first processed as a set of unrelated visual elements, and only after some exposure to these videos did the dogs show signs of perceiving the unusual configuration of the depicted scene. Our most important conclusion was that dogs process the same type of artificial visual stimuli in different ways, depending on the familiarity of the depicted scene, and that the processing mode can change with exposure to unfamiliar stimuli.

Of "what" and "where" in a natural search task: Active object handling supports object location memory beyond the object's identity.

Looking for as well as actively manipulating objects that are relevant to ongoing behavioral goals are intricate parts of natural behavior. It is, however, not clear to what degree these two forms of interaction with our visual environment differ with regard to their memory representations. In a real-world paradigm, we investigated if physically engaging with objects as part of a search task influences identity and position memory differently for task-relevant versus irrelevant objects. Participants equipped with a mobile eye tracker either searched for cued objects without object interaction (Find condition) or actively collected the objects they found (Handle condition). In the following free-recall task, identity memory was assessed, demonstrating superior memory for relevant compared to irrelevant objects, but no difference between the Handle and Find conditions. Subsequently, location memory was inferred via times to first fixation in a final object search task. Active object manipulation and task-relevance interacted in that location memory for relevant objects was superior to irrelevant ones only in the Handle condition. Including previous object recall performance as a covariate in the linear mixed-model analysis of times to first fixation allowed us to explore the interaction between remembered/forgotten object identities and the execution of location memory. Identity memory performance predicted location memory in the Find but not the Handle condition, suggesting that active object handling leads to strong spatial representations independent of object identity memory. We argue that object handling facilitates the prioritization of relevant location information, but this might come at the cost of deprioritizing irrelevant information.

I saw where you have been—The topography of human demonstration affects dogs’ search patterns and perseverative errors

Performance in object search tasks is not only influenced by the subjects’ object permanence ability. For example, ostensive cues of the human manipulating the target markedly affect dogs’ choices. However, the interference between the target’s location and the spatial cues of the human hiding the object is still unknown.In a five-location visible displacement task, the experimental groups differed in the hiding route of the experimenter. In the ‘direct’ condition he moved straight towards the actual location, hid the object and returned to the dog. In the ‘indirect’ conditions, he additionally walked behind each screen before returning. The two ‘indirect’ conditions differed from each other in that the human either visited the previously baited locations before (proactive interference) or after (retroactive interference) hiding the object.In the ‘indirect’ groups, dogs’ performance was significantly lower than in the ‘direct’ group, demonstrating that for dogs, in an ostensive context, spatial cues of the hider are as important as the observed location of the target. Based on their incorrect choices, dogs were most attracted to the previously baited locations that the human visited after hiding the object in the actual trial. This underlines the importance of retroactive interference in multiple choice tasks.

There's Waldo! A Normalization Model of Visual Search Predicts Single-Trial Human Fixations in an Object Search Task

When searching for an object in a scene, how does the brain decide where to look next? Visual search theories suggest the existence of a global "priority map" that integrates bottom-up visual information with top-down, target-specific signals. We propose a mechanistic model of visual search that is consistent with recent neurophysiological evidence, can localize targets in cluttered images, and predicts single-trial behavior in a search task. This model posits that a high-level retinotopic area selective for shape features receives global, target-specific modulation and implements local normalization through divisive inhibition. The normalization step is critical to prevent highly salient bottom-up features from monopolizing attention. The resulting activity pattern constitues a priority map that tracks the correlation between local input and target features. The maximum of this priority map is selected as the locus of attention. The visual input is then spatially enhanced around the selected location, allowing object-selective visual areas to determine whether the target is present at this location. This model can localize objects both in array images and when objects are pasted in natural scenes. The model can also predict single-trial human fixations, including those in error and target-absent trials, in a search task involving complex objects.

Eye movements as probes of lexico-semantic processing in a patient with primary progressive aphasia

Eye movement trajectories during a verbally cued object search task were used as probes of lexico-semantic associations in an anomic patient with primary progressive aphasia. Visual search was normal on trials where the target object could be named but became lengthy and inefficient on trials where the object failed to be named. The abnormality was most profound if the noun denoting the object could not be recognized. Even trials where the name of the target object was recognized but not retrieved triggered abnormal eye movements, demonstrating that retrieval failures can have underlying associative components despite intact comprehension of the corresponding noun.

Maternal docosahexaenoic acid intake levels during pregnancy and infant performance on a novel object search task at 22 months.

This study investigated maternal prenatal docosahexaenoic acid (DHA) intake and infant cognitive development at 22 months. Estimates for second- and third-trimester maternal DHA intake levels were obtained using a comprehensive Food Frequency Questionnaire. Infants (n = 67) were assessed at 22 months on a novel object search task. Mothers' DHA intake levels were divided into high or low groups, with analyses revealing a significant positive effect of third-trimester DHA on object search task performance. The third trimester appears to be a critical time for ensuring adequate maternal DHA levels to facilitate optimum cognitive development in late infancy.