Visual Localization Research Articles

Sighted humans show enhanced sensitivity for horizontal but not vertical audio localization

Introduction Experiments on humans and animals indicate that auditory localization develops through the association of sets of monaural and binaural cues and locations in space: this calibration process in guided by vision (Knudsen, 1989).The auditory system lacks a straightforward correspondence between external spatial locations and sensory receptive fields (Ahveninen et al ., 2014), so the development of auditory localization is more complex than visual localization. Aim The main aim is to assess if the development of audio spatial localization in sighted humans:Differs for horizontal and vertical dimensionsDepends on: a)Type of sound: static or motion b)Type of task: verbal or motor Results and DiscussionWe tested audio spatial localization for horizontal and vertical dimensions in sighted humans taking into consideration both the type of sound (static or motion) and the type of response requested by the task (verbal or motor). In audio spatial localization, sighted humans show enhanced sensitivity for the horizontal dimension, irrespective of the type of sound and the type of task. The static sound condition tend to be easier than the sound motion condition and motor responses more accurate than verbal responses. Developmental improvement between 10-15 years of age. Future direction: since vision has a predominant role in the maturation of space cognition in the brain, we aim to assess the development of audio spatial localization for horizontal and vertical dimensions in blind humans.

Speed of visual attention and localization of motion onset

Speed of visual attention and localization of motion onset

Vision-based localization for on-orbit servicing of a partially cooperative satellite

This paper proposes ground-in-the-loop, model-based visual localization system based on transmitted images to ground, to aid rendezvous and docking maneuvers between a servicer and a target satellite. In particular, we assume to deal with a partially cooperative target, i.e. passive and without fiducial markers, but supposed at least to keep a controlled attitude, up to small fluctuations, so that the approach mainly involves translational motion. For the purpose of localization, video cameras provide an effective and relatively inexpensive solution, working at a wide range of distances with an increasing accuracy and robustness during the approach. However, illumination conditions in space are especially challenging, due to the direct sunlight exposure and to the glossy surface of a satellite, that creates strong reflections and saturations and therefore a high level of background clutter and missing detections. We employ a monocular camera for mid-range tracking (20–5m) and stereo camera at close-range (5–0.5m), with the respective detection and tracking methods, both using intensity edges and robustly dealing with the above issues. Our tracking system has been extensively verified at the facility of the European Proximity Operations Simulator (EPOS) of DLR, which is a very realistic ground simulation able to reproduce sunlight conditions through a high power floodlight source, satellite surface properties using multilayer insulation foils, as well as orbital motion trajectories with ground-truth data, by means of two 6 DOF industrial robots. Results from this large dataset show the effectiveness and robustness of our method against the above difficulties.

Manifold based map representation for mobile robot using Euclidean data difference dimension reduction

In this paper, a novel method for mobile robot map representation is presented. We introduce a manifold based map which is constructed by mapping the color histogram of omni-directional camera images into a low dimensional space while preserving local and global geometry. The geometry is preserved using orthogonal and independent feature vectors consisting of the Euclidean data difference and the nearest and farthest neighbor data vectors. The introduced dimension reduction method has shown superior performance compared to LLE and PCA in benchmark data sets providing a hard dimension reduction of 768 into 3. This mapping matrix has been used for real time robot localization and mapping in real world experiments. In large environments, adjacent places share the same set of objects and features therefore dimension reduction may end up in a non-smooth manifold. To apply the method in large environments, first a K-Means method is used to cluster the environment and then each obtained cluster is processed separately. Successful experiments on 7 different data sets show effectiveness of this method for robotic applications such as mapping, visual place classification and kidnaped-robot localization.

Learning place-dependant features for long-term vision-based localisation

This paper presents an alternative approach to the problem of outdoor, persistent visual localisation against a known map. Instead of blindly applying a feature detector/descriptor combination over all images of all places, we leverage prior experiences of a place to learn place-dependent feature detectors (i.e., features that are unique to each place in our map and used for localisation). Furthermore, as these features do not represent low-level structure, like edges or corners, but are in fact mid-level patches representing distinctive visual elements (e.g., windows, buildings, or silhouettes), we are able to localise across extreme appearance changes. Note that there is no requirement that the features posses semantic meaning, only that they are optimal for the task of localisation. This work is an extension on previous work (McManus et al. in Proceedings of robotics science and systems, 2014b) in the following ways: (i) we have included a landmark refinement and outlier rejection step during the learning phase, (ii) we have implemented an asynchronous pipeline design, (iii) we have tested on data collected in an urban environment, and (iv) we have implemented a purely monocular system. Using over 100 km worth of data for training, we present localisation results from Begbroke Science Park and central Oxford.

Relative position determination between Chang’E-3 lander and rover using in-beam phase referencing

Chang’E-3 is China’s first lunar surface exploration mission, as a part of the second phase of China’s Lunar Exploration Program (CLEP). It landed successfully in the northwest of Mare Imbrium on December 14, 2013, incorporating two probes, a lander and a rover named Yutu. The execution of the cooperative science tasks required an accurate determination of the relative position between the lander and the rover. In-beam phase referencing experiments have been conducted with the Chinese VLBI Network (CVN) to accurately image the rover in reference to the stationary lander in order to obtain their offsets in Right Ascension and Declination, which were then transformed into the lunar local coordinate system to acquire their relative position on the lunar surface. The observations and the data processing procedures are described in this paper. Images of the rover in six epochs between December 15 and 21, 2013 were made, resulting in accurate positions of the rover that are consistent with the visual localization approach. Through measurement of the locations of the rover’s omnidirectional antenna with respect to the lander’s directional antenna before their separation, and the recognition of the rover’s turning actions during its exploratory moves, we demonstrate that the accuracy of relative position determined in this way could reach the meter level and the resolution is better than 1 meter.

DATA ASSOCIATION OF RF-VSLAM FOR OCEAN OBSERVATION USING BLIMP

This paper describes a selection of features for potential landmarks for ocean observation system using radio frequency visual simultaneous localization and mapping (RF-VSLAM) framework. Due to dynamic changes of the ocean surface caused by the ocean gyres, the features selection is difficult. Therefore, the tendency for vehicles to drift is high. As a solution, we introduced the beacons as an anchor node as an aid to correct the navigation and improve data association. We investigated the data association stage of the RF-VSLAM system which improved the state estimator for the aerial vehicle. The goal is to produce a correct association to the landmarks, since wrong data association will produce inaccurate maps. The points features were extracted from a monocular camera using SIFT as detector and descriptor. The experimental data of the dynamic changes of water surface has been evaluated. The result showed that the data association method was able to produce correct and accurate landmarks selection.

Research on visual segmentation and localization of multi-robots under different lighting conditions

In the vision-based segmentation and localization of mobile robots, a change in light will greatly influence the accuracy of segmentation and localization. To obtain accurate positions of multi-robots under different lighting conditions, a threshold segmentation combining histogram equalization and cluster analysis algorithm based on the designed color label plate is presented. First, the local regions of interest of mobile robots are obtained and the histograms of the regions of interest are equalized, which normalizes the video image under different lighting conditions. Then, the accurate segmentation threshold values for the color label plates of multi-robots are calculated based on cluster analysis in the YUV color space of regions of interest. After that, the segmentation and localization of multi-robots based on the segmentation threshold value are realized. Compared with other segmentation algorithms, the proposed threshold segmentation combining histogram equalization and cluster analysis algorithm can not only guarantee fast segmentation but also is characterized by better segementation robustness and higher localization accuracy. Finally, based on the proposed visual segmentation and localization algorithm, a trajectory tracking experiment of a mobile robot is carried out, which further supports the real-time performance of the proposed threshold segmentation combining histogram equalization and cluster analysis algorithm in the practical application.

Surgical aspects of recurrent oropharyngeal cancer treatment.

Oropharynx is the complex anatomical and topographical area, the boundaries of which begin behind oral cavity, and despite the visual localization, it not always easily to provide its appropriate inspection. This is particularly evident in cases of mandible dysfunction such as a limitation of mouth opening, certain anatomical features, pronounced vomiting reflex and hypertrophy of the tonsils. These factors, as well as low cancer alertness of general practitioners, are largely the reason for the neglect of oropharynx tumors.

Imaging of myocardial inflammation with somatostatin receptor based PET/CT — A comparison to cardiac MRI

BackgroundAcute myocarditis as well as post-ischemic myocardial inflammation are generally associated with a profound activation of the immune system. Current established imaging techniques such as cardiac MRI reliably demonstrate signs of acute myocardial injury. However, detection of mediating cells such as macrophages is currently limited to experimental settings. We aimed to investigate the feasibility of somatostatin receptor (SSTR) based positron emission tomography/computed tomography (PET/CT) for detecting inflammatory lesions in patients after acute myocardial infarction or acute peri-/myocarditis. Methods12 patients with active peri-/myocarditis (n=6) or sub-acute myocardial infarction (n=6) underwent SSTR-PET/CT and cardiac MRI within 3–10days after onset of symptoms. The AHA 17-segment model of the left myocardium was used for visual localization of inflamed myocardium for both imaging modalities. Tracer uptake of infarcted/inflamed myocardium was assessed as mean and maximum standardized uptake value (SUVmean and SUVmax) and compared with both remote myocardium and left ventricular (LV) cavity. ResultsSSTR-PET/CT revealed areas with increased cardiac tracer uptake in all patients. In the 17-segment model, PET/CT yielded 55 and MRI 47 positive segments. Overall, concordance of the 2 modalities was 85.3% (174/204 segments analyzed). In 9.3% (19/204), more positive segments were identified by PET/CT, whereas in 5.4% (11/204), MRI detected more positive segments. ConclusionsThe imaging patterns of SSTR-directed radiotracers and MRI in vivo show a close spatial relation of macrophage concentration and structural changes. This suggests the possibility of a new potential biomarker that predicts cardiac remodeling and, hence, progression towards heart failure. Prospective trials are warranted.

Virtual auditory display validation using transaural techniques

Validation of headphone-based virtual auditory display technology is inherently difficult, given the potential for display hardware to interfere with normal sound field listening conditions due to occlusion of the pinna. This difficulty is eliminated if loudspeaker-based transaural techniques are instead used for the virtual auditory display. Transaural techniques also offer the advantage of being able to use the display loudspeakers, or other additional loudspeakers, as individual reference locations in order to validate the display under natural listening conditions in the same sound field. Such validation can also include explicit comparison to visual localization of the sound sources. In this way, the accuracy and precision of virtual source localization can be directly compared to real source localization, and optionally to visual localization. Results of a validation experiment of this type using non-individualized head-related transfer functions are reported and compared to analogous data from a headphone-based virtual auditory display. [Work supported by NEI.]

Auditory and visual localization accuracy in young children and adults

ObjectiveThis study aimed to measure and compare sound and light source localization ability in young children and adults who have normal hearing and normal/corrected vision in order to determine the extent to which age, type of stimuli, and stimulus order affects sound localization accuracy. MethodsTwo experiments were conducted. The first involved a group of adults only. The second involved a group of 30 children aged 3 to 5 years. Testing occurred in a sound-treated booth containing a semi-circular array of 15 loudspeakers set at 10° intervals from −70° to 70° azimuth. Each loudspeaker had a tiny light bulb and a small picture fastened underneath. Seven of the loudspeakers were used to randomly test sound and light source identification. The sound stimulus was the word “baseball”. The light stimulus was a flashing of a light bulb triggered by the digital signal of the word “baseball”. Each participant was asked to face 0° azimuth, and identify the location of the test stimulus upon presentation. Adults used a computer mouse to click on an icon; children responded by verbally naming or walking toward the picture underneath the corresponding loudspeaker or light. A mixed experimental design using repeated measures was used to determine the effect of age and stimulus type on localization accuracy in children and adults. A mixed experimental design was used to compare the effect of stimulus order (light first/last) and varying or fixed intensity sound on localization accuracy in children and adults. ResultsLocalization accuracy was significantly better for light stimuli than sound stimuli for children and adults. Children, compared to adults, showed significantly greater localization errors for audition. Three-year-old children had significantly greater sound localization errors compared to 4- and 5-year olds. Adults performed better on the sound localization task when the light localization task occurred first. ConclusionsYoung children can understand and attend to localization tasks, but show poorer localization accuracy than adults in sound localization. This may be a reflection of differences in sensory modality development and/or central processes in young children, compared to adults.

Guest editorial: selected papers from ICIMCS 2012

These papers cover key issues in Internet multimedia computing and services, including large-scale visual search, mobile media processing, Internet image tagging, video transmission, as well as fundamental technologies for intelligent multimedia computing, such as image segmentation, logo recognition, and object tracking. The first article “Visual word expansion and BSIFT verification for large-scale image search” by Zhou et al. presents a visual word expansion method improving the recall of retrieval on large-scale image collections and a binary SIFT verification method boosting the retrieval precision. The former represents a query image as the aggregation of visual words of local features in the query and the neighboring words of these query words. The latter verifies feature matching between images based on the binarized SIFT feature. The work is experimented on two public image data sets. The second article “Accurate sensing of scene geo-context via mobile visual localization” by Liu et al. proposes a new mobile image geo-tagging approach toward accurate location tagging for images captured in urban areas. For a landmark image, the approach infers comprehensive and accurate geo-context of the image, including camera location, view direction, and scene location, by exploiting large-scale image retrieval and 3D reconstruction techniques. Experimental results on the San Francisco street view data set, which consists of approximately 150 k panoramic images in San Francisco at about 4-m intervals, have demonstrated the effectiveness of the approach. The third article entitled “Tag ranking based on salient region graph propagation” by Tang et al. presents a new tag ranking solution to rank user-contributed tags associated with Internet images. The solution exploits salient regions in images and ranks tags by integrating visual clues from entire images and salient regions of images. It constructs two sparse graphs over images and salient sub-images, International Conference on Internet Multimedia Computing and Services (ICIMCS) is an annual conference sponsored by ACM SIGMM China Chapter. The conference is especially interested in the latest technologies and applications that deal with the Web-scale processing and management of heterogeneous data from the Internet for multimedia computing and service. ICIMCS 2012 was held in Wuhan, China. The conference attracted around 90 participants, including researchers from academia and industries across ten countries/regions, for sharing their recent works on the topics ranging from visual feature representation and analysis to visual recognition and classification, and from social and mobile media analysis to multimedia services. This special issue comprises the extended versions of seven papers, including three best papers and four papers from the regular and special sessions of ICIMCS 2012.

Spatial Representation to Support Visual Impaired People

The rapid development of Information and Communication Technology (ICT) enhances the government services to its citizen. As a consequence affects the exchange of information between citizen and government. However, some groups can experience some difficulties accessing the government services due to disabilities, such as blind people as well as material and geographical constraints. This paper aims to introduce a novel approach for place recognition to help the blind people to navigate inside a government building based on the correlation degree for the covariance feature vectors. However, this approach faces several challenges. One of the fundamental challenges inaccurate indoor place recognition for the visual impaired people is the presence of similar scene images in different places in the environmental space of the mobile robot system, such as a computer or office table in many rooms. This problem causes bewilderment and confusion among different places. To overcome this, the local features of these image scenes should be represented in more discriminatory and robustly way. However, to perform this, the spatial relation of the local features should be considered. The findings revealed that this approach has a stable manner due to its reliability in the place recognition for the robot localization. Finally, the proposed Covariance approach gives an intelligent way for visual place people localization through the correlation of Covariance feature vectors for the scene images.

Design of an enhanced visual odometry by building and matching compressive panoramic landmarks online

Efficient and precise localization is a prerequisite for the intelligent navigation of mobile robots. Traditional visual localization systems, such as visual odometry (VO) and simultaneous localization and mapping (SLAM), suffer from two shortcomings: a drift problem caused by accumulated localization error, and erroneous motion estimation due to illumination variation and moving objects. In this paper, we propose an enhanced VO by introducing a panoramic camera into the traditional stereo-only VO system. Benefiting from the 360° field of view, the panoramic camera is responsible for three tasks: (1) detecting road junctions and building a landmark library online; (2) correcting the robot’s position when the landmarks are revisited with any orientation; (3) working as a panoramic compass when the stereo VO cannot provide reliable positioning results. To use the large-sized panoramic images efficiently, the concept of compressed sensing is introduced into the solution and an adaptive compressive feature is presented. Combined with our previous two-stage local binocular bundle adjustment (TLBBA) stereo VO, the new system can obtain reliable positioning results in quasi-real time. Experimental results of challenging long-range tests show that our enhanced VO is much more accurate and robust than the traditional VO, thanks to the compressive panoramic landmarks built online.

Trajectory-based visual localization in underwater surveying missions.

We present a new vision-based localization system applied to an autonomous underwater vehicle (AUV) with limited sensing and computation capabilities. The traditional EKF-SLAM approaches are usually expensive in terms of execution time; the approach presented in this paper strengthens this method by adopting a trajectory-based schema that reduces the computational requirements. The pose of the vehicle is estimated using an extended Kalman filter (EKF), which predicts the vehicle motion by means of a visual odometer and corrects these predictions using the data associations (loop closures) between the current frame and the previous ones. One of the most important steps in this procedure is the image registration method, as it reinforces the data association and, thus, makes it possible to close loops reliably. Since the use of standard EKFs entail linearization errors that can distort the vehicle pose estimations, the approach has also been tested using an iterated Kalman filter (IEKF). Experiments have been conducted using a real underwater vehicle in controlled scenarios and in shallow sea waters, showing an excellent performance with very small errors, both in the vehicle pose and in the overall trajectory estimates.

Error aware multiple vertical planes based visual localization for mobile robots in urban environments

A novel error-aware visual localization method is proposed that utilizes vertical planes, such as vertical building facades in urban areas as landmarks. Vertical planes, reconstructed from coplanar vertical lines, are robust high-level features if compared with point features or line features. Firstly, the error models of vertical lines and vertical planes are built, where maximum likelihood estimation (MLE) is employed to estimate all vertical planes from coplanar vertical lines. Then, the closed-form representation of camera location error variance is derived. Finally, the minimum variance camera pose estimation is formulated into a convex optimization problem, and the weight for each vertical plane is obtained by solving this well-studied problem. Experiments are carried out and the results show that the proposed localization method has an accuracy of about 2 meters, at par with commercial GPS operating in open environments.

Cross-modal and multisensory training may distinctively shape restored senses.

Cross-modal and multisensory training may distinctively shape restored senses.

Design of the Independent Rescue System Based on the Passive Sound Source Localization and Machine Vision

The purpose of this paper is to design a family or battlefield independent rescue system which is based on passive sound source localization and machine vision, The system consists of five modules: the sound acquisition and processing module based on FPGA, image acquisition and processing module based on TMS320DM6437, video capture and communication control module based on TMS320DM355, motion control module based on MSP430F149, PC server and smart handheld mobile client. It can track target person who need help automatically by recognizing his voice and image, to provide emergency medicines, communication tools, and transmit real-time video information to a server on PC and a mobile client on Andriod platform by wireless network, and be controlled remotely.

3D Visual SLAM Based on Multiple Iterative Closest Point

With the development of novel RGB-D visual sensors, data association has been a basic problem in 3D Visual Simultaneous Localization and Mapping (VSLAM). To solve the problem, a VSLAM algorithm based on Multiple Iterative Closest Point (MICP) is presented. By using both RGB and depth information obtained from RGB-D camera, 3D models of indoor environment can be reconstructed, which provide extensive knowledge for mobile robots to accomplish tasks such as VSLAM and Human-Robot Interaction. Due to the limited views of RGB-D camera, additional information about the camera pose is needed. In this paper, the motion of the RGB-D camera is estimated by a motion capture system after a calibration process. Based on the estimated pose, the MICP algorithm is used to improve the alignment. A Kinect mobile robot which is running Robot Operating System and the motion capture system has been used for experiments. Experiment results show that not only the proposed VSLAM algorithm achieved good accuracy and reliability, but also the 3D map can be generated in real time.