Stereo Data Research Articles

Disparity Image Plane Segmentation Based Obstacle Map Construction for Mobile Robot

As a part of GPOD (ground plane obstacle detection) technology of autonomous mobile robot, a method for creating front obstacle grid map is presented which utilizes disparity image of binocular camera to obtain information. This method combines both the stereo vision and the traditional 2D image processing technique. The former implements autonomous plane segmentation based on the histogram of disparity image, and regards each plane as scene slice of 3D scenario to extract obstacle’s 3D information. The later extracts obstacle information by calculating obstacle profile for each plane, and transforms stereo data from disparity image space to 2D obstacle map space. The obstacle map construction in this method is presented. The experiment results prove the validity and accuracy of this method.

CME evolution and 3D reconstruction with STEREO Data

AbstractWe describe a CME event, occurred in NOAA 11059 on April 3 2010, using STEREO and MDI/SOHO data. We analyze the CME evolution using data provided by SECCHI-EUVI and COR1 onboard STEREO satellites, and we perform a 3D reconstruction of the CME using the LCT-TP method. Using MDI/SOHO line-of-sight magnetograms we analyze the magnetic configuration of NOAA 11059 and we determine the magnetic helicity trend.

Large scale graph-based SLAM using aerial images as prior information

The problem of learning a map with a mobile robot has been intensively studied in the past and is usually referred to as the simultaneous localization and mapping (SLAM) problem. However, most existing solutions to the SLAM problem learn the maps from scratch and have no means for incorporating prior information. In this paper, we present a novel SLAM approach that achieves global consistency by utilizing publicly accessible aerial photographs as prior information. It inserts correspondences found between stereo and three-dimensional range data and the aerial images as constraints into a graph-based formulation of the SLAM problem. We evaluate our algorithm based on large real-world datasets acquired even in mixed in- and outdoor environments by comparing the global accuracy with state-of-the-art SLAM approaches and GPS. The experimental results demonstrate that the maps acquired with our method show increased global consistency.

Comparison of Automatic DSM Generation Modules by Processing IKONOS Stereo Data of an Urban Area

This study deals with the evaluation of four different image-processing software modules for the generation of digital surface models from very high-resolution stereo satellite data. The analysis was done in an urban area due to the growing interest in 3-D information over built-up areas. Depending on the different geometric model approaches used by the different software packages, shifts between 3.06-3.27 m between the digital surface models (DSMs) and the reference DSM were measured. The vertical RMSE of the four tested software packages range between 2.96-14.01 m. However, the visual evaluation resulted in a different ranking and does not confirm the quantitative results entirely. The results show that, depending on the building type to be extracted, the choice of software package may vary. The challenges of automatic DSM extraction in urban areas and the performance of current software package modules to address them are discussed. Potential improvements for automatic DSM extraction in urban areas are identified.

Vast-scale Outdoor Navigation Using Adaptive Relative Bundle Adjustment

In this paper we describe a relative approach to simultaneous localization and mapping, based on the insight that a continuous relative representation can make the problem tractable at large scales. First, it is well known that bundle adjustment is the optimal non-linear least-squares formulation for this problem, in that its maximum-likelihood form matches the definition of the Cramer—Rao lower bound. Unfortunately, computing the maximum-likelihood solution is often prohibitively expensive: this is especially true during loop closures, which often necessitate adjusting all parameters in a loop. In this paper we note that it is precisely the choice of a single privileged coordinate frame that makes bundle adjustment costly, and that this expense can be avoided by adopting a completely relative approach. We derive a new relative bundle adjustment which, instead of optimizing in a single Euclidean space, works in a metric space defined by a manifold. Using an adaptive optimization strategy, we show experimentally that it is possible to solve for the full maximum-likelihood solution incrementally in constant time, even at loop closure. Our approach is, by definition, everywhere locally Euclidean, and we show that the local Euclidean estimate matches that of traditional bundle adjustment. Our system operates online in realtime using stereo data, with fast appearance-based loop closure detection. We show results on over 850,000 images that indicate the accuracy and scalability of the approach, and process over 330 GB of image data into a relative map covering 142 km of Southern England. To demonstrate a baseline sufficiency for navigation, we show that it is possible to find shortest paths in the relative maps we build, in terms of both time and distance. Query images from the web of popular landmarks around London, such as the London Eye or Trafalgar Square, are matched to the relative map to provide route planning goals.

EKF-Based Recursive Dual Estimation of Structure and Motion From Stereo Data

The estimation of motion and structure from stereo-video streams is revisited for applications in ocean exploration and seaflooΓ mapping. Operational constraints require real-time processing to enable adaptive trajectory planning, and robust estimation to navigate optimal paths for collection of useful underwater stereo data. While the traditional joint estimation of motion and structure by way of an extended Kalman filter (EKF) provides a suitable recursive framework, the cubic computation growth with the number of feature tracks is a serious bottleneck. By treating the motion and structure as the states of two coupled filters with stereo feature correspondences as observations, a dual estimator is devised with the performance of joint estimation and computational complexity proportional to the number of features. We favor a sequential implementation to ensure unbiased estimation, in contrast to two parallel dual estimation schemes that generally produce biased updates. Stochastic stability can be established in terms of conditions on initial estimation error, bound on observation noise covariance, observation nonlinearity, and modeling error. Moreover, dynamic features can be treated effectively and efficiently by the removal or addition to a bank of filters, one assigned per feature. Experimental results with synthetic and several real data sets are presented to demonstrate the merits of the proposed recursive dual EKF-based estimator.

ANALYSIS OF LARGE-SCALE ANISOTROPY OF ULTRA-HIGH ENERGY COSMIC RAYS IN HiRes DATA

Stereo data collected by the HiRes experiment over a six year period are examined for large-scale anisotropy related to the inhomogeneous distribution of matter in the nearby Universe. We consider the generic case of small cosmic-ray deflections and a large number of sources tracing the matter distribution. In this matter tracer model the expected cosmic ray flux depends essentially on a single free parameter, the typical deflection angle theta. We find that the HiRes data with threshold energies of 40 EeV and 57 EeV are incompatible with the matter tracer model at a 95% confidence level unless theta is larger than 10 degrees and are compatible with an isotropic flux. The data set above 10 EeV is compatible with both the matter tracer model and an isotropic flux.

Processing Dense Stereo Data Using Elevation Maps: Road Surface, Traffic Isle, and Obstacle Detection

A new approach for the detection of the road surface and obstacles is presented. The high accuracy of the method allows the detection of traffic isles as a distinct class. The 3-D data inferred from dense stereo are transformed into a rectangular digital elevation map (DEM). Two classifiers are proposed, namely, density based and road surface based. The density-based obstacle classifier marks DEM cells as road or obstacles, using the density of 3-D points as a criterion. A quadratic road surface model is initially fitted by a random sample consensus (RANSAC) approach to the region in front of the ego vehicle. A region growing-like process refines this primary solution, driven by the 3-D uncertainty model of the stereo sensor. A robust global solution for the road surface is obtained. The road surface is used for discrimination between road, traffic isle, and obstacle points. Fusion and error filtering is performed on the results of the two classifiers. The proposed real-time algorithm was evaluated in an urban scenario and can be used in complex applications from collision avoidance to path planning.

Assessment of the Stereo-Radargrammetric Mapping Potential of TerraSAR-X Multibeam Spotlight Data

TerraSAR-X can acquire image data in various resolutions down to a range of about 1 m. Moreover, the sensor can operate at various imaging beams and thus acquire image data at different off-nadir viewing angles. These circumstances led to a stimulation of the traditional stereo-mapping approach, as TerraSAR-X image pairs became available in high resolution and in various geometric dispositions. With respect to 3-D surface mapping, TerraSAR-X stereo data processing, therefore, is a serious alternative to synthetic aperture radar interferometry, which can be addressed as the evolving mapping technique of the last decade. Within the TerraSAR-X science program of the German Aerospace Center (DLR), high-resolution multibeam data sets in Spotlight mode were acquired for several Austrian test sites. In general, three images were obtained from either ascending or descending orbits. In order to exploit the 3-D mapping accuracy of TerraSAR-X, stereo-radargrammetric mapping techniques were applied to the data sets, thereby utilizing stereo pairs as well as multi-image data sets in various dispositions. This paper focuses on one of the selected test sites and refers to the issues of 2-D and 3-D mapping-accuracy assessment as well as to surface model and vegetation-height-model generation. Validation of these products was widely restricted to visual analysis due to the lack of adequate high-quality reference products.

Statistics of counter-streaming solar wind suprathermal electrons at solar minimum: STEREO observations

Abstract. Previous work has shown that solar wind suprathermal electrons can display a number of features in terms of their anisotropy. Of importance is the occurrence of counter-streaming electron patterns, i.e., with "beams" both parallel and anti-parallel to the local magnetic field, which is believed to shed light on the heliospheric magnetic field topology. In the present study, we use STEREO data to obtain the statistical properties of counter-streaming suprathermal electrons (CSEs) in the vicinity of corotating interaction regions (CIRs) during the period March–December 2007. Because this period corresponds to a minimum of solar activity, the results are unrelated to the sampling of large-scale coronal mass ejections, which can lead to CSE owing to their closed magnetic field topology. The present study statistically confirms that CSEs are primarily the result of suprathermal electron leakage from the compressed CIR into the upstream regions with the combined occurrence of halo depletion at 90° pitch angle. The occurrence rate of CSE is found to be about 15–20% on average during the period analyzed (depending on the criteria used), but superposed epoch analysis demonstrates that CSEs are preferentially observed both before and after the passage of the stream interface (with peak occurrence rate >35% in the trailing high speed stream), as well as both inside and outside CIRs. The results quantitatively show that CSEs are common in the solar wind during solar minimum, but yet they suggest that such distributions would be much more common if pitch angle scattering were absent. We further argue that (1) the formation of shocks contributes to the occurrence of enhanced counter-streaming sunward-directed fluxes, but does not appear to be a necessary condition, and (2) that the presence of small-scale transients with closed-field topologies likely also contributes to the occurrence of counter-streaming patterns, but only in the slow solar wind prior to CIRs.

Automatic Detection of Limb Prominences in 304 Å EUV Images

A new algorithm for automatic detection of prominences on the solar limb in 304 Å EUV images is presented, and results of its application to SOHO/EIT data discussed. The detection is based on the method of moments combined with a classifier analysis aimed at discriminating between limb prominences, active regions, and the quiet corona. This classifier analysis is based on a Support Vector Machine (SVM). Using a set of 12 moments of the radial intensity profiles, the algorithm performs well in discriminating between the above three categories of limb structures, with a misclassification rate of 7%. Pixels detected as belonging to a prominence are then used as the starting point to reconstruct the whole prominence by morphological image-processing techniques. It is planned that a catalogue of limb prominences identified in SOHO and STEREO data using this method will be made publicly available to the scientific community.

Identification of cryovolcanism on Titan using fuzzy cognitive maps

Future planetary exploration of Titan will require higher degrees of on-board automation, including autonomous determination of sites where the probability of significant scientific findings is the highest. In this paper, a novel Artificial Intelligence (AI) method for the identification and interpretation of sites that yield the highest potential of cryovolcanic activity is presented. We introduce the theory of fuzzy cognitive maps (FCM) as a tool for the analysis of remotely collected data in planetary exploration. A cognitive model embedded in a fuzzy logic framework is constructed via the synergistic interaction of planetary scientists and AI experts. As an application example, we show how FCM can be employed to solve the challenging problem of recognizing cryovolcanism from Synthetic Aperture Radar (SAR) Cassini data. The fuzzy cognitive map is constructed using what is currently known about cryovolcanism on Titan and relies on geological mapping performed by planetary scientists to interpret different locales as cryovolcanic in nature. The system is not conceived to replace the human scientific interpretation, but to enhance the scientists’ ability to deal with large amounts of data, and it is a first step in designing AI systems that will be able, in the future, to autonomously make decisions in situations where human analysis and interpretation is not readily available or could not be sufficiently timely. The proposed FCM is tested on Cassini radar data to show the effectiveness of the system in reaching conclusions put forward by human experts and published in the literature. Four tests are performed using the Ta SAR image (October 2004 fly-by). Two regions (i.e. Ganesa Macula and the lobate high backscattering region East of Ganesa) are interpreted by the designed FCM as exhibiting cryovolcanism in agreement with the initial interpretation of the regions by Stofan et al. (2006). Importantly, the proposed FCM is shown to be flexible and adaptive as new data and knowledge are acquired during the course of exploration. Subsequently, the FCM has been modified to include topographic information derived from SAR stereo data. With this additional information, the map concludes that Ganesa Macula is not a cryovolcanic region. In conclusion, the FCM methodology is shown to be a critical and powerful component of future autonomous robotic spacecraft (e.g., orbiter(s), balloon(s), surface/lake lander(s), rover(s)) that will be deployed for the exploration of Titan.

Omnidirectional texturing based on robust 3D registration through Euclidean reconstruction from two spherical images

We propose a semi-automatic omnidirectional texturing method that maps a spherical image onto a dense 3D model obtained by a range sensor. For accurate texturing, accurate estimation of the extrinsic parameters is inevitable. In order to estimate these parameters, we propose a robust 3D registration-based method between a dense range data set and a sparse spherical image stereo data set. For measuring the distances between the two data sets, we introduce generalized distances taking account of 3D error distributions of the stereo data. To reconstruct 3D models by images, we use two spherical images taken at arbitrary positions in arbitrary poses. Then, we propose a novel rectification method for spherical images that is derived from E matrix and facilitates the estimation of the disparities. The experimental results show that the proposed method can map the spherical image onto the dense 3D models effectively and accurately.

Cartosat-1 height product and ICESat/GLAS data for digital elevation surface generation

Space born systems like Geoscience Laser Altimeter System (GLAS) onboard collect data for ice, cloud and Land. Elevation satellite (ICESat) collects an unparalleled data set as waveform over terrestrial targets, helps in evaluating the global elevation data. In this study we compared the Digital Elevation Surface (DES) generated by Cartosat-1 point data and DES generated by merging the Cartosat-1 data with ICESat data. Outputs in the form of interpolated surfaces were evaluated with the help of differential global positioning system (DGPS) points collected from study area. The study showed the results that the DES generated from Cartosat — 1 data had less elevation accuracy when compared with the DGPS data. While merging Cartosat-1 point height data with ICESat/GLAS data resulted in better accuracy. On the practical side for processing the interpolation, based on the research the ICESat /GLAS with Cartosat-1 height data can produce better DES compared to the Cartosat-1 stereo data. The DES was generated using geostatistical interpolation methods in which the global polynomial method proved to be the better for generating the surface compare to other interpolation techniques studied in this work. For co-kriging method, the accuracy decreases compare to the kriging interpolation, due to the complexity of parameters that were used for interpolation. On the theory side, based on this research the statement of which interpolation technique is better than the other cannot be mentioned easily, because these are based on the data type, parameters and also on method of interpolation. So research experiment should be more intensely and with more focused.

Validation of aircraft height extraction from Worldview-1 stereo data

The satellite Worldview-1 was launched in September 2007; the optical sensor is acquiring panchromatic data of 0.5 m geometric resolution at nadir. A stereo data set covering an urban area was provided by DigitalGlobe to evaluate the potential of object height extraction and digital surface model (DSM) generation. Because of a lack of reference elevation data, an accuracy assessment of eight aircraft at Ontario airport was undertaken. The object heights were compared with the known height of every aircraft type. They were well estimated with a mean absolute error (MAE) for all measured positions between 0.59 and 0.62 m. One aircraft was analysed in more detail and its shape evaluated qualitatively. The results are promising and show that the sensor is suitable for DSM generation and object height extraction. It has the potential for improved three-dimensional (3D) information extraction and thus can contribute to more accurate, detailed and realistic 3D models. Future studies will provide more information about the absolute height accuracies to be expected for different surfaces of the Earth.

Multi-resolution topographic data extraction from Martian stereo imagery

The acquisition of high resolution topographic data such as Digital Terrain Models (DTMs) and orthoimages from various Martian stereo imagery is now readily available. The very successful European Space Agency (ESA) Mars Express mission includes the High Resolution Stereo Camera (HRSC) which has provided, since March 2004, an increasingly global set of along-track stereo coverage at relatively high spatial resolution (<100 m, mainly 12.5–25 m) over the Martian surface. Previous DTM generation was only accomplished with planned or serendipitous Viking Orbiter and more recently from Mars Orbiter Camera Narrow Angle (MOC-NA) stereo-pairs. Neither system was designed for stereo photogrammetric DTM generation. Recently, the NASA Mars Reconnaissance Orbiter (MRO) deployed two major optical pushbroom CCD cameras which are capable of across-track stereo targeting. Both Context Camera (CTX) and High Resolution Imaging Science Experiment (HiRISE) provide very high resolution stereo imagery at 6 m and submetre, respectively. A stereo processing chain has been developed which uses a non-rigorous sensor model with geodetic control derived from a reference stereo data source (HRSC) and is here shown to be successfully applied to CTX and HiRISE stereo imagery for 3 test areas. This processing chain is here demonstrated to generate excellent quality DTMs (up to a maximum grid spacing of 0.7 m with HiRISE and 10 m with CTX) and associated ortho images. The photogrammetric quality of these products is here verified using inter-comparisons with HRSC and Mars Orbiter Laser Altimeter (MOLA) data sets and shows good agreement.

Signatures of interchange reconnection: STEREO, ACE and Hinode observations combined

Abstract. Combining STEREO, ACE and Hinode observations has presented an opportunity to follow a filament eruption and coronal mass ejection (CME) on 17 October 2007 from an active region (AR) inside a coronal hole (CH) into the heliosphere. This particular combination of "open" and closed magnetic topologies provides an ideal scenario for interchange reconnection to take place. With Hinode and STEREO data we were able to identify the emergence time and type of structure seen in the in-situ data four days later. On the 21st, ACE observed in-situ the passage of an ICME with "open" magnetic topology. The magnetic field configuration of the source, a mature AR located inside an equatorial CH, has important implications for the solar and interplanetary signatures of the eruption. We interpret the formation of an "anemone" structure of the erupting AR and the passage in-situ of the ICME being disconnected at one leg, as manifested by uni-directional suprathermal electron flux in the ICME, to be a direct result of interchange reconnection between closed loops of the CME originating from the AR and "open" field lines of the surrounding CH.

Leaving Flatland: Efficient real‐time three‐dimensional perception and motion planning

AbstractIn this article we present the complete details of the architecture and implementation of Leaving Flatland, an exploratory project that attempts to surmount the challenges of closing the loop between autonomous perception and action on challenging terrain. The proposed system includes comprehensive localization, mapping, path planning, and visualization techniques for a mobile robot to operate autonomously in complex three‐dimensional (3D) indoor and outdoor environments. In doing so we integrate robust visual odometry localization techniques with real‐time 3D mapping methods from stereo data to obtain consistent global models annotated with semantic labels. These models are used by a multiregion motion planner that adapts existing two‐dimensional planning techniques to operate in 3D terrain. All the system components are evaluated on a variety of real‐world data sets, and their computational performance is shown to be favorable for high‐speed autonomous navigation. © 2009 Wiley Periodicals, Inc.

Stereo-Based Stochastic Mapping for Robust Speech Recognition

We present a stochastic mapping technique for robust speech recognition that uses stereo data. The idea is based on constructing a Gaussian mixture model for the joint distribution of the clean and noisy features and using this distribution to predict the clean speech during testing. The proposed mapping is called stereo-based stochastic mapping (SSM). Two different estimators are considered. One is iterative and is based on the maximum <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">posteriori</i> (MAP) criterion while the other uses the minimum mean square error (MMSE) criterion. The resulting estimators are effectively a mixture of linear transforms weighted by component posteriors, and the parameters of the linear transformations are derived from the joint distribution. Compared to the uncompensated result, the proposed method results in 45% relative improvement in word error rate (WER) for digit recognition in the car. In the same setting, SSM outperforms SPLICE and gives similar results to MMSE compensation of Huang A 66% relative improvement in word error rate (WER) is observed when applied in conjunction with multistyle training (MST) for large vocabulary English speech recognition in a real environment. Also, the combination of the proposed mapping with CMLLR leads to about 38% relative improvement in performance compared to CMLLR alone for real field data.

Evaluation of Stereo Matching Costs on Images with Radiometric Differences

Stereo correspondence methods rely on matching costs for computing the similarity of image locations. We evaluate the insensitivity of different costs for passive binocular stereo methods with respect to radiometric variations of the input images. We consider both pixel-based and window-based variants like the absolute difference, the sampling-insensitive absolute difference, and normalized cross correlation, as well as their zero-mean versions. We also consider filters like LoG, mean, and bilateral background subtraction (BilSub) and nonparametric measures like Rank, SoftRank, Census, and Ordinal. Finally, hierarchical mutual information (HMI) is considered as pixelwise cost. Using stereo data sets with ground-truth disparities taken under controlled changes of exposure and lighting, we evaluate the costs with a local, a semiglobal, and a global stereo method. We measure the performance of all costs in the presence of simulated and real radiometric differences, including exposure differences, vignetting, varying lighting, and noise. Overall, the ranking of methods across all data sets and experiments appears to be consistent. Among the best costs are BilSub, which performs consistently very well for low radiometric differences; HMI, which is slightly better as pixelwise matching cost in some cases and for strong image noise; and Census, which showed the best and most robust overall performance.