Stereo Image Matching Research Articles

RESEARCH ON DIFFERENTIAL CODING METHOD FOR SATELLITE REMOTE SENSING DATA COMPRESSION

Abstract. Data compression, in the process of Satellite Earth data transmission, is of great concern to improve the efficiency of data transmission. Information amounts inherent to remote sensing images provide a foundation for data compression in terms of information theory. In particular, distinct degrees of uncertainty inherent to distinct land covers result in the different information amounts. This paper first proposes a lossless differential encoding method to improve compression rates. Then a district forecast differential encoding method is proposed to further improve the compression rates. Considering the stereo measurements in modern photogrammetry are basically accomplished by means of automatic stereo image matching, an edge protection operator is finally utilized to appropriately filter out high frequency noises which could help magnify the signals and further improve the compression rates. The three steps were applied to a Landsat TM multispectral image and a set of SPOT-5 panchromatic images of four typical land cover types (i.e., urban areas, farm lands, mountain areas and water bodies). Results revealed that the average code lengths obtained by the differential encoding method, compared with Huffman encoding, were more close to the information amounts inherent to remote sensing images. And the compression rates were improved to some extent. Furthermore, the compression rates of the four land cover images obtained by the district forecast differential encoding method were nearly doubled. As for the images with the edge features preserved, the compression rates are average four times as large as those of the original images.

MULTI-LEVEL BUILDING RECONSTRUCTION FOR AUTOMATIC ENHANCEMENT OF HIGH RESOLUTION DSMS

Abstract. In this article a multi-level approach is proposed for reconstruction-based improvement of high resolution Digital Surface Models (DSMs). The concept of Levels of Detail (LOD) defined by CityGML standard has been considered as basis for abstraction levels of building roof structures. Here, the LOD1 and LOD2 which are related to prismatic and parametric roof shapes are reconstructed. Besides proposing a new approach for automatic LOD1 and LOD2 generation from high resolution DSMs, the algorithm contains two generalization levels namely horizontal and vertical. Both generalization levels are applied to prismatic model of buildings. The horizontal generalization allows controlling the approximation level of building footprints which is similar to cartographic generalization concept of the urban maps. In vertical generalization, the prismatic model is formed using an individual building height and continuous to included all flat structures locating in different height levels. The concept of LOD1 generation is based on approximation of the building footprints into rectangular or non-rectangular polygons. For a rectangular building containing one main orientation a method based on Minimum Bounding Rectangle (MBR) in employed. In contrast, a Combined Minimum Bounding Rectangle (CMBR) approach is proposed for regularization of non-rectilinear polygons, i.e. buildings without perpendicular edge directions. Both MBRand CMBR-based approaches are iteratively employed on building segments to reduce the original building footprints to a minimum number of nodes with maximum similarity to original shapes. A model driven approach based on the analysis of the 3D points of DSMs in a 2D projection plane is proposed for LOD2 generation. Accordingly, a building block is divided into smaller parts according to the direction and number of existing ridge lines. The 3D model is derived for each building part and finally, a complete parametric model is formed by merging all the 3D models of the individual parts and adjusting the nodes after the merging step. In order to provide an enhanced DSM, a surface model is provided for each building by interpolation of the internal points of the generated models. All interpolated models are situated on a Digital Terrain Model (DTM) of corresponding area to shape the enhanced DSM. Proposed DSM enhancement approach has been tested on a dataset from Munich central area. The original DSM is created using robust stereo matching of Worldview-2 stereo images. A quantitative assessment of the new DSM by comparing the heights of the ridges and eaves shows a standard deviation of better than 50cm.

HIGH-PRECISION DEM PRODUCTION FOR SPACEBORNE STEREO SAR IMAGES BASED ON SIFT MATCHING AND REGION-BASED LEAST SQUARES MATCHING

Abstract. Generally, there are two ways to generate Digital Elevation Model (DEM) using synthetic aperture radar (SAR) data, which are Interferometer Synthetic Aperture Radar(InSAR) and radargrammetry. Considering the disadvantages of InSAR data, such as the limit of terrain and the influence of water content, the application field of InSAR is relatively limited, while radargrammetry is more widely applied since it does not have such limits. However, for high-precision stereo SAR imagery, since the terrain distortion caused by shooting angle cannot be eliminated and the speckle noises are obvious, the classical matching algorithms for optical stereo images do not have the same effect on SAR data. Based on the experience of optical stereo image matching, this paper proposes a new algorithm which combines the feature of SIFT image matching, region-based least squares matching and TIN . First, SIFT matching is used as the initial matching to obtain the sparse DEM, then by using TIN the matching points are forecast, finally the region-based least squares matching is adopted to get accurate matching points. In this paper, COSMO-SkyMed and TSX stereo images of Lanzhou area are used to validate the proposed method. Experiment results show that the algorithm can be effectively used in stereo SAR matching and high-precision DEM production.

Stereo-Image Matching Using a Speeded Up Robust Feature Algorithm in an Integrated Vision Navigation System

Visual navigation is comparatively advanced without a Global Positioning System (GPS). It obtains environmental information via real-time processing of the data gained through visual sensors. Compared with other methods, visual navigation is a passive method that does not launch light or other radiation applications, thus making it easier to hide. The novel navigation system described in this paper uses stereo-matching combined with Inertial Measurement Units (IMU). This system applies photogrammetric theory and a matching algorithm to identify the matching points of two images of the same scene taken from different views and obtains their 3D coordinates. Integrated with the orientation information output by the IMU, the system reduces model-accumulated errors and improves the point accuracy.

Stereo Image Matching for Vehicle-Borne Mobile Mapping System Based on Digital Parallax Model

Considering automatic and effective stereo image matching for vehicle-borne mobile mapping system (VMMS), a new stereo image matching algorithm based on digital parallax model (DPM) is proposed in this paper. The new matching propagation strategy is designed in this algorithm, which includes two processes as DPM construction and parallax prediction. With some known matched points, the DPM of stereo image pairs is firstly constructed, and parameters for confirming conjugate epipolar line is also worked out. Then searching range during dense matching can be confirmed under constraints of DPM and epipolar line, which can improve matching speed and accuracy. Furthermore, to improve matching robustness, the computation model of similarity measurement combined with local structure feature and global color feature is designed. The new algorithm is applied to actual stereo images taken by VMMS to verify its validity. Experimental results show that the proposed approach has higher reliability and accuracy.

3D head pose tracking using a particle filter with nose template matching

AbstractIn this paper, we propose a method for real‐time tracking of the driver's head pose in a real vehicle environment by using multiple NIR cameras. In order to achieve real‐time performance and high accuracy, the head pose is estimated with six degrees of freedom by a particle filter with a restricted state space. First, the 3D position of the nostrils is measured by template matching of stereo images. Because the nostrils are the darkest area in the captured NTR images, we can detect the position of the nose robustly and very precisely. From the 3D position of the nose, we can make an initial estimate of the head pose. Then, each hypothesis is updated by prior probability with the constraint of the nose position. Thus, it is possible to reduce the number of particles while maintaining accuracy. Experimental results indicate that this method is effective for head pose tracking. © 2010 Wiley Periodicals, Inc. Electron Comm Jpn, 94(1): 34–42, 2011; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ecj.10268

IMAGE MATCHING OPTIMIZATION VIA VISION AND INERTIAL DATA FUSION: APPLICATION TO NAVIGATION OF THE VISUALLY IMPAIRED

Autonomous navigation by humans or robots is usually based on stereo image matching. However, classic image matching methods are not suitable for wearable real-time navigation systems. This paper proposes a new image-matching algorithm which fuses information from both images and inertial data. A preliminary evaluation of the algorithm shows that it is effective for indoor navigation.

Robust and Efficient Surface Reconstruction From Range Data

Abstract We describe a robust but simple algorithm to reconstruct a surface from a set of merged range scans. Our key contribution is the formulation of the surface reconstruction problem as an energy minimisation problem that explicitly models the scanning process. The adaptivity of the Delaunay triangulation is exploited by restricting the energy to inside/outside labelings of Delaunay tetrahedra. Our energy measures both the output surface quality and how well the surface agrees with soft visibility constraints. Such energy is shown to perfectly fit into the minimum s − t cuts optimisation framework, allowing fast computation of a globally optimal tetrahedra labeling, while avoiding the “shrinking bias” that usually plagues graph cuts methods. The behaviour of our method confronted to noise, undersampling and outliers is evaluated on several data sets and compared with other methods through different experiments: its strong robustness would make our method practical not only for reconstruction from range data but also from typically more difficult dense point clouds, resulting for instance from stereo image matching. Our effective modeling of the surface acquisition inverse problem, along with the unique combination of Delaunay triangulation and minimum s − t cuts, makes the computational requirements of the algorithm scale well with respect to the size of the input point cloud.

Binocular Spherical Stereo

A fish-eye camera has a wide field of view (FOV), and the realization of a binocular fish-eye stereo for sensing the surrounding 3-D information of the environment around a vehicle is useful for safe driving. However, since a fish-eye camera may have a wider-than-hemispherical FOV, the conventional stereo approach of obtaining a perspective image based on the pinhole camera model cannot directly be applied. However, using a spherical camera model and defining the disparity of a spherical stereo, the conventional binocular stereo problem is reformulated as a <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">binocular</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">spherical</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stereo</i> problem. A binocular spherical stereo is a generalized paradigm that can cope with cameras having any FOV, including conventional cameras and fish-eye cameras. Moreover, by transforming the rectified spherical images to latitude-longitude representation, the feature point matching of the spherical stereo images can be sped up by using the processing used for perspective stereo images. The effectiveness of this approach is demonstrated by realizing a binocular spherical stereo using a pair of fish-eye cameras. Finally, the application of the proposed approach to vehicles in the future is considered.

3D Head Pose Tracking Using a Particle Filter with Nose Template Matching

In this paper, we propose a real-time tracking method for driver's head pose in real vehicle environment by using multiple NIR cameras. In order to achieve real-time performance and high-accuracy, 6-DOF of head pose is estimated by a particle filter with restricted state space. Firstly, the 3D position of nose hole is measured by template matching of stereo images. Because nose hole is the most dark area in the captured NIR images, we can robustly detect the position of the nose very precisely. From the 3D position of the nose, we can estimate the head pose as an initial estimate. Then, each hypothesis is updated by prior probability with a constraint of nose position. Thus it can reduce the number of particles with maintained accuracy. The experimental results indicate that this method is effective for head pose tracking.

Robust Stereo Image Matching for Spaceborne Imagery

This paper presents a computationally efficient method for stereo image matching of remote sensing images. Traditional correlation methods often fail to give correct matching when the slope of the terrain is very high. A novel technique called aspect-based estimation is proposed here to find the best match points and to estimate the slope of the terrain. Stereo image matching is implemented in a hierarchical manner using wavelets to reduce the computational cost. A robust technique for identifying outliers based on local statistics is also presented here. Experimental results with CARTOSAT-1 images indicate that the aspect-based correlation and blunder detection works very efficiently and effectively in stereo image matching of remote sensing images. It is observed that this algorithm is able to give a high density of reliable match points of the order of 1 for every 5 times 5 pixels. The technique is demonstrated here for CARTOSAT-1 images. However, the proposed algorithm can be applied to a remote sensing stereo pair of any spaceborne imagery.

Propagation strategies for stereo image matching based on the dynamic triangle constraint

For the purpose of reliable stereo image matching, this paper discusses a novel propagation strategy of image matching under the dynamic triangle constraint. Firstly, the construction and the dynamic updating method for the corresponding triangulations on the stereo pairs are introduced, which are used as both constraints and carriers during the matching propagation. Then, three propagation strategies: the stochastic propagation, the adjacent propagation based on the topological relationship of triangles, and the self-adaptive propagation, which considers the texture features are proposed. The detailed algorithms of these three propagation strategies are also presented. To compare these strategies, a stereo pair with typic texture features is employed to describe the different propagation manners of these three strategies, and an experimental analysis is illustrated with different aerial stereo pairs. From test results, the following has been found: (1) stochastic propagation gives the worst matching results; (2) self-adaptive propagation performs better than the adjacent propagation by making use of the global “best first” strategy. From these conclusions, the self-adaptive propagation strategy is recommended for reliable stereo image matching under the dynamic triangle constraint.

Rock modeling and matching for autonomous long‐range Mars rover localization

AbstractThis paper introduces the concept and design of a new integrated approach to long‐range autonomous Mars rover localization based on the incremental bundle adjustment and visual odometry technologies that have been individually experimented with during the 2003 Mars Exploration Rover mission. The design result indicates that a rover would have a varying performance in traversing from 7.5 to 118 m within one traverse leg under various scenarios including camera systems and traverse geometry, while maintaining the onboard rover localization accuracy at 1%. To implement the proposed integrated approach, the key is to develop autonomous cross‐site tie point selection algorithms for automatic generation of a sufficient number of high quality tie points to link all the images and to form the image network. New methods of rock extraction, rock modeling, and rock matching from multiple rover sites are developed to automate cross‐site tie point selection. Rocks are extracted from three‐dimensional ground points generated by stereo image matching, and then modeled using analytical surfaces such as hemispheroid, semiellipsoid, cone, and tetrahedron. Rocks extracted and modeled from two rover sites are matched by a combination of rock model matching and rock distribution pattern matching. The matched rocks are used as cross‐site tie points for a subsequent bundle adjustment. The presented results show that the proposed cross‐site tie point selection approach functions successfully for medium‐range (up to 26 m) traverse legs. © 2007 Wiley Periodicals, Inc.

The Infection Algorithm: An Artificial Epidemic Approach for Dense Stereo Correspondence

We present a new bio-inspired approach applied to a problem of stereo image matching. This approach is based on an artificial epidemic process, which we call the infection algorithm. The problem at hand is a basic one in computer vision for 3D scene reconstruction. It has many complex aspects and is known as an extremely difficult one. The aim is to match the contents of two images in order to obtain 3D information that allows the generation of simulated projections from a viewpoint that is different from the ones of the initial photographs. This process is known as view synthesis. The algorithm we propose exploits the image contents in order to produce only the necessary 3D depth information, while saving computational time. It is based on a set of distributed rules, which propagate like an artificial epidemic over the images. Experiments on a pair of real images are presented, and realistic reprojected images have been generated.

Automatic Building Detection Using the Dempster-Shafer Algorithm

An approach and strategy for automatic detection of buildings from aerial images using combined image analysis and interpretation techniques is described in this paper. It is undertaken in several steps. A dense DSM is obtained by stereo image matching and then the results of multi-band classification, the DSM, and Normalized Difference Vegetation Index (NDVI) are used to reveal preliminary building interest areas. From these areas, a shape modeling algorithm has been used to precisely delineate their boundaries. The Dempster-Shafer data fusion technique is then applied to detect buildings from the combination of three data sources by a statistically-based classification. A number of test areas, which include buildings of different sizes, shape, and roof color have been investigated. The tests are encouraging and demonstrate that all processes in this system are important for effective building detection.

3D Motion parameters determination based on binocular sequence images

Exactly capturing three dimensional (3D) motion information of an object is an essential and important task in computer vision, and is also one of the most difficult problems. In this paper, a binocular vision system and a method for determining 3D motion parameters of an object from binocular sequence images are introduced. The main steps include camera calibration, the matching of motion and stereo images, 3D feature point correspondences and resolving the motion parameters. Finally, the experimental results of acquiring the motion parameters of the objects with uniform velocity and acceleration in the straight line based on the real binocular sequence images by the mentioned method are presented.

An Initial Study on Automatic Reconstruction of Ground DEMs from Airborne IfSAR DSMs

The demands for ground digital elevation models (DEMs) have been increasing rapidly. Many modern mapping technologies, such as lidar scanning, interferometric SAR (IfSAR), and stereo image matching, are capable of generating digital surface models (DSMs) directly. Research on deriving DEM sf romDSM si s a topical area. Airborne IfSAR technology has been in commercial operation since 1996. However, it is still technically challenging to automate the process of generating DEM sf romIfSAR DSMs. The lack of scientific studies in this area is mainly caused by the sensitivity and availability of commercial IfSAR data. In this study, researchers have been working with the commercial IfSAR system developer to perform an initial study on automatic reconstruction of ground DEM sf romIfSAR DSMs. This paper presents a compact and yet efficient version of our developed approach. This approach has been tested by using many IfSAR DSM si ndifferent hilly urban areas with light and moderate variations in relief. An accuracy analysis of the reconstructed ground DEMs against the laser DSMs and DEMs has also been performed to test the reliability and efficiency of our approach.

Auto-Generating DEM with Stereo Image Matching and Mathematical Morphologic Transformation

Auto-Generating DEM with Stereo Image Matching and Mathematical Morphologic Transformation

Multiresolution Phase-Based Bidirectional Stereo Matching with Provision for Discontinuity and Occlusion

Stereo image matching is the most robust and domain-independent way of reconstructing object surfaces from perspective images. Dense feature space matching is desirable because it constructs a disparity field without arbitrary selection of interest points for individual matching. Multiresolution or coarse-to-fine strategies have been shown to be effective in overcoming the inherent ambiguities of dense feature space matching. We use Gabor phase as the basis for dense multiresolution matching, as it is a stable, ubiquitous feature of a signal. The use of Gabor-like complex wavelets enables the efficient transformation of images to Gabor-phase-based feature pyramids. Disparity discontinuity and occlusions are the major potential disruptions to coarse-to-fine image matching. Through perspective geometry, we show that depth discontinuities are the direct cause of the discontinuity of disparity field, and bidirectional matching is required to detect disparity discontinuities (and their dual counterpart, occlusions) in both views. To handle these, a global objective function incorporating feature similarity, disparity smoothness, and discontinuity/occlusions is established under the maximuma posterioriprobability criterion and equivalently transformed into a minimum description length criterion. A general procedure, using stochastic relaxation with special provision for occlusions, is developed for minimising the objective function. The result is a disparity field which is regularised in the continuous regions, while the discontinuities and occlusions are detected and preserved. Some results from an aerial terrain image pair indicate the applicability of this approach.

Reconstruction and Simulation of 3D Open Pits: A Critical Path Towards Practical Virtual Mine

Virtual reality is a promising environment for future mine design, planning and monitoring. One of the major challenges in the development of future practical Virtual Mine is the safe, detailed and efficient data acquisition and automatic reconstruction of the object (either underground mine or open pit). The approach proposed by the authors is the utilisation of close-range digital terrestrial photogrammetric technology. This paper gives a brief description of the methods, techniques and procedures developed specifically for the reconstruction and simulation of complete 3D open pits. An important conclusion drawn from this research is that complete mine open pits can be reconstructed accurately and efficiently by using the close-range digital terrestrial photogrammetric techniques developed by CSIRO Exploration and Mining. The case study on the Eureka pit in Western Australia demonstrates that the close-range digital terrestrial photogrammetry approach has clear advantages over conventional surveying methods in the level of texture detail, accuracy, efficiency and operating costs. The image matching techniques, specifically developed for the reconstruction of complete 3D open pits, have proved to be effective tools for automatic matching of stereo images with large distortions. The mosaicking techniques, specifically developed in this research have proved to be powerful tools in handling special objects such as open pits, pillars and tunnels. In addition, the field procedures developed are important to the data acquisition and subsequent image processing of complete open pits.