Set Of Stereo Images Research Articles

BUNDLE ADJUSTMENT OF SPACEBORNE DOUBLE-CAMERA PUSH-BROOM IMAGERS AND ITS APPLICATION TO LROC NAC IMAGERY

Abstract. The TU Berlin group of the Lunar Reconnaissance Orbiter Camera (LROC) team has implemented a Bundle Adjustment (BA) for spaceborne multi-lenses line scan imagers, by rigorously modeling the geometric properties of the image acquisition. The BA was applied to stereo image sets of the LROC Narrow Angle Camera (NAC) and first results show, that the overall geometry of the stereo models were significantly improved. Ray intersection accuracies of initially up to several meters were homogenized within the integrated stereo models and improved to 0.14 m on average. The mean point error of the adjusted 3D object points was estimated by the BA to be 0.95 m. The inclusion of available Lunar Orbiter Laser Altimeter (LOLA) shots as 3D ground control to the BA, accurately tied to the image space by an aforegoing co-registration, allowed to register the final adjusted NAC DTM to the currently most accurate global lunar reference frame. The BA also provides accuracy assessments of the individual LOLA tracks used for georeference during the adjustment, which will be useful to further assess LOLA derived products.

Study of convolutional autoencoder models for feature extraction from stereo image sets

The modeling process of convolutional autoencoder unsupervised learning for feature extraction from stereo images sets is provided. The number of learning filters and topology of autoencoder influence on quality of image reconstruction is researched. Autoencoder configurations of high efficiency reconstruction is defined. Figs.: 2. Tabl.: 1. Refs.: 13.Keywords: unsupervised learning, convolutional autoencoder; feature extraction; stereo pair; image reconstruction.

Modified adaptive support weight and disparity search range estimation schemes for stereo matching processors

Recently, to obtain three-dimensional depth information from a set of stereo images, stereo matching processors are widely used in intelligent robots, autonomous vehicles, and the Internet of things environment, all of which require real-time processing capability with minimal hardware resources. In this paper, we propose a modified adaptive support weight scheme with rectangular ring-type window configurations that minimize hardware resources while maintaining matching accuracy. In addition, to reduce the computational overhead of window-based local stereo matching algorithms, we present a robust disparity search range estimation scheme based on stretched depth histograms. To evaluate the performance of the proposed schemes, we implemented them using C language and performed experiments. In addition, to show the feasibility of the hardware implementation of the proposed schemes, we also describe them using Verilog hardware description language and implemented them using a field-programmable gate array-based platform. Experimental results show that compared to conventional method, the proposed schemes reduced up to 57% of hardware resources and 33% of computational overhead, respectively.

Extracting accurate and precise topography from LROC narrow angle camera stereo observations

The Lunar Reconnaissance Orbiter Camera (LROC) includes two identical Narrow Angle Cameras (NAC) that each provide 0.5 to 2.0m scale images of the lunar surface. Although not designed as a stereo system, LROC can acquire NAC stereo observations over two or more orbits using at least one off-nadir slew. Digital terrain models (DTMs) are generated from sets of stereo images and registered to profiles from the Lunar Orbiter Laser Altimeter (LOLA) to improve absolute accuracy. With current processing methods, DTMs have absolute accuracies better than the uncertainties of the LOLA profiles and relative vertical and horizontal precisions less than the pixel scale of the DTMs (2–5m).We computed slope statistics from 81 highland and 31 mare DTMs across a range of baselines. For a baseline of 15m the highland mean slope parameters are: median=9.1°, mean=11.0°, standard deviation=7.0°. For the mare the mean slope parameters are: median=3.5°, mean=4.9°, standard deviation=4.5°. The slope values for the highland terrain are steeper than previously reported, likely due to a bias in targeting of the NAC DTMs toward higher relief features in the highland terrain.Overlapping DTMs of single stereo sets were also combined to form larger area DTM mosaics that enable detailed characterization of large geomorphic features. From one DTM mosaic we mapped a large viscous flow related to the Orientale basin ejecta and estimated its thickness and volume to exceed 300m and 500km3, respectively. Despite its ∼3.8 billion year age the flow still exhibits unconfined margin slopes above 30°, in some cases exceeding the angle of repose, consistent with deposition of material rich in impact melt.We show that the NAC stereo pairs and derived DTMs represent an invaluable tool for science and exploration purposes. At this date about 2% of the lunar surface is imaged in high-resolution stereo, and continued acquisition of stereo observations will serve to strengthen our knowledge of the Moon and geologic processes that occur across all of the terrestrial planets.

Implementation of Real-Time Post-Processing for High-Quality Stereo Vision

We propose a novel post-processing algorithm and its very-large-scale integration architecture that simultaneously uses the passive and active stereo vision information to improve the reliability of the three-dimensional disparity in a hybrid stereo vision system. The proposed architecture consists of four steps — left-right consistency checking, semi-2D hole filling, a tiny adaptive variance checking, and a 2D weighted median filter. The experimental results show that the error rate of the proposed algorithm (5.77%) is less than that of a raw disparity (10.12%) for a real-world camera image having a resolution and maximum disparity of 256. Moreover, for the famous Middlebury stereo image sets, the proposed algorithm's error rate (8.30%) is also less than that of the raw disparity (13.7%). The proposed architecture is implemented on a single commercial field-programmable gate array using only 13.01% of slice resources, which achieves a rate of 60 fps for stereo images with a disparity range of 256.

Probabilistic Integration of Intensity and Depth Information for Part-Based Vehicle Detection

In this paper, an object class recognition method is presented. The method uses local image features and follows the part-based detection approach. It fuses intensity and depth information in a probabilistic framework. The depth of each local feature is used to weigh the probability of finding the object at a given distance. To train the system for an object class, only a database of images annotated with bounding boxes is required, thus automatizing the extension of the system to different object classes. We apply our method to the problem of detecting vehicles from a moving platform. The experiments with a data set of stereo images in an urban environment show a significant improvement in performance when using both information modalities.

A comparison of dense matching algorithms for scaled surface reconstruction using stereo camera rigs

Photogrammetric methods for dense 3D surface reconstruction are increasingly available to both professional and amateur users who have requirements that span a wide variety of applications. One of the key concerns in choosing an appropriate method is to understand the achievable accuracy and how choices made within the workflow can alter that outcome. In this paper we consider accuracy in two components: the ability to generate a correctly scaled 3D model; and the ability to automatically deliver a high quality data set that provides good agreement to a reference surface. The determination of scale information is particularly important, since a network of images usually only provides angle measurements and thus leads to unscaled geometry. A solution is the introduction of known distances in object space, such as base lines between camera stations or distances between control points. In order to avoid using known object distances, the method presented in this paper exploits a calibrated stereo camera utilizing the calibrated base line information from the camera pair as an observational based geometric constraint. The method provides distance information throughout the object volume by orbiting the object.In order to test the performance of this approach, four topical surface matching methods have been investigated to determine their ability to produce accurate, dense point clouds. The methods include two versions of Semi-Global Matching as well as MicMac and Patch-based Multi-View Stereo (PMVS). These methods are implemented on a set of stereo images captured from four carefully selected objects by using (1) an off-the-shelf low cost 3D camera and (2) a pair of Nikon D700 DSLR cameras rigidly mounted in close proximity to each other. Inter-comparisons demonstrate the subtle differences between each of these permutations. The point clouds are also compared to a dataset obtained with a Nikon MMD laser scanner. Finally, the established process of achieving accurate point clouds from images and known object space distances are compared with the presented strategies.Results from the matching demonstrate that if a good imaging network is provided, using a stereo camera and bundle adjustment with geometric constraints can effectively resolve the scale. Among the strategies for dense 3D reconstruction, using the presented method for solving the scale problem and PMVS on the images captured with two DSLR cameras resulted in a dense point cloud as accurate as the Nikon laser scanner dataset.

An improved feature matching technique for stereo vision applications with the use of self-organizing map

Stereo vision cameras are widely used for finding a path for obstacle avoidance in autonomous mobile robots. The Scale Invariant Feature Transform (SIFT) algorithm proposed by Lowe is used to extract distinctive invariant features from images. While it has been successfully applied to a variety of computer vision problems based on feature matching including machine vision, object recognition, image retrieval, and many others, this algorithm has high complexity and long computational time. In order to reduce the computation time, this paper proposes a SIFT improvement technique based on a Self-Organizing Map (SOM) to perform the matching procedure more efficiently for feature matching problems. Matching for multi-dimension SIFT features is implemented with a self-organizing map that introduces competitive learning for matching features. Experimental results on real stereo images show that the proposed algorithm performs feature group matching with lower computation time than the SIFT algorithm proposed by Lowe. We performed experiments on various set of stereo images under dynamic environment with different camera viewpoints that is based on rotation and illumination conditions. The numbers of matched features were increased to double as compared to the algorithm developed by Lowe. The results showing improvement over the SIFT proposed by Lowe are validated through matching examples between different pairs of stereo images. The proposed algorithm can be applied to stereo vision based robot navigation for obstacle avoidance, as well as many other feature matching and computer vision applications.

Accuracy of stereoscopic navigation system in flights above natural landscapes and urban land

Accuracy of stereoscopic navigation system designated for solution of problems of high accuracy navigation, guidance, and remote landscape probing is estimated. Linearization of photometric equations of the stereoscopic navigation system in the regime of stabilized horizontal flight of an aircraft is performed. Observation equations of the stereoscopic navigation system with account of the only stereo pair and a set of stereo images are determined. New algorithms of image processing in the stereoscopic navigation system based on the following two proved results: direct and inverse photogrammetry equations and the theory of joint optimal filtering, identification, and verification of hypotheses in a discrete dynamic system, are used to obtain formulas characterizing the accuracy of the stereoscopic navigation system in flights above natural landscapes and urban land. Procedure of choosing correction segments is considered.

Bayesian stereo matching

A Bayesian framework is proposed for stereo vision where solutions to both the model parameters and the disparity map are posed in terms of predictions of latent variables, given the observed stereo images. A mixed sampling and deterministic strategy is adopted to balance between effectiveness and efficiency: the parameters are estimated via Markov Chain Monte Carlo sampling techniques and the Maximum A Posteriori (MAP) disparity map is inferred by a deterministic approximation algorithm. Additionally, a new method is provided to evaluate the partition function of the associated Markov random field model. Encouraging results are obtained on a standard set of stereo images as well as on synthetic forest images.

변이공간에서 스네이크 알고리즘을 이용한 객체분할에 관한 연구

본 논문에서는 2차원 영상의 객체분할 방법으로 잘 알려져 있는 Snake 알고리즘을 스테레오 영상(Stereo Image)에 적용할 수 있는 새로운 알고리즘을 제안한다. 기존의 2차원 단일영상에 적용한 Snake 알고리즘은 만족한 결과를 얻기 위해선 분할하려는 객체의 주변 배경이 단순하고, 다른 객체들과 중첩되어 있지 않으며, 초기 Snake 포인트를 객체의 윤곽(Boundary) 가까이에 사용자가 설정해야 하는 문제점이 있다. 본 논문에서는 이러한 문제점을 해결하기 위해 스테레오 영상에서 얻을 수 있는 변이정보(Disparity Iformation)를 이용하여 변이공간(Disparity Space)에서 새로운 Snake 에너지 함수를 정의하였으며, 정의한 에너지 함수를 사용하여 스테레오 영상에 적용할 수 있는 새로운 객체 분할 알고리즘을 제안하였다. 제안한 알고리즘은 복잡도가 다양한 실험영상에 적용하여 성능을 분석하였다. Object segmentation is a challenging Problem when the background is cluttered and the objects are overlapped one another. Recent develop-ment using snake algorithms proposed to segment objects from a 2-D Image presents a higher possibilityfor getting better contours. However, the performance of those snake algorithms degrades rapidly when the background is cluttered and objects are overlapped one another, Moreover, the initial snake point placement is another difficulty to be resolved. Here, we propose a novel snake algorithm for object segmentation using disparity information taken from a set of stereo images. By applying our newly designed snake energy function defined in the disparity space, our algorithmeffectively circumvents the limitations found in the previous methods. The performance of the proposed algorithm has been verified by computer simulation using various stereo image sets. The experiment results have exhibited a better performance over the well-known snake algorithm in terms of segmentation accuracy.

Toward effectively generating environment maps

We explore a cost-effective approach to creating envi- ronment maps. Unlike other environment map generating systems that use specially designed mirrors or sensing devices, our system uses off-the-shelf digital cameras. This makes our system poten- tially acquirable by average consumers. In our design, the two digital cameras are displaced vertically with parallel optical axes, and the rotation axis passes through the focal points of the two cameras. This vertical displacement arrangement overcomes the problem of having inconsistent occlusions in two adjacent stereo image sets that happen in a horizontally displaced stereo camera system. To obtain dense spatial images, our system uses color-information- based algorithms to propagate disparity values from matched fea- ture points to unmatched feature points and featureless areas. Schemes of transforming local 3-D (X,Y,Z) data to a world coordi- nate system are also devised to smoothly stitch 3-D data from indi- vidual frames for virtual reality modeling language graphics display.

Three‐dimensional display utilizing a diffractive optical element and an active matrix liquid crystal display

We describe the design, construction, and performance of the first real-time autostereoscopic three-dimensional (3-D) display based on the partial pixel 3-D display architecture. The primary optical components of the 3-D display are an active-matrix liquid crystal display and a diffractive optical element (DOE). The display operates at video frame rates and is driven with a conventional VGA signal. Three-dimensional animations with horizontal motion parallax are readily viewable as sets of stereo images. Formation of the virtual viewing slits by diffraction from the partial pixel apertures is experimentally verified. The measured contrast and perceived brightness of the display are excellent, but there are minor flaws in image quality due to secondary images. The source of these images and how they may be eliminated is discussed. The effects of manufacturing-related systematic errors in the DOE are also analyzed.