Epipolar Lines Research Articles

Stereoscopic Correspondence of Particles for 3-Dimensional Particle Tracking Velocimetry by using Genetic Algorithm

The genetic algorithm (GA) based stereo particle-pairing algorithm has been developed and applied to the spatial particle-pairing problem of the stereoscopic three-dimensional (3-D) PTV system. In this 3 D PTV system, particles viewed by two (or more than two) stereoscopic cameras with a parallax have to be correctly paired at every synchronized time step. This is important because the 3-D coordinates of individual particles cannot be computed without the knowledge of the correct stereo correspondence of the particles. In the present study, the GA algorithm is applied to the epipolar line proximity analysis for establishing correspondence of particles pairs between two co-instantaneous stereoscopic particles images, in order to compute the 3-D coordinates of every individual particle. The results are tested with various standard images and it’s found that the new strategy using GA works better than conventional particle pairing methods of 3-D particle tracking velocimetry for steoroscopic PTV. Journal of the Institute of Engineering, 2016, 12(1): 10-26

UAV Low Altitude Photogrammetry for Power Line Inspection

When the distance between an obstacle and a power line is less than the discharge distance, a discharge arc can be generated, resulting in the interruption of power supplies. Therefore, regular safety inspections are necessary to ensure the safe operation of power grids. Tall vegetation and buildings are the key factors threatening the safe operation of extra high voltage transmission lines within a power line corridor. Manual or laser intensity direction and ranging (LiDAR) based inspections are time consuming and expensive. To make safety inspections more efficient and flexible, a low-altitude unmanned aerial vehicle (UAV) remote-sensing platform, equipped with an optical digital camera, was used to inspect power line corridors. We propose a semi-patch matching algorithm based on epipolar constraints, using both the correlation coefficient (CC) and the shape of its curve to extract three dimensional (3D) point clouds for a power line corridor. We use a stereo image pair from inter-strip to improve power line measurement accuracy by transforming the power line direction to an approximately perpendicular to epipolar line. The distance between the power lines and the 3D point cloud is taken as a criterion for locating obstacles within the power line corridor automatically. Experimental results show that our proposed method is a reliable, cost effective, and applicable way for practical power line inspection and can locate obstacles within the power line corridor with accuracy better than ±0.5 m.

Epipolar Plane Image Rectification and Flat Surface Detection in Light Field

Flat surface detection is one of the most common geometry inferences in computer vision. In this paper we propose detecting printed photos from original scenes, which fully exploit angular information of light field and characteristics of the flat surface. Unlike previous methods, our method does not need a prior depth estimation. The algorithm rectifies the mess epipolar lines in the epipolar plane image (EPI). Then feature points are extracted from light field data and used to compute an energy ratio in the depth distribution of the scene. Based on the energy ratio, a feature vector is constructed and we obtain robust detection of flat surface. Apart from flat surface detection, the kernel rectification algorithm in our method can be expanded to inclined plane refocusing and continuous depth estimation for flat surface. Experiments on the public datasets and our collections have demonstrated the effectiveness of the proposed method.

Geometric analysis of influence of fringe directions on phase sensitivities in fringe projection profilometry.

In fringe projection profilometry, phase sensitivity is one of the important factors affecting measurement accuracy. A typical fringe projection system consists of one camera and one projector. To gain insight into its phase sensitivity, we perform in this paper a strict analysis in theory about the dependence of phase sensitivities on fringe directions. We use epipolar geometry as a tool to derive the relationship between fringe distortions and depth variations of the measured surface, and further formularize phase sensitivity as a function of the angle between fringe direction and the epipolar line. The results reveal that using the fringes perpendicular to the epipolar lines enables us to achieve the maximum phase sensitivities, whereas if the fringes have directions along the epipolar lines, the phase sensitivities decline to zero. Based on these results, we suggest the optimal fringes being circular-arc-shaped and centered at the epipole, which enables us to give the best phase sensitivities over the whole fringe pattern, and the quasi-optimal fringes, being straight and perpendicular to the connecting line between the fringe pattern center and the epipole, can achieve satisfyingly high phase sensitivities over whole fringe patterns in the situation that the epipole locates far away from the fringe pattern center. The experimental results demonstrate that our analyses are practical and correct, and that our optimized fringes are effective in improving the phase sensitivities and, further, the measurement accuracies.

Automatic Trimap Generation and Consistent Matting for Light-Field Images

In this paper, we introduce an automatic approach to generate trimaps and consistent alpha mattes of foreground objects in a light-field image. Our method first performs binary segmentation to roughly segment a light-field image into foreground and background based on depth and color. Next, we estimate accurate trimaps through analyzing color distribution along the boundary of the segmentation using guided image filter and KL-divergence. In order to estimate consistent alpha mattes across sub-images, we utilize the epipolar plane image (EPI) where colors and alphas along the same epipolar line must be consistent. Since EPI of foreground and background are mixed in the matting area, we propagate the EPI from definite foreground/background regions to unknown regions by assuming depth variations within unknown regions are spatially smooth. Using the EPI constraint, we derive two solutions to estimate alpha when color samples along epipolar line are known, and unknown. To further enhance consistency, we refine the estimated alpha mattes by using the multi-image matting Laplacian with an additional EPI smoothness constraint. In experimental evaluations, we have created a dataset where the ground truth alpha mattes of light-field images were obtained by using the blue screen technique. A variety of experiments show that our proposed algorithm produces both visually and quantitatively high-quality alpha mattes for light-field images.

An Image Matching Algorithm Integrating Global SRTM and Image Segmentation for Multi-Source Satellite Imagery

This paper presents a novel image matching method for multi-source satellite images, which integrates global Shuttle Radar Topography Mission (SRTM) data and image segmentation to achieve robust and numerous correspondences. This method first generates the epipolar lines as a geometric constraint assisted by global SRTM data, after which the seed points are selected and matched. To produce more reliable matching results, a region segmentation-based matching propagation is proposed in this paper, whereby the region segmentations are extracted by image segmentation and are considered to be a spatial constraint. Moreover, a similarity measure integrating Distance, Angle and Normalized Cross-Correlation (DANCC), which considers geometric similarity and radiometric similarity, is introduced to find the optimal correspondences. Experiments using typical satellite images acquired from Resources Satellite-3 (ZY-3), Mapping Satellite-1, SPOT-5 and Google Earth demonstrated that the proposed method is able to produce reliable and accurate matching results.

A EUCLIDEAN FORMULATION OF INTERIOR ORIENTATION COSTRAINTS IMPOSED BY THE FUNDAMENTAL MATRIX

Epipolar geometry of a stereopair can be expressed either in 3D, as the relative orientation (i.e. translation and rotation) of two bundles of optical rays in case of calibrated cameras or, in case of unclalibrated cameras, in 2D as the position of the epipoles on the image planes and a projective transformation that maps points in one image to corresponding epipolar lines on the other. The typical coplanarity equation describes the first case; the Fundamental matrix describes the second. It has also been proven in the Computer Vision literature that 2D epipolar geometry imposes two independent constraints on the parameters of camera interior orientation. In this contribution these constraints are expressed directly in 3D Euclidean space by imposing the equality of the dihedral angle of epipolar planes defined by the optical axes of the two cameras or by suitably chosen corresponding epipolar lines. By means of these constraints, new closed form algorithms are proposed for the estimation of a variable or common camera constant value given the fundamental matrix and the principal point position of a stereopair.

A EUCLIDEAN FORMULATION OF INTERIOR ORIENTATION COSTRAINTS IMPOSED BY THE FUNDAMENTAL MATRIX

Abstract. Epipolar geometry of a stereopair can be expressed either in 3D, as the relative orientation (i.e. translation and rotation) of two bundles of optical rays in case of calibrated cameras or, in case of unclalibrated cameras, in 2D as the position of the epipoles on the image planes and a projective transformation that maps points in one image to corresponding epipolar lines on the other. The typical coplanarity equation describes the first case; the Fundamental matrix describes the second. It has also been proven in the Computer Vision literature that 2D epipolar geometry imposes two independent constraints on the parameters of camera interior orientation. In this contribution these constraints are expressed directly in 3D Euclidean space by imposing the equality of the dihedral angle of epipolar planes defined by the optical axes of the two cameras or by suitably chosen corresponding epipolar lines. By means of these constraints, new closed form algorithms are proposed for the estimation of a variable or common camera constant value given the fundamental matrix and the principal point position of a stereopair.

Automated architectural reconstruction using reference planes under convex optimization

In this paper, a method for the automated reconstruction of architectures from two views of a monocular camera is proposed. While this research topic has been studied over the last few decades, we contend that a satisfactory approach has not yet been devised. Here, a new method to solve the same problem with several points of novelty is proposed. First, reference planes are automatically detected using color, straight lines, and edge/vanishing points. This approach is quite robust and fast even when different views and complicated conditions are presented. Second, the camera pose and 3D points are accurately estimated by a two-view geometry constraint in the convex optimization approach. It has been demonstrated that camera rotations are appropriately estimated, while translations induce a significant error in short baseline images. To overcome this problem, we rely only on reference planes to estimate image homography instead of using the conventional camera pose estimation method. Thus, the problem associated with short baseline images is adequately addressed. The 3D points and translation are then simultaneously triangulated. Furthermore, both the homography and 3D point triangulation are computed via the convex optimization approach. The error of back-projection and measured points is minimized in L ∞-norm so as to overcome the local minima problem of the canonical L 2-norm method. Consequently, extremely accurate homography and point clouds can be achieved with this scheme. In addition, a robust plane fitting method is introduced to describe a scene. The corners are considered as properties of the plane in order to limit the boundary. Thus, it is necessary to find the exact corresponding corner positions by searching along the epipolar line in the second view. Finally, the texture of faces is mapped from 2D images to a 3D plane. The simulation results demonstrate the effectiveness of the proposed method for scenic images in an outdoor environment.

An External Parameter Optimize Method for 3D Optical Measurement System

A simple and effective external parameter calibration and optimize method based on epipolar geometry for improving the accuracy and stabilization of 3D optical measurement system are present. First, the internal parameters of two cameras are calibrated using the planar calibration method. Then using the parameters as initial value calibrates the external parameter of the 3D optical measurement system in every view. Optimization result is computed through minimizing epipolar line error of the same pair of points in every view. Experimental results show that the measurement precision is high and stable in the measurement volume.

Fundamental Matrix of a Stereo Pair, with A Contrario Elimination of Outliers

In a stereo image pair, the fundamental matrix encodes the rigidity constraint of the scene. It combines the internal parameters of both cameras (which can be the same) and their relative position and orientation. It associates to image points in one view the so-called epipolar line in the other view, which is the locus of projection of the same 3D point, whose particular position on the straight line is determined by its depth. Reducing the correspondence search to a 1D line instead of the 2D image is a large benefit enabling the computation of the dense 3D scene. The estimation of the matrix depends on at least seven pairs of corresponding points in the images. The algorithm discarding outliers presented here is a variant of the classical RANSAC (RANdom SAmple Consensus) based on a contrario methodology and proposed first by Moisan and Stival in 2004 under the name ORSA. The distinguishing feature of this algorithm compared to other RANSAC variants is that the measure of validity of a set of point pairs is not its sheer number, but a combination of this number and the geometric precision of the points.

A Multi-View Dense Point Cloud Generation Algorithm Based on Low-Altitude Remote Sensing Images

This paper presents a novel multi-view dense point cloud generation algorithm based on low-altitude remote sensing images. The proposed method was designed to be especially effective in enhancing the density of point clouds generated by Multi-View Stereo (MVS) algorithms. To overcome the limitations of MVS and dense matching algorithms, an expanded patch was set up for each point in the point cloud. Then, a patch-based Multiphoto Geometrically Constrained Matching (MPGC) was employed to optimize points on the patch based on least square adjustment, the space geometry relationship, and epipolar line constraint. The major advantages of this approach are twofold: (1) compared with the MVS method, the proposed algorithm can achieve denser three-dimensional (3D) point cloud data; and (2) compared with the epipolar-based dense matching method, the proposed method utilizes redundant measurements to weaken the influence of occlusion and noise on matching results. Comparison studies and experimental results have validated the accuracy of the proposed algorithm in low-altitude remote sensing image dense point cloud generation.

Achieving invariance to the temporal offset of unsynchronized cameras through epipolar point-line triangulation

This paper addresses the stereo camera synchronization problem for dynamic scenes by proposing a new triangulation method which is invariant to the temporal offset of the cameras. Contrary to spatio-temporal alignment approaches, our method estimates the correct positions of the tracked points without explicitly estimating the temporal offset of the cameras. The method relies on epipolar geometry. In the presence of a temporal delay, a tracked point does not lie on its corresponding epipolar line, thereby inducing triangulation errors. We propose to solve this problem by intersecting its motion trajectory with the corresponding epipolar line. The method we propose does not require calibrated cameras, since it can rely on the fundamental matrix. One major advantage of our approach is that the temporal offset can change throughout the sequence. Evaluated with synthetic and real data, our method proves to be stable and robust to varying temporal offset as well as complex motions.

Fast stereo matching based on probability assignment and ORB operator

A stereo matching algorithm based on probability assignment is proposed to meet the real-time demand of stereo matching using SURF(Speeded Up Robust Features). In order to accelerate search of matching points, probability assignment of feature points is acquired based on the correlation between fuzzy clustering and probability assignment, as well as the necessary conditions of information entropy; then, when extracting ORB feature points, a data set is used to record the sequence of feature points after decreasing the edge effect; epipolar line, uniqueness and sequence constraints narrow down the search space; the Hamming distance between descriptors are calculated and the nearest neighbor matching becomes the stereo matching result. Experimental results show that the proposed algorithm accelerates matching and maintains high accuracy, even when applied to non-standard pictures.

AN INTEGRATED RANSAC AND GRAPH BASED MISMATCH ELIMINATION APPROACH FOR WIDE-BASELINE IMAGE MATCHING

Abstract. In this paper we propose an integrated approach in order to increase the precision of feature point matching. Many different algorithms have been developed as to optimizing the short-baseline image matching while because of illumination differences and viewpoints changes, wide-baseline image matching is so difficult to handle. Fortunately, the recent developments in the automatic extraction of local invariant features make wide-baseline image matching possible. The matching algorithms which are based on local feature similarity principle, using feature descriptor as to establish correspondence between feature point sets. To date, the most remarkable descriptor is the scale-invariant feature transform (SIFT) descriptor , which is invariant to image rotation and scale, and it remains robust across a substantial range of affine distortion, presence of noise, and changes in illumination. The epipolar constraint based on RANSAC (random sample consensus) method is a conventional model for mismatch elimination, particularly in computer vision. Because only the distance from the epipolar line is considered, there are a few false matches in the selected matching results based on epipolar geometry and RANSAC. Aguilariu et al. proposed Graph Transformation Matching (GTM) algorithm to remove outliers which has some difficulties when the mismatched points surrounded by the same local neighbor structure. In this study to overcome these limitations, which mentioned above, a new three step matching scheme is presented where the SIFT algorithm is used to obtain initial corresponding point sets. In the second step, in order to reduce the outliers, RANSAC algorithm is applied. Finally, to remove the remained mismatches, based on the adjacent K-NN graph, the GTM is implemented. Four different close range image datasets with changes in viewpoint are utilized to evaluate the performance of the proposed method and the experimental results indicate its robustness and capability.

AUTOMATED DSM EXTRACTION FROM UAV IMAGES AND PERFORMANCE ANALYSIS

Abstract. As technology evolves, unmanned aerial vehicles (UAVs) imagery is being used from simple applications such as image acquisition to complicated applications such as 3D spatial information extraction. Spatial information is usually provided in the form of a DSM or point cloud. It is important to generate very dense tie points automatically from stereo images. In this paper, we tried to apply stereo image-based matching technique developed for satellite/aerial images to UAV images, propose processing steps for automated DSM generation and to analyse the possibility of DSM generation. For DSM generation from UAV images, firstly, exterior orientation parameters (EOPs) for each dataset were adjusted. Secondly, optimum matching pairs were determined. Thirdly, stereo image matching was performed with each pair. Developed matching algorithm is based on grey-level correlation on pixels applied along epipolar lines. Finally, the extracted match results were united with one result and the final DSM was made. Generated DSM was compared with a reference DSM from Lidar. Overall accuracy was 1.5 m in NMAD. However, several problems have to be solved in future, including obtaining precise EOPs, handling occlusion and image blurring problems. More effective interpolation technique needs to be developed in the future.

On the Accuracy Potential in Underwater/Multimedia Photogrammetry.

Underwater applications of photogrammetric measurement techniques usually need to deal with multimedia photogrammetry aspects, which are characterized by the necessity of handling optical rays that are refracted at interfaces between optical media with different refractive indices according to Snell’s Law. This so-called multimedia geometry has to be incorporated into geometric models in order to achieve correct measurement results. The paper shows a flexible yet strict geometric model for the handling of refraction effects on the optical path, which can be implemented as a module into photogrammetric standard tools such as spatial resection, spatial intersection, bundle adjustment or epipolar line computation. The module is especially well suited for applications, where an object in water is observed by cameras in air through one or more planar glass interfaces, as it allows for some simplifications here. In the second part of the paper, several aspects, which are relevant for an assessment of the accuracy potential in underwater/multimedia photogrammetry, are discussed. These aspects include network geometry and interface planarity issues as well as effects caused by refractive index variations and dispersion and diffusion under water. All these factors contribute to a rather significant degradation of the geometric accuracy potential in underwater/multimedia photogrammetry. In practical experiments, a degradation of the quality of results by a factor two could be determined under relatively favorable conditions.

영상 내 건설인력 위치 추적을 위한 등극선 기하학 기반의 개체 매칭 기법

Vision-based tracking has been proposed as a means to efficiently track a large number of construction resources operating in a congested site. In order to obtain 3D coordinates of an object, it is necessary to employ stereo-vision theories. Detecting and tracking of multiple objects require an entity matching process that finds corresponding pairs of detected entities across the two camera views. This paper proposes an efficient way of entity matching for tracking of construction workers. The proposed method basically uses epipolar geometry which represents the relationship between the two fixed cameras. Each pixel coordinate in a camera view is projected onto the other camera view as an epipolar line. The proposed method finds the matching pair of a worker entity by comparing the proximity of the all detected entities in the other view to the epipolar line. Experimental results demonstrate its suitability for automated entity matching for 3D vision-based tracking of construction workers.

A MODULAR GEOMETRIC MODEL FOR UNDERWATER PHOTOGRAMMETRY

Abstract. Underwater applications of photogrammetric measurement techniques usually need to deal with multimedia photogrammetry aspects, which are characterized by the necessity of handling optical rays that are broken at interfaces between optical media with different refrative indices according to Snell’s Law. This so-called multimedia geometry has to be incorporated into geometric models in order to achieve correct measurement results. The paper shows a flexible yet strict geometric model for the handling of refraction effects on the optical path, which can be implemented as a module into photogrammetric standard tools such as spatial resection, spatial intersection, bundle adjustment or epipolar line computation. The module is especially well suited for applications, where an object in water is observed by cameras in air through one or more plane parallel glass interfaces, as it allows for some simplifications here.

Volume Measurement of Deposits Based on Stereo Vision and SURF

Based on the stereo vision principle, a volume measurement method of deposit is realized. The principle can be applied for the measurement of deposits with messy texture, such as rubbish, construction waste, and small mounds. After camera calibration and image rectification, the disparity of key points are calculated using SURF feature point matching, and the mismatched point pairs are kick out by the epipolar line constrain. Delaunay triangular meshes are created on the final point pairs. Then the volumes of deposit heap are calculated according to the pinhole imaging model. Experiments show that, despite the use of ordinary USB camera, the result is still relatively accurate. If cameras with high quality and high definition are used, the method can be applied to practical applications.