Affine Camera Research Articles

HIGH-RESOLUTION SATELLITE IMAGE TRIFOCAL TENSOR SOLUTION

Abstract. This study presents an efficient algorithm for solving the affine trifocal tensor problem in high-resolution satellite imagery. Because the imaging mode of high-resolution satellites is usually a line-center projection and the field of view is small, it can be approximated as an affine camera for orientation. However, the nature of high-resolution satellite trifocal tensors has not been effectively studied or discussed. To address the shortcomings of the current trifocal tensor direct linear algorithm, this study proposes a minimally parameterized trifocal calculation method that can stably calculate the affine trifocal tensors of high-resolution satellites. Efficacy was demonstrated using ZY3-02 and Pleiades-Neo data.

Autocalibration method for scanning electron microscope using affine camera model

This paper deals with the task of autocalibration of scanning electron microscope (SEM), which is a technique allowing to compute camera motion and intrinsic parameters. In contrast to classical calibration, which implies the use of a calibration object and is known to be a tedious and rigid operation, auto- or selfcalibration is performed directly on the images acquired for the visual task. As autocalibration represents an optimization problem, all the steps contributing to the success of the algorithm are presented: formulation of the cost function incorporating metric constraints, definition of bounds, regularization, and optimization algorithm. The presented method allows full estimation of camera matrices for all views in the sequence. It was validated on virtual images as well as on real SEM images (pollen grains, cutting tools, etc.). The results show a good convergence range and low execution time, notably compared to classical methods, and even more in the context of the calibration of SEM.

Quantitative 3D Reconstruction from Scanning Electron Microscope Images Based on Affine Camera Models.

Scanning electron microscopes (SEMs) are versatile imaging devices for the micro- and nanoscale that find application in various disciplines such as the characterization of biological, mineral or mechanical specimen. Even though the specimen’s two-dimensional (2D) properties are provided by the acquired images, detailed morphological characterizations require knowledge about the three-dimensional (3D) surface structure. To overcome this limitation, a reconstruction routine is presented that allows the quantitative depth reconstruction from SEM image sequences. Based on the SEM’s imaging properties that can be well described by an affine camera, the proposed algorithms rely on the use of affine epipolar geometry, self-calibration via factorization and triangulation from dense correspondences. To yield the highest robustness and accuracy, different sub-models of the affine camera are applied to the SEM images and the obtained results are directly compared to confocal laser scanning microscope (CLSM) measurements to identify the ideal parametrization and underlying algorithms. To solve the rectification problem for stereo-pair images of an affine camera so that dense matching algorithms can be applied, existing approaches are adapted and extended to further enhance the yielded results. The evaluations of this study allow to specify the applicability of the affine camera models to SEM images and what accuracies can be expected for reconstruction routines based on self-calibration and dense matching algorithms.

Calibration routine for a telecentric stereo vision system considering affine mirror ambiguity

A robust calibration approach for a telecentric stereo camera system for three-dimensional (3-D) surface measurements is presented, considering the effect of affine mirror ambiguity. By optimizing the parameters of a rigid body transformation between two marker planes and transforming the two-dimensional (2-D) data into one coordinate frame, a 3-D calibration object is obtained, avoiding high manufacturing costs. Based on the recent contributions in the literature, the calibration routine consists of an initial parameter estimation by affine reconstruction to provide good start values for a subsequent nonlinear stereo refinement based on a Levenberg–Marquardt optimization. To this end, the coordinates of the calibration target are reconstructed in 3-D using the Tomasi–Kanade factorization algorithm for affine cameras with Euclidean upgrade. The reconstructed result is not properly scaled and not unique due to affine ambiguity. In order to correct the erroneous scaling, the similarity transformation between one of the 2-D calibration plane points and the corresponding 3-D points is estimated. The resulting scaling factor is used to rescale the 3-D point data, which then allows in combination with the 2-D calibration plane data for a determination of the start values for the subsequent nonlinear stereo refinement. As the rigid body transformation between the 2-D calibration planes is also obtained, a possible affine mirror ambiguity in the affine reconstruction result can be robustly corrected. The calibration routine is validated by an experimental calibration and various plausibility tests. Due to the usage of a calibration object with metric information, the determined camera projection matrices allow for a triangulation of correctly scaled metric 3-D points without the need for an individual camera magnification determination.

Motion Segmentation Based on Model Selection in Permutation Space for RGB Sensors.

Motion segmentation is aimed at segmenting the feature point trajectories belonging to independently moving objects. Using the affine camera model, the motion segmentation problem can be viewed as a subspace clustering problem—clustering the data points drawn from a union of low-dimensional subspaces. In this paper, we propose a solution for motion segmentation that uses a multi-model fitting technique. We propose a data grouping method and a model selection strategy for obtaining more distinguishable data point permutation preferences, which significantly improves the clustering. We perform extensive testing on the Hopkins 155 dataset, and two real-world datasets. The experimental results illustrate that the proposed method can deal with incomplete trajectories and the perspective effect, comparing favorably with the current state of the art.

A magnification-continuous calibration method for SEM-based nanorobotic manipulation systems.

Calibration for scanning electron microscope (SEM) based nanorobotic manipulation systems is important and difficult. Most current calibration methods are cumbersome because they require customized high precision calibration boards and repeated calibration procedures in different magnifications. This paper presents a convenient magnification-continuous calibration method with high precision for SEM-based nanorobotic manipulation systems. The projection matrix containing a continuous magnification factor is obtained by modifying the affine camera model. This facilitates the simplification of the parameter computing process. Movement features are used to align the moving axes of micropositioning stages and calibrate the system, which benefits for the realization of efficient automatic calibration. Three experiments are carried out, and the results demonstrate that the proposed method is effective and practical for calibrating SEM-based nanorobotic manipulation systems under a wide range of continuous magnifications. Experiments also confirm that high precision measurements can be conducted in different magnifications with only once calibration and the relative error is within 1%.

Optimization of Line Extraction Based on an Affine Camera Model

Line feature constitutes important geometric structure information in image processing and is significant in visual navigation and 3-D structure contour extraction of 3-D objects. The current state-of-the-art line extraction methods exhibit fast extracting speed and good extracting results. However, most line extraction methods face a problem, wherein a long line is easily fragmented. Thus, a long line is broken into several short segments due to local small changes or disturbances. To obtain longer and more useful line segments without the aid of external information, this paper proposes an improved line extraction method called affine-lines, which is based on an affine camera model. First, the affine camera model is used to simulate affine projections from different viewing angles, and a sequence of affine simulated images is obtained via affine transformation matrices. Subsequently, line segments are extracted from the original image and each simulated image. Each set of line segments on the simulated image is back-projected on the original image based on its corresponding inversely affine transformation matrix. Finally, the line segments on the original image are used as references to sequentially purify and optimize subsequent lines. Several defined geometric constraints are used to eliminate pseudo lines and combine short lines. Thus, fragmented lines are connected, and short lines are converted into long lines. Given several sets of close-range and aerial images, experiments are performed and compared via a state-of-the-art line extraction algorithm, called the line segment detector (LSD). The results indicate that the proposed method significantly increases the lengths of the obtained line segments, significantly reduces the fragmentation effect, and obtains more useful line segments. The extracted improved line segments are applied to line matching for building structures, and promising results are obtained.

The Effect of Using Projective Cameras on View-Independent Gait Recognition Performance

Gait as a biometric can be used to identify subjects at a distance and thus it receives great attention from the research community for security and surveillance applications. One of the challenges that affects gait recognition performance is view variation. Much work has been done to tackle this challenge. However, the majority of the work assumes that gait silhouettes are captured by affine cameras where only the height of silhouettes changes and the difference in viewing angle of silhouettes in one gait cycle is relatively small. In this paper, we analyze the variation in gait recognition performance when using silhouettes from projective cameras and from affine cameras with different distance from the center of a walking path. This is done by using 3D models of walking people in the gallery set and 2D gait silhouettes from independent (single) cameras in the probe set. Different factors that affect matching 3D human models with 2D gait silhouettes from single cameras for view-independent gait recognition are analyzed. In all experiments, we use 258 multi-view sequences belong to 46 subjects from Multi-View Soton gait dataset. We evaluate the matching performance for 12 different views using Gait Energy Image (GEI) as gait features. Then, we analyze the effect of using different camera configurations for 3D model reconstruction, the GEI from cameras with different settings, the upper and lower body parts for recognition and different GEI resolutions. The results illustrate that low recognition performance is achieved when using gait silhouettes from affine cameras while lower recognition performance is obtained when using gait silhouettes from projective cameras.

Plane-Based Resection for Metric Affine Cameras

We study the problem of resecting the metric affine camera models from at least three non-colinear point correspondences. A direct application is plane pose estimation. We consider the three most popular metric affine cameras, namely the paraperspective, weak-perspective and orthographic cameras. For each model, we give an algebraic procedure which finds the optimal solution, where by optimal we mean the global minimizer of the reprojection error under the Euclidean norm. Our algebraic procedures cover both the minimal case of three points and the redundant cases of more than three points. They always return two solutions, as the problem has a two-way ambiguity on the rotation and translation for the three cameras in the general case. The scale of the paraperspective and weak-perspective cameras is, however, recovered uniquely. The orthographic case is the most involved and has not been solved analytically in the literature. We characterize its intrinsic complexity by showing that it reduces to finding the roots of an irreducible and non-solvable by radicals sextic polynomial. The previous algorithms for the paraperspective and weak-perspective cases have singularities, while, in contrast, our algebraic procedures do not.

Factorization based structure from motion with object priors

This paper presents an efficient framework to include the information of objects position in classical multi-view geometry problems for 3D reconstruction. In particular, we present two main contributions to Structure from Motion (SfM) using factorization methods for the affine camera case. First, we introduce a method based on factorization that extends the classical 3D point cloud reconstruction based on 2D point correspondences to objects using detection correspondences. In this case, objects are approximated as quadrics in 3D (or more specifically as ellipsoids) and therefore projected as conics in 2D onto the image plane. Therefore, instead of having 2D point to point correspondences, we solve a conic to conic correspondence problem in the setting of affine factorization methods. The solution to this problem provides a 3D location/occupancy of the object together with an affine camera calibration. This is shown to be a generalisation of the standard Tomasi and Kanade factorization method with rigid objects. Secondly, we use the estimated object locations/occupancies to robustly estimate the 3D point cloud from 2D point correspondences by constructing a prior that relates 2D points locations and the positions of the object ellipsoids in 3D. This is done by recasting the problem as a probabilistic matrix factorization where the priors are not generic but truly representative of the scene structure as a composition of objects. In particular we show that by using objects to points relations, we achieve compelling results with high rate of missing data and noisy 2D data, a common occurrence when dealing with man-made textureless objects.

Calibration method for projector-camera-based telecentric fringe projection profilometry system.

By combining a fringe projection setup with a telecentric lens, a fringe pattern could be projected and imaged within a small area, making it possible to measure the three-dimensional (3D) surfaces of micro-components. This paper focuses on the flexible calibration of the fringe projection profilometry (FPP) system using a telecentric lens. An analytical telecentric projector-camera calibration model is introduced, in which the rig structure parameters remain invariant for all views, and the 3D calibration target can be located on the projector image plane with sub-pixel precision. Based on the presented calibration model, a two-step calibration procedure is proposed. First, the initial parameters, e.g., the projector-camera rig, projector intrinsic matrix, and coordinates of the control points of a 3D calibration target, are estimated using the affine camera factorization calibration method. Second, a bundle adjustment algorithm with various simultaneous views is applied to refine the calibrated parameters, especially the rig structure parameters and coordinates of the control points forth 3D target. Because the control points are determined during the calibration, there is no need for an accurate 3D reference target, whose is costly and extremely difficult to fabricate, particularly for tiny objects used to calibrate the telecentric FPP system. Real experiments were performed to validate the performance of the proposed calibration method. The test results showed that the proposed approach is very accurate and reliable.

Inextensible Non-Rigid Structure-from-Motion by Second-Order Cone Programming.

We present a global and convex formulation for the template-less 3D reconstruction of a deforming object with the perspective camera. We show for the first time how to construct a Second-Order Cone Programming (SOCP) problem for Non-Rigid Structure-from-Motion (NRSfM) using the Maximum-Depth Heuristic (MDH). In this regard, we deviate strongly from the general trend of using affine cameras and factorization-based methods to solve NRSfM, which do not perform well with complex nonlinear deformations. In MDH, the points' depths are maximized so that the distance between neighbouring points in camera space are upper bounded by the geodesic distance. In NRSfM both geodesic and camera space distances are unknown. We show that, nonetheless, given point correspondences and the camera's intrinsics the whole problem can be solved with SOCP. This is the first convex formulation for NRSfM with physical constraints. We further present how robustness and temporal continuity can be included in the formulation to handle outliers and decrease the problem size, respectively. We show with extensive experiments that our methods accurately reconstruct quasi-isometric objects from partial views under articulated and strong deformations. Compared to the previous methods, our approach gives better or similar accuracy. It naturally handles missing correspondences, non-smooth objects and is very simple to implement compared to previous methods, with only one free parameter (the neighbourhood size).

Planar Structure-from-Motion with Affine Camera Models: Closed-Form Solutions, Ambiguities and Degeneracy Analysis.

Planar Structure-from-Motion (SfM) is the problem of reconstructing a planar object or surface from a set of 2D images using motion information. The problem is well-understood with the perspective camera model and can be solved with Homography Decomposition (HD). However when the structure is small and/or viewed far from the camera the perspective effects diminish, and in the limit the projections become affine. In these situations HD fails because the problem itself becomes ill-posed. We propose a stable alternative using affine camera models. These have been used extensively to reconstruct non-planar structures, however a general, accurate and closed-form method for planar structures has been missing. The problem is fundamentally different with planar structures because the types of affine camera models one can use are more restricted and it is inherently more ambiguous and non-linear. We provide a closed-form method for the orthographic camera model that solves the general problem (three or more views with three or more correspondences and missing correspondences) and returns all metric structure solutions and corresponding camera poses. The method does not require initialisation, and optimises an objective function that is very similar to the reprojection error. In fact there is no clear benefit in refining its solutions with bundle adjustment, which is a remarkable result. We also present a new theoretical analysis that deepens our understanding of the problem. The main result is the necessary and sufficient geometric conditions for the problem to be degenerate with the orthographic camera. We also show there can exist up to two solutions for metric structure with four or more views (previously it was assumed to be unique), and we give the necessary and sufficient geometric conditions for disambiguation. Other theoretical results include showing that in the case of three images the optimal reconstruction (with respect to reprojection error) can usually be found in closed-form, and additional prior knowledge needed to solve with non-orthographic affine cameras.

Multiple View Geometry with Multiple Dynamic Affine Cameras

Multiple View Geometry with Multiple Dynamic Affine Cameras

Flexible calibration method for telecentric fringe projection profilometry systems.

A newly developed flexible calibration algorithm for telecentric 3D measurement systems is presented in this paper. We theoretically analyzed the similarities and differences between the telecentric and entocentric system. The telecentric system can be calibrated with the aid of the traditional 2D planar calibration method. An additional two-step refining process is proposed to improve the calibration accuracy effectively. With the calibration and refining algorithm, an affine camera can be calibrated with a reprojection error of 0.07 pixel. A projector with small field of view (FOV) is applied to achieve a full 3D reconstruction in our profilometry system. Experiments with a prototype demonstrate the validation and accuracy of the proposed calibration algorithm and system configuration. The reconstruction accuracy can achieve 5 µm with a measurement FOV of 28.43 mm×21.33 mm and a working distance of 110 mm.

Metric corrections of the affine camera

Given a general affine camera, we study the problem of finding the closest metric affine camera, where the latter is one of the orthographic, weak-perspective and paraperspective projection models. This problem typically arises in stratified Structure-from-Motion methods such as factorization-based methods. For each type of metric affine camera, we give a closed-form solution and its implementation through an algebraic procedure. Using our algebraic procedure, we can then provide a complete analysis of the problem’s generic ambiguity space. This also gives the means to generate the other solutions if any.

Using In-frame Shear Constraints for Monocular Motion Segmentation of Rigid Bodies

It is a well known result in the vision literature that the motion of independently moving objects viewed by an affine camera lie on affine subspaces of dimension four or less. As a result a large number of the recently proposed motion segmentation algorithms model the problem as one of clustering the trajectory data to its corresponding affine subspace. While these algorithms are elegant in formulation and achieve near perfect results on benchmark datasets, they fail to address certain very key real-world challenges, including perspective effects and motion degeneracies. Within a robotics and autonomous vehicle setting, the relative configuration of the robot and moving object will frequently be degenerate leading to a failure of subspace clustering algorithms. On the other hand, while gestalt-inspired motion similarity algorithms have been used for motion segmentation, in the moving camera case, they tend to over-segment or under-segment the scene based on their parameter values. In this paper we present a principled approach that incorporates the strengths of both approaches into a cohesive motion segmentation algorithm capable of dealing with the degenerate cases, where camera motion follows that of the moving object. We first generate a set of prospective motion models for the various moving and stationary objects in the video sequence by a RANSAC-like procedure. Then, we incorporate affine and long-term gestalt-inspired motion similarity constraints, into a multi-label Markov Random Field (MRF). Its inference leads to an over-segmentation, where each label belongs to a particular moving object or the background. This is followed by a model selection step where we merge clusters based on a novel motion coherence constraint, we call in-frame shear, that tracks the in-frame change in orientation and distance between the clusters, leading to the final segmentation. This oversegmentation is deliberate and necessary, allowing us to assess the relative motion between the motion models which we believe to be essential in dealing with degenerate motion scenarios.We present results on the Hopkins-155 benchmark motion segmentation dataset [27], as well as several on-road scenes where camera and object motion are near identical. We show that our algorithm is competitive with the state-of-the-art algorithms on [27] and exceeds them substantially on the more realistic on-road sequences.

Optical measuring equipments. Part I: Calibration model and uncertainty estimation

This study aims to propose and describe a procedure for calibrating the “vision subsystem” of digital optical machines, based on the affine camera model, valid for telecentric optical equipment. From this procedure, it is possible to systematise the calculation of the uncertainty of the calibration parameters associated with the “vision subsystem” by using the Monte Carlo method. The identification and characterisation of the calibration parameters of the “vision subsystem” and its associated uncertainty, obtained through this study, make it possible to characterise the metrological properties of the indicated optical equipment and ensure the metrological traceability of the measurements that have been subsequently taken.This work is divided into two parts. Part I focuses on developing a mathematical model of the “vision subsystem” and calculating the uncertainties of its parameters. Part II develops an example of measuring and calculating measurement uncertainty, based on the theoretical development of Part I.

Affine calibration based on invariable extrinsic parameters for stereo light microscope

The stereo light microscope (SLM) plays an important role in the measurement of three-dimensional geometry on the microscopic scale. We propose a fast and precise affine calibration algorithm based on the invariable extrinsic parameters for the SLM. This calibration algorithm with a free planar reference consists of three steps: first, derive the extrinsic parameters based on their invariable definition in the pinhole and affine models; second, calculate the intrinsic parameters through homography matrix; finally, refine all the model parameters by global optimization with the previous closed-form solutions as the initial values. The effectiveness of assuming a noncoaxial optical system as an affine camera is also verified to affinely model all types of SLMs. The calibration experiments show that the affine calibration is preferable for multicriteria including running time, relative positioning precision, and absolute positioning precision. With PlanApo S 1.5× and a total magnification of 3.024×, the proposed affine calibration algorithm achieves a distance error of 0.423 μm and a positioning error of 0.195 mm within 10.6 s.

3D Reconstruction Method of Bent Tube with NURBS Curve

To solve the difficulty and low efficiency of three-dimensional reconstruction of complex spatial form bent tube, a stereo vision method with NURBS curve to achieve three-dimensional reconstruction of bent tube is presented. This method first extracts central axis and the edge line of bent tube by the grayscale image, and the outer diameter of bent tube is calculated through the edge line extracted. On the premise of affine camera model, NURBS curve has been chosen as the primitive of reconstruction, and by using its affine invariance, according to the principle of binocular vision, the central axis of bent tube is reconstructed through reconstructing the control points. Finally, the three-dimensional model of bent tube is reconstructed by three-dimensional modeling technique. Experimental results show that, relative to the traditional methods, this method makes a big improvement on the efficiency and accuracy of three-dimensional reconstruction of bent tube.