Color-coded pattern for non metric camera calibration

This paper considers the joint geometric and photometric image registration problem. The inverse compositional (IC) algorithm and the efficient second-order minimization (ESM) algorithm are two typical efficient methods applied to the geometric registration problem. Their efficiency stems from the utilization of the group structure of geometric transformations. To allow for photometric variations, the dual IC algorithm (DIC) proposed by Bartoli performs joint geometric and photometric image registration by extending the IC algorithm. The group structures of both geometric and photometric transformations are exploited. Despite the robustness to large photometric variations, DIC is vulnerable to large geometric deformations. The ESM algorithm is extended by Silveira et al. to address photometric variations. In their approach, the photometric transformations are modeled in Euclidean space. Their approach is robust to relatively large geometric and photometric transformations; however, it is not efficient for large photometric variations. We propose a new efficient and robust image registration method by exploiting the non-Euclidean Lie group structure of joint geometric and photometric transformations for both grayscale and color images. The image registration is formulated as a nonlinear least squares problem. In our method, the geometric and photometric transformations are jointly parameterized by their corresponding Lie algebras. Based on this parameterization approach, the second-order approximation strategy of ESM is employed to optimize the joint geometric and photometric parameters. The error function in the nonlinear least squares problem is approximated by a second-order Taylor expansion with respect to joint geometric and photometric parameters without computing the Hessian matrix. For further efficiency, independent convergence criteria for geometric and photometric parameters are used in the iterative optimization process. The superiority of our proposed method over the previous methods, in terms of efficiency, accuracy, and robustness, is demonstrated through extensive experiments on synthetic and real data.

The problem of sequential estimation of the exterior orientation of imaging sensors and the three‐dimensional environment reconstruction in real time is commonly known as visual simultaneous localisation and mapping (vSLAM). Omnidirectional optical sensors have been increasingly used in vSLAM solutions, mainly for providing a wider view of the scene, allowing the extraction of more features. However, dealing with unmodelled points in the hyperhemispherical field poses challenges, mainly due to the complex lens geometry entailed in the image formation process. To address these challenges, the use of rigorous photogrammetric models that appropriately handle the geometry of fisheye lens cameras can overcome these challenges. Thus, this study presents a real‐time vSLAM approach for omnidirectional systems adapting ORB‐SLAM with a rigorous projection model (equisolid‐angle). The implementation was conducted on the Nvidia Jetson TX2 board, and the approach was evaluated using hyperhemispherical images captured by a dual‐fisheye camera (Ricoh Theta S) embedded into a mobile backpack platform. The trajectory covered a distance of 140 m, with the approach demonstrating accuracy better than 0.12 m at the beginning and achieving metre‐level accuracy at the end of the trajectory. Additionally, we compared the performance of our proposed approach with a generic model for fisheye lens cameras.

Abstract. Fisheye lens cameras are becoming increasingly popular for vSLAM applications due to their wide field of view (FoV), providing more features to be tracked in a single image shot. However, the complex lens geometry involved in the image formation process still limits their full potential, especially when points in the hyperhemispherical field are unmodeled. In this paper, we compare two adaptations of ORB-SLAM for fisheye lens cameras, considering the use of the rigorous projection model (equisolid-angle) versus the use of the generic projection model (EUCM). The ORB-SLAM versions were adapted for real-time processing on the Nvidia Jetson TX2 board. The experiment was conducted using hyperhemispherical images obtained with a Ricoh Theta S camera. Our results showed that the trajectory estimated with the equisolid-angle ORB-SLAM had smaller discrepancies, compared to the reference trajectory, than the EUCM ORB-SLAM. This suggests that a rigorous photogrammetric model with a suitable treatment of hyperhemispherical points is beneficial for trajectory estimation.

The vision system is always applied to industrial in many applications. There are many applications that integrated with industrial robots by using vision system such as detecting position, and matching objects. Therefore, it is necessary to transfer from camera position in pixel coordination to robot position in world coordination. After calibration, robot can know the objects position and orientation, which be detected from vision system. In basic camera calibration to robot coordination, it needs at least 3 points of camera and robot positions. However, the accuracy from the algorithm will be low from human error, internal hardware such as intrinsic, extrinsic camera parameters, and installation error (Ex: tilt etc.). Thus, origianally, the process of calibration must have 3 steps, intrinsic camera calibration, extrinsic camera calibration and camera to robot base calibration. Moreover, the basic calibration cannot calculate TCP offset (Tool coordinate point offset). If tool has been installed to robot, the robot must change final position from MIF (Mechanical interface) to TIF (Tool interface). Thus, user must calculate the TCP offset before calculate camera calibration. However, there are many processes of calibration. In this paper, it will show the 9 points calibration's algorithm, which be applied from other applications of vision system. The paper will explain step-by-step how to solve the equation and how to apply with SCARA robot. Moreover, the paper will explain not only camera calibration, but also SCARA robot and TCP offset calculation.

Fish-eye lens provide ultra-wide field of view, however cause obvious image distortion. The paper provides a close range fisheye lens camera model which reflects the relationship between imaging distortion and object distance. In the model, three camera images shot at various object distances in close range are sufficient to achieve the camera calibration. The experiment demonstrated that the accuracy of image correction was obviously promoted because influence of object distances was well considered, especially in macroshot.

With the developing technology in recent years, there has been a significant increase in 3D spatial data needs and data production. 3D spatial data occupy an essential place in almost every field. As a result, there is a need for 3D spatial data production and the produced data's reliability. Like all spatial measuring equipment, cameras, which are data production units used in photogrammetry, must also be calibrated to produce reliable data. Calibration for non-metric cameras is critical to ensure that the measurements made are of high accuracy and at standards acceptable to everyone. In this study, to determine the factors affecting the calibration, terrestrial photogrammetric images were taken with a Canon EOS 600D non-metric camera with different point diameters on the calibration paper. These images were evaluated in the camera calibration software, and the photogrammetric result accuracies were investigated. The effects of the point diameters on the camera calibration paper on the calibration results have been observed. For both calibration papers, A4, A3, A0 sizes were printed and calibrated. As a result of the printouts, as can be predicted, the largest point sizes are A0, which has the largest paper size, and according to the results, it was seen that the highest accuracy was achieved in the A0 dimension. The relationship between the accuracy obtained in the calibration process and the point diameters in the test area was examined in this study. When the 144-point test area presented by the software is printed in A0 size, the point diameters were measured as 1.00 cm. In this study, calibration processes were performed as 1.00 cm, 1.20 cm, 1.40 cm, 1.60 cm, 1.80 cm and 2.00 cm by enlarging the point diameters by 20%, and more accurate results were obtained more easily than other paper sizes and calibration methods. As a result, since the calibration of the device to be used in the studies directly affects the accuracy of the model to be obtained at the end of the study, it is necessary to take the maximum accuracy obtained in the calibration process. In this research, the calibration process's highest accuracy is aimed and how the calibration process can be performed more effortless and more accurately by increasing the size of the point diameter on the test area.

Multicamera calibration is an important technique for generating free-view videos. By arranging multiple cameras in a scene after camera calibration and image processing, a multidimensional viewing experience can be presented to the audience. To address the problem that low texture cannot be robustly self-calibrated in common sports scenes when placing artificial markers or towers in the calibration process is impractical, this article proposes a robust multicamera calibration method based on sequence feature matching and fusion. Additionally, to validate the effectiveness of the proposed calibration algorithm, a virtual axis fast bullet-time synthesis algorithm is proposed for generating a free-view video. First, camera self-calibration is performed in low-texture situations by fusing dynamic objects in time series to enrich geometric constraints in scenes without the use of calibration panels or additional artificial markers. Second, a virtual-axis bullet-time video synthesis method based on the calibration result is proposed. In the calibrated multicamera scenario, a fast bullet-time video is generated by constructing a virtual axis. Qualitative and quantitative experiments in comparison with a state-of-the-art calibration method demonstrate the validity and robustness of the proposed calibration algorithm for free-view video synthesis tasks.

This paper concerns the incorporation of geometric information in camera calibration and 3D modeling. Using geometric constraints enables more stable results and allows us to perform tasks with fewer images. Our approach is motivated and developed within a framework of semi-automatic 3D modeling, where the user defines geometric primitives and constraints between them. It is based on the observation that constraints, such as coplanarity, parallelism, or orthogonality, are often embedded intuitively in parallelepipeds. Moreover, parallelepipeds are easy to delineate by a user and are well adapted to model the main structure of, e.g., architectural scenes. In this paper, first a duality that exists between the shape parameters of a parallelepiped and the intrinsic parameters of a camera is described. Then, a factorization-based algorithm exploiting this relation is developed. Using images of parallelepipeds, it allows us to simultaneously calibrate cameras, recover shapes of parallelepipeds, and estimate the relative pose of all entities. Besides geometric constraints expressed via parallelepipeds, our approach simultaneously takes into account the usual self-calibration constraints on cameras. The proposed algorithm is completed by a study of the singular cases of the calibration method. A complete method for the reconstruction of scene primitives that are not modeled by parallelepipeds is also briefly described. The proposed methods are validated by various experiments with real and simulated data, for single-view as well as multiview cases.

This research aimed to optimize the camera calibration process by identifying the optimal distance and angle for capturing checkered board images, with a specific focus on understanding the factors that influence the reprojection error (ϵRP). The objective was to improve calibration efficiency by exploring the impacts of distance and orientation factors and the feasibility of independently manipulating these factors. The study employed Zhang's camera calibration method, along with the 2k full-factorial analysis method and the Latin Hypercube Sampling (LHS) method, to identify the optimal calibration parameters. Three calibration methods were devised: calibration with distance factors (D, H, V), orientation factors (R, P, Y), and the combined two influential factors from both sets of factors. The calibration study was carried out with three different stereo cameras. The results indicate that D is the most influential factor, while H and V are nearly equally influential for method A; P and R are the two most influential orientation factors for method B. Compared to Zhang's method alone, on average, methods A, B, and C reduce ϵRP by 25%, 24%, and 34%, respectively. However, method C requires about 10% more calibration images than methods A and B combined. For applications where lower value of ϵRP is required, method C is recommended. This study provides valuable insights into the factors affecting ϵRP in calibration processes. The proposed methods can be used to improve the calibration accuracy for stereo cameras for the applications in object detection and ranging. The findings expand our understanding of camera calibration, particularly the influence of distance and orientation factors, making significant contributions to camera calibration procedures.

The accuracy of the results in stereo image-based 3D reconstruction is very sensitive to the intrinsic and extrinsic camera parameters determined during camera calibration. The existing camera calibration algorithms induce a significant amount of error due to poor estimation accuracies in camera parameters when they are used for long-range scenarios such as mapping civil infrastructure. This leads to unusable results, and may result in the failure of the whole reconstruction process. This paper proposes a novel way to address this problem. Instead of incremental improvements to the accuracy typically induced by new calibration algorithms, the authors hypothesize that a set of multiple calibrations created by videotaping a moving calibration pattern along a specific path can increase overall calibration accuracy. This is achieved by using conventional camera calibration algorithms to perform separate estimations for some predefined distance values. The result, which is a set of camera parameters for different distances, is then uniquely input in the Structure from Motion process to improve the Euclidean accuracy of the reconstruction. The proposed method has been tested on infrastructure scenes and the experimental analyses indicate the improved performance.

The current study aimed to quantify the value of color spaces and channels as a potential superior replacement for standard grayscale images, as well as the relative performance of open-source detectors and descriptors for general feature-based image registration purposes, based on a large benchmark dataset. The public dataset UDIS-D, with 1106 diverse image pairs, was selected. In total, 21 color spaces or channels including RGB, XYZ, Y'CrCb, HLS, L*a*b* and their corresponding channels in addition to grayscale, nine feature detectors including AKAZE, BRISK, CSE, FAST, HL, KAZE, ORB, SIFT, and TBMR, and 11 feature descriptors including AKAZE, BB, BRIEF, BRISK, DAISY, FREAK, KAZE, LATCH, ORB, SIFT, and VGG were evaluated according to reprojection error (RE), root mean square error (RMSE), structural similarity index measure (SSIM), registration failure rate, and feature number, based on 1,950,984 image registrations. No meaningful benefits from color space or channel were observed, although XYZ, RGB color space and L* color channel were able to outperform grayscale by a very minor margin. Per the dataset, the best-performing color space or channel, detector, and descriptor were XYZ/RGB, SIFT/FAST, and AKAZE. The most robust color space or channel, detector, and descriptor were L*a*b*, TBMR, and VGG. The color channel, detector, and descriptor with the most initial detector features and final homography features were Z/L*, FAST, and KAZE. In terms of the best overall unfailing combinations, XYZ/RGB+SIFT/FAST+VGG/SIFT seemed to provide the highest image registration quality, while Z+FAST+VGG provided the most image features.

Fish-eye lenses are convenient in such applications where a very wide angle of view is needed, but their use for measurement purposes has been limited by the lack of an accurate, generic, and easy-to-use calibration procedure. We hence propose a generic camera model, which is suitable for fish-eye lens cameras as well as for conventional and wide-angle lens cameras, and a calibration method for estimating the parameters of the model. The achieved level of calibration accuracy is comparable to the previously reported state-of-the-art.

II.3 - ASPECTS OF INVARIANT PATTERN AND OBJECT RECOGNITION

We propose an algorithm for finding out the single or multiple camera calibration planar coded patterns from an image with a complicate background, and provide a kind of patterns design for multi-pattern calibration accordingly. Until recently, the camera calibration planar pattern recognition methods proposed are mainly the recognition of the calibration pattern from images with a single pattern and a simple background. Our approach expands the scope of application of object reconstruction in two aspects. The first is that the reconstruction can be operated in a natural environment instead of an arranged scene. The second is that the large objects, or objects immovable, can be reconstructed conveniently and accurately. We begin by contour detecting with geometric constraints in the given image. Then we build the contours' adjacent relations by generating planar points set Delaunay triangular subdivision. Thirdly, we obtain the correspondence between image contours and pattern elements based on the Delaunay net, pattern geometric constraints and perspective projection invariants. We demonstrate the validity of our approach by providing recognition results on real scene images.

The authors present a technique for 3-D camera calibration using a pyramid calibration frame. The focus is on camera calibration and correspondence analysis. A rule is presented to set up precorrespondence automatically before the calibration process begins. To strengthen practicability, a careful discussion on the change of the parameters of a calibration matrix when the camera moves is provided. Five constraints are integrated into a feature-point-based matching algorithm: epipolar, common view, uniqueness, compatibility, and model. The calibration process equips the stereo system with common view and model constraints. Uniqueness and compatibility constraints have their origins in Marr's (1982) stereo theory. Based on the calibration process, a stereo algorithm synthesizes general constraints and is applied to a real situation. The experimental results have shown its practical use in a general situation. >