Camera Calibration Process Research Articles

Real‐time displacement measurement for long‐span bridges using a compact vision‐based system with speed‐optimized template matching

AbstractThis paper introduces a new accelerating algorithm, efficient match slimmer (EMS), specifically designed to lighten computational loads of sophisticated template matching algorithms, enabling these algorithms to be effectively run on single‐board computers. Utilizing EMS in conjunction with a robust template matching algorithm, we have developed Raspberry Vision—a compact, cost‐effective, and real‐time vision‐based system. Its compactness and portability facilitate a practical measurement strategy that not only minimizes the camera‐to‐target distance but also simplifies the camera calibration process in bridge displacement monitoring, thereby enhancing measurement accuracy. The performance of the system is estimated on two operational suspension bridges. The results demonstrate that Raspberry Vision, equipped with the measurement strategy, can significantly improve the measurement accuracy in the long‐span bridge test and is also suitable for cross‐sea bridge measurements.

A Systematic Stereo Camera Calibration Strategy: Leveraging Latin Hypercube Sampling and 2k Full-Factorial Design of Experiment Methods.

This research aimed to optimize the camera calibration process by identifying the optimal distance and angle for capturing checkered board images, with a specific focus on understanding the factors that influence the reprojection error (ϵRP). The objective was to improve calibration efficiency by exploring the impacts of distance and orientation factors and the feasibility of independently manipulating these factors. The study employed Zhang's camera calibration method, along with the 2k full-factorial analysis method and the Latin Hypercube Sampling (LHS) method, to identify the optimal calibration parameters. Three calibration methods were devised: calibration with distance factors (D, H, V), orientation factors (R, P, Y), and the combined two influential factors from both sets of factors. The calibration study was carried out with three different stereo cameras. The results indicate that D is the most influential factor, while H and V are nearly equally influential for method A; P and R are the two most influential orientation factors for method B. Compared to Zhang's method alone, on average, methods A, B, and C reduce ϵRP by 25%, 24%, and 34%, respectively. However, method C requires about 10% more calibration images than methods A and B combined. For applications where lower value of ϵRP is required, method C is recommended. This study provides valuable insights into the factors affecting ϵRP in calibration processes. The proposed methods can be used to improve the calibration accuracy for stereo cameras for the applications in object detection and ranging. The findings expand our understanding of camera calibration, particularly the influence of distance and orientation factors, making significant contributions to camera calibration procedures.

Time-Series Fusion-Based Multicamera Self-Calibration for Free-View Video Generation in Low-Texture Sports Scene

Multicamera calibration is an important technique for generating free-view videos. By arranging multiple cameras in a scene after camera calibration and image processing, a multidimensional viewing experience can be presented to the audience. To address the problem that low texture cannot be robustly self-calibrated in common sports scenes when placing artificial markers or towers in the calibration process is impractical, this article proposes a robust multicamera calibration method based on sequence feature matching and fusion. Additionally, to validate the effectiveness of the proposed calibration algorithm, a virtual axis fast bullet-time synthesis algorithm is proposed for generating a free-view video. First, camera self-calibration is performed in low-texture situations by fusing dynamic objects in time series to enrich geometric constraints in scenes without the use of calibration panels or additional artificial markers. Second, a virtual-axis bullet-time video synthesis method based on the calibration result is proposed. In the calibrated multicamera scenario, a fast bullet-time video is generated by constructing a virtual axis. Qualitative and quantitative experiments in comparison with a state-of-the-art calibration method demonstrate the validity and robustness of the proposed calibration algorithm for free-view video synthesis tasks.

Razvoj SCARA robotske ruke za aplikacije za paletizaciju zasnovane na kompjuterskom vidu

This paper develops a computer vision system integrated with a SCARA robot arm to pick and place objects. A novel method to calculate the 3D coordinates of the objects from a camera is proposed. This method helps simplify the camera calibration process. It requires no knowledge of camera modeling and mathematical knowledge of coordinate transformations. The least square method will predate the Equation describing the relationship between pixel coordinates and 3D coordinates. An image processing algorithm is presented to detect objects by color or pixel intensity (thresholding method). The pixel coordinates of the objects are then converted to 3D coordinates. The inverse kinematic Equation is applied to find the joint angles of the SCARA robot. A palletizing application is implemented to test the accuracy of the proposed method. The kinematic Equation of the robot arm is presented to convert the 3D position of the objects to the robot joint angles. So, the robot moves exactly to the required positions by providing suitable rotational movements for each robot joint. The experiment results show that the robot can pick and place 27 boxes on the conveyor to the pallet with an average time of 2.8s per box. The positions of the boxes were determined with an average error of 0.5112mm and 0.6838mm in the X and Y directions, respectively.

Accuracy Assessment of Establishing 3D Real Scale Model in Close-Range Photogrammetry with Digital Camera

Three-dimensional (3D) real scale models delivered from digital photogrammetric techniques have rapidly increased to meet the requirements of many applications in different fields of daily life. This paper deals with the establishment of a 3D real scale model from a block of images (18 images) that were captured by using Canon EOS 500D digital camera to cover a test field area consisting of 90 artificial target points, 25 of them are ground control points (GCPs) while the remains are checkpoints (CPs). The analytical photogrammetric processes including the calculation of interior orientation parameters (IOPs) of the camera during the camera calibration process, exterior orientation parameters (EOPs) of the camera in each capturing, and the object space (ground) coordinates of the model are calculated simultaneously based on collinearity equation using bundle block adjustment method (BBA). Assessment and validation of the accuracy of the results is an important task in this study that was implemented to determine and analyze the errors of 3D coordinates through linear regression analysis (LRA). Root mean square error (RMSE) is the statistical parameter that was used in the statistical analysis of results. The standard error is another statistical parameter which also used to evaluate the accuracy of locations and rotation angles (EOPs) of cameras. The total RMSE (RMSE)xyz of GCPs is ± 2.530 mm while the total RMSE (RMSExyz) of CPs is ± 2.740 mm. The overall accuracy of the work is 5.000 mm.

Reinforcement learning-based calibration method for cameras with large FOV

The accuracy of camera calibration is of great importance to vision measurements. Target-based calibration methods should cover the whole field of view (FOV), which leads to complex operation in large FOVs and rely on personal experiences. To overcome the difficulty in accurate calibration, a calibration method based on reinforcement learning is proposed. First, a Markov decision process (MDP) model of the calibration procedure is established. Then, the reward function is designed, combining the requirement of calibration accuracy and the state-space constraint. Finally, the optimized target locations and poses are obtained by continuous interaction with the calibration environment based on Q-learning, which plays a key guiding role in camera calibration. Simulation experiments and real experiments are performed, which indicate that the proposed method effectively improves the success rate of large FOV camera calibration, and solves the problems of relying on personal experiences, low calibration accuracy, and poor stability caused by target placement in the camera calibration process.

Mechanical movement data acquisition method based on the multilayer neural networks and machine vision in a digital twin environment

Background: Digital twin requires virtual reality mapping and optimization iteration between physical devices and virtual models. The mechanical movement data collection of physical equipment is essential for the implementation of accurate virtual and physical synchronization in a digital twin environment. However, the traditional approach relying on PLC (programmable logic control) fails to collect various mechanical motion state data. Additionally, few investigations have used machine visions for the virtual and physical synchronization of equipment. Thus, this paper presents a mechanical movement data acquisition method based on multilayer neural networks and machine vision. Methods: Firstly, various visual marks with different colors and shapes are designed for marking physical devices. Secondly, a recognition method based on the Hough transform and histogram feature is proposed to realize the recognition of shape and color features respectively. Then, the multilayer neural network model is introduced in the visual mark location. The neural network is trained by the dropout algorithm to realize the tracking and location of the visual mark. To test the proposed method, 1000 samples were selected. Results: The experiment results shows that when the size of the visual mark is larger than 6mm, the recognition success rate of the recognition algorithm can reach more than 95%. In the actual operation environment with multiple cameras, the identification points can be located more accurately. Moreover, the camera calibration process of binocular and multi-eye vision can be simplified by the multilayer neural networks. Conclusions: This study proposes an effective method in the collection of mechanical motion data of physical equipment in a digital twin environment. Further studies are needed to perceive posture and shape data of physical entities under the multi-camera redundant shooting.

3D Shape Estimation for Smooth Surfaces Using Double Mirror System

In this paper, a low cost, single camera, double mirror system that can be built in a desktop nail printer will be described. The usage of this system is to capture an image of a fingernail and to generate the 3D shape of the nail. The nail’s depth map will be estimated from this rendered 3D nail shape. The paper will describe the camera calibration process and explain the calibration theory for this proposed system. Then a 3D reconstruction method will be introduced, as well. Experimental results will be shown in the paper, which illustrate the accuracy of the system to handle the rendering task.

The Influence of Autofocus Lenses in the Camera Calibration Process

Camera calibration is a crucial step in robotics and computer vision. Accurate camera parameters are necessary to achieve robust applications. Nowadays, camera calibration process consists of adjusting a set of data to a pin-hole model, assuming that with a reprojection error close to zero, camera parameters are correct. Since all camera parameters are unknown, computed results are considered true. However, the pin-hole model does not represent the camera behavior accurately if the autofocus is considered. Real cameras with autofocus lenses change the focal length slightly to obtain sharp objects in the image, and this feature skews the calibration result if a unique pin-hole model is computed with a constant focal length. In this article, a deep analysis of the camera calibration process is done to detect and strengthen its weaknesses when autofocus lenses are used. To demonstrate that significant errors exist in computed extrinsic parameters, the camera is mounted in a robot arm to know true extrinsic camera parameters with an accuracy under 1 mm. It is also demonstrated that errors in extrinsic camera parameters are compensated with bias in intrinsic camera parameters. Since significant errors exist with autofocus lenses, a modification of the widely accepted camera calibration method using images of a planar template is presented. A pin-hole model with distance-dependent focal length is proposed to improve the calibration process substantially.

Automatic Roadblock Identification Algorithm for Unmanned Vehicles Based on Binocular Vision

In order to improve the accuracy of automatic obstacle recognition algorithm for driverless vehicles, an automatic obstacle recognition algorithm for driverless vehicles based on binocular vision is constructed. Firstly, the relevant parameters of the camera are calibrated around the new car coordinate system to determine the corresponding obstacle position of the vehicle. At the same time, the three‐dimensional coordinates of obstacle points are obtained by binocular matching method. Then, the left and right cameras are used to capture the feature points of obstacles in the image to realize the recognition of obstacles. Finally, the experimental results show that for obstacle 1, the recognition error of the algorithm is 0.03 m; for obstacle 2, the recognition error is 0.02 m; for obstacle 3, the recognition error is 0.01 m. The algorithm has small recognition error. The vehicle coordinate system is added in the camera calibration process, which can accurately measure the relative position information between the vehicle and the obstacle.

Retracted] Pose Estimation under Visual Sensing Technology and Its Application in Art Design

The study is aimed at solving the problem of large measurement errors caused by the binocular camera in traditional 3D art design, which leads to inaccurate 3D information of the target. The contour information extraction in the process of human motion pose reconstruction is easily affected by the noise in the image. Therefore, a binocular stereo vision system is built first and it integrates image acquisition, camera calibration, and image processing. The dedistortion method is used to process the image because it can reduce errors. Second, a three‐dimensional human motion pose reconstruction model is implemented, the Gaussian template is used to remove the noise in the image frame, and the change detection template (CDM) is used to solve the problem of background “exposure” and “occlusion.” Finally, simulation experiments are designed to verify the system and model designed. Since the research on the application of pose estimation based on visual sensing technology in art design is still blank, such research has great significance and provides a reference for the research in the field. The literature analysis is used to expound and analyze the application of pose estimation based on visual sensing technology in visual communication design and environmental art design: (1) although the binocular stereo vision system causes some errors in the measurement, the overall error is controlled within 2% and the accuracy is high, which proves that it can be applied to the acquisition of three‐dimensional information of the target in art design; (2) there is a high degree of fitting between the video sequence data created by the three‐dimensional human motion pose reconstruction model designed and the real motion data, which indicates that this method has high accuracy in processing image sequences and the feasibility of applying it to human pose reconstruction in three‐dimensional art design is high; (3) through the analysis of the existing literature, it is found that most of the current visual‐based attitude assessment studies are carried out by using network cameras combined with computers, and the quality of the obtained images is low. The combination of binocular stereo sensor and attitude estimation technology can be applied to the design of advertising, animation, games, and packaging, making the behavior of virtual characters in animation and games more vivid. The combination provides convenience for the collection of environmental spatial information and object attitude information, the formulation of a design scheme, and real‐time monitoring of construction in environmental art design. The purpose of this study is to provide an important theoretical basis for the technical upgrading of art design.

Longitudinal-Scanline-Based Arterial Traffic Video Analytics with Coordinate Transformation Assisted by 3D Infrastructure Data

High-resolution vehicle trajectory data can be used to generate a wide range of performance measures and facilitate many smart mobility applications for traffic operations and management. In this paper, a Longitudinal Scanline LiDAR-Camera model is explored for trajectory extraction at urban arterial intersections. The proposed model can efficiently detect vehicle trajectories under the complex, noisy conditions (e.g., hanging cables, lane markings, crossing traffic) typical of an arterial intersection environment. Traces within video footage are then converted into trajectories in world coordinates by matching a video image with a 3D LiDAR (Light Detection and Ranging) model through key infrastructure points. Using 3D LiDAR data will significantly improve the camera calibration process for real-world trajectory extraction. The pan-tilt-zoom effects of the traffic camera can be handled automatically by a proposed motion estimation algorithm. The results demonstrate the potential of integrating longitudinal-scanline-based vehicle trajectory detection and the 3D LiDAR point cloud to provide lane-by-lane high-resolution trajectory data. The resulting system has the potential to become a low-cost but reliable measure for future smart mobility systems.

Research on Camera Calibration Method for Visual Inspection of Excavator Working Object

In order to realize the identification and positioning of the working object when the excavator works independently, it is necessary to calibrate the binocular camera to determine that the binocular is the internal and external parameters of the camera. This paper focuses on the calibration of binocular camera, and determines the transformation relationship among world coordinate system, camera coordinate system and image coordinate system based on binocular stereo vision ranging principle. The checkerboard images that meet the calibration requirements are selected by using the visual toolbox of MATLAB. The camera calibration process is carried out to determine the camera parameters to be determined. The calibration parameters are analyzed and compared to preliminarily verify the accuracy of the calibration parameters. Finally, the error source is analyzed in QT5.14.2 programming environment combined with OpenCV3.4.10, the calibration method meets the application requirements.

Welding Seam Detection and Tracking Based on Laser Vision for Robotic Arc Welding

The welding robots are widely employed in the manufacturing field. In order to measure up to the developing need of automation and intelligent production, real-time welding process sensing and controlling are necessary, in which the welding seam detection and tracking is representative task. Among the kinds of sensors applied in the robotic welding process, vision sensors is one of the most widely used for automatic welding seam tracking and quality control in robotic arc welding. This paper is to present a new framework and the data process algorithm of the sensor. A novel two-step calibration approach is proposed in this paper based on the “Calibration toolbox in Matlab”, the proposed approach acquires the laser light plane by project the linear stripe on the calibration pattern plate during camera calibration process. Experiment result indicates that the fitting error of the detected 3D point data of the weld groove is less than 0.1mm. The tracking error in robotic welding process is no more than 0.2mm. Which is acceptable for robotic welding system.

Camera calibration method for solid spheres based on triangular primitives

A new camera calibration method based on triangular primitives for solid grid sphere objects are proposed in this work to address the contradiction among the number of cameras, calibration accuracy, and algorithm stability in the current camera calibration process. In the proposed method, a sphere is used as the base, and the surface coverage curve of the sphere forms a calibration object as the grid based on triangular primitives. We use the absolute quadratic curve and vanishing point to calculate the intrinsic and extrinsic parameters of the camera, respectively. The proposed method can simplify the element correspondence in the calibration and calibrate multiple cameras in different positions simultaneously. Experimental results show that the new 3D calibrator exceeds the calibration accuracy and stability of traditional algorithms. Moreover, the comprehensive evaluation of the algorithm is improved significantly.

Three-dimensional mandibular motion trajectory-tracking system based on BP neural network.

The aim of this study was to develop a prototype three-dimensional optical motion capture system based on binocular stereo vision, Back-propagation (BP) Neural Network and 3D compen-sation method for accurate and real-time recording of mandibular movement. A specialized 3D method of compensation to eliminate the involuntary vibration motions by human heart beating and respiration. A kind of binocular visual 3D measurement method based on projection line and a calibration method based on BP neural network is proposed to solve the problem of the high complexity of camera calibration process and the low accuracy of 3D measurement. The accuracy of the proposed system is systematically evaluated by means of electric platform and clinical trials, and the root-mean-square is 0.0773 mm. Finally, comparisons with state-of-the-art methods demonstrate that our system has higher reliability and accuracy. Meanwhile, the motion trajectory-tracking system is expected to be used in the diagnosis of clinical oral diseases and digital design of restoration.

Assessing Stability Performance of Non-Metric Camera’s Lens Distortion Model during UAV Flight Missions

The process and definition of camera calibration have change greatly over recent years. Aerial metric cameras calibration, for which laboratory and field calibration procedures were a separate process, was performed before and independent of any actual mapping data collection using precise calibration fixtures with an assumption that the camera parameters determined would remain valid for a significant period. In contrast, non-metric cameras are characterized by unstable intrinsic parameters over the times and they are vulnerable to the engine and other vibrations during flight data acquisitions. Moreover, there is no standard calibration procedures exist for these cameras. But, since non-metric camera self-calibration augments the concept of calibration as a part of the measurement process, it can determine the camera intrinsic parameters at the time of the data acquisition as long as highly-convergent geometry and the use of multiple exposures are employed. Therefore, this paper investigates variations of the lens distortion components with object distance within the photographic field by using the self-calibration method. The use of redundant flight paths and tilted camera is also incorporated to ascertain the stability of the principal distance and the principal points during the flight mission. During the experiments, a series of flight mission is conducted to photograph test field areas from over a relatively flat area to highly mountainous one. It is revealed that the radial, decentering, and affinity distortion parameters are more stable than that of the principal distance and principal points against vibrations.

Camera calibration using a planar target with pure translation.

This paper presents a novel camera calibration method using a planar target with pure translation and a known translation distance. It only requires the straightness and position accuracy of the one-axis translational platform. There are theoretically no constraints among the normal direction of the planar target, the optical axis direction of the camera, and the moving direction of the translational platform. The paper analyzes the closed-form solution, followed by a nonlinear refinement based on the maximum likelihood criterion. Both computer simulation and real data are implemented to verify the effectiveness of the proposed method. Compared with Zhang's method, the proposed method realizes the camera calibration process automatically and evaluates the calibration process via the measurement accuracy of the calibrated camera, so it is a key factor to advance 3D computer vision one more step from expert to novice use.

An OGM-Based Visual Navigation System with Reduced Homographies and Accurate Computations for Mobile Robots

This article presents a novel approach to calculate and accelerate the Occupancy Grid Mapping (OGM) for real-time mobile robot navigation. Here, we have improved the main process involved in the construction of OGMs in two ways. On one hand, we propose a new method to perform the camera calibration (CC) process, in which we reduce the number of intermediate steps (homographies) when the reference system is transformed from the real world into a navigation map, reducing consequently the computational costs. On the other hand, the OGM is constructed by an online probabilistic method, which simultaneously performs both the camera calibration and maps construction, thus reducing cumulative errors when these two processes are separately treated. Furthermore, the proposed system can be easily parallelized and mapped onto a digital system (e.g., Field Programmable Gate Arrays, FPGAs) for real-time robot navigation.

A FREQUENCY-DRIFT COMPENSATED CLOSED-FORM SOLUTION FOR STEREO RGB-D MAPPING

Abstract. In this work, we present a frequency-drift compensated (Fd-C) closed-form solution for stereo RGB-D SLAM. The intrinsic parameters for each sensor are first obtained with a standard camera calibration process and the extrinsic orientation parameters achieved through a coarse-to-fine scheme that solves the initial exterior orientation parameters (EoPs) from control markers and further refines the initial value by an iterative closest point (ICP) variant minimizing the distance between the RGB-D point clouds and the referenced laser point clouds. With the assumption of fix transformation between the frames with the same timestamp, we define one sensor as reference sensor and the other sensor as slave sensor and the slave frames can be mapped to the timeline of the references sensor. Rather than endow the camera pose of the nearest frame to the slave frames, we derive the accurate camera pose for slave frames in a spatially variant way. Therefore, the pose relations between the slave frame and the adjacent reference frame can be derived, which provided opportunity to use the more accuracy observations from multiple frames for better tracking and global optimization. We present the mathematical analysis of the iterative optimizations for pose tracking in multi-RGB-D camera cases. Finally, the experiments in complex indoor scenarios demonstrate the efficiency of the proposed multiple RGB-D slam algorithm.