Calibration Board Research Articles

Robust calibration of surround-view camera systems with Planar-Eccentric Constraint: A novel Apriltag approach

This research tackles calibrating surround view camera systems for automated driving, where traditional methods require extensive manual measurements and precise conditions. We introduce a novel automatic calibration method that ensures portability and accuracy using only custom-designed calibration boards placed in the cameras’ common view. To address visual interference from extraneous objects, we design a unique board with AprilTag 2D codes, offering clear advantages over traditional checkerboards. Additionally, we’ve developed an Adaptive Angle Projection to counteract fisheye lens distortions. We also introduce a hierarchical pose optimization strategy that transitions from local to global, incorporating a “Planar-Eccentric constraint” to leverage the calibration board’s geometric positioning. Our method achieved 4.01 cm measurement error, with 0.075°angle error and 1.18 cm translation error. Our method’s efficacy and robustness were confirmed through extensive testing on an experimental vehicle and simulation dataset, proving its superiority in achieving precise calibration and measurement with minimal manual input.

A LiDAR-Camera Joint Calibration Algorithm Based on Deep Learning.

Multisensor (MS) data fusion is important for improving the stability of vehicle environmental perception systems. MS joint calibration is a prerequisite for the fusion of multimodality sensors. Traditional calibration methods based on calibration boards require the manual extraction of many features and manual registration, resulting in a cumbersome calibration process and significant errors. A joint calibration algorithm for a Light Laser Detection and Ranging (LiDAR) and camera is proposed based on deep learning without the need for other special calibration objects. A network model constructed based on deep learning can automatically capture object features in the environment and complete the calibration by matching and calculating object features. A mathematical model was constructed for joint LiDAR-camera calibration, and the process of sensor joint calibration was analyzed in detail. By constructing a deep-learning-based network model to determine the parameters of the rotation matrix and translation matrix, the relative spatial positions of the two sensors were determined to complete the joint calibration. The network model consists of three parts: a feature extraction module, a feature-matching module, and a feature aggregation module. The feature extraction module extracts the image features of color and depth images, the feature-matching module calculates the correlation between the two, and the feature aggregation module determines the calibration matrix parameters. The proposed algorithm was validated and tested on the KITTI-odometry dataset and compared with other advanced algorithms. The experimental results show that the average translation error of the calibration algorithm is 0.26 cm, and the average rotation error is 0.02°. The calibration error is lower than those of other advanced algorithms.

From microscope to head-mounted display: integrating hand tracking into microsurgical augmented reality.

The operating microscope plays a central role in middle and inner ear procedures that involve working within tightly confined spaces under limited exposure. Augmented reality (AR) may improve surgical guidance by combining preoperative computed tomography (CT) imaging that can provide precise anatomical information, with intraoperative microscope video feed. With current technology, the operator must manually interact with the AR interface using a computer. The latter poses a disruption in the surgical flow and is suboptimal for maintaining the sterility of the operating environment. The purpose of this study was to implement and evaluate free-hand interaction concepts leveraging hand tracking and gesture recognition as an attempt to reduce the disruption during surgery and improve human-computer interaction. An electromagnetically tracked surgical microscope was calibrated using a custom 3D printed calibration board. This allowed the augmentation of the microscope feed with segmented preoperative CT-derived virtual models. Ultraleap's Leap Motion Controller 2 was coupled to the microscope and used to implement hand-tracking capabilities. End-user feedback was gathered from a surgeon during development. Finally, users were asked to complete tasks that involved interacting with the virtual models, aligning them to physical targets, and adjusting the AR visualization. Following observations and user feedback, we upgraded the functionalities of the hand interaction system. User feedback showed the users' preference for the new interaction concepts that provided minimal disruption of the surgical workflow and more intuitive interaction with the virtual content. We integrated hand interaction concepts, typically used with head-mounted displays (HMDs), into a surgical stereo microscope system intended for AR in otologic microsurgery. The concepts presented in this study demonstrated a more favorable approach to human-computer interaction in a surgical context. They hold potential for a more efficient execution of surgical tasks under microscopic AR guidance.

Research on visual servo strategy for the uncalibrated non-standard manipulator

Abstract Aiming at the problem that the hand-eye calibration of the uncalibrated non-standard manipulator cannot be addressed by the general calibration method, i.e., the eye-in-hand and eye-to-hand calibration method of the visual servo system was proposed to guide the manipulator without calibration. First, a camera was installed on the manipulator to complete the eye-in-hand calibration and achieve transformation from the end coordinate system to the camera coordinate system. Second, a second camera was used to achieve eye-to-hand calibration and obtain the transformation relationship from the end coordinate system to the calibration board. The entire calibration process only requires the collection of images from two scenarios, i.e., the docking and the completion of the docking processes. The implementation results show that the rotation error between groups is less than 1.5°, and the translation error between groups is less than 8 mm. Therefore, the proposed visual servo strategy can accurately guide the uncalibrated non-standard manipulator.

A closed-loop kinematic approach to Self-Calibration of joint offset for multi-branched robots

Kinematic accuracy is fundamental for the performance of multi-branched robots, which are composed of multiple branches that collaborate to manipulate. Among those various factors affecting kinematic accuracy, the joint offset is the most significant contributor due to misalignment in component assembly. In this work, we present a general, fast, and low-cost approach based on closed-loop kinematics to calibrate joint offsets of multi-branched robots. A dedicated calibration board connects the distal ends of branches to form closed-loop configurations. The identification model is derived based on the closed-loop kinematic differential model. The observable indexes of multiple closed-loop are integrated to optimize the calibration board’s poses, and joint offsets are identified simultaneously. To enhance the robustness of the calibration outcomes against joint backlash noise, the Kolmogornov-Smirnov(K-S) test is performed on multiple sets of identification results to verify that joint offset obeys normal distribution. Experiments on a quadruped showed significantly improved foot-tip position accuracy.

Research on a Machine Vision-based Blade Measurement Method

A high-precision visual measurement method for the geometric dimensions of oil pump blades is proposed, utilizing an auxiliary measurement mechanism to assist in sub-pixel edge positioning. Based on images of calibration boards, the camera distortion is corrected by partitioning the image into regions and establishing a mapping index between pixel coordinates and real-world coordinates. Under fixed object and image distances, images of the oil pump blades and the contact between the blades and the pump are separately captured. The Canny operator and Sigmoid function are employed to fit the edges, extracting sub-pixel coordinates, which are then mapped to real-world coordinates. Finally, a least squares edge fitting is conducted to compute the dimensional parameters. Experimental results indicate that the use of auxiliary measurement mechanisms effectively enhances the precision of blade measurements, as compared to measurements solely based on oil pump blade images and images of the blade in contact with the pump.

Grounding rod hanging and removing robot with hand-eye self-calibration capability in substation

In this paper, a robot system for hanging and removing grounding rods is designed by integrating 3D recognition technology, hand-eye self-calibration technology, and automatic operation technology. Specifically, a novel hand-eye self-calibration algorithm is proposed that only uses common objects in the actual scene, which differs from traditional robot hand-eye calibration in that it requires a dedicated calibration board to assist with offline completion. Addressing the problem that existing self-calibration methods cannot be optimized as a whole, resulting in low accuracy and instability of the solution, a multi-stage objective function optimization self-calibration algorithm is proposed. An optimization method based on the minimization of re-projection error is designed to compensate for the results, which uses an efficient Oriented fast and rotated brief (ORB) feature extraction algorithm and introduces a scoring mechanism to retain more correct matching points in the feature matching stage. Experimental verifications are conducted using both public dataset and practical robot system. In the dataset experiment, our method demonstrates superior accuracy and robustness compared to existing self-calibration methods. Furthermore, the practical robot platform experiment confirms the feasibility and efficacy of our approach across various wind speeds and lighting conditions.

A coarse to fine calibration method for the q-X-Z type wafer pre-alignment device

Abstract The wafer pre-alignment device is a crucial component of the equipment front-end modules (EFEM) in the integrated circuit (IC) manufacturing industry. It corrects the wafer position and orientation errors before the wafer is transferred to the subsequent fine alignment wafer stage. To solve the calibration problem of a q-X-Z type wafer pre-alignment device, the study establishes its kinematic model and constructs the relationship between each axis and a laser sensor. Compared to the conventional method of directly selecting 4 points’ wafer edge data to achieve calibration, this paper proposes a method that makes full use of the edge data of a square calibration board. This method solves the device parameters through iteration based on coarse to fine thoughts and obtains a calibration accuracy within 10 microns. The statistical accuracy and operational repeatability of the proposed method are verified through a series of calibration experiments of the wafer pre-alignment device.

Camera calibration with active standard Gaussian stripes for 3D measurement

The extraction accuracy of the mark points is directly related to the calibration accuracy of the camera. However, traditional ceramic calibration boards with checkerboard or circle dot markers are not suitable for defocused conditions and lack adaptability to changes in the measuring field. Therefore, liquid crystal displays (LCDs) that display phase-shifting patterns are widely used as phase active targets for cameral calibration. While phase-shifting patterns accurately extract feature points in defocused conditions, they are susceptible to LCD screen nonlinear errors. To address these shortcomings, we propose a practical method that involves displaying standard Gaussian stripes on LCD. Specifically, this method displays Gaussian stripes in two directions and extracts their centerline subpixel coordinates. The intersection points of the two orthogonal centerlines are then considered as the feature points. Simulation and experimental results demonstrate that the standard Gaussian stripes still accurately extract feature points when affected by LCD nonlinearity or image defocus.

Simple calibration of fisheye lenses for hemispherical photography of the forest canopy

Canopy hemispherical photography (HP) is a technology for vegetation surveying that usually employs consumer-oriented devices. Variations on those devices are caused by technological advances and design. These variations, despite providing the potential for more reliable surveying, make the measurements challenging. The knowledge about how to standardize devices output is available, what is missing is a simple and inexpensive methodology to accomplish it. This is needed since the use of consumer-oriented devices creates an expectation of simplicity and affordability. To fill this methodology gap, we developed an easy-to-assemble calibration board of near-zero material cost (simple board) and outlined how to use it to develop device-specific functions: the projection function (fp) and the vignetting function (fv). The former relates view zenith angle (θ) with distance from image center, while fv models the radial decreasing of brightness known as vignetting effect. Explanations are accompanied by tailored R code. Altogether, we call it “the simple method”. Three devices were used to test it: a Nikon Coolpix 5700 with a FC-E9 converter (D1), a Nikon D610 with a Nikkor 8 mm fisheye lens (D2), and an iPhone 6 Plus with an Olloclip fisheye converter (D3). The reference data was taken with D1, corrected for vignetting effect with a photometric sphere and reprojected to the standard equidistant projection with a fv developed by Delta-T Devices (www.delta-t.co.uk). The root mean square error (RMSE) of the simple method were 0.55° θ and 0.021 relative radiance (RR) for D1; 1.8° θ and 0.1 RR for D2; and 2.1° θ and 0.066 RR for D3. This level of standardization will help to simplify the classification of canopy gaps, a key processing step in HP. And, if it is broadly adopted, will facilitate comparisons between independent studies and large time series.

Automatic extrinsic calibration for structured light camera and repetitive LiDARs

AbstractThe integration of camera and LiDAR technologies has the potential to significantly enhance construction robots’ perception capabilities by providing complementary construction information. Structured light cameras (SLCs) are a desirable alternative as they provide comprehensive information on construction defects. However, fusing these two types of information depends largely on the sensors’ relative positions, which can only be established through extrinsic calibration. This paper introduces a novel calibration algorithm considering a customized board for SLCs and repetitive LiDARs, which are designed to facilitate the automation of construction robots. The calibration board is equipped with four symmetrically distributed hemispheres, whose centers are obtained by fitting the spheres and adoption with the geometric constraints. Subsequently, the spherical centers serve as reference features to estimate the relationship between the sensors. These distinctive features enable our proposed method to only require one calibration board pose and minimize human intervention. We conducted both simulation and real-world experiments to assess the performance of our algorithm. And the results demonstrate that our method exhibits enhanced accuracy and robustness.

Calibration for endoscopic 3D shape measurement with cone beam projection

We demonstrate a calibration method for endoscopic three-dimensional shape measurement with cone beam projection. In this method, changes in the shape of the optical sectioning profiles are quantified and fitted while scanning a calibration board in the depth direction, using a cubic function. In accuracy tests using a flat plate and a ring reference gauge, the proposed method obtains an accuracy of 0.02 mm in the depth dimension and 0.09 mm in the radial dimension. These results represent 88% and 55% improvements compared to previous analysis. For medical applications, an ear examination simulator was employed, and our measurement results were compared to ground truth data obtained by microfocus X-ray computed tomography. The surface deviation of our method relative to the ground truth data was ±0.36mm during manual operation. A comparison of the measurement results before and after calibration revealed an improvement in the peak agreement with the ground truth data, with the deviation shifting from 0.2 mm to −0.05mm. Our strategy achieves a digital transformation of 3D endoscopy, which would benefit a number of medical fields.

Adaptive adjustment of brightness and blur of the camera for high precision internal parameter calibration

Camera calibration is a prerequisite for many computer vision tasks, such as visual measurement and visual localization. This paper proposes a new method for high-precision calibration of camera intrinsic parameters. By dynamically adjusting image brightness and blur to cope with changes in external lighting conditions and camera working distance, high-precision internal parameter measurement is achieved. This method solves the problems of the traditional camera intrinsic parameter calibration process being cumbersome and susceptible to changes in the placement of calibration boards and fluctuations in external lighting conditions. Firstly, an adaptive exposure adjustment algorithm is proposed to adjust the camera exposure value by statistically analyzing the average gray value of randomly selected rectangular regions on the calibration board, thereby solving the problem of the camera calibration accuracy being affected by overly bright or dark environments. Subsequently, an evaluation criterion and adaptive adjustment strategy for adjusting image blur are proposed. By extracting edge points of feature circles and obtaining the slope of the fitted curve, the relationship between edge sharpness and slope is utilized to adjust the image blur. Finally, the proposed method achieves high-precision semi-automatic calibration. Experimental results demonstrate that the proposed method has significant advantages in terms of correctness, robustness, and flexibility, with an average reprojection error of 0.0167.

A correction method for radial distortion and nonlinear response of infrared cameras.

The key feature of non-contact temperature measurement provided by infrared (IR) cameras underpins their versatility. However, the accuracy of temperature measurements with IR cameras depends on imaging quality due to their non-contact nature, such as the lens, body temperature, and measurement environment. This paper addresses the correction of radial distortion and nonlinear response issues in IR cameras. To address radial distortion, we have designed a passive checkerboard calibration board specifically for infrared cameras. This board is used to calibrate the IR camera and derive the necessary camera parameters. Subsequently, these parameters are applied during the actual measurement process to rectify radial distortion effectively. Building on the radial distortion correction method mentioned above, we propose a multi-point segmented calibration approach that considers different temperature ranges and imaging regions. This method alleviates the issue of reduced temperature measurement accuracy due to variations in camera responses by computing gain and offset coefficient matrices for each temperature range. Experimental results demonstrate the effectiveness of the calibration board in correcting radial distortion in IR cameras, with a mean reprojection error of less than 0.16 pixels. Regarding the nonlinear response problem, the introduced method significantly reduces the relative error in temperature measurement. In the verification phase, spanning from 100 to 500 °C, the average relative error in temperature measurement decreases by 0.49% from 1.61% before and after correction, which highlights a substantial improvement in temperature measurement accuracy. This work gives a useful reference to improve the imaging quality and temperature measurement accuracy using infrared cameras.

Improved Shi-Tomasi sub-pixel corner detection based on super-wide field of view infrared images.

In this work, we propose a new, to the best of our knowledge, corner point detection method for a super-wide field of view infrared imaging system. First, the edge of the checkerboard calibration board is detected at the pixel level by morphological operation. Second, the interpolation technique is used to refine the edge so that the edge has sub-pixel accuracy. We obtain the four checkerboard unit corners near the real corner point and average the coordinates of the four corners to indirectly obtain the coordinates of the real corner point. Meanwhile, we take pictures of the same calibration board at different angles for repeatability verification. It is proved that the improvement of our algorithm for the detection of corners of super-wide field of view infrared images is more feasible compared to the traditional algorithms.

Development of the ATLAS Liquid Argon Calorimeter Readout Electronics for the HL-LHC

The ATLAS Liquid Argon Calorimeter readout electronics will be upgraded to withstand higher radiation doses and harsher conditions during the HL-LHC. In this context, the upgraded readout chain will include front-end, calibration, processing, and timing control boards. The front-end boards will amplify and digitize the ionization signal with high precision and low noise. Custom made chips for amplifying, shaping and digitize the calorimeter signals have been developed, demonstrating stability under irradiation tests. Calibration boards will ensure precise calibration across all channels by using custom ASICs which will be also stable under irradiation. The digitized signal will be processed for energy estimation and time stamping purposes by using boards featuring FPGAs connected via high-speed links. To enhance their functionality, we are exploring using artificial neural networks within the FPGAs. Lastly, the timing and control system will ensure the synchronization across the different boards. The current status of the prototype boards and testing efforts are presented.

Edge-guided two-stage feature matching for infrared and visible image registration in electric power scenes

Collaborative processing of infrared and visible (IR–VS) images is essential in identifying potential equipment failures during power inspections. However, due to significant differences in spectrum, resolution, and intensity between IR–VS images, it is still challenging to obtain an accurate spatial registration for them. To address these issues, we propose the edge-guided two-stage registration strategy for various kinds of real-world applications. We first develop a heated, hollow calibration board tailored to IR–VS cameras, thus conducting offline calibration to estimate the scale and resolution relationships. The IR–VS images are pretransformed into a similar scale to improve registration accuracy. Furthermore, we propose the edge-guided feature matching network (EGFMNet) to extract feature points and build associated descriptors end-to-end. Specifically, benefiting from the excellent description ability of the network for image intensity changes, we directly adopt multimodal edge maps to learn more consistent features for accurate registration. Meanwhile, the local–global consistent features from edge maps are extracted to eliminate the nonlinear radiation distortion. By jointly learning the repeatability and reliability information, robust feature points with distinctive descriptors can be extracted and matched for homography estimation. Comparative and ablation experiments demonstrate the effectiveness and practicality of the proposed registration strategy in real-world power inspection scenarios. Extended experiments on several public datasets further demonstrate the superiority of the proposed method.

Suppression for Phase Error of Fringe Projection Profilometry Using Outlier-Detection Model: Development of an Easy and Accurate Method for Measurement

Fringe projection is an important technology in three-dimensional measurement and target recognition. The measurement accuracy depends heavily on the calibration of the absolute phase and projector pixels. An easy-to-implement calibration method based on the Random Sample Consensus (RANSAC) algorithm is proposed to exterminate the phase error data and elevate the measurement accuracy in a fringe projection system. The reconstruction experiments of a double-sphere standard demonstrate that the uncertainties in radius and sphere-distance measurement are reduced to one thousandth of the measured value or even less, and the standard deviation in multiple measurements is restricted to within 50 μm. The measurement accuracy provided by the proposed RANSAC method can be improved by up to 44% compared with that provided by traditional least squared method (LSM). The proposed calibration method is easy and simple to implement, and it does not need additional hardware, but rather a calibration board.

Development of Water-Wheel Tail Measurement System Based on Image Projective Transformation

Fishery is vital for Taiwan’s economy, and over 40% of the fishery products come from aquaculture. Traditional aquaculture relies on the visual observation of a water-wheel tail length to assess water quality. However, the aging population, lack of young labor, and difficulty in passing down experience pose challenges. There is currently no systematic method to determine the correlation between the water quality and water-wheel tail length, and adjustments are made based on visual inspection, relying heavily on experience without substantial data for transmission. To address the challenge, a precise and efficient water quality control system is proposed. This study proposes a water-wheel tail length measurement system that corrects input images through image projective transformation to obtain the transformed coordinates. By utilizing known conditions of the water-wheel, such as the length of the base, the actual water-wheel tail length is deduced based on proportional relationships. Validated with two different calibration boards, the projective transformation performance of specification A is found to be better, with an average error percentage of less than 0.25%. Data augmentation techniques are employed to increase the quantity and diversity of the dataset, and the YOLO v8 deep learning model is trained to recognize water-wheel tail features. The model achieves a maximum mAP50 value of 0.99013 and a maximum mAP50-95 value of 0.885. The experimental results show that the proposed water-wheel tail length measurement system can be used feasibly to measure water-wheel tail length in fish farms.

Calibration method for monocular laser speckle projection system.

This paper proposes a novel calibration method for the monocular laser speckle projection system. By capturing images of a calibration board with speckles under different poses, projector's optical axis is fitted and utilized to calibrate the rotation between the camera and projector. The translation is solved in closed form subsequently and projector's virtual image is recovered via homography. After calibration, the system can be regarded and operated as a binocular stereo vision system with speckle pattern. The proposed method is efficient and convenient, without need of reference image or high-precision auxiliary equipment. Validated by experiments on Astra-s and Astra-pro, it presents significant improvement in depth-estimation compared to the traditional method.