Camera Calibration Parameters Research Articles

A high accuracy method for the sintering condition recognition of rotary kiln

According to the different material sintering conditions, the sintering conditions of alumina rotary kiln can be divided into: super-heated, super-chilled, and normal. In this paper, based on Local Binary Pattern (LBP) and the primary-color method, a novel feature extraction method is proposed to obtain information about temperature in flame images without calibrating camera parameters. Through the analysis of the problem as well as the experimental phenomena, a new classification procedure is devised: in the first step, the super-chilled condition is first separated, in the second step, the normal and super-heat condition are classified. Different feature extraction methods are used in the two steps mentioned above. One is to extract the texture features of pseudo temperature images, and short-time energy is used to describe the dynamic features. The other is to extract the texture features of grey-images by our improved LBPP,Rriu2, and characterize the dynamics with sample entropy and variance. Finally, experimental results show that the our feature extraction method can effectively reduce intra-class variation, increase inter-class variation and receive a high classification accuracy.

Background-oriented Schlieren tomography using gated recurrent unit.

Current Background-oriented schlieren tomography (BOST) methods rely primarily on iterative algorithms for reconstruction. Before reconstruction, a weight projection matrix was generated by performing 3D ray tracing using the projection relationship between the cameras, depending on the camera calibration parameters and large weight projection matrix which introduce artifacts and greatly reduce computational efficiency in the reconstruction. Considering that CT reconstruction uses spatial projection sequences from multiple directions, this study draws inspiration from the Recurrent Neural network(RNN) and utilizes spatial correlation between adjacent projection data to propose a background-oriented schlieren reconstruction method based on a gated recurrent unit (GRU) neural network. First, the model architecture is designed and implemented. Subsequently, numerical simulations were conducted using a methane combustion model to evaluate the proposed method, which achieved an average mean relative error (MRE) of 0.23%. Finally, reconstruction experiments were performed on the actual flow-field data above a candle flame, with a reprojection correlation coefficient of 89% and an average reconstruction time of only 1.04 s per frame. The results demonstrate that the proposed method outperforms traditional iterative reconstruction methods in terms of reconstruction speed and accuracy. This provides a feasible solution for the real-time reconstruction of three-dimensional instantaneous flow fields.

SurgT challenge: Benchmark of soft-tissue trackers for robotic surgery

This paper introduces the “SurgT: Surgical Tracking” challenge which was organized in conjunction with the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2022). There were two purposes for the creation of this challenge: (1) the establishment of the first standardized benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated data in surgery. A dataset of 157 stereo endoscopic videos from 20 clinical cases, along with stereo camera calibration parameters, have been provided. Participants were assigned the task of developing algorithms to track the movement of soft tissues, represented by bounding boxes, in stereo endoscopic videos. At the end of the challenge, the developed methods were assessed on a previously hidden test subset. This assessment uses benchmarking metrics that were purposely developed for this challenge, to verify the efficacy of unsupervised deep learning algorithms in tracking soft-tissue. The metric used for ranking the methods was the Expected Average Overlap (EAO) score, which measures the average overlap between a tracker’s and the ground truth bounding boxes. Coming first in the challenge was the deep learning submission by ICVS-2Ai with a superior EAO score of 0.617. This method employs ARFlow to estimate unsupervised dense optical flow from cropped images, using photometric and regularization losses. Second, Jmees with an EAO of 0.583, uses deep learning for surgical tool segmentation on top of a non-deep learning baseline method: CSRT. CSRT by itself scores a similar EAO of 0.563. The results from this challenge show that currently, non-deep learning methods are still competitive. The dataset and benchmarking tool created for this challenge have been made publicly available at https://surgt.grand-challenge.org/. This challenge is expected to contribute to the development of autonomous robotic surgery and other digital surgical technologies.

The impact of oblique images and flight‐planning scenarios on the accuracy of UAV 3D mapping

AbstractThe developments in lightweight unmanned aerial vehicles (UAVs) and structure‐from‐motion (SfM)‐based software have opened a new era in 3D mapping which is notably cost‐effective and fast, though the photogrammetric blocks lead to systematic height error due to inaccurate camera calibration parameters particularly when the ground control points (GCPs) are few and unevenly distributed. The use of onboard Global Navigation Satellite System (GNSS) receivers (such as RTK‐ or PPK‐based devices which use the DGNSS technique) to obtain the accurate coordinates of camera perspective centres has reduced the need for ground surveys, nevertheless, the aforementioned systematic error was reported in the UAV photogrammetric blocks. In this research, three flight‐planning scenarios with oblique imagery in addition to the traditional nadir block were evaluated and processed with four different processing cases. Therefore, 16 various blocks with different overlaps, direct and indirect georeferencing approaches as well as flight‐planning scenarios were tested to examine and offer the best imaging network. The results denote that the combination of oblique images located on a circle in the centre of the block with the nadir block provides the best self‐calibration functionality and improves the final accuracy by 50% (from 0.163 to 0.085 m) for direct georeferenced blocks and by 40% (from 0.042 to 0.026 m) for indirect georeferenced blocks.

Novel Bayesian Inference-Based Approach for the Uncertainty Characterization of Zhang's Camera Calibration Method.

Camera calibration is necessary for many machine vision applications. The calibration methods are based on linear or non-linear optimization techniques that aim to find the best estimate of the camera parameters. One of the most commonly used methods in computer vision for the calibration of intrinsic camera parameters and lens distortion (interior orientation) is Zhang's method. Additionally, the uncertainty of the camera parameters is normally estimated by assuming that their variability can be explained by the images of the different poses of a checkerboard. However, the degree of reliability for both the best parameter values and their associated uncertainties has not yet been verified. Inaccurate estimates of intrinsic and extrinsic parameters during camera calibration may introduce additional biases in post-processing. This is why we propose a novel Bayesian inference-based approach that has allowed us to evaluate the degree of certainty of Zhang's camera calibration procedure. For this purpose, the a prioriprobability was assumed to be the one estimated by Zhang, and the intrinsic parameters were recalibrated by Bayesian inversion. The uncertainty of the intrinsic parameters was found to differ from the ones estimated with Zhang's method. However, the major source of inaccuracy is caused by the procedure for calculating the extrinsic parameters. The procedure used in the novel Bayesian inference-based approach significantly improves the reliability of the predictions of the image points, as it optimizes the extrinsic parameters.

A Perceptual Measure for Deep Single Image Camera and Lens Calibration.

Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences. To produce beautiful composites, the camera needs to be geometrically calibrated, which can be tedious and requires a physical calibration target. In place of the traditional multi-image calibration process, we propose to infer the camera calibration parameters such as pitch, roll, field of view, and lens distortion directly from a single image using a deep convolutional neural network. We train this network using automatically generated samples from a large-scale panorama dataset, yielding competitive accuracy in terms of standard l2 error. However, we argue that minimizing such standard error metrics might not be optimal for many applications. In this work, we investigate human sensitivity to inaccuracies in geometric camera calibration. To this end, we conduct a large-scale human perception study where we ask participants to judge the realism of 3D objects composited with correct and biased camera calibration parameters. Based on this study, we develop a new perceptual measure for camera calibration and demonstrate that our deep calibration network outperforms previous single-image based calibration methods both on standard metrics as well as on this novel perceptual measure. Finally, we demonstrate the use of our calibration network for several applications, including virtual object insertion, image retrieval, and compositing.

Towards a Guideline for UAV-Based Data Acquisition for Geomorphic Applications

Recent years have seen a rapid rise in the generation of high-resolution topographic data using custom-built or commercial-grade Unmanned Aerial Vehicles (UAVs). Though several studies have demonstrated the application potential of UAV data, significant knowledge gaps still exist in terms of proper documentation of protocols for data acquisition, post-flight data processing, error assessments, and their mitigation. This work documents and provides guidelines for UAV data acquisition and processing from several years of field experience in diverse geomorphic settings across India, including undulating topography (~17 km2), alluvial plains (~142 km2), lowland-river basin (~66 km2), and a highly urbanized area (~5 km2). A total of 37,065 images with 16 and 20 Megapixels and 604 ground control points (GCPs) were captured with multiple UAV systems and processed to generate point clouds for a total area of ~230 km2. The Root Mean Square Error (RMSE) for each GCP for all sites ranged from 6.41 cm to 36.54 cm. This manuscript documents a comprehensive guideline for (a) pre-field flight planning and data acquisition, (b) generation and removal of noise and errors of the point cloud, and (c) generation of orthoimages and digital elevation models. We demonstrate that a well-distributed and not necessarily uniformly distributed GCP placement can significantly reduce doming error and other artifacts. We emphasize the need for using separate camera calibration parameters for each flight and demonstrate that errors in camera calibration can significantly impact the accuracy of the point cloud. Accordingly, we have evaluated the stability of lens calibration parameters between consumer-grade and professional cameras and have suggested measures for noise removal in the point cloud data. We have also identified and analyzed various errors during point cloud processing. These include systematic doming errors, errors during orthoimage and DEM generation, and errors related to water bodies. Mitigation strategies for various errors have also been discussed. Finally, we have assessed the accuracy of our point cloud data for different geomorphic settings. We concluded that the accuracy is influenced by Ground Sampling Distance (GSD), topographic features, and the placement, density, and distribution of GCPs. This guideline presented in this paper can be extremely beneficial to both experienced long-term users and newcomers for planning the UAV-based topographic survey and processing the data acquired.

Multi-camera BEV video-surveillance system for efficient monitoring of social distancing

The current sanitary emergency situation caused by COVID-19 has increased the interest in controlling the flow of people in indoor infrastructures, to ensure compliance with the established security measures. Top view camera-based solutions have proven to be an effective and non-invasive approach to accomplish this task. Nevertheless, current solutions suffer from scalability problems: they cover limited range areas to avoid dealing with occlusions and only work with single camera scenarios. To overcome these problems, we present an efficient and scalable people flow monitoring system that relies on three main pillars: an optimized top view human detection neural network based on YOLO-V4, capable of working with data from cameras at different heights; a multi-camera 3D detection projection and fusion procedure, which uses the camera calibration parameters for an accurate real-world positioning; and a tracking algorithm which jointly processes the 3D detections coming from all the cameras, allowing the traceability of individuals across the entire infrastructure. The conducted experiments show that the proposed system generates robust performance indicators and that it is suitable for real-time applications to control sanitary measures in large infrastructures. Furthermore, the proposed projection approach achieves an average positioning error below 0.2 meters, with an improvement of more than 4 times compared to other methods.

Research on vehicle speed identification method based on time interpolation method and feature point recognition

Based on camera calibration in detection area, an improved method of time interpolation for vehicle speed identification is proposed. Firstly, calibration of camera parameters during photogrammetry is carried out through preset rectangular detection area. Secondly, the key frames are extracted from the video images and the target vehicle feature points and their image coordinates are identified by the image morphological processing. Then the actual coordinates of the characteristic points are determined through the coordinate conversion. Finally, the improved time interpolation method considering the acceleration between frames is used to identify the vehicle speed, so as to improve the accuracy of the vehicle speed measurement. The results proved that the vehicle speed calculated by improved time interpolation method is more reliable, and the error is reduced by 0.7%.

Ispitivanje uticaja broja orijentacionih tačaka na geometrijsku tačnost UAV fotogrametrijskih proizvoda formiranih pomoću pristupa strukture iz pokreta

The positional and vertical accuracy of UAV aerial photogrammetry products generated using the Structure from Motion (SfM) approach depends on various factors, such as flight plan parameters, camera quality, camera calibration, the SfM algorithm used, and the georeferencing process. The influence of the quantity of Ground Control Points (GCPs) on the geometric quality of generated models and the stability of camera calibration parameters assessed through self-calibration in the block-aerotriangulation process was investigated in this study. Three software systems were used to process the collected UAV photogrammetry images: Pix4D Mapper, Agisoft Metashape, and Trimble Inpho UASMaster. Standard statistical quality assessments were employed to assess the accuracy of the block-aerotriangulation. The research findings indicate that augmenting the quantity of GCPs enhances model reliability and decreases the RMSE values of vertical deviation on the control points. The RMSE values of vertical deviation on the check points for all three used software systems converged to approximately twice the value of the average spatial resolution. Additionally, the RMSE values of positional deviation on check points converged to the value of the average spatial resolution.

Програмний пакет для оцінки похибки калібрування стереокамери в системі комп’ютерного зору

The approach for accuracy assessment of the object model a for the problem of stable grasping in the combined system of the proposal of grasping and the reconstruction of the three-dimensional model of the object was considered. Such a combined system allows stable capture of objects of any shape without restrictions on the types of shapes in the training data set. Novel approaches to surface reconstruction of the object are based on restoring the depth of points from a pair of images from two cameras. The quality of the 3D reconstruction is affected by several factors: the movement of the camera and environmental objects, spatial quantization of the image coordinates, correspondence of key points, camera calibration parameters, unaccounted camera distortions, as well as numerical and statistical properties of the selected reconstruction method. Camera parameter errors can be minimized by improving the calibration procedure, so the impact of errors on the quality of the 3D model was investigated. The deviation of the model from the plane is chosen as a metric for quality assessment. For its calculation, the point cloud is processed by plane identification and segmentation, for which an algorithm based on the RANSAC method is considered. The software package for accuracy estimation was developed. An experiment was conducted to obtain the dependence of the accuracy of the reconstructed planes on the errors of the camera parameters. The impact of calibration errors on 3D reconstruction was evaluated by comparing metrics for individual planes at different levels of artificial error and evaluating the impact of the error on these metrics. Modeling the error of the camera calibration parameters with a given noise level shows that the calibration parameters deteriorate as the noise level increases. In particular, it was established that an increase in error contributes to an increase in the error of estimation of calibration parameters. In addition, orientation parameters (rotation and translation) are more complex and therefore more sensitive to measurement noise than other parameters.

Comprehensive Analysis of the Object Detection Pipeline on UAVs

An object detection pipeline comprises a camera that captures the scene and an object detector that processes these images. The quality of the images directly affects the performance of the object detector. Current works focus on independently improving the image quality or object detection models but neglect the importance of joint optimization of the two subsystems. This paper aims to tune the detection throughput and accuracy of existing object detectors in the remote sensing scenario by optimizing the input images tailored to the object detector. We empirically analyze the influence of two selected camera calibration parameters (camera distortion correction and gamma correction) and five image parameters (quantization, compression, resolution, color model, and additional channels) for these applications. For our experiments, we utilize three Unmanned Aerial Vehicle (UAV) data sets from different domains and a mixture of large and small state-of-the-art object detector models to provide an extensive evaluation of the influence of the pipeline parameters. Finally, we realize an object detection pipeline prototype on an embedded platform for a UAV and give a best practice recommendation for building object detection pipelines based on our findings. We show that not all parameters have an equal impact on detection accuracy and data throughput. Using a suitable compromise between parameters, we can achieve higher detection accuracy for lightweight object detection models while keeping the same data throughput.

Three-Dimensional Reconstruction of Shoe Soles via Binocular Vision Based on Improved Matching Cost

Aiming at the problem that the toe cap and upper part of the sole of a shoe easily appear missing when using binocular vision to reconstruct the shoe sole in the industrial production process, an improved matching cost calculation method is proposed to reconstruct shoe soles in three dimensions. Firstly, a binocular vision platform is built, and Zhang’s calibration method is used to obtain the calibration parameters. Secondly, the method of fusing Census and BT costs is used to calculate the matching cost of the image, so that the matching cost calculation result is more accurate. On this basis, 4-path aggregation is performed on the obtained cost, and the optimal matching cost is selected in combination with the WTA algorithm. Finally, left–right consistency detection and median filtering are used to optimize the disparity map and combine the camera calibration parameters to reconstruct the shoe sole in three dimensions. The experimental results show that the average mismatch rate of the four images on the Middlebury website in this method is about 6.57%, the reconstructed sole point cloud contour information is complete, and there is no material missing at the toe and heel.

High-precision visual imaging model and calibration method for multi-depth-of-field targets

In the field of high-precision large-depth 3D reconstruction, some bottlenecks exist in the improvement of visual measurement accuracy. This is because traditional vision measurement methods simplify the imaging model of a fixed-focus camera to a pinhole imaging model with fixed parameters accompanied by various distortion information. In fact, according to the optical properties of camera imaging, the position of the optical center in the pinhole imaging model varies with the depth of the imaging target. In this paper, in order to achieve the objective of improving visual measurement accuracy, a high-precision visual imaging model for multi-depth-of-field targets and the corresponding camera parameters calibration method are proposed based on the optical properties of camera imaging. When using this imaging model for 3D measurements, different pixel points in the image use different camera internal parameters depending on the depth of the imaging target, thus enabling highly accurate 3D reconstruction. The experimental results show that the imaging model and calibration method proposed in this paper greatly improve the accuracy and robustness of visual 3D measurement and have a promising future in the field of large-depth 3D reconstruction.

Investigation of the effects of different flight parameters on the accuracy of DEM generated using UAV systems

Unmanned Aerial Vehicles (UAVs), can be controlled automatically or with the help of a remote control system, provide high spatial and temporal resolution images in addition to its advantages in terms of cost and time. Due to these features, UAV’s have been widely used in studies such as map production, 3D modeling and volume calculation in recent years. In this study, it is aimed to investigate the effects of different flight parameters on the accuracy of Orthomosaic and DEM produced with UAV Systems. In addition, the rolling shutter effect and the use of pre-calibrated and automatic camera calibration parameters were also tested. For this purpose, a total of seven UAV flights were carried out at different heights, overlap ratios, angles and grids at a mining area located within the borders of Aksaray province. Agisoft MetaShape and Pix4D Mapper, which are widely used commercial software, were used to process the UAV images. In this study, 27 points were established and 3D coordinates of these points were measured using GNSS technique. 15 of these points were used as reference points and the remaining 12 points were used as check points. Accuracy analysis was performed by comparing the produced Orthomosaics and DEMs with the check points in the study area. CloudCompare and ArcGIS software were used in the comparison of the products as a result of the evaluation of the images. It is concluded that the Orthomosaics produced using 450 angle and DEMs with 600 angle has higher accuracy than the others. Point clouds and DEMs’ produced using automatic and pre-calibrated camera calibration parameters show that the pre-calibrated images provide higher accuracy. Also, it is seen that when the rolling shutter effect is modeled, the horizontal and vertical accuracy is increased in all three flights with different flight parameters in this study.

Correcting projector lens distortion in real time with a scale-offset model for structured light illumination.

In fringe projection profilometry, inevitable distortion of optical lenses decreases phase accuracy and decreases the quality of 3D point clouds. For camera lens distortion, existing compensation methods include real time look-up tables derived from the related parameters of camera calibration. However, for projector lens distortion, so far, post-undistortion methods iteratively correcting lens distortion are relatively time-consuming while, despite avoiding iteration, pre-distortion methods are not suitable for binary fringe patterns. In this paper, we aim to achieve real-time phase correction for the projector by means of a scale-offset model that characterizes projector distortion by four correction parameters within a small-enough area, and thus we can speed up the post-undistortion by looking up tables. Experiments show that the proposed method can suppress the distortion error by a factor of 20 ×, i.e., the error of root mean square is less than 45 µm/0.7‰, while also proposed improving the computation speed by a factor of 50× over traditional iterative post-undistortion.

Research on the Hand–Eye Calibration Method of Variable Height and Analysis of Experimental Results Based on Rigid Transformation

In view of the phenomenon that camera imaging will appear large up close and small from afar in the eye-to-hand hand-calibration system, one hand–eye calibration is carried out. The manipulator is only suitable for grasping objects of the same height, and the calibration results cannot be applied to grasping products with variable height. Based on the study of the pinhole camera model and the rigid transformation model between coordinate systems, the introduction of the calibration height parameters, the relationship between parameters of the rigid transformation matrix between image the coordinate system and the robot coordinate system, and sampling height are established. In the experiment, firstly, through the calibration of camera parameters, the influence of camera distortion on imaging quality is eliminated, and the influence of calibration height is ignored. Then, the machine coordinate system and image coordinate system of the calibration plate at different heights are calibrated using the four-point calibration method. The parameters of the rigid transformation matrix at different heights (H) are calculated. Finally, through experimental analysis, the high linear relationship between the parameters of the rigid transformation matrix from the image coordinate system to the robot coordinate system and the calibration height is fitted. By analyzing the random error of the experiment, the linear relationship between calibration height and pixel density is further established, and the systematic error of the experimental process is deeply analyzed. The experimental results show that the hand–eye calibration system based on this linear relationship is precise and suitable for grabbing products of any height, and the positioning error is less than 0.08%.

Camera Parameter Calibration Based on Directional Layered Target

Camera Parameter Calibration Based on Directional Layered Target

Calibration of intrinsic camera parameters with a conic and its asymptotes.

This study proposes a method for camera calibration using the properties of conic asymptotes, which are combined with the projective invariance, images of the asymptotes, and image of the absolute points to obtain the vanishing line. Constraints for the image of the absolute conic are obtained from a set of orthogonal vanishing points, the homography, and images of the circular points. The proposed calibration algorithm is compared with several other research methods, and its effectiveness and robustness are verified within a certain error range from the results of simulated and real experiments for different eccentricities without loss of generality.

Conic tangents based high precision extraction method of concentric circle centers and its application in camera parameters calibration

A high-precision camera intrinsic parameters calibration method based on concentric circles was proposed. Different from Zhang’s method, its feature points are the centers of concentric circles. First, the collinearity of the projection of the center of concentric circles and the centers of two ellipses which are imaged from the concentric circles was proved. Subsequently, a straight line passing through the center of concentric circles was determined with four tangent lines of concentric circles. Finally, the projection of the center of concentric circles was extracted with the intersection of the straight line and the line determined by the two ellipse centers. Simulation and physical experiments are carried out to analyze the factors affecting the accuracy of circle center coordinate extraction and the results show that the accuracy of the proposed method is higher. On this basis, several key parameters of the calibration target design are determined through simulation experiments and then the calibration target is printed to calibrate a binocular system. The results show that the total reprojection error of the left camera is reduced by 17.66% and that of the right camera is reduced by 21.58% compared with those of Zhang’s method. Therefore, the proposed calibration method has higher accuracy.