Camera Intrinsic Parameters Research Articles

Autocalibration method for scanning electron microscope using affine camera model

This paper deals with the task of autocalibration of scanning electron microscope (SEM), which is a technique allowing to compute camera motion and intrinsic parameters. In contrast to classical calibration, which implies the use of a calibration object and is known to be a tedious and rigid operation, auto- or selfcalibration is performed directly on the images acquired for the visual task. As autocalibration represents an optimization problem, all the steps contributing to the success of the algorithm are presented: formulation of the cost function incorporating metric constraints, definition of bounds, regularization, and optimization algorithm. The presented method allows full estimation of camera matrices for all views in the sequence. It was validated on virtual images as well as on real SEM images (pollen grains, cutting tools, etc.). The results show a good convergence range and low execution time, notably compared to classical methods, and even more in the context of the calibration of SEM.

Deep end-to-end rolling shutter rectification.

CMOS sensors employ a row-wise acquisition mechanism while imaging a scene, which can result in undesired motion artifacts known as rolling shutter (RS) distortions in the captured image. Existing single image RS rectification methods attempt to account for these distortions by using either algorithms tailored for a specific class of scenes that warrants information of intrinsic camera parameters or a learning-based framework with known ground truth motion parameters. In this paper, we propose an end-to-end deep neural network for the challenging task of single image RS rectification. Our network consists of a motion block, a trajectory module, a row block, an RS rectification module, and an RS regeneration module (which is used only during training). The motion block predicts the camera pose for every row of the input RS distorted image, while the trajectory module fits estimated motion parameters to a third-order polynomial. The row block predicts the camera motion that must be associated with every pixel in the target, i.e., RS rectified image. Finally, the RS rectification module uses motion trajectory and the output of a row block to warp the input RS image to arrive at a distortion-free image. For faster convergence during training, we additionally use an RS regeneration module that compares the input RS image with the ground truth image distorted by estimated motion parameters. The end-to-end formulation in our model does not constrain the estimated motion to ground truth motion parameters, thereby successfully rectifying the RS images with complex real-life camera motion. Experiments on synthetic and real datasets reveal that our network outperforms prior art both qualitatively and quantitatively.

Fast automatic camera network calibration through human mesh recovery

Camera calibration is a necessary preliminary step in computer vision for the estimation of the position of objects in the 3D world. Despite the intrinsic camera parameters can be easily computed offline, extrinsic parameters need to be computed each time a camera changes its position, thus not allowing for fast and dynamic network re-configuration. In this paper we present an unsupervised and automatic framework for the estimation of the extrinsic parameters of a camera network, which leverages on optimised 3D human mesh recovery from a single image, and which does not require the use of additional markers. We show how it is possible to retrieve the real-world position of the cameras in the network together with the floor plane, exploiting regular RGB images and with a weak prior knowledge of the internal parameters. Our framework can also work with a single camera and in real-time, allowing the user to add, re-position, or remove cameras from the network in a dynamic fashion.

PASMVS: A perfectly accurate, synthetic, path-traced dataset featuring specular material properties for multi-view stereopsis training and reconstruction applications.

A Perfectly Accurate, Synthetic dataset for Multi-View Stereopsis (PASMVS) is presented, consisting of 400 scenes and 18,000 model renderings together with ground truth depth maps, camera intrinsic and extrinsic parameters, and binary segmentation masks. Every scene is rendered from 45 different camera views in a circular pattern, using Blender's path-tracing rendering engine. Every scene is composed from a unique combination of two camera focal lengths, four 3D models of varying geometrical complexity, five high definition, high dynamic range (HDR) environmental textures to replicate photorealistic lighting conditions and ten materials. The material properties are primarily specular, with a selection of more diffuse materials for reference. The combination of highly specular and diffuse material properties increases the reconstruction ambiguity and complexity for MVS reconstruction algorithms and pipelines, and more recently, state-of-the-art architectures based on neural network implementations. PASMVS serves as an addition to the wide spectrum of available image datasets employed in computer vision research, improving the precision required for novel research applications.

Disturbance observer based adaptive model predictive control for uncalibrated visual servoing in constrained environments

This paper presents an adaptive model predictive control (MPC) method based on disturbance observer (DOB) to improve the disturbance rejection performance of the image-based visual servoing (IBVS) system. The proposed control method is developed based on the depth-independent interaction matrix, which can simultaneously handle unknown camera intrinsic and extrinsic parameters, unknown depth parameters, system constraints, as well as external disturbances. The proposed control scheme includes two parts which are the feedback regulation part based on the adaptive MPC and the feedforward compensation part based on the modified DOB. Unlike the traditional DOB that is based on the fixed nominal plant model, the modified DOB here is based on the estimated plant model. The adaptive MPC controller consists of an iterative identification algorithm, which not only can provide the model parameters for both the controller and the modified DOB, but also can be used to control plant dynamics and to minimize the effects of DOB. Simulations for both the eye-in-hand and eye-to-hand camera configurations are conducted to illustrate the effectiveness of the proposed method.

Improved genetic algorithm for intrinsic parameters estimation of on-orbit space cameras

Computer vision plays a key role to measure the relative posture and position between the spacecrafts, especially in various important space tasks. As one of the essential steps for computer vision, camera calibration is important for obtaining precise three-dimensional contours of the space target. However, it is impossible to use the traditional calibration targets to calibrate the space camera in orbit. To solve this problem, in this paper, we attack the on-orbit space camera calibration problem by using two steps. First, we only use two images of the solar panel, which is a commonly used element of majority human-made spacecraft, to generate an approximate initial estimation of the camera intrinsic parameters. In order to improve the robustness and accuracy of our method, the second step optimizes the initial solution by using an improved genetic algorithm (IGA). Simulated and real experiments prove that the proposed method is accurate and flexible, and shows good robust performance. Therefore, our method has realistic significance for various space tasks.

A novel three-dimensional surface reconstruction method for the complex fabrics based on the MVS

To solve the existing problems occurring in the field of digital fabric appearance analysis, such as information loss, visual expression difficulty and low accuracy, a novel three-dimensional (3-D) surface reconstruction method based on the multi-view stereo (MVS) technology was proposed for the appearance evaluation of complex fabrics in this paper. Initially, the fabric images were captured by the self-developed multiple image acquisition system. Subsequently, the dense point cloud of fabric surface could be obtained by the operations of feature point detection and matching, camera intrinsic and extrinsic parameter calculation, image pre-processing, sparse point cloud generation, patch expansion and patch filtering. For the sake of acquiring the high accuracy and great completeness of dense point cloud, planar patches with different scales were used to fit the fabric surface, the normalized cross correlation (NCC) algorithm was adopted to improve the accuracy of photo-consistency measurement, and besides, a derivative-free particle swarm optimization (PSO) method was applied to obtain the optimal patches. Our experimental results show that the fabric surface model could be reconstructed by this method accurately, which can be used to characterize the surface texture and 3D weaving profile of woven fabrics. The reconstructed 3D fabric surface model with high precision can be used not only for three-dimensional illustration or rendering but also for structural analysis, quality evaluation of fabric appearance and etc., it provides a useful solution for artificial intelligence production in the textile industry.

Inertial measurement unit aided extrinsic parameters calibration for stereo vision systems

Calibration of a stereo vision system is essential for three-dimensional shape and deformation measurements. Although the target-based calibration technique is a good solution, it comes with limitations in a large field of view (FOV) measurement due to the target size and difficulties in operation. In this paper, a method using the inertial measurement unit (IMU) has been proposed to calibrate stereo vision systems for large-scale measurements. The prerequisite for this method is to calibrate the camera intrinsic parameters in advance. With the alignment of IMU sensor with the camera, the relative rotation between the two cameras can be determined through coordinate transform. An angular differential method has been developed to improve accuracy in determining the yaw angle. After the epipolar geometry is constructed upon stereo scene correspondences, the translation vector can be obtained by solving a set of linear equations, since the intrinsic parameters and the relative rotation are predefined. The proposed method does not require any known calibration target, leading to easy operation in practice. Two laboratory experiments are presented to demonstrate the use of the method. The experimental results show that the accuracy of the proposed method is comparable to that resulted from the well-known Zhang's method, and it can also be applied to measurement with a large FOV.

Calibration methods for a camera with a tilted lens and a three-dimensional laser scanner in the Scheimpflug condition.

A three-dimensional laser scanner has been designed and widely utilized in many fields. The lens plane is tilted according to the Scheimpflug condition and the optical axis is not perpendicular to the charge-coupled device plane. In this case, depth of view can be extended significantly. In this paper, analytical models for a camera with a tilted lens and a laser scanner meeting the Scheimpflug condition are presented. Based on these models, a calibration procedure is detailed. We propose a simple calibration method to determine the intrinsic parameters of a Scheimpflug camera. Meanwhile, two calibration methods for a laser scanner in the Scheimpflug condition are detailed. According to the obtained intrinsic parameters, the laser scanner can be calibrated directly. Moreover, a simple calibration for a three-dimensional laser scanner without the help of a precise positioning system is described. Experimental results show the effectiveness and high measurement accuracy of our calibration methods.

Camera calibration based on the common pole-polar properties between two coplanar circles with various positions.

Computer vision camera calibration is widely performed using parallel circles. Various cases of two coplanar circles are algebraically explained, proving that the common pole is located at the line at infinity for all relative positions, and the corresponding polar passes through the centers of the two circles. The two common poles of the two coplanar circles are the points at infinity when concentric; one common pole of the two coplanar circles is a point at infinity when nonconcentric. Accordingly, the vanishing line can be obtained by using the common pole-polar properties of two groups of two coplanar circles, and the camera's intrinsic parameters are solved according to the constraints between the image of the circular points and the imaged absolute conic. The camera calibration can be solved using only three images of two coplanar circles. Simulation and experiments verify that the proposed algorithms are effective.

Confidence-based camera calibration with modified census transform

Most camera calibration methods assume that the camera is nearly in-focus for precisely estimating the intrinsic and extrinsic camera parameters. However, the camera is generally out-of-focus in real calibration applications. This paper presents a method that is capable of accurately calibrating an out-of-focus camera. A confidence-based camera calibration with modified census transform (MCT) is proposed for a checkerboard calibration pattern. No additional devices or special calibration targets are required. First, the MCT utilizes the intensity information and local image structure to construct a descriptor. Then, the dissimilarity between the control point on the real image and the one on the sharp image is evaluated using the Hamming distance. This similarity metric is treated as the confidence of each control point. Finally, the confidence of each control point is added to the calibration energy minimization procedure to enhance the calibration performance. The experimental results on real images demonstrate that the proposed method achieves a more accurate calibration result than conventional methods.

Purely Image-Based Pose Stabilization of Nonholonomic Mobile Robots With a Truly Uncalibrated Overhead Camera

Although many vision-based control methods have been proposed for nonholonomic mobile robots, in their implementation, it is usually necessary to calibrate the camera intrinsic and/or extrinsic parameters using offline/online parameter estimation algorithms or online adaptation laws. To avoid the tediousness of camera calibration and to make the system performance highly robust to camera parameter uncertainties, in this article, we propose novel image-based pose stabilization control approaches for nonholonomic mobile robots with a truly uncalibrated overhead fixed camera. In the proposed approaches, only image position information of three feature points from an overhead camera is used for controller design, while information from other sensors (such as wheel encoders) is not required. Furthermore, either offline or online camera calibration is not necessary, and no knowledge about the camera intrinsic and extrinsic parameters is needed, which also can greatly simplify the controller implementation. Simulation and experimental results are given to demonstrate the feasibility and effectiveness of the proposed purely image-based pose stabilization approaches.

Camera Intrinsic Parameters Estimation by Visual–Inertial Odometry for a Mobile Phone With Application to Assisted Navigation

The increasing computing and sensing capabilities of modern mobile phones have spurred research interests in developing new visual–inertial odometry (VIO) techniques to turn a smartphone into a self-contained vision-aided inertial navigation system for various applications. Smartphones nowadays use cameras with optical image stabilization (OIS) technology to reduce image blurs. However, the mechanism may result in varying camera intrinsic parameters (CIP), which must be taken into account in VIO computation. In this article, we first develop a linear model to relate the CIP with the inertial measurement unit measured acceleration. Based on the model, we introduce a new VIO method, called CIP-VMobile, which treats CIP as state variables and tightly couples them with other state variables in a graph optimization process to estimate the optimal state. The method uses the linear model to construct a factor graph and uses the linear-model-computed values as initial CIP estimates to speed up the VIO computation and attain a better pose estimation result. Simulation and experimental results with an iPhone 7 validate the method's efficacy. Based on CIP-VMobile, we fabricated a robotic navigation aid (RNA) based on an iPhone 7 for assisted navigation. Experimental results with the RNA demonstrate CIP-VMobile's promise in real-world navigation applications.

ZoomNet: Part-Aware Adaptive Zooming Neural Network for 3D Object Detection

3D object detection is an essential task in autonomous driving and robotics. Though great progress has been made, challenges remain in estimating 3D pose for distant and occluded objects. In this paper, we present a novel framework named ZoomNet for stereo imagery-based 3D detection. The pipeline of ZoomNet begins with an ordinary 2D object detection model which is used to obtain pairs of left-right bounding boxes. To further exploit the abundant texture cues in rgb images for more accurate disparity estimation, we introduce a conceptually straight-forward module – adaptive zooming, which simultaneously resizes 2D instance bounding boxes to a unified resolution and adjusts the camera intrinsic parameters accordingly. In this way, we are able to estimate higher-quality disparity maps from the resized box images then construct dense point clouds for both nearby and distant objects. Moreover, we introduce to learn part locations as complementary features to improve the resistance against occlusion and put forward the 3D fitting score to better estimate the 3D detection quality. Extensive experiments on the popular KITTI 3D detection dataset indicate ZoomNet surpasses all previous state-of-the-art methods by large margins (improved by 9.4% on APbv (IoU=0.7) over pseudo-LiDAR). Ablation study also demonstrates that our adaptive zooming strategy brings an improvement of over 10% on AP3d (IoU=0.7). In addition, since the official KITTI benchmark lacks fine-grained annotations like pixel-wise part locations, we also present our KFG dataset by augmenting KITTI with detailed instance-wise annotations including pixel-wise part location, pixel-wise disparity, etc.. Both the KFG dataset and our codes will be publicly available at https://github.com/detectRecog/ZoomNet.

Robust uncalibrated visual stabilization for nonholonomic mobile robots

ABSTRACT This paper proposes a new image-based visual servoing scheme for stabilizing the nonholonomic mobile robot equipped with a monocular vision system to the desired configuration. In the vision system, neither intrinsic nor extrinsic camera parameters are calibrated. In addition, the lack of depth information should be tackled when using a monocular camera. The main difficulty encountered lies in the simultaneous existence of nonholonomic constraints, uncalibrated camera parameters and unknown depth. To deal with these problems, a dual-loop control structure is utilized. For the inner loop, an extended state observer is constructed to observe the system uncertainties caused by the uncalibrated camera parameters and unknown depth. For the outer loop, a switching controller is synthesized based on the compensated system model to maneuver the mobile robot to the desired configuration. Simulations and experimental results illustrate the effectiveness and robustness of the proposed control scheme.

An Effective Adversarial Attack on Person Re-Identification in Video Surveillance via Dispersion Reduction

Person re-identification across a network of cameras, with disjoint views, has been studied extensively due to its importance in wide-area video surveillance. This is a challenging task due to several reasons including changes in illumination and target appearance, and variations in camera viewpoint and camera intrinsic parameters. The approaches developed to re-identify a person across different camera views need to address these challenges. More recently, neural network-based methods have been proposed to solve the person re-identification problem across different camera views, achieving state-of-the-art performance. In this paper, we present an effective and generalizable attack model that generates adversarial images of people, and results in very significant drop in the performance of the existing state-of-the-art person re-identification models. The results demonstrate the extreme vulnerability of the existing models to adversarial examples, and draw attention to the potential security risks that might arise due to this in video surveillance. Our proposed attack is developed by decreasing the dispersion of the internal feature map of a neural network to degrade the performance of several different state-of-the-art person re-identification models. We also compare our proposed attack with other state-of-the-art attack models on different person re-identification approaches, and by using four different commonly used benchmark datasets. The experimental results show that our proposed attack outperforms the state-of-art attack models on the best performing person re-identification approaches by a large margin, and results in the most drop in the mean average precision values.

基于改进粒子群算法的相机内参优化方法

基于改进粒子群算法的相机内参优化方法

Cadastral Surveys with Non-metric Camera Using Uav: A Feasibility Study

Orthophoto maps are believed by mapping communities as a favorable media to generate land parcel boundaries for cadaster survey related projects. However, since burgeoning off-the-shelf cameras mounted on the unmanned aerial vehicle (UAV) are commonly utilized for photographing such boundaries, unreliable and unstable intrinsic parameters of these non-metric cameras impede good quality orthophoto productions. This paper presents an alternative method to measure the boundaries reliably without an existence of the orthophoto. A degraded quality of the orthophoto can be circumvented by our newly proposed method so called direct visual referencing. This method comprises two stages. The first step is to perform on the fly camera calibration to minimize instabilities of the intrinsic components of the non-metric camera. Modifying common and widely known flying paths for aerial photogrammetry mission is enabling a block variant self-calibrating bundle adjustments to proceed. The second step is a digitation process. Carefully selected Premark or prominent features along the boundaries are digitized on arbitrary selected images. These features are then matched to the similar ones onto all available images by performing multi photo geometrically constraint least squares image matching. Final results are 3D coordinates from the multi photos triangulation process. These boundaries coordinates are compared against the GPS-RTK measurements on the field. Deviations of these types of coordinates are around 1 cm. It is obvious that this method meets the precision requirement of the GPS-RTK measurements. Therefore we firmly believed that conducting UAV’s cadastral surveys using direct visual referencing is very promising in the near future.

Passive vision measurement for robust reconstruction of molten pool in wire and arc additive manufacturing

Robust measurement of molten pool in wire and arc additive manufacturing can help better understand the geometrical size of fabricated parts and even offer feedback to control this process. Most of existing detection methods are not sufficient to express full geometric features of molten pool. In this paper, a novel passive vision sensing system using a single camera is designed based on the refraction principle of a biprism. Intrinsic and external parameters of left-right virtual cameras and their relative position are solved through a planar pattern, and an image pair is rectified by transmission projection. A global-based iterative matching algorithm is optimized for calculating the left-right parallax images of the image pair, and coordinates of space points are calculated via the triangle measuring method. The proposed sensing system and algorithms are validated by a standard cylinder. The results show that the width error of the calculated cylinder appearance is approximately 3.02% and the maximum height error is 4.15%. Finally, molten pool geometry in gas metal arc (GMA) additive manufacturing is restructured, proving the effectiveness of the virtual binocular vision system and the accuracy of the reconstruction algorithms.

Assessing Stability Performance of Non-Metric Camera’s Lens Distortion Model during UAV Flight Missions

The process and definition of camera calibration have change greatly over recent years. Aerial metric cameras calibration, for which laboratory and field calibration procedures were a separate process, was performed before and independent of any actual mapping data collection using precise calibration fixtures with an assumption that the camera parameters determined would remain valid for a significant period. In contrast, non-metric cameras are characterized by unstable intrinsic parameters over the times and they are vulnerable to the engine and other vibrations during flight data acquisitions. Moreover, there is no standard calibration procedures exist for these cameras. But, since non-metric camera self-calibration augments the concept of calibration as a part of the measurement process, it can determine the camera intrinsic parameters at the time of the data acquisition as long as highly-convergent geometry and the use of multiple exposures are employed. Therefore, this paper investigates variations of the lens distortion components with object distance within the photographic field by using the self-calibration method. The use of redundant flight paths and tilted camera is also incorporated to ascertain the stability of the principal distance and the principal points during the flight mission. During the experiments, a series of flight mission is conducted to photograph test field areas from over a relatively flat area to highly mountainous one. It is revealed that the radial, decentering, and affinity distortion parameters are more stable than that of the principal distance and principal points against vibrations.