Pinhole Imaging Model Research Articles

Pupil-centric imaging model for zoom camera calibration.

Zoom camera calibration has always been challenging, as arbitrary zoom/focus settings change the camera parameters. Current calibration methods are based on the pinhole imaging model, which results in coupled calibration parameters due to the lack of geometric/physical constraints. In this Letter, we present a novel, to the best of our knowledge, pupil-centric imaging model that accounts for the camera's radial, decentering, and mustache distortions at various zoom settings using exit pupil offsets and a non-frontal lens-sensor model. Therefore, we provide a reasonable physical explanation for the different distortion effects. Global optimization is performed based on the proposed initial camera calibration and bundle adjustment under several zoom and autofocus setting combinations. Experiments using three representative zoom cameras demonstrate the effectiveness of the proposed method. Its relative measurement accuracy is better than that of current state-of-the-art methods.

A Novel Method for Extracting DBH and Crown Base Height in Forests Using Small Motion Clips

The diameter at breast height (DBH) and crown base height (CBH) are important indicators in forest surveys. To enhance the accuracy and convenience of DBH and CBH extraction for standing trees, a method based on understory small motion clips (a series of images captured with slight viewpoint changes) has been proposed. Histogram equalization and quadtree uniformization algorithms are employed to extract image features, improving the consistency of feature extraction. Additionally, the accuracy of depth map construction and point cloud reconstruction is improved by minimizing the variance cost function. Six 20 m × 20 m square sample plots were selected to verify the effectiveness of the method. Depth maps and point clouds of the sample plots were reconstructed from small motion clips, and the DBH and CBH of standing trees were extracted using a pinhole imaging model. The results indicated that the root mean square error (RMSE) for DBH extraction ranged from 0.60 cm to 1.18 cm, with relative errors ranging from 1.81% to 5.42%. Similarly, the RMSE for CBH extraction ranged from 0.08 m to 0.21 m, with relative errors ranging from 1.97% to 5.58%. These results meet the accuracy standards required for forest surveys. The proposed method enhances the efficiency of extracting tree structural parameters in close-range photogrammetry (CRP) for forestry. A rapid and accurate method for DBH and CBH extraction is provided by this method, laying the foundation for subsequent forest resource management and monitoring.

Multi-view fringe projection profilometry for surfaces with intricate structures and high dynamic range

Fringe projection profilometry plays an important role for quality control in production line. However, it is facing challenges in the measurement of objects with intricate structures and high dynamic range that involved in precision manufacturing and semiconductor packaging. In this paper, a multi-view fringe projection profilometry system, which deploys a vertical telecentric projector and four oblique tilt-shift cameras, is presented to address the “blind spots” caused by shadowing, occlusion and local specular reflection. A flexible and accurate system calibration method is proposed, in which the corrected pinhole imaging model is used to calibrate the telecentric projection, and the unified calibration is performed by bundle adjustment. Experimental results show that the 3D repeated measurement error and standard deviation are no more than 10 μm within a measurable volume of 70 × 40 × 20 mm3. Furthermore, a group of experiments prove that the developed system can achieve complete and accurate 3D measurement for high dynamic range surfaces with complex structures.

Real-time analysis of star sensor attitude accuracy

In the conventional methods of star sensor accuracy evaluation, neither the accuracy calculation formula nor the Monte Carlo method can comprehensively reflect the attitude measurement accuracy of star sensors in real-time. In this paper, a real-time analysis and calculation model for the attitude measurement accuracy of star sensors is proposed. Firstly, the basic attitude measurement model of the star sensor is established through the pinhole imaging model. Moreover, a moving frame of star sensor attitude measurement is established to avoid the defects of attitude representation by the Euler angle in the star sensor accuracy analysis. Then, combined with the error transfer of implicit overdetermined equations proposed in this paper, the error transfers of starspot extraction random error, star catalog random error, and intrinsic parameter system error are provided. Finally, simulation and field experiments are conducted to verify the accuracy of the proposed error analysis theory. The experimental results show that the attitude measurement accuracy of star sensors can be accurately estimated by using single-frame data of guide stars.

A high-accuracy measurement method for shield tail clearance based on line structured light

A high-accuracy measurement method for shield tail clearance based on line structured light is presented in this article. The method depends on monocular vision imaging and laser triangulation, which utilizes a line laser and an industrial camera. Firstly, the laser is used to project a narrow line-shaped indicator laser between shield shell and segment. The camera captures two laser lines in the image to acquire their feature information. Then, the 3D coordinates of the clearance endpoints are calculated by the pinhole imaging model and the spatial position relation. Subsequently, the shield tail clearance is measured by calculating the distance between the two endpoints. Several experiments are designed to verify the measurement performance in this article. The results indicate that within the measurement range, the repeatability accuracy is better than 0.29 mm and the deviation is less than 1.7 mm, which can meet the construction demands for shield tail clearance.

A refractive three-dimensional reconstruction method for fringe projection with a planar medium

In special measurement situations, a vision measurement system needs to measure objects through a viewing observation window made of a planar transparent medium. The refraction at the interface between a medium and air leads to large measurement errors in the pin-hole imaging model. Most existing solutions require complex processes of calculating refracted light equations and/or matching stereo images. Here, we propose a refractive three-dimensional (3D) reconstruction method for fringe projection with a planar medium. We derive a coordinate conversion relationship from a pseudo-object point caused by light refraction to a real-object point based on flat refractive geometry. Then, we integrate the relationship into regular fringe projection framework for unbiased 3D reconstruction. Two experiments, including 3D shape measurements of a step and a regular sphere are performed. The results verify the effectiveness and accuracy of the proposed refractive 3D reconstruction method.

Measurement Technologies of Light Field Camera: An Overview.

Visual measurement methods are extensively used in various fields, such as aerospace, biomedicine, agricultural production, and social life, owing to their advantages of high speed, high accuracy, and non-contact. However, traditional camera-based measurement systems, relying on the pinhole imaging model, face challenges in achieving three-dimensional measurements using a single camera by one shot. Moreover, traditional visual systems struggle to meet the requirements of high precision, efficiency, and compact size simultaneously. With the development of light field theory, the light field camera has garnered significant attention as a novel measurement method. Due to its special structure, the light field camera enables high-precision three-dimensional measurements with a single camera through only one shot. This paper presents a comprehensive overview of light field camera measurement technologies, including the imaging principles, calibration methods, reconstruction algorithms, and measurement applications. Additionally, we explored future research directions and the potential application prospects of the light field camera.

Accurate and flexible calibration method for a 3D microscopic structured light system with telecentric imaging and Scheimpflug projection.

3D imaging and metrology of complex micro-structures is a critical task for precision manufacturing and inspection. In this paper, an accurate and flexible calibration method for 3D microscopic structured light system with telecentric imaging and Scheimpflug projector is proposed. Firstly, a fringe projection 3D microscopy (FP-3DM) system consisting of a telecentric camera and a Scheimpflug projector is developed, which can take full advantage of the depth of field (DOF) and increase the measurement depth range. Secondly, an accurate and flexible joint calibration method is proposed to calibrate the developed system, which utilizes the established pinhole imaging model and Scheimpflug distortion model to calibrate telecentric imaging, and fully considers the correction and error optimization of the Scheimpflug projection model. Meanwhile, the optimized local homography is calculated to obtain more accurate sub-pixel correspondence between the camera and the projector, and the perspective-n-point (PnP) method make the 3D coordinate estimation of the feature point more accurate. Finally, a prototype and a dedicated calibration program are developed to realize high-resolution and high-precision 3D imaging. The experimental results demonstrate that the re-projection error is less than 1µm, and the 3D repeated measurement error based on feature fitting is less than 4µm, within the calibrated volume of 10(H)mm × 50(W)mm × 40(D)mm.

Generalized Scale Factor Calibration Method for an Off-Axis Digital Image Correlation-Based Video Deflectometer.

When using off-axis digital image correlation (DIC) for non-contact, remote, and multipoint deflection monitoring of engineering structures, accurate calibration of the scale factor (SF), which converts image displacement to physical displacement for each measurement point, is critical to realize high-quality displacement measurement. In this work, based on the distortion-free pinhole imaging model, a generalized SF calibration model is proposed for an off-axis DIC-based video deflectometer. Then, the transversal relationship between the proposed SF calibration method and three commonly used SF calibration methods was discussed. The accuracy of these SF calibration methods was also compared using indoor rigid body translation experiments. It is proved that the proposed method can be degraded to one of the existing calibration methods in most cases, but will provide more accurate results under the following four conditions: (1) the camera's pitch angle is more than 20°, (2) the focal length is more than 25 mm, (3) the pixel size of the camera sensor is more than 5 um, and (4) the image y-coordinate corresponding to the measurement point after deformation is far from the image center.

High-precision visual imaging model and calibration method for multi-depth-of-field targets

In the field of high-precision large-depth 3D reconstruction, some bottlenecks exist in the improvement of visual measurement accuracy. This is because traditional vision measurement methods simplify the imaging model of a fixed-focus camera to a pinhole imaging model with fixed parameters accompanied by various distortion information. In fact, according to the optical properties of camera imaging, the position of the optical center in the pinhole imaging model varies with the depth of the imaging target. In this paper, in order to achieve the objective of improving visual measurement accuracy, a high-precision visual imaging model for multi-depth-of-field targets and the corresponding camera parameters calibration method are proposed based on the optical properties of camera imaging. When using this imaging model for 3D measurements, different pixel points in the image use different camera internal parameters depending on the depth of the imaging target, thus enabling highly accurate 3D reconstruction. The experimental results show that the imaging model and calibration method proposed in this paper greatly improve the accuracy and robustness of visual 3D measurement and have a promising future in the field of large-depth 3D reconstruction.

A linear pushbroom satellite image epipolar resampling method for digital surface model generation

Epipolar resampling, also known as stereo rectification, is a prerequisite for optical satellite image stereo matching in digital surface model (DSM) generation. In epipolar resampling, image pairs are warped and thus can be matched pixel-to-pixel along the epipolar line. Different from the pinhole imaging model of frame cameras, the multi-center projection model of linear pushbroom sensors does not obey the epipolar geometric constraint, making it impossible to conjugate epipolar lines from stereo images. The epipolar resampling methods can resolve some of the inherent epipolar error, but not completely. The topographic relief brings high disparity values between epipolar image pairs, posing a challenge to the subsequent stereo matching to recover disparity map. In this paper, we introduce a novel epipolar resampling method for satellite imagery based on ortho rectification. This method takes full advantage of a rational function model to establish a connection between image space and object space coordinates, and thereby transform the two-dimensional disparity in ortho rectification products into one dimension. We propose a coarse-to-fine DSM generation pipeline to overcome the high-disparity-problem without additional time costs or precision decrement. We conducted experiments on various satellite stereo images, and validated both the extreme suppression on epipolar error as well as the feasibility of the proposed coarse-to-fine DSM generation pipeline. The experimental results show that the generated epipolar image pairs from our proposed method effectively solved the high disparity problem. Furthermore, the novel DSM generation method obtained an internal error below one pixel for a rectified stereo image pair in mountain area, and the lowest vertical error in the IARPA MVS dataset when compared with the common open-source programs and commercial software.

Underwater large field of view 3D imaging based on fisheye lens

As for the 3D imaging of seabed topography, seabed sunken ships and large underwater buildings, the field of view is a significant technical index of the 3D imaging system. In this paper, a binocular vision imaging system based on fisheye lens is applied to satisfy the requirements for 3D imaging of large underwater scenes. Considering the severe image distortion caused by the large wide-angle of fisheye lens and the failure of pinhole imaging model underwater, the imaging process of underwater fisheye camera is re-described through the parametric representation of four-dimensional light field, and the multi-layer refraction imaging model of underwater binocular stereo vision is established based on fisheye camera. A conversion imaging model is constructed to convert the corrected underwater fisheye image into the equivalent image taken in the air. Combined with the existing stereo matching methods, the converted images realize the 3D reconstruction of a large-scale underwater scene. Experimental results show that the corrected underwater fisheye images have a small average reprojection error, which evidences the effectiveness of the proposed calibration method. In the experiment of 3D reconstruction for the large scene composed of multiple underwater ceramic discs, the reconstructed targets have good 3D imaging shapes, which indicates the developed underwater binocular fisheye camera imaging system has the ability to perform 3D imaging and make observations of large underwater scenes.

Vision-based pose estimation algorithm with four coplanar feature points via an incident-ray-based camera model

The pinhole imaging model assumes that all the projecting rays intersect at the effective pinhole point. The restriction of this camera model will result in low accuracy of vision-based pose estimation. Furthermore, the camera calibration gets the global optimal solution of intrinsic and extrinsic parameters, which will affect the calibration accuracy of intrinsic parameters and then pose estimation accuracy is also affected. In this paper we completed the pose estimation with four coplanar feature points via an incident-ray-based camera model. Two parallel planes are used to perform the ray-pixel mapping. The projecting ray could be defined by its two intersections with the two planes. In this way, the camera is parametrized by the image plane and the two parallel planes. Based on this camera model, through four coplanar feature points, the object pose could be estimated. The whole process consists of both a non-iterative step and iterative step. In the non-iterative step, the initial estimation of the pose is obtained and then it is refined in the iterative step. Experiment results on pose estimation by the proposed algorithm and the other existing algorithms demonstrate the superiority of our algorithm.

A real-time, continuous pedestrian tracking and positioning method with multiple coordinated overhead-view cameras

This paper proposes a real-time, continuous pedestrian tracking and positioning method with multiple coordinated overhead-view cameras. First, a discriminative correlation filter with detection and space reliability (DSR-DCF) is presented for continuous, real-time pedestrian tracking with single cameras. Then, the frame difference method is employed to detect moving pedestrians when they walk from one camera’s view to another’s. Motion-area images are cropped to establish a pedestrian image library (PIL). Finally, this PIL is relied upon to reidentify pedestrians. Pedestrian positioning is realized based on the pinhole imaging model. Tracking and positioning experiments showed the following. DSR-DCF is stable with a high success rate. Its tracking speed exceeds 24 fps. It produces a success rate higher than 90% for cross-camera pedestrian tracking and misses no pedestrians. It yields stable pedestrian positioning results with an accuracy better than 0.2 m. The proposed algorithm will be used in airport terminals.

An Image Super-Resolution Reconstruction Method with Single Frame Character Based on Wavelet Neural Network in Internet of Things

The application of the traditional single frame character image super-resolution reconstruction method has some problems, such as noise can not be removed completely and anti-interference performance is poor. A new method for the super-resolution reconstruction of single frame character image based on wavelet neural network is proposed. The structure and interface of image acquisition unit of solid state image sensor are designed. Combined with pinhole imaging model and camera self-calibration, image acquisition of Internet of Things is completed. An image degradation model was established to simulate the degradation process of ideal high-resolution image to low-resolution image. Wavelet threshold denoising method is used to remove the noise in a single frame character image and improve the anti-interference performance of the method. The wavelet neural network reflection model is used to reconstruct the single frame feature image and improve the resolution of the image. The experimental results show that the blur degree of the reconstructed image is always less than 5%. In the whole experiment, the accuracy of this method can be maintained at 80% ~ 90%. The image detail retention rate of the research method is relatively stable. With the increase of the number of experimental images, the retention rate of image details remains between 80% and 95%, indicating that the method is effective in practical application.

A Rapid Method of Monocular Image Measurement Based on Rectangle Information

A method of rapid measurement of monocular image based on rectangle information is proposed in this paper. Based on the pinhole imaging model, a fast method of calculating the object size in monocular image is derived by fitting the straight line and then finding the intersection point of the straight line. The algorithm can not only measure the size and absolute depth of the object in monocular image, but also meet the conditions of the specific pinhole imaging model with a distance far greater than the focal length. The complexity of the algorithm is low, and it can meet the requirements of real-time calculation. It provides a new idea for the use of small and micro-computing platform.

Optical-CT Dual-Modality Mapping Base on DRR Registration

Optical-CT dual-modality imaging requires the mapping between 2D fluorescence images and 3D body surface light flux. In this paper, we proposed an optical-CT dual-modality image mapping algorithm based on the Digitally Reconstructed Radiography (DRR) registration. In the process of registration, a series of DRR images were computed from CT data using the ray casting algorithm. Then, the improved HMNI similarity strategy based on Hausdorff distance was used to complete the registration of the white-light optical images and DRR virtual images. According to the corresponding relationship obtained by the image registration and the Lambert’s cosine law based on the pin-hole imaging model, the 3D light intensity distribution on the surface of the object could be solved. The feasibility and effectiveness of the mapping algorithm are verified by the irregular phantom and mouse experiments.

A Survey of Calibration Methods for Traditional Cameras Based on Line Structure Light

Abstract The line structure light three-dimensional reconstruction system is a kind of three-dimensional non-contact measurement system, which has the advantages of high precision, high speed, small damage to objects and strong adaptability. Camera calibration is a major factor that constrains the accuracy of 3D measurement systems. The camera calibration is based on the pinhole imaging model, and through a series of complex calculations, the camera’s internal parameters (focal length, distortion coefficient) and external parameters (rotation matrix and translation vector). The different calibration methods use different calibration targets, which can be divided into 3D calibration targets, 2D calibration targets, and one-dimensional calibration targets according to the characteristics of the calibration targets. This paper mainly discusses: calibration content and significance, calibration methods for different targets and evaluation methods for calibration of different targets. Firstly, the content and significance of calibration are expounded. Then, according to different calibration targets, the calibration algorithm is analyzed. Finally, the calibration algorithm is analyzed and summarized, and the development trends, advantages and disadvantages of different calibration methods are pointed out.

Efficient block matching using improved particle swarm optimization with application to displacement measurement for nano motion systems

This paper is devoted to the micro vision based displacement measurement for a nano-positioning stage with the aid of a single camera. For this purpose, a pinhole imaging model as well as corresponding Jacobin matrix is established, with which an efficient block matching (BM) algorithm is developed based on the particle swarm optimization (PSO) algorithm. In particular, an improved PSO algorithm is developed by proposing a new fitness updating strategy (FUS) such that the particles’ migration can be performed with an efficient way in the whole optimization process. The improved PSO can significantly speed up the optimization process, but meanwhile resulting in a coarse-level estimation result. The full search algorithm (FSA) is thus performed on the coarse-level result in order to achieve the final fine-level result. The effectiveness of the presented method was demonstrated using the testing data acquired from an experimental platform, where comparative studies with those existing benchmark methods were also provided. The comparison results show that the method achieves a significant improvement on computational efficiency without loss of accuracy, which implies it has a better balance/trade-off between the measurement accuracy and computation efficiency in real applications.

Calibration Method of Orthogonally Splitting Imaging Pose Sensor Based on General Imaging Model

Orthogonally splitting imaging pose sensor is a new sensor with two orthogonal line array charge coupled devices (CCDs). Owing to its special structure, there are distortion correction and imaging model problems during the calibration procedure. This paper proposes a calibration method based on the general imaging model to solve these problems. The method introduces Plücker Coordinate to describe the mapping relation between the image coordinate system and the world coordinate system. This paper solves the mapping relation with radial basis function interpolation and adaptively selecting control points with Kmeans clustering method to improve the fitting accuracy. This paper determines the appropriate radial basis function and its shape parameter by experiments. And these parameters are used to calibrate the orthogonally splitting imaging pose sensor. According to the calibration result, the root mean square (RMS)of calibration dataset and the RMS of test dataset are 0.048 mm and 0.049 mm. A comparative experiment is conducted between the pinhole imaging model and the general imaging model. Experimental results show that the calibration method based on general imaging model applies to the orthogonally splitting imaging pose sensor. The calibration method requires only one image corresponding to the target in the world coordinates and distortion correction is not required to be taken into account. Compared with the calibration method based on the pinhole imaging model, the calibration procedure based on the general imaging model is easier and accuracy is greater.