Lens Distortion Research Articles

Distortion correction algorithm based on absolute phase image in structured light 3D reconstruction

Abstract This paper proposes a distortion correction algorithm based on absolute phase maps for a dual-projector, single-camera structured light 3D shape measurement system. Distortion correction is performed separately for the projector-camera pairs on the left and right. By projecting a ninth-order complementary Gray code combined with the eight-step phase-shifting method onto a white board, absolute phase maps are obtained, solving the impact of radial and tangential lens distortion. A sub-pixel level distortion error lookup table is introduced to address the influence of residual error after distortion correction on measurement accuracy, improving the overall precision by at least 95.7%. Experiments demonstrate that the dual-projector, single-camera structured light 3D shape measurement system expands the system’s measurement range while reducing shadowing issues caused by single-projector illumination. The proposed distortion correction and error compensation algorithms effectively enhance the overall measurement accuracy of the system and significantly improve issues like warping caused by distortion.

A fisheye image correction method based on deep learning

Purpose Fisheye camera is a wide-angle camera with a large field of view, which can obtain more information than conventional cameras, but its own disadvantage of large distortion can lead to shape distortion of objects in the captured image. To correct lens distortion, most scholars use manual calibration methods. The traditional manual calibration method relies on complex data computation and specialized mathematical knowledge, but the method is complex and not universal. Considering the nonlinear features of neural networks can be used to fit the nonlinear distortion of distorted images, this paper aims to correct the distortion of a fisheye image based on convolutional neural networks. Design/methodology/approach By generating fisheye distortion images using readily available cityscape road data sets, this paper designs an effective correction model by taking advantage of the fact that the relationship between the pixel coordinates is basically stable after uniform distortion. In our training process, the corresponding distortion is firstly synthesized using the original image, the loss function is constructed using the geometric means of the Hough transform, the whole model is then trained with the help of the loss of the linear slope, and finally the predicted parameters are used to correct the fisheye image, which realizes the overall end-to-end framework. Findings Experimental results show that the method proposed in this paper outperforms similar methods in terms of correction performance. Originality/value This paper proposed a fisheye distortion correction algorithm based on convolutional neural network, which uses distortion-free images based on a generalized fisheye imaging model and maps to generate a fisheye distortion picture data set. It also builds a loss function training model based on line reconstruction error loss of Hough transforms.

A transferable approach for quantifying benthic fish sizes and densities in annotated underwater images

Abstract Benthic fishes are a common target of scientific monitoring but are difficult to quantify because of their close association to bottom habitats that are hard to access. Advances in image‐acquisition technologies, machine vision and deep learning have made capturing and quantifying fishes with cameras increasingly feasible. We present a method and open‐source software called ‘FishScale’ to estimate benthic fish lengths, numeric abundance and biomass density in underwater environments assessed with down‐looking monocular images. ‘FishScale’ estimates fish abundances and size frequencies from near‐nadir monocular images where fish have already been semantically segmented. The software accounts for lens distortion, underwater magnification effects and fish body curvature to automatically estimate fish lengths and the areas of images where they were captured. Numeric and biomass density are estimated through a deterministic machine vision algorithm that requires a user‐provided length‐weight relationship for species of interest and calibration images. Results from validation studies show that lengths and weights can be estimated with high accuracy and precision for round goby (Neogobius melanostomus) captured in distorted action camera images, and from large‐bodied lake trout (Salvelinus namaycush) imaged with a machine vision camera. The real‐world utility of the approach is demonstrated in a case study estimating round goby abundances and size frequencies along a 10.7‐km transect surveyed with an autonomous underwater vehicle in Lake Michigan, USA. Our validation studies demonstrate that the approach estimates benthic and benthopelagic fish lengths and weights with little bias and good accuracy and precision for species with much different body shapes and sizes. The method is applicable to data collected using a variety of nadir imaging approaches with widespread applications to fisheries monitoring and quantification of any species or object for which nadir images and working distances between the camera and feature of interest are available.

Visualizing Plant Responses: Novel Insights Possible Through Affordable Imaging Techniques in the Greenhouse.

Efficient and affordable plant phenotyping methods are an essential response to global climatic pressures. This study demonstrates the continued potential of consumer-grade photography to capture plant phenotypic traits in turfgrass and derive new calculations. Yet the effects of image corrections on individual calculations are often unreported. Turfgrass lysimeters were photographed over 8 weeks using a custom lightbox and consumer-grade camera. Subsequent imagery was analyzed for area of cover, color metrics, and sensitivity to image corrections. Findings were compared to active spectral reflectance data and previously reported measurements of visual quality, productivity, and water use. Results confirm that Red-Green-Blue imagery effectively measures plant treatment effects. Notable correlations were observed for corrected imagery, including between yellow fractional area with human visual quality ratings (r = -0.89), dark green color index with clipping productivity (r = 0.61), and an index combination term with water use (r = -0.60). The calculation of green fractional area correlated with Normalized Difference Vegetation Index (r = 0.91), and its RED reflectance spectra (r = -0.87). A new chromatic ratio correlated with Normalized Difference Red-Edge index (r = 0.90) and its Red-Edge reflectance spectra (r = -0.74), while a new calculation correlated strongest to Near-Infrared (r = 0.90). Additionally, the combined index term significantly differentiated between the treatment effects of date, mowing height, deficit irrigation, and their interactions (p < 0.001). Sensitivity and statistical analyses of typical image file formats and corrections that included JPEG, TIFF, geometric lens distortion correction, and color correction were conducted. Findings highlight the need for more standardization in image corrections and to determine the biological relevance of the new image data calculations.

Robust calibration of surround-view camera systems with Planar-Eccentric Constraint: A novel Apriltag approach

This research tackles calibrating surround view camera systems for automated driving, where traditional methods require extensive manual measurements and precise conditions. We introduce a novel automatic calibration method that ensures portability and accuracy using only custom-designed calibration boards placed in the cameras’ common view. To address visual interference from extraneous objects, we design a unique board with AprilTag 2D codes, offering clear advantages over traditional checkerboards. Additionally, we have developed an Adaptive Angle Projection to counteract fisheye lens distortions. We also introduce a hierarchical pose optimization strategy that transitions from local to global, incorporating a “Planar-Eccentric constraint” to leverage the calibration board’s geometric positioning. Our method achieved 4.01 cm measurement error, with 0.075°angle error and 1.18 cm translation error. Our method’s efficacy and robustness were confirmed through extensive testing on an experimental vehicle and simulation dataset, proving its superiority in achieving precise calibration and measurement with minimal manual input.

Vision-guided robot calibration using photogrammetric methods

We propose novel photogrammetry-based robot calibration methods for industrial robots that are guided by cameras or 3D sensors. Compared to state-of-the-art methods, our methods are capable of calibrating the robot kinematics, the hand–eye transformations, and, for camera-guided robots, the interior orientation of the camera simultaneously. Our approach uses a minimal parameterization of the robot kinematics and hand–eye transformations. Furthermore, it uses a camera model that is capable of handling a large range of complex lens distortions that can occur in cameras that are typically used in machine vision applications. To determine the model parameters, geometrically meaningful photogrammetric error measures are used. They are independent of the parameterization of the model and typically result in a higher accuracy. We apply a stochastic model for all parameters (observations and unknowns), which allows us to assess the precision and significance of the calibrated model parameters. To evaluate our methods, we propose novel procedures that are relevant in real-world applications and do not require ground truth values. Experiments on synthetic and real data show that our approach improves the absolute positioning accuracy of industrial robots significantly. By applying our approach to two different uncalibrated UR3e robots, one guided by a camera and one by a 3D sensor, we were able to reduce the RMS evaluation error by approximately 85% for each robot.

Determining The Lens Aberations on Positive Lenses Through Science Process Skill

Abstract This research aims to determine aberrations or image distortions in a positive lens using an optics experiment kit for image distortions. The experimental tool consists of a diaphragm, a positive lens, a light source, lens path, an image capture screen, and a flint glass used to reduce lens distortions. The main tool is made of acrylic, which is durable and easy to use. The lens used is a strong positive lens (+100). The diaphragms used vary, including a slit diaphragm with five slits and a four-ring diaphragm. These diaphragms are used to produce image shapes and observe the distortions that occur. The work process refers to the skills of scientific processes such as observation, classification, measurement, prediction, communication, and conclusion, which are implemented in the work instructions for creating aberrations or image distortions by placing the positive lens parallel to the light source and the image capture screen. Then, the distortions that occur are observed, and a flint glass is placed in front of the light source to reduce the distortions that are formed. By observing the resulting image, it can be seen that the distortions that occur on the positive lens and how the flint glass works to reduce these aberrations. This is in line with the theory that aberrations on a positive lens occur due to imperfect light bias, either because of the different refractive index biases from different media in the lens, or because of the lens shape not being ideal. Flint glass is one of the methods used to reduce these aberrations.

An Image Distortion Correction Method Based on Distortion Centre and Parameter Estimation

Abstract We propose a radial distortion correction method based on straight-line feature calibration to address the challenge of image radial distortion when measuring mechanical motion parameters using computer vision technology. This method first independently estimates the distortion center, and then transforms the distortion parameter estimation into an arc fitting problem. The least squares optimization algorithm is applied to quickly and accurately estimate the arc parameter of distorted lines, ultimately achieving precise correction of distortion in wide-angle camera images. This method can reduce lens distortion by 89% to 92%, laying a solid foundation for the application of computer vision technology in the field of mechanical motion parameter measurement.

Analysis of Nogunt&amp;apos;an (Emerald Cloud Pool) from Chŏng Sŏn’s Album of Views in the Capital Region (Kyŏnggyo myŏngsŭng ch&amp;apos;ŏp) Using Virtual Reality Analysis Tool

The Scenic Views of Kyŏnggyo (Kyŏnggyo myŏngsŭng ch'ŏp 京郊名勝帖) is a two-volume album of poetry and paintings in 1740 by Chŏng Sŏn (sobriquet: Kyŏmjae) during his time serving as magistrate of Yangch'ŏn. It captures scenic views from his travels along the Han River. Among these, three paintings (Nogunt'an 綠雲灘, Tokpaekt'an 獨栢灘, and Uch'ŏn 牛川), feature major viewpoints along the South Han River. Each has encouraged scholarly debates on the exact locations depicted in the album due to the scarcity of historical records and topographical changes of the area’s natural landscape.In particular, there has been no research that identifies the viewpoint and pictorial subject of Nogunt'an. This difficulty arises from two factors: first, the absence of architectural structures depicted in the painting, and second, the submersion of a rock that had once been significant landmark due to the construction of the P'altang Dam. These factors have made it impossible to fully match the painting’s historical landscape with its present-day appearance.This study aims to overcome these limitations by reconstructing the landscape in a virtual space. By analyzing topographical changes and utilizing image similarity evaluation techniques from the field of computer graphics, this study also propose the precise viewpoint of Nogunt'an, which previous studies were unable to clearly identify. Additionally, this research examines the fisheye lens distortion and panoramic composition techniques that Chŏng Sŏn frequently employed in Kyŏnggyo myŏngsŭng ch'ŏp and his other landscape paintings.

A calibration method for telecentric line-scan cameras with an enhanced dynamic imaging model and initial estimates derived from direct linear transformation

This study presents a calibration method for telecentric line-scan cameras, incorporating a dynamic imaging model and initial estimates derived from direct linear transformation. The enhanced dynamic imaging model includes an inclination parameter instead of velocities in three orthogonal spatial directions. It is applicable to any direction of motion, considers lens distortion, and accommodates the line sensor at an arbitrary position relative to the optical axis. Employing the enhanced camera model, the initial values of telecentric line-scan camera parameters are determined derived from direct linear transformation. Sensitivity of our method with respect to the noise level of image points and the number of pattern planes are assessed through numerical simulations, while effectiveness and robustness are validated through experiments using real-world data, achieving a reprojection error of 0.6386 pixels and a standard deviation of 0.0160 pixels. The versatility of the proposed method can be extended to measurement applications utilizing flatbed scanners.

Systematical and universal calibration scheme for division-of-aperture polarimetric camera

Division-of-aperture polarimetric (DoAP) camera has been widely used to obtain polarization images of scene owing to its feature in simultaneous acquisition of polarization information, real-time measurement and compact system. However, practical applications of DoAP imaging inevitably involve calibrating the camera from several aspects, which is very essential but the related reports are relatively lacking in current research. In this paper, we proposed a systematical and universal calibration scheme for the DoAP camera, which includes the lens distortion correction, the image registration, the polarization channel calibration and the Stokes parameters modification. Accordingly, a software for calibration and imaging processing, which consists of four modules, is designed to collectively facilitate the efficient and accurate execution of all calibration steps. In order to achieve real-time polarimetric imaging, the lens distortion correction and the image registration were done by means of lookup tables. Experimental results by using a full-Stokes DoAP camera we fabricated indicate that, via the lens distortion correction and the image registration, the alignment error between two images with a mean absolute error (MAE) less than 0.16 pixels can be obtained, resulting in the aligned images having an average increase of 64.53 % for the structural similarity index (SSIM) and that of 84.97 % for the peak signal to noise ratio (PSNR), respectively; and further via the polarization channel calibration and the Stokes parameters modification, the MAE of 0.0146 for the degree of polarization (DoP) and that of 0.3544° for the angle of polarization (AoP) are obtained, respectively. This proposed calibration scheme for DoAP camera is systematic, automatic and robust, which is benefit of improving the polarization detection accuracy, enhancing the camera calibration efficiency and thus promoting its polarimetric imaging performance.

Measurement of bending deformation of sheet metal using machine vision

ABSTRACT The aim of this study is to propose a fast and effective method to measure the curvature distribution and bending angle of sheet metal in the bending deformation using machine vision. An open-source computer vision library (OpenCV) was used for image processing. First, the image is calibrated for perspective and lens distortion. After that, the image is subjected to color extraction and binary conversion. The outer geometry of the material was obtained using edge detection. With the proposed method, the bending angle and curvature distribution in the four-point bending and V-bending tests are measured. The proposed machine vision method exhibits errors of 0.55% and 0.12% compared with the 3D scanner measurements for the bending angle in the four-point bend and V-bend tests after springback, respectively. The curvature distribution in bending deformation was also obtained. As an application of the proposed method, the moment – curvature diagram of the material was determined based on the obtained curvature at the center. Additionally, the optimal stroke was calculated for the target bend angle during the V-bending process. The proposed method proved to be an accurate and efficient way to measure bending deformation of sheet metal.

Georectifying drone image data over water surfaces without fixed ground control: Methodology, uncertainty assessment and application over an estuarine environment

Light-weight consumer-grade drones have the potential to provide geospatial image data to study a broad range of oceanic processes. However, rigorously tested methodologies to effectively and accurately geolocate and rectify these image data over mobile and dynamic water surfaces, where temporally fixed points of reference are unlikely to exist, are limited. We present a simple to use automated workflow for georectifying individual aerial images using position and orientation data from the drone's on-board sensor (i.e. direct-georectification). The presented methodology includes correcting for camera lens distortion and viewing angle and exploits standard mathematics and camera data processing techniques. The method is used to georectify image datasets from test flights with different combinations of altitude and camera angle. Using a test site over land, directly-georectified images, as well as the same images georectified using standard photogrammetry software, are evaluated using a network of known ground control points. The novel methodology performs well with the camera at nadir (both 10 m and 25 m above ground level) and exhibits a mean spatial accuracy of ±1 m. The same accuracy is achieved when the camera angle is 30° at 10 m above ground level but decreases to ±2.9 m at 30° and 25 m. The accuracy changes because the uncertainties are a function of the altitude and angle of the camera versus the ground. Drone in-flight positioning errors can reduce the accuracy further to ±5 m with the camera at 30° and 25 m. An ensemble approach is used to map the uncertainties within the camera field-of-view to show how they change with viewing distance and drone position and orientation. The complete approach is demonstrated over an estuarine environment that includes the shoreline and open water, producing results consistent with the land-based field-tests of accuracy. Overall, the workflow presented here provides a low cost and agile solution for direct-georectification of drone-captured image data over water surfaces. This approach could be used for collecting and processing image data from drones or ship-mounted cameras to provide observations of ocean colour, sea-ice, ocean glitter, sea surface roughness, white-cap coverage, coastal water quality, and river plumes. The Python scripts for the complete image georectification workflow, including uncertainty map generation, are available from https://github.com/JamieLab/SArONG.

Human height estimation using AI-assisted computer vision for intelligent video surveillance system

In urban areas, technological advancements have led to an increased focus on height as a critical human characteristic for surveillance purposes. Face recognition often encounters challenges due to occlusion and masks, necessitating the use of height, build, and torso. Accurately estimating human height in surveillance scenarios is complex due to camera calibration, posture variations, and movement patterns. This research introduces a novel human height estimation method for surveillance systems, along with a dedicated dataset. The process begins with camera calibration to rectify lens distortions. A deep learning-based YOLOv7-Occlusion Aware (YOLOv7-OA) target detection technique is employed to precisely locate individuals within the frame. The study assesses the impact of camera height and deflection angle on height estimation across different areas of the field of vision (FOV). The proposed method yields a mean absolute error of 0.02 cm to 0.8 cm across various FOV zones, surpassing the previous 1.39 cm benchmark findings.

54‐1: Development of UDC Image Restoration Technology Using Space Variant CNN

With the continuous evolution of mobile device technologies, the integration of under‐display cameras has emerged as a groundbreaking innovation in the pursuit of bezel‐less and immersive visual experiences. As this technology becomes increasingly prevalent, it introduces a unique set of challenges, particularly in the realm of photography. In particular, the flare cannot be restored by conventional image processing due to saturated pixels. Therefore, a deep learning network capable of restoring such deterioration is proposed. The network is trained by a synthetic datasets generated by accurate optical simulation. However, the distortion of the camera lens distorts the shape of flare differently depending on its position in the photo, and it degrades restore quality. In this paper, we propose space‐variant CNN for solving distortion problem.

Improving radial lens distortion correction with multi-task learning

With computer vision and machine learning, sports image analysis can enhance viewer engagement and contribute to the overall understanding of sports events. However, beneath the apparent simplicity of sports images lies a complex challenge of radial distortion, which can impede their accurate interpretation. The need for high precision and real-time performance is paramount. We present a new regression-based method for radial distortion correction that improves the accuracy of distortion model coefficient prediction by introducing a secondary learning task that compares the distortion level of two random training samples. The secondary task requires no additional annotation, and because it is also related to radial distortion, it encourages the common feature extractor component to learn more general and robust features while preventing overfitting and improving the efficiency of training data. We have evaluated our proposed method using two public datasets and compared it against five other methods. Our method surpassed them all, both in the accuracy of the radial distortion correction and speed as well. You can find the source code and trained models at https://vgg.fiit.stuba.sk/improving-radial.

High-throughput fluorescence quantification method based on inner filter effect and fluorescence imaging analysis

The application of the inner filter effect (IFE) in fluorescent substance determination is gaining popularity. In this paper, a theory of the fluorescence distribution along with the excitation light path is derived from our previous research about the spatial micro-element method. According to the relationship between the summation of fluorescence intensities along the vertical direction at a certain position on the excitation light path and the position, a high-concentration and wide-range fluorescent substance quantification method based on the IFE and fluorescence imaging analysis is proposed. Correspondingly, a high-throughput fluorescent substance quantification detection system is constructed. In order to validate the method, solutions of rhodamine B in different concentrations are used for principle validation, concentration prediction, and experimental investigation on the influence of integration time and lens distortion. The high-throughput system enables the simultaneous measurement of six samples, realizing the high-concentration and wide-range quantification of rhodamine B (100–600 mg/L) with high precision (R2 = 0.9992, MRE = 2.34 %). By setting the filter wheel, the system can measure the concentration of fluorescent substances with different emission wavelengths. The improvement of experimental device is expected to reduce the single sample capacity to tens of microliters and increase the overall sample quantity to tens or even hundreds. The proposed method and system are beneficial to fluorescence measurement in fields such as biomedicine and dye research and to the improvement of high-throughput fluorescence quantitative PCR instruments.

Simultaneous geometric calibration and orbit-attitude determination of Hayabusa2’s deployable camera (DCAM3)

Hayabusa2’s deployable camera (DCAM3) was deployed from the spacecraft near the asteroid Ryugu to successfully monitor the artificial cratering experiment. Scientific analyses of the observed impact ejecta require accurate determination of the camera’s position and orientation. However, in contrast to conventional spacecraft operations, DCAM3 lacked the capability to acquire orbit tracking data and attitude sensor data due to its limited onboard resources. Even though the optical images obtained by DCAM3 itself are the only source of geometric information, they were subject to significant distortion caused by the rolling shutter effect and lens distortion. This research is, therefore, designed to simultaneously estimate the image distortion and the orbital and attitude motions of DCAM3, relying solely on its image data. The relative geometry between the camera and the asteroid is reconstructed from the geographic information of the observed feature points by incorporating distortion effects. The proposed method involves segmenting the geometry reconstruction, with more than ten thousand feature measurements, into smaller-scale least-squares problems. The image-based estimation yields consistent distortion, orbit, and attitude solutions with pixel-scale accuracy. The analysis results indicate that DCAM3 experienced significant nutation of its optical axis during ballistic flight in a semi-elliptical orbit. In addition, these motions were sensitive enough to simultaneously determine system parameters, such as the gravitational parameter of Ryugu and the inertia moment of DCAM3. This paper illustrates the potential of deployable camera systems as a promising option for enhancing the scientific and engineering aspects of asteroid exploration.

Minimizing optical attribute errors for a lane departure warning system using an ultra-wide-angle camera

Advanced driver assistance systems (ADAS) rely on lane departure warning (LDW) technology to enhance safety while driving. However, the current LDW method is limited to cameras with standard angles of view, such as mono cameras and black boxes. In recent times, more cameras with ultra-wide-angle lenses are being used to save money and improve accuracy. However, this has led to some challenges such as fixing optical distortion, making the camera process images faster, and ensuring its performance. To effectively implement LDW, we developed three technologies: (i) distortion correction using error functions based on the projection characteristics of optical lenses, (ii) automatic vanishing point estimation using geometric characteristics, and (iii) lane tracking and lane departure detection using constraints. The proposed technology improves system stability and convenience through automatic calculation and updating of parameters required for LDW function operation. By performing automatic distortion correction and vanishing point estimation, it has also been proven that fusion with other ADAS systems including front cameras is possible. Existing systems that use vanishing point information do not consider lens distortion and have slow and inaccurate vanishing point estimation, leading to a deterioration of system performance. The proposed method enables fast and accurate vanishing point estimation, allowing for adaptive responses to changes in the road environment.

A Distortion Correction Method Based on Actual Camera Imaging Principles.

In the human-robot collaboration system, the high-precision distortion correction of the camera as an important sensor is a crucial prerequisite for accomplishing the task. The traditional correction process is to calculate the lens distortion with the camera model parameters or separately from the camera model. However, in the optimization process calculate with the camera model parameters, the mutual compensation between the parameters may lead to numerical instability, and the existing distortion correction methods separated from the camera model are difficult to ensure the accuracy of the correction. To address this problem, this study proposes a model-independent lens distortion correction method based on the image center area from the perspective of the actual camera lens distortion principle. The proposed method is based on the idea that the structured image preserves its ratios through perspective transformation, and uses the local image information in the central area of the image to correct the overall image. The experiments are verified from two cases of low distortion and high distortion under simulation and actual experiments. The experimental results show that the accuracy and stability of this method are better than other methods in training and testing results.