Camera Model Research Articles

PaIRS-UniNa: A Robust And Accurate Free Tool For Digital Particle Image Velocimetry And Optical Camera Calibration

This contribution presents PaIRS-UniNa (PArticle Image Reconstruction Software – University of Naples “Federico II”), a free software application designed to perform digital particle image velocimetry (PIV). PaIRS-UniNa relies on a powerful C library designed for speed, robustness and accuracy, and a user-friendly graphical interface (Figure 1) running in Python environment on all operating systems, including Windows, Linux and MacOS. At the current stage, it features modules to perform two-component (2C) PIV, stereoscopic PIV and optical camera calibration for multi-camera systems. The processing of PIV and stereoscopic PIV data is based on an iterative image deformation method, which can be customized by the user varying different parameters in such a way to find the best compromise between accuracy and computational speed. These parameters include the size, shape and number of interrogation windows, the number of process iterations and the type of interpolating functions for both image deformation and displacement interpolation (such as bilinear, bicubic, methods based on simplexes or the FFT shift theorem and B-splines of varying orders). Additionally, it is possible to select a range of weighting windows for cross-correlation estimation and velocity field filtering, including rectangular, Blackman, Gaussian, Harris, Hann windows, among others, and several vector validation strategies for outlier detection and replacement. Such a level of customization enables to effectively adjust the modulation transfer function of the PIV process to meet specific application requirements. For the stereoscopic PIV process two different approaches are possible: mapping, which involves applying PIV to de-warped images for 2D-2C displacement fields, and warping, which applies PIV to raw images and combine 3C reconstruction and image dewarping in a single subsequent step. In order to accurately estimate the laser sheet plane location and correct errors due to misalignment and inaccurate displacement vector positions, a disparity correction procedure is also available. The calibration module (CalVi) supports accurate calibration with the camera models mostly used in the PIV community: polynomials, rational functions and the pinhole camera model. Moreover, it supports the integration of the pinhole camera model with a refractive correction model for cylindrical geometries and camera calibration procedures working with unknown positions and orientations of the calibration target. In such a way, the target may be moved by hand without losing accuracy in comparison with a conventional calibration, in which the target is displaced with the aid of translational and rotational stages and knowledge of the target positions is strictly required. Already used in numerous experimental studies by the Experimental Thermo-Fluid Dynamics group of University of Naples “Federico II” and international collaborating groups, PaIRS-UniNa offers a rapid and effective free alternative to commercial PIV software for research and industrial use.

Constraining the Geometry of NeRFs for Accurate DSM Generation from Multi-View Satellite Images

Neural Radiance Fields (NeRFs) are an emerging approach to 3D reconstruction that use neural networks to reconstruct scenes. However, its applications for multi-view satellite photogrammetry, which aim to reconstruct the Earth’s surface, struggle to acquire accurate digital surface models (DSMs). To address this issue, a novel framework, Geometric Constrained Neural Radiance Field (GC-NeRF) tailored for multi-view satellite photogrammetry, is proposed. GC-NeRF achieves higher DSM accuracy from multi-view satellite images. The key point of this approach is a geometric loss term, which constrains the scene geometry by making the scene surface thinner. The geometric loss term alongside z-axis scene stretching and multi-view DSM fusion strategies greatly improve the accuracy of generated DSMs. During training, bundle-adjustment-refined satellite camera models are used to cast rays through the scene. To avoid the additional input of altitude bounds described in previous works, the sparse point cloud resulting from the bundle adjustment is converted to an occupancy grid to guide the ray sampling. Experiments on WorldView-3 images indicate GC-NeRF’s superiority in accurate DSM generation from multi-view satellite images.

Se-PSD: A sequence-based parking-slot detection approach for indoor parking

Accurate parking slot detection is crucial for autonomous vehicles to navigate automatic parking. In recent years, significant progress has been made in research in this field, with some methods achieving high accuracy and efficiency on specific datasets. However, existing methods still face some challenges in practical applications, such as: when the ground reflection covers the parking-slot marking lines, the recognition accuracy of existing methods will significantly decrease; The generalization performance of existing methods is not strong enough, and the recognition accuracy will decrease after changing the place or camera model. To address these limitations, this paper introduces Se-PSD, a novel parking slot detection method utilizing image sequences. Se-PSD analyzes a series of images to predict individual marking point locations, shapes, and orientations. Finally, through geometric rules, parking-slots can be found on the last image of the image sequence. Se-PSD prioritizes generalizability without sacrificing accuracy compared to existing methods. While real-time performance may be slightly impacted, the relaxed time constraints of automatic parking applications make Se-PSD a promising solution.

Large depth-of-field fringe projection profilometry applied to defocused scenes

Fringe projection profilometry (FPP) is a widely adopted technique for three-dimensional (3D) reconstruction. However, its depth-of-field (DOF) is constrained when reconstructing defocused scenes, mainly due to limitations in the camera model and image blur. This study introduces a camera model based on the ideal optical system, which effectively reduces the systematic errors associated with the conventional pinhole camera model. A calibration method to determine the optical system parameters of the improved camera model is proposed. Additionally, the point spread function (PSF) of the camera is calibrated and the image is deblurred through non-blind deconvolution, thereby minimizing the phase aliasing resulting from defocus. Experimental results validate the potential of the proposed method for accurate 3D reconstruction in scenes with a wide depth range.

Deep learning model for automatic limb detection and gripping in a novel meat factory cell

The goal of this work has been the development of a deep-learning model which may be used in conjunction with a novel so-called “Meat Factory Cell” platform, namely to enable an industrial robot within the system to identify and successfully grip the limbs of an entire pig carcass. The model consists of three main components: (1) a U-Net-based deep learning model that predicts heatmaps with a probability distribution of gripping and key point locations on the limbs, within RGB-D images; (2) a post-processing element for the extraction of keypoints from heatmaps, and transferral of these points into 3D space using a pinhole camera model; and (3) gripper orientation estimation, which uses the predicted limb key points to define gripper orientation in 3D space. The proposed system demonstrates high precision and robustness in estimating gripping points on pig limbs based on a data test set, which includes two gripping definitions: Norwegian and Danish. These gripping definitions account for variation in the slaughter process in two different European countries. The Norwegian definition gives mAP(0.5…0.95) = 0.971, mAR(0.5…0.95) = 0.982, and distance error 13 mm, while the Danish definition gives mAP(0.5…0.95) = 0.985, mAR(0.5…0.95) = 0.995, and distance error 14 mm. The model was validated in practice during experimental trials at the Meat Factory Cell test facility at the Norwegian University of Life Science (Ås, Norway), with whole pig carcasses (n=25).

Real-Time Seamless Multi-Projector Displays on Deformable Surfaces.

Prior works on multi-projector displays have focused primarily on static rigid objects, some focusing on dynamic rigid objects. However, works on projection based displays on deformable dynamic objects have focused only on small scale single projector displays. Tracking a deformable dynamic surface and updating projections precisely in real time on it is a significantly challenging task, even for a single projector system. In this paper, we present the first end-to-end solution for achieving a real-time, seamless display on deformable surfaces using mutliple unsychronized projectors without requiring any prior knowledge of the surface or device parameters. The system first accurately calibrates multiple RGB-D cameras and projectors using the deformable display surface itself, and then using those calibrated devices, tracks the continuous changes in the surface shape. Based on the deformation and projector calibration, the system warps and blends the image content in real-time to create a seamless display on a surface that continuously changes shape. Using multiple projectors and RGB-D cameras, we provide the much desired aspect of scale to the displays on deformable surfaces. Most prior dynamic multi-projector systems assume rigid objects and depend critically on the constancy of surface normals and non-existence of local shape deformations. These assumptions break in deformable surfaces making prior techniques inapplicable. Point-based correspondences become inadequate for calibration, exacerbated with no synchronization between the projectors. A few works address non-rigid objects with several restrictions like targeting semi-deformable surfaces (e.g. human face), or using single coaxial (optically aligned) projector-camera pairs, or temporally synchronized cameras. We break loose from such restrictions and handle multiple projector systems for dynamic deformable fabric-like objects using temporally unsynchronized devices. We devise novel methods using ray and plane-based constraints imposed by the pinhole camera model to address these issues and design new blending methods dependent on 3D distances suitable for deformable surfaces. Finally, unlike all prior work with rigid dynamic surfaces that use a single RGB-D camera, we devise a method that involve all RGB-D cameras for tracking since the surface is not seen completely by a single camera. These methods enable a seamless display at scale in the presence of continuous movements and deformations. This work has tremendous applications on mobile and expeditionary systems where environmentals (e.g. wind, vibrations, suction) cannot be avoided. One can create large displays on tent walls in remote, austere military or emergency operations in minutes to support large scale command and control, mission rehearsal or training operations. It can be used to create displays on mobile and inflatable objects for tradeshows/events and touring edutainment applications.

SADNet: Generating immersive virtual reality avatars by real‐time monocular pose estimation

SummaryGenerating immersive virtual reality avatars is a challenging task in VR/AR applications, which maps physical human body poses to avatars in virtual scenes for an immersive user experience. However, most existing work is time‐consuming and limited by datasets, which does not satisfy immersive and real‐time requirements of VR systems. In this paper, we aim to generate 3D real‐time virtual reality avatars based on a monocular camera to solve these problems. Specifically, we first design a self‐attention distillation network (SADNet) for effective human pose estimation, which is guided by a pre‐trained teacher. Secondly, we propose a lightweight pose mapping method for human avatars that utilizes the camera model to map 2D poses to 3D avatar keypoints, generating real‐time human avatars with pose consistency. Finally, we integrate our framework into a VR system, displaying generated 3D pose‐driven avatars on Helmet‐Mounted Display devices for an immersive user experience. We evaluate SADNet on two publicly available datasets. Experimental results show that SADNet achieves a state‐of‐the‐art trade‐off between speed and accuracy. In addition, we conducted a user experience study on the performance and immersion of virtual reality avatars. Results show that pose‐driven 3D human avatars generated by our method are smooth and attractive.

A photogrammetric approach for real‐time visual SLAM applied to an omnidirectional system

AbstractThe problem of sequential estimation of the exterior orientation of imaging sensors and the three‐dimensional environment reconstruction in real time is commonly known as visual simultaneous localisation and mapping (vSLAM). Omnidirectional optical sensors have been increasingly used in vSLAM solutions, mainly for providing a wider view of the scene, allowing the extraction of more features. However, dealing with unmodelled points in the hyperhemispherical field poses challenges, mainly due to the complex lens geometry entailed in the image formation process. To address these challenges, the use of rigorous photogrammetric models that appropriately handle the geometry of fisheye lens cameras can overcome these challenges. Thus, this study presents a real‐time vSLAM approach for omnidirectional systems adapting ORB‐SLAM with a rigorous projection model (equisolid‐angle). The implementation was conducted on the Nvidia Jetson TX2 board, and the approach was evaluated using hyperhemispherical images captured by a dual‐fisheye camera (Ricoh Theta S) embedded into a mobile backpack platform. The trajectory covered a distance of 140 m, with the approach demonstrating accuracy better than 0.12 m at the beginning and achieving metre‐level accuracy at the end of the trajectory. Additionally, we compared the performance of our proposed approach with a generic model for fisheye lens cameras.

Study of Underwater Large-Target Localization Based on Binocular Camera and Laser Rangefinder

Currently, for underwater close-range large-target localization, visual localization techniques fail since large targets completely occupy the camera’s field of view at ultraclose ranges. To address the issue, a multi-stage optical localization method combining a binocular camera and a single-point laser rangefinder is proposed in this paper. The proposed method comprises three parts. First, the imaging model of the underwater camera is modified, and a laser rangefinder is used to further correct the underwater calibration results of the binocular camera. Second, YOLOv8 is applied to recognize the targets to prepare for target localization. Third, extrinsic calibration of the binocular camera and laser rangefinder is performed, and a Kalman filter is employed to fuse the target position information measured by the binocular camera and laser rangefinder. The experimental results show that, compared with using a binocular camera alone, the proposed method can accurately and stably locate the target at close ranges with an average error of only 2.27 cm, without the risk of localization failure, and reduces binocular localization error by 90.57%.

A Novel Progressive Enhancement of Low Light Raw Images

Low-light imaging on mobile devices is often difficult due to the issue of not enough light passing through the small aperture, resulting in poor quality images. Most previous work on low-resolution images has focused on a single task, such as illumination, color correction, or noise removal; Noise removal function based on short and long images of the camera model. This technique is less efficient and general in real-world environments that require special camera integration and restoration. In this paper, we propose a lighting system that can integrate illumination variation, color correction, and noise removal to solve this problem. Considering the difficulty of obtaining model-specific data and the maximum content of the resulting image, we created two branches: the coefficient estimation branch and the integration branch. While the computer estimator operates in the sparsely resolved space and estimates the coefficients developed by pairwise learning, the collaborative system operates in the fully resolved space, providing step-by-step integration and denoising. Compared to existing methods, our framework does not need to remember as much information when switching to another camera model, which reduces the effort required to benefit our usage pattern. Through extensive testing, we demonstrate its great potential in real-world low-light applications.

Camera trap surveys of Atlantic Forest mammals: A data set for analyses considering imperfect detection (2004-2020).

Camera traps became the main observational method of a myriad of species over large areas. Data sets from camera traps can be used to describe the patterns and monitor the occupancy, abundance, and richness of wildlife, essential information for conservation in times of rapid climate and land-cover changes. Habitat loss and poaching are responsible for historical population losses of mammals in the Atlantic Forest biodiversity hotspot, especially for medium to large-sized species. Here we present a data set from camera trap surveys of medium to large-sized native mammals (>1 kg) across the Atlantic Forest. We compiled data from 5380 ground-level camera trap deployments in 3046 locations, from 2004 to 2020, resulting in 43,068 records of 58 species. These data add to existing data sets of mammals in the Atlantic Forest by including dates of camera operation needed for analyses dealing with imperfect detection. We also included, when available, information on important predictors of detection, namely the camera brand and model, use of bait, and obstruction of camera viewshed that can be measured from example pictures at each camera location. Besides its application in studies on the patterns and mechanisms behind occupancy, relative abundance, richness, and detection, the data set presented here can be used to study species' daily activity patterns, activity levels, and spatiotemporal interactions between species. Moreover, data can be used combined with other data sources in the multiple and expanding uses of integrated population modeling. An R script is available to view summaries of the data set. We expect that this data set will be used to advance the knowledge of mammal assemblages and to inform evidence-based solutions for the conservation of the Atlantic Forest. The data are not copyright restricted; please cite this paper when using the data.

A Distortion Correction Method Based on Actual Camera Imaging Principles.

In the human-robot collaboration system, the high-precision distortion correction of the camera as an important sensor is a crucial prerequisite for accomplishing the task. The traditional correction process is to calculate the lens distortion with the camera model parameters or separately from the camera model. However, in the optimization process calculate with the camera model parameters, the mutual compensation between the parameters may lead to numerical instability, and the existing distortion correction methods separated from the camera model are difficult to ensure the accuracy of the correction. To address this problem, this study proposes a model-independent lens distortion correction method based on the image center area from the perspective of the actual camera lens distortion principle. The proposed method is based on the idea that the structured image preserves its ratios through perspective transformation, and uses the local image information in the central area of the image to correct the overall image. The experiments are verified from two cases of low distortion and high distortion under simulation and actual experiments. The experimental results show that the accuracy and stability of this method are better than other methods in training and testing results.

Automated cleaning of tie point clouds following USGS guidelines in Agisoft Metashape professional (ver. 2.1.0)

The U.S. Geological Survey (USGS) has published a guideline to improve the quality of digital photogrammetric reconstructions created with the widely used Agisoft Metashape Professional software. The suggested workflows aim at filtering out low-quality tie points from the tie point cloud to optimize the camera model. However, the optimization procedure relies on an iteratively performed trial-and-error approach. If manually performed, the time expenditure required from the operator can be significant and the optimization process can be affected by the degree of diligence that is applied.To minimize the time expenditure and attentiveness required from the operator and to provide a framework for an improved reproducibility of camera model optimization workflows, we present here a python script serving as an extension for Agisoft Metashape Professional (tested on version 2.1.0) that automatizes the iterative point filtering procedure proposed by the USGS. As a result, the entire processing cycle can be performed largely unattended.•A graphical user interface allows to individually adjust important camera model optimization parameters.•Main tie point cloud quality measures can be directly assessed.•The reproducibility of the automated camera model optimization as tested in this study generally is above 99%.

An innovative sCMOS based autonomous astronomical camera dedicated to universal use for SST and other fields of optical astronomy

Creotech Instruments is advancing a game-changing sCMOS camera series. The Final Prototype Model of an astronomical camera for Space Surveillance and Tracking (SST) is in the test campaign phase. Designed for SST, NEO, and debris detection, its adaptable platform suits quantum tech and biological microscopy. Edge computing sets it apart, leveraging FPGA-based SoC for real-time processing and Linux-based pre-processing. Operating autonomously, it supports on-camera ML algorithms, revolutionizing astronomy. Data pre-processing, like frame stacking, reduces data load. This paper introduces the camera's concept, architecture, and prototype test results, emphasizing specific use cases and future product line development.

A geometric calibration method for thermal cameras using a ChArUco board

Accurate and efficient geometric calibration is important to precisely acquire geometric information in thermal cameras. Typically, the calibration procedure involves obtaining defined control points from the images of a calibration target and utilizing them to estimate the geometric calibration parameters based on a camera model. Calibration targets include checkerboards, circle grids, Hermann grids, and specially designed patterns. Thermal cameras typically have lower resolution than visible cameras and the control points are localized by creating contrast through heating. These differences make the calibration process of thermal cameras more complex. Conventional thermal camera calibration methods require that the entire calibration target is captured and that all the control points from each image are identified. Due to these limitations, geometric calibration of a thermal camera traditionally requires advanced image processing or substantial manual intervention.This work presents a calibration method using a ChArUco board as the calibration target. The proposed method allows the calibration to be performed with partial-view images and does not require that all the control points from each image are extracted, thereby allowing the calibration to be performed without requiring complex image processing. The proposed calibration method was successfully implemented to estimate the calibration parameters of two different thermal cameras; in both cases achieving an overall mean reprojection error below 0.4 pixels. The effectiveness of the approach was further demonstrated by performing the calibration with a lower mean reprojection error than the MATLAB camera calibrator app. Furthermore, the effectiveness of using different heat sources to create contrast was evaluated.

Density and abundance estimation of unmarked ungulates using camera traps in the Mudumu National Park, Namibia

Density and abundance estimates are critical to effective wildlife management and are essential for monitoring population trends and setting effective quotas for harvesting. Management of roan (Hippotragus equinus) and sable (H. niger) antelopes in Mudumu National Park (MNP), Namibia, is challenging because they are elusive, naturally unmarked, and believed to occur at low densities. The species are threatened by habitat fragmentation, human population growth, and illegal hunting, and reliable density and abundance estimates have not been quantified, hampering management and conservation plans. Our objective was to estimate roan and sable densities and abundances using the time in front of the camera model (TIFC) and the Poisson-binomial N-mixture model (PB), respectively. We also evaluated the effects of environmental and ecological variables on roan and sable abundance. We used data from two camera trap surveys conducted between March and September 2021 in the MNP. Results showed that the TIFC model provided low-density estimates of 1.62 (95% CI 1.61–1.64) roans/km2 and 2.46 (95% CI 2.42–2.50) sables/km2, consistent with trends reported in Africa where these species occur at low densities. In addition, the total abundance of roans and sables in the MNP from the PB model were 57 and 242, respectively. Higher roan abundance occurred in sites with higher grass cover. This study provides the first accurate camera trap-derived density and abundance estimates for roan and sable in the MNP, which will be critical for developing comprehensive conservation programs and strategies that are likely to reduce the risk of extinction for both species.

Level0 to Level1B processor for MethaneAIR

Abstract. This work presents the development of the MethaneAIR Level0–Level1B processor, which converts raw L0 data to calibrated and georeferenced L1B data. MethaneAIR is the airborne simulator for MethaneSAT, a new satellite under development by MethaneSAT LLC, a subsidiary of the Environmental Defense Fund (EDF). MethaneSAT's goals are to precisely map over 80 % of the production sources of methane from oil and gas fields across the globe to an accuracy of 2–4 ppb on a 2 km2 scale. Efficient algorithms have been developed to perform dark corrections, estimate the noise, radiometrically calibrate data, and correct stray light. A forward model integrated into the L0–L1B processor is demonstrated to retrieve wavelength shifts during flight accurately. It is also shown to characterize the instrument spectral response function (ISRF) changes occurring at each sampled spatial footprint. We demonstrate fast and accurate orthorectification of MethaneAIR data in a three-step process: (i) initial orthorectification of all observations using aircraft avionics, a simple camera model, and a medium-resolution digital elevation map; (ii) registration of oxygen (O2) channel grayscale images to reference Multispectral Instrument (MSI) band 11 imagery via Accelerated-KAZE (A-KAZE) feature extraction and linear transformation, with similar co-registration of methane (CH4) channel grayscale images to the registered O2 channel images; and finally (iii) optimization of the aircraft position and attitude to the registered imagery and calculation of viewing geometry. This co-registration technique accurately orthorectifies each channel to the referenced MSI imagery. However, in the pixel domain, radiance data for each channel are offset by almost 150–200 across-track pixels (rows) and need to be aligned for the full-physics or proxy retrievals where both channels are simultaneously used. We leveraged our orthorectification tool to identify tie points with similar geographic locations in both CH4 and O2 images in order to produce shift parameters in the across-track and along-track dimensions. These algorithms described in this article will be implemented into the MethaneSAT L0–L1B processor.

Spot the difference: optimising camera trap use to detect and identify individuals of a medium-sized carnivorous marsupial

Camera trap model and setup choice can significantly affect data collection. This study investigated standard-angle Reconyx and wide-angle Swift camera traps in different setups and how the choice of equipment influenced detection and individual identification of the chuditch (Dasyurus geoffroii). Camera models were placed side-by-side in four setups: (i) single and (ii) paired camera traps 30 cm above ground; and single cameras (iii) 1 m and (iv) 2 m above ground, with a lure (tuna oil) in the centre of all setups. Swifts had a higher Detection probability than Reconyx, although the number of detections with identified individuals did not significantly differ between camera models. The 30 cm paired camera setups had the highest Detection and Identification probabilities for both camera models, with both probabilities decreasing as cameras were positioned higher. Camera model and setup choice are important in obtaining detection and identification data, and should be considered when planning studies and interpreting results.

3D Modeling of Façade Elements Using Multi-View Images from Mobile Scanning Systems

There is a growing demand for detailed building façade models (Level-of-Detail 3: LoD 3) in a variety of applications. Despite the increasing number of papers addressing this issue in the literature, occlusions are still a significant problem when processing building façade elements. Conversely, depending on the view angle of the images, the detected elements might not be projected to their accurate locations causing uncertainties in their 3D locations. In this paper, we address the aforementioned issues utilizing multi-view images. Using a building footprint layer, we first locate the points belonging to buildings. We then detect the 2D windows and doors in images by combining Faster R-CNN and Segment Anything (SAM) deep learning models. The 2D borders are projected into the 3D object space using a pinhole camera model and collinearity equations. Utilizing the multi-view capabilities of mobile scanning systems, this method effectively mitigates uncertainties associated with occlusion and exterior orientation parameters (EOP). This study provides a comprehensive evaluation of 3D spatial accuracy, achieving an average of 84% Intersection over Union (IoU) accuracy for 12 different single-sided façades over 750 multi-view images for 312 windows and doors of various sizes with rectangular and curved shapes.

A 6-DOF camera motion correction method using IMU sensors for photogrammetry and optical measurements

Environmental conditions such as wind and ground traffic introduce motion in camera measurement systems and affect measurement accuracy. Conventional camera motion correction methods track static reference points with one or multiple cameras, reducing applicability. This study proposes a novel 6-degree-of-freedom (DOF) camera motion correction method using only an inertial measurement unit (IMU) sensor. A Kalman filter is adopted as a data fusion method to estimate the camera orientation and translation using IMU data. Six pinhole camera models are built to evaluate and correct 6-DOF camera motions. The motion correction efficiency and robustness are tested for different object distances and focal lengths of optical lenses. The motion correction ratio is statistically analysed and reaches approximately 80%. The object distance has little effect on the motion correction ratio. The rotation-induced pixel movement is independent of the object distance. More than 90% of the pixel movement noise is caused by camera rotation. The translation-induced pixel movement is inversely correlated with the object distance.