Virtual Control Points Research Articles

Generation of Virtual Ground Control Points Using a Binocular Camera

This paper presents a methodology for generating virtual ground control points (VGCPs) using a binocular camera mounted on a drone. We compare the measurements of the binocular and monocular cameras between the classical method and the proposed one. This work aims to decrease human processing times while maintaining a reduced root mean square error (RMSE) for 3D reconstruction. Additionally, we propose utilizing COLMAP to enhance reconstruction accuracy by solely utilizing a sparse point cloud. The results demonstrate that implementing COLMAP for pre-processing reduces the RMSE by up to 16.9% in most cases. We prove that VGCPs further reduce the RMSE by up to 61.08%.

Multizone sound field reproduction using pressure matching with sparse equivalent source

Multizone sound field reproduction aims to create different acoustic environments in multiple spatial zones, allowing listeners to enjoy their individual sounds without being disturbed by sound from other zones. The conventional pressure matching method is used in practice to reproduce the desired sound field by minimizing the sound field error with a limited number of control points, whose reproduction performance decreases above the cutoff frequency. In this paper, a multizone sound field reproduction method based on a sparse equivalent source method is proposed to reproduce the sound field in the whole target region. The acoustic transfer function of each loudspeaker and the desired sound field are represented by the sparse equivalent sources. The whole target region is controlled by interpolating fine virtual control points within the region of the uncontrolled points, and the acoustic transfer functions between the virtual control points and the loudspeaker array are interpolated by the sparse equivalent sources. An optimization reproduction model with the objective of reproducing a desired sound field in the bright zone and constraining the acoustic energy in the dark zone is formulated to find the sparse loudspeaker weights. The simulation results and experimental results show that the proposed method achieves better performance than the conventional pressure matching method in free field and reverberant environments.

Trajectory Tracking Control for Vector Propulsion Unmanned Surface Vehicle With Incomplete Underactuated Inputs

Considering different propulsion systems of unman- ned surface vehicles (USVs), it is necessary to design corresponding vessel motion controllers according to propulsion characteristics. This article devotes to solve the problem of trajectory tracking control for vector propulsion USV. By analyzing actuator distribution of vector propulsion, the vessel can be regarded as an incomplete underactuated system, and the control inputs adopt thruster speed and propulsion angle which are closer to reality. On this basis, the guidance trajectory and adaptive sliding mode controller based on virtual control point theory are proposed to realize trajectory tracking control of vector propulsion USV with system uncertainty and external disturbance. The designed guidance trajectory can guide the vessel to return to the desired trajectory when position error is large, and the system stability is illustrated by stability proof. Then in the design process of the controller, to mitigate chattering for sliding mode, a continuously derivable approximate saturation function is used instead of the signum function. Next, by zero-dynamics stability analysis, the position of the dynamic virtual control point is obtained with the relationship between the virtual control point and vessel speed. Finally, numerical simulations are carried out with two kinds of trajectory tracking scenarios to verify the correctness and feasibility of the proposed control strategy.

New Methodology for Precise UAV Surveys with a Single Ground Control Point

The advances in aerial mapping using unmanned aerial vehicles (UAVs) are increasingly allowing for the surveying of areas that are difficult or unsafe to access. Considering that the cameras boarded on UAVs are non-photogrammetric sensors, positional accuracy and precision are paramount to ensuring that the final product is geometrically consistent and that its coordinates are precise enough within an acceptable range (which depends on the purpose of each application). With this in mind, this work evaluates the positional accuracy of aerial surveys with UAVs supported by the post- processed kinematic (PPK) technique for the generation of cartographic products in compliance with the Brazilian cartographic accuracy standard PEC-PCD. Furthermore, this study proposes a methodology for inserting virtual control points using coordinates adjusted and inserted into the main point of selected images, using the package of solutions for GNSS RTKLIB processing, to forgo Ground Control Points (GCPs) without loss of quality of the cartographic product. This research used a UAV from DJI (model Mavic 2 Pro), equipped with a dual frequency GNSS receiver, where positional records were synchronized with the camera’s shooting times. To validate the technique, 20 checkpoints with known coordinates were selected, and these points were used in statistical tests to assess the accuracy and precision of the survey. The orthomosaic and the digital elevation model were used as the reference for positional coordinates to be compared against the PEC-PCD standards. The results show that the discrepancy is, on average, 0.126 m for planimetric coordinates and 0.066 m for altimetry. No outliers were found, suggesting a Gaussian sampling distribution. The application of Student’s t- test indicated the existence of a small bias, suggesting the displacement of coordinates along the three axes. The chi-square test showed results below the tabulated limits, attesting to the high precision of the survey. Finally, the root mean squared error (RMSE) was lower than the standard error limit for PEC-PCD class A planialtimetric cartographic products on a 1:1,000 scale. The viability of this technique is thus confirmed.

Geometric Quality Improvement Method of Optical Remote Sensing Satellite Images Based on Rational Function Model

The high-precision geometric positioning of optical remote sensing satellites is the prerequisite to determine the application capability of satellite image products. Its positioning accuracy is related to the observation accuracy of each link in the imaging process, including satellite attitude, orbit measurement accuracy, time synchronization accuracy, camera measurement accuracy, and so on. Untimely and inaccurate on-orbit calibration will lead to great geometric positioning errors. To optimize the positioning accuracy of satellite images with the rational function model (RFM) under low positioning accuracy, our paper proposes an improved geometric quality model based on the reorientation of internal and external orientation elements in the RFM model of remote sensing images. By establishing the rational function positioning model, the external orientation model, and the internal orientation model, the original image can be reorientated. Then, we use the improved model to generate uniformly distributed virtual ground control points. By analyzing and verifying the relationship between each rational polynomial coefficient (RPC) and its influence on geometric positioning accuracy, we propose an RPC coefficients optimization method based on image offset correction and positioning dominant coefficients. Finally, we use the small satellite “MN200Sar-1” with low geometric accuracy for experimental verification. The results show that the model can effectively eliminate the errors of internal and external elements in the on-orbit calibration, and the positioning accuracy is improved from one hundred pixels to one pixel. At the same time, the rational polynomial dominant coefficient optimization method can improve geometric positioning accuracy without introducing additional compensation parameters.

UAV-Based Multitemporal Remote Sensing Surveys of Volcano Unstable Flanks: A Case Study from Stromboli

UAV-based photogrammetry is becoming increasingly popular even in application fields that, until recently, were deemed unsuitable for this technique. Depending on the characteristics of the investigated scenario, the generation of three-dimensional (3D) topographic models may in fact be affected by significant inaccuracies unless site-specific adaptations are implemented into the data collection and processing routines. In this paper, an ad hoc procedure to exploit high-resolution aerial photogrammetry for the multitemporal analysis of the unstable Sciara del Fuoco (SdF) slope at Stromboli Island (Italy) is presented. Use of the technique is inherently problematic because of the homogeneous aspect of the gray ash slope, which prevents a straightforward identification of match points in continuous frames. Moreover, due to site accessibility restrictions enforced by local authorities after the volcanic paroxysm in July 2019, Ground Control Points (GCPs) cannot be positioned to constrain georeferencing. Therefore, all 3D point clouds were georeferenced using GCPs acquired in a 2019 (pre-paroxysm) survey, together with stable Virtual Ground Control Points (VGCPs) belonging to a LiDAR survey carried out in 2012. Alignment refinement was then performed by means of an iterative algorithm based on the closest points. The procedure succeeded in correctly georeferencing six high-resolution point clouds acquired from April 2017 to July 2021, whose time-focused analysis made it possible to track several geomorphological structures associated with the continued volcanic activity. The procedure can be further extended to smaller-scale analyses such as the estimation of locally eroded/accumulated volumes and pave the way for rapid UAV-based georeferenced surveys in emergency conditions at the SdF.

Exploiting High Geopositioning Accuracy of SAR Data to Obtain Accurate Geometric Orientation of Optical Satellite Images

Accurate geopositioning of optical satellite imagery is a fundamental step for many photogrammetric applications. Considering the imaging principle and data processing manner, SAR satellites can achieve high geopositioning accuracy. Therefore, SAR data can be a reliable source for providing control information in the orientation of optical satellite images. This paper proposes a practical solution for an accurate orientation of optical satellite images using SAR reference images to take advantage of the merits of SAR data. Firstly, we propose an accurate and robust multimodal image matching method to match the SAR and optical satellite images. This approach includes the development of a new structural-based multimodal applicable feature descriptor that employs angle-weighted oriented gradients (AWOGs) and the utilization of a three-dimensional phase correlation similarity measure. Secondly, we put forward a general optical satellite imagery orientation framework based on multiple SAR reference images, which uses the matches of the SAR and optical satellite images as virtual control points. A large number of experiments not only demonstrate the superiority of the proposed matching method compared to the state-of-the-art methods but also prove the effectiveness of the proposed orientation framework. In particular, the matching performance is improved by about 17% compared with the latest multimodal image matching method, namely, CFOG, and the geopositioning accuracy of optical satellite images is improved, from more than 200 to around 8 m.

INTEGRATION OF PHOTOGRAMMETRY, COMPUTED TOMOGRAPHY AND ENDOSCOPY FOR GYROSCOPE 3D DIGITIZATION

Abstract. Cultural heritage preservation via 3D digitization is becoming more and more important. Besides conventional buildings and landmarks, many technical instruments and artifacts, which belong to tech heritage (TH), are also of great importance, historically and didactically. Gyroscopes, which can be dated back for 200 years, are fascinating instruments with complex structures and different working principles. With such properties, any 3D digitization of Gyroscopes could not be realized by simply using conventional solutions of photogrammetry or laser scanning. In our work, we introduce photogrammetry, endoscopy and computed tomography (CT) for an integrated 3D digitization solution. Though photogrammetry has been widely used for the purpose of cultural heritage preservation, 3D reconstructions using the other two sensor systems have their own challenges. For an endoscope, a pre-calibration solution has been put forward and the Structure-from-Motion (SfM) process has been optimized to deal with the drift caused by a long imaging trajectory. Regarding the CT 3D reconstruction, we mainly focus on the 3D representation’s completeness and the denoising process. In the section of data integration, we designed different methods according to the characteristics of the objects as well as the 3D models from different sonsors. In case of limited overlap between the pair of point clouds, the Gauss-Helmert model with manually picked control points is applied for the estimation of the transformation matrix. CT point clouds, which hold only the intensity values representing the material attenuation, could be integrated with photogrammetry data via a surface color mapping method using the photogrammetric images or the primitive based corresponding virtual control points. Through our research, the concept of integrating photogrammetry, endoscopy and CT for 3D digitization of Gyroscopes is validated. Furthermore, advantages and disadvantages involved in the complete process are discussed and a solid foundation has been laid for further research.

Collision avoidance guidance and control scheme for vector propulsion unmanned surface vehicle with disturbance

In this paper, a collision avoidance guidance and control (CAGC) scheme for fully autonomous unmanned surface vehicle (USV) according to the guidance, navigation and control (GNC) architecture is studied. The CAGC scheme is divided into guidance layer and control layer. To avoid the possibility of collision in tracking guidance, the proposed guidance layer needs remove the tracking guidance and realize the direct guidance to the control layer, which is designed according to the theory of finite control set model predictive control (FCS-MPC) with control ability. Next, to balance the contradiction between calculation and guidance accuracy, the methods of delay compensation and control set normal distribution are designed. In addition, in the control layer, the vector propulsion USV as the control object is non-minimum phase system because of its small size and significant sway thrust, so the virtual control point is used to solve this system, and the relationship between virtual control point and vessel speed is determined by zero-dynamics stability analysis. And the control layer adopts adaptive sliding mode control theory to resist the influence of strong disturbances. Finally, simulation results are provided to illustrate the efficacy of the CAGC scheme.

PHOTOGRAMMETRY AND COMPUTED TOMOGRAPHY POINT CLOUD REGISTRATION USING VIRTUAL CONTROL POINTS

Abstract. In this paper we propose a virtual control point based method for the registration of photogrammetry and computed tomography (CT) data. Because of the fundamentally different two data sources, conventional registration methods, such as manual control points registration or 3D local feature-based registration, are not suitable. The registration objective of our application is about 3D reconstructions of gyroscopes, which contain abundant geometric primitives to be fitted in the point clouds. In the first place, photogrammetry and CT scanning are applied, respectively, for 3D reconstructions. Secondly, our workflow implements a segmentation after obtaining the surface point cloud from the complete CT volumetric data. Then geometric primitives are fitted in this point cloud benefitting from the less complex cluster segments. In the next step, intersection operations of the parametrized primitives generates virtual points, which are utilized as control points for the transformation parameters estimation. A random sample consensus (RANSAC) method is applied to find the correspondences of both virtual control point sets using corresponding descriptors and calculates the transformation matrix as an initial alignment for further refining the registration. The workflow is invariant to pose, resolution, completeness and noise within our validation process.

Optimal selection of virtual control points with planar constraints for large‐scale block adjustment of satellite imagery

AbstractIn large‐scale block adjustment of high‐resolution satellite imagery (HRSI) without ground control points (GCPs), the selection of virtual control points (VCPs) is an important factor in determining the block adjustment accuracy. The traditional VCP generation method is based on a regular grid and uses the initial rational polynomial coefficient (RPC) files but without considering topographic factors. To further improve the accuracy, this paper proposes an approach for the optimal selection of VCPs with planar constraints, which ensures that the VCPs are mostly in areas with small elevation differences and are not influenced by vegetation and buildings. A three‐step approach generates the 3D coordinates of tie points based on forward intersection, calculates their mean values and finally selects optimal VCPs using a threshold value. Experiments conducted using ZY3‐01 satellite images show the accuracy of the proposed method without GCPs outperforms the existing method.

Research on the Fast Solution Method of Pilots’ Head Attitude

Aiming at the problem of poor accuracy and poor real-time performance in the process of head pose calculation in pilot cabin simulation, a high accuracy and fast algorithm for head pose calculation is proposed. On the basis of traditional EPNP algorithm was improved, and combining with POSIT algorithm’s speed advantage, select face five key feature points as virtual control points, according to the mapping relationship between space and virtual control point coordinates, establish a system of linear equations, then the internal camera parameters is calculated using Zhengyou Zhang camera calibration method, finally using POSIT algorithm of calculating to get the head rotation matrix and translation vector, by conversion from the head of Euler angle in real time. The experimental results show that compared with DLT, EPNP, POSIT and other traditional methods, the proposed method has higher accuracy, better real-time performance and better robustness in solving pilots’ head attitude in continuous video.

A new control framework for flapping-wing vehicles based on 3D pendulum dynamics

In this paper, a new control framework for an insect-scale flapping-wing vehicle is presented that exploits passive aerodynamic effects to stabilize the attitude dynamics. Many flapping-wing robotic flyers and flying insects share a common morphological feature in that the center of mass (CoM) is below the center of pressure (CoP), which makes the hovering configuration intrinsically unstable with open-loop control. Motivated by the fact that the CoM should be ahead of the CoP to ensure the longitudinal stability of the flight dynamics, a new coordinate system is proposed by placing a virtual control point (VCP) above the CoP. The dynamics in the new coordinates are derived using a near-identity diffeomorphism which admits a partial feedback linearization with stable zero dynamics. The behavior of the zero dynamics resembles the dynamics of a 3D pendulum with an aerodynamic damper. An adaptive controller is proposed to make the upright orientation almost globally asymptotically stable over a bounded uncertainty of the aerodynamic drag coefficient. The controller is evaluated in simulation with a Harvard RoboBee following a virtual control point reference trajectory.

A generic framework for improving the geopositioning accuracy of multi-source optical and SAR imagery

To date, numerous Earth observation datasets from different types of satellites have been widely used in photogrammetric fields, including urban 3D modelling and geographic information systems. The development of small satellites has provided a new way to obtain repeated observations in a short period. However, compared with that of standard satellite imagery, the geometric performance of imagery from small satellites is relatively poor, restricting their photogrammetric applications. Traditional methods can improve the accuracy of optical images with the addition of well-distributed ground control points (GCPs), which require considerable financial and human resources. The collection of multi-view datasets is an alternative method for geometric processing without GCPs, but relies heavily on the stability and revisit period of satellite platforms. Therefore, this paper presents a framework for improving the geopositioning accuracy of multi-source datasets obtained from optical and synthetic aperture radar (SAR) satellites, and a novel heterogeneous weight strategy is proposed based on an analysis of the geometric error sources of SAR and optical images. The geometric performance of multi-source optical imagery from the Jilin-1 (JL-1) small satellite constellation is evaluated and analysed first, and then Gaofen-3 (GF-3) SAR images are calibrated based on statistical analysis for the production of virtual control points (VCPs). Based on our proposed heterogeneous weight strategy, multi-source optical and SAR images are integrated to improve the geopositioning accuracy. Experimental results indicate that our proposed model can achieve the best performance compared with other popular models, producing an accuracy of approximately 3 m in planimetry and 2 m in height, thereby providing a generic way to synergistically use multi-source remote sensing data.

Direct Georeferencing for the Images in an Airborne LiDAR System by Automatic Boresight Misalignments Calibration.

Airborne Light Detection and Ranging (LiDAR) system and digital camera are usually integrated on a flight platform to obtain multi-source data. However, the photogrammetric system calibration is often independent of the LiDAR system and performed by the aerial triangulation method, which needs a test field with ground control points. In this paper, we present a method for the direct georeferencing of images collected by a digital camera integrated in an airborne LiDAR system by automatic boresight misalignments calibration with the auxiliary of point cloud. The method firstly uses an image matching to generate a tie point set. Space intersection is then performed to obtain the corresponding object coordinate values of the tie points, while the elevation calculated from the space intersection is replaced by the value from the LiDAR data, resulting in a new object point called Virtual Control Point (VCP). Because boresight misalignments exist, a distance between the tie point and the image point of VCP can be found by collinear equations in that image from which the tie point is selected. An iteration process is performed to minimize the distance with boresight corrections in each epoch, and it stops when the distance is smaller than a predefined threshold or the total number of epochs is reached. Two datasets from real projects were used to validate the proposed method and the experimental results show the effectiveness of the method by being evaluated both quantitatively and visually.

Block Adjustment of Large-scale Domestic Optical Satellite Remote Sensing Imagery without GCP in Antarctic

Aiming to the difficult area of surveying and mapping, such as Antarctic, it is difficult to guarantee favourable observation condition for traditional mapping methods. Large-scale block adjustment (BA) without ground control point (GCP) is carried out by using domestic high-resolution satellite remote sensing imagery. The key technologies of BA model construction based on virtual control point (VCP), gross error detection and elimination, and robust and fast method of large-scale BA are studied in this paper. On this basis, the experimental analysis and validation of 13pairs (39 images) of ZY-3 satellite imagery in Antarctic are carried out. The results show that the sparse matrix technology can effectively reduce the memory requirement. The combined matrix block and GPU parallel technology can solve the problem of large-scale BA computational efficiency. In addition, after BA, the maximum residual is 3.920 pixel, the root mean square error (RMSE) is 0.169 pixel in the X (flight) direction, the maximum residual error is 5.933 pixel, and the RMSE is 0.191 pixel in the Y (scan) direction. The proposed method has certain accuracy and stability in large-scale BA without GCP using high-resolution satellite remote sensing imagery in Antarctica. The relative positioning accuracy can reach sub-pixel level, which can meet the requirements of cartographic splicing.

Robust Radiometric Normalization of Multitemporal Satellite Images Via Block Adjustment Without Master Images

Determining appropriate master images, reducing radiometric error accumulation, and eliminating outliers from the cloud, water, and land changes, are three main issues in radiometric normalization of multitemporal high-resolution satellite images (HRSI) during mosaicking. However, these three issues have not been simultaneously considered by the existing methods. This article presents a comprehensive radiometric normalization method for multitemporal HRSI using a radiometric block adjustment without master images. Pseudoinvariant features (PIFs) extracted from image pairs using the iteratively reweighted multivariate alteration detection are used as the corresponding pixel observations and organized to form radiometric tie points according to the corresponding horizontal space coordinates. Radiometric error equations are subsequently constructed, and the linear radiometric transformation parameters are solved by a global adjustment. The time-invariant PIFs generally represent the true corresponding features and naturally avoid the cloud, water, and land changes, which can eliminate the effects of outliers. Furthermore, the pixel values of tie points calculated from the weighted average of the corresponding pixel observations are used as virtual radiometric control points to eliminate the dependency on master images. Moreover, a global optimum can be achieved by the global adjustment, effectively overcoming the error accumulation, which is severe in large datasets. Four groups of HRSI datasets from various satellites are used to validate the performance of the proposed method. Experimental results demonstrate that the proposed method outperforms two state-of-the-art methods and has good applicability and stability, considering both visual effects and quantitative performance.

Model Identification and Trajectory Tracking Control for Vector Propulsion Unmanned Surface Vehicles

To promote the development of military and civilian applications for marine technology, more and more scientific research around the world has begun to develop unmanned surface vehicles (USVs) technology with advanced control capabilities. This paper establishes and identifies the model of vector propulsion USV, which is widely used at present. After analyzing its actuator distribution, we consider that the more realistic vessel model should be an incomplete underactuated system. For this system, a virtual control point method is adopted and an adaptive sliding mode trajectory tracking controller with neural network minimum learning parameter (NNMLP) theory is designed. Finally, in the simulation experiment, the thruster speed and propulsion angle are used as the inputs of the controller, and the linear and circular trajectory tracking tests are carried out considering the delay effect of the actuator, system uncertainty, and external disturbance. The results show that the proposed tracking control framework is reasonable.

BLOCK ADJUSTMENT OF LARGE-SCALE HIGH-RESOLUTION OPTICAL SATELLITE IMAGERY WITHOUT GCPS BASED ON THE GPU

Abstract. Precise geo-positioning of high-resolution optical satellite imagery without ground control points (GCPs) has always been the goal pursued by photogrammetry scholars. This paper introduces the block adjustment (BA) without GCPs based on rational function model (RFM) model and its practical application in high-precision geo-positioning of optical satellite imagery. The mainly key technologies of BA model construction based on virtual control points (VCPs), gross error detection and elimination, and GPU parallel computing method of large-scale adjustment are studied. On this basis, experimental analysis and validation of 123 images of ZY-3 satellite in Taihu are carried out. The results show that the sparse matrix compression can reduce the memory requirement effectively. The GPU parallel computing can solve the problem of large-scale BA computational efficiency. In addition, after BA, the maximum residual is 3.79 pixel, the root mean square error (RMSE) is 0.37 pixel in the x (flight) direction, the maximum residual error is 7.18 pixel, and the RMSE is 0.66 pixel in the y (scan) direction. The proposed method has certain accuracy and stability in large-scale BA without GCPs. The relative positioning accuracy can reach sub-pixel level, which could meet the requirements of cartographic mosaicking.

Robust binaural rendering with the time-domain underdetermined multichannel inverse prefilters.

In this paper, a robust binaural audio rendering system is proposed on the basis of a time-domain underdetermined multichannel inverse filtering approach. The celebrated multiple-input/output inverse theorem is reformulated into a general multichannel model-matching problem with the emphasis on binaural audio reproduction. Robustness with widened sweet spots is achieved by selecting multiple control points in the reproduction zones. The model-matching problem is formulated in the time domain as an underdetermined system, where the number of channels is selected in relation to the number of virtual sources and control points. Under the full-rank condition, exact solutions of inverse filters always exist to fulfill the ideal model-matching criterion. However, the gains of prefilters need to be limited in the design stage by using the Tikhonov regularization at a minor expense of matching performance. The proposed binaural audio system has been implemented on a six-element linear loudspeaker array. Three problems of binaural rendering, cross talk cancellation, source widening, and 5.1 virtual surround, are adopted to validate the proposed approach. Results of objective and subjective tests have demonstrated the efficacy of the proposed approach for binaural audio rendering.