Least-squares Matching Research Articles

Single-Point Least Square Matching Embedded Method for Improving Visual SLAM

Visual simultaneous location and mapping (VSLAM) has been widely used today, but for some tough cases with insufficient feature points (such as weak texture and motion blur), it is still an ongoing challenge to further boost the corresponding robustness and accuracy. In this paper, to address the problem of inadequate feature points, for conventional visual SLAM, we improve the original feature matching mechanism by providing more correspondences with higher position precision. Firstly, based on the original feature matching results in the map initialization stage, a single-point least square matching (LSM) which iteratively refines position of the matching points by considering geometric and photometric consistency, is presented to improve the map initialization result. Secondly, to improve the tracking performance, given the motion model, the single-point LSM is again used to yield new correspondences from the unsuccessfully matched features of original matching mechanism between current frame and previous frame in the tracking thread, and these extra correspondences participate in and boost the subsequent pose optimization. Lastly, for newly inserted keyframe, we focus on identifying the corresponding 3D map points’ reprojections on relevant local neighbor keyframes, these reprojections are added in the local bundle adjustment (BA) optimization for further improving the estimation of camera pose and local map. The popular VSLAM package, ORB-SLAM2, is used to demonstrate the efficacy of the proposed improvements. An ablation study on accuracy and tracking stability on public datasets are reported, furthermore, the comparisons with several state of the arts (namely, VINS-Mono, ORB-SLAM3, DSO, PL-SLAM) are investigated. The experimental results show that our approach can improve accuracy in weak texture scenes and motion blur cases, reduce the number of tracking loss and improve tracking robustness.

Comparison of image operators for time-based photogrammetry

AbstractReal-time deformation measurements have the potential to be used in a wide range of civil and structural engineering applications. This paper introduces an algorithm that utilizes time-based photogrammetry to measure deformations in real-time. The hardware used in the algorithm consists of a camera and a computer that runs camera-control software and a MATLAB code. The code is responsible for detecting the approximate locations of the circular targets using normalized 2D cross-correlation and performing the sub-pixel measurement of the center by fitting an ellipse on the edges of the circular target. During an experimental load test of a concrete slab outfitted with Ringed Automated Detection targets, a series of images were taken at different loads. The coordinates of the targets were measured using the edge-ellipse operator, least-squares matching, and digital image correlation methods. The edge-ellipse operator has proven to be an effective method of measuring the image coordinates of circular targets.

Wireless Binocular Stereovision Measurement System Based on Improved Coarse-to-Fine Matching Algorithm

To cope with the challenges some commonly used displacement sensors may face in structural health monitoring, we proposed a wireless binocular stereovision measurement system based on the improved coarse-to-fine matching algorithm. The vision measurement system can perceive multipoint three-dimensional displacement and consists of two cameras, one remote control unit, and a solar panel. The improved coarse-to-fine matching algorithm only requires a flat surface with some painted textures instead of designed markers with refined patterns (circle, square, and chessboard). Omitting the textures keeps a balance between measurement accuracy and the burden of the target installation in the actual field of structural health monitoring. The proposed coarse-to-fine matching algorithm utilizes scale-invariant feature transform (SIFT) points to generate a robust initial guess and combines improved least squares matching (LSM) algorithm for advanced corresponding point matching. A precalculated look-up table for interpolation and parallel computation is used for higher matching precision and computational efficiency. Several experiments were conducted to present the accuracy and feasibility of the proposed metering system. A simulation test, an indoor shaking table test with several commonly used displacement sensors, and an outdoor experiment consider the natural condition. Experimental results verified that the proposed matching methodology could achieve 0.2-pixel matching accuracy, 0.1 mm measuring precision in a laboratory, and millimeter-level precision in real conditions.

Quality analysis of Worldview-4 DSMs generated by least squares matching and semiglobal matching

For the estimation of a dense disparity map from a rectified space-borne stereo image pair and generation of the most qualified digital surface model (DSM), new image matching algorithms are being developed. Our study has two main goals of the DSM quality validation of Worldview-4 (WV-4), which offers the highest ground sampling distance (GSD) (30 cm) of civilian optical space-borne missions together with Worldview-3 (WV-3), and performance comparison of semiglobal matching (SGM) and least squares matching (LSM), two of the most preferred image matching algorithms for space-borne data. In the Istanbul study area with a rough topographic structure, 1 m gridded DSMs were derived from geometrically corrected WV-4 stereo image pairs. The qualities were estimated by well-rounded visual and statistical approaches based on standard deviation and normalized median absolute deviation. In model-to-model comparisons, a very high-resolution airborne laser scanning (ALS) DSM was utilized as the reference model. The results demonstrated that the WV-4 DSM derived by SGM has the standard deviation of ±0.51 m (1.7 GSD) in height in open areas, whereas the LSM DSM has ±1.46 m. In addition, the visual quality of SGM is much better than LSM through building description potential.

A Unified Framework of Bundle Adjustment and Feature Matching for High-Resolution Satellite Images

Bundle adjustment (BA) is a technique for refining sensor orientations of satellite images, while adjustment accuracy is correlated with feature matching results. Feature matching often contains high uncertainties in weak/repeat textures, while BA results are helpful in reducing these uncertainties. To compute more accurate orientations, this article incorporates BA and feature matching in a unified framework and formulates the union as the optimization of a global energy function so that the solutions of the BA and feature matching are constrained with each other. To avoid a degeneracy in the optimization, we propose a comprised solution by breaking the optimization of the global energy function into two-step suboptimizations and compute the local minimums of each suboptimization in an incremental manner. Experiments on multi-view high-resolution satellite images show that our proposed method outperforms state-of-the-art orientation techniques with or without accurate least-squares matching.

The Epoch of Giant Planet Migration Planet Search Program. I. Near-infrared Radial Velocity Jitter of Young Sun-like Stars

Abstract We present early results from the Epoch of Giant Planet Migration program, a precise radial velocity (RV) survey of more than 100 intermediate-age (∼20–200 Myr) G and K dwarfs with the Habitable Zone Planet Finder spectrograph (HPF) at McDonald Observatory’s Hobby–Eberly Telescope. The goals of this program are to determine the timescale and dominant physical mechanism of giant planet migration interior to the water ice line of Sun-like stars. Here, we summarize results from the first 14 months of this program, with a focus on our custom RV pipeline for HPF, a measurement of the intrinsic near-infrared RV activity of young Solar analogs, and modeling the underlying population-level distribution of stellar jitter. We demonstrate on-sky stability at the sub-2 m s−1 level for the K2 standard HD 3765 using a least-squares matching method to extract precise RVs. Based on a subsample of 29 stars with at least three RV measurements from our program, we find a median rms level of 34 m s−1. This is nearly a factor of 2 lower than the median rms level in the optical of 60 m s−1 for a comparison sample with similar ages and spectral types as our targets. The observed near-infrared jitter measurements for this subsample are well reproduced with a log-normal parent distribution with μ = 4.15 and σ = 1.02. Finally, by compiling rms values from previous planet search programs, we show that near-infrared jitter for G and K dwarfs generally decays with age in a similar fashion to optical wavelengths, albeit with a shallower slope and lower overall values for ages ≲1 Gyr.

Real-time mosaicking for infrared videos from an oblique sweeping camera

Image mosaicking is widely used in Geographic Information Systems (GISs) for large-scale ground surface analysis. However, most existing mosaicking methods can only be used in offline processing due to the enormous amounts of computation. In this paper, we propose a novel and practical algorithm for real-time infrared video mosaicking. To achieve this, a novel fast template matching algorithm based on Sum of Cosine Differences (SCD) is proposed to coarsely match the sequential images. The high speed of the proposed template matching algorithm is obtained by computing correlation with Fast Fourier Transform (FFT). We also propose a novel fast Least Squares Matching (LSM) algorithm for inter-frame fine registration, which can significantly reduce the computation without degrading the matching accuracy. In addition, the proposed fast LSM can effectively adapt for noise degradation and geometric distortion. Based on the proposed fast template matching algorithm and fine registration algorithm, we develop a practical real-time mosaicking approach which can produce seamless mosaic image highly efficiently. Experiments on synthetic and real-world datasets demonstrate that the proposed algorithm is not just computationally efficient but also robust against various noise distortions.

Registrating Oblique SAR Images Based on Complementary Integrated Filtering and Multilevel Matching

This paper presents a novel registration method for oblique synthetic aperture radar (SAR) images based on complementary integrated filtering (CIF) and multilevel matching. Our algorithm is divided into three steps. First, we considered different type of noises and employed the CIF to increase the signal-to-noise ratio of SAR images. Second, complementary affine invariant features, namely maximally stable extremal regions and Harris-affine features, were extracted simultaneously from image pairs, and then the initial matches were obtained based on the scale invariant feature transform (SIFT) descriptor and Euclidean distance. Therefore, the fundamental and homography matrixes could be calculated between image pairs, and then more matches were obtained under quasi-affine invariant SIFT (QAISIFT) and the hybrid geometric constraints. We further implemented the least square matching (LSM) based on the second-polynomial geometric model (SPGM), and thus the matching error of each corresponding point can be compensated according to the optimal SPGM. Third, the precise registration was achieved based on the matches of the second step. Experiments on four groups of oblique SAR images demonstrated the effectiveness of the proposed method. The contribution of this paper includes three aspects. One is that the proposed CIF can remove SAR image noise as much as possible; another is that the proposed QAISIFT can achieve near affine invariance across viewpoint change images; the third is that the advanced LSM can notably improve the accuracy of feature matches.

Least-square Matching for Mobile Robot SLAM Based on Line-segment Model

This study proposes an efficient simultaneous localization and mapping (SLAM) algorithm for a mobile robot. The proposed algorithm consists of line-segment feature extraction from a set of points measured by a LIDAR, association and matching between the line-segments and a map database for position estimation, and the registration of the line-segments into the map database for the incremental construction of the map database. The line-segment features help reduce the amount of data required for map representation. The matching algorithm for position estimation is efficient in computation owing to the use of a number of inliers as the weights in the least-squares method. Experiments are conducted to demonstrate the performance of the proposed SLAM algorithm, and the results show that the proposed algorithm is effective in the map representation and the localization of a mobile robot.

Radargrammetric DSM generation in mountainous areas through adaptive-window least squares matching constrained by enhanced epipolar geometry

Radargrammetry is a powerful tool to construct digital surface models (DSMs) especially in heavily vegetated and mountainous areas where SAR interferometry (InSAR) technology suffers from decorrelation problems. In radargrammetry, the most challenging step is to produce an accurate disparity map through massive image matching, from which terrain height information can be derived using a rigorous sensor orientation model. However, precise stereoscopic SAR (StereoSAR) image matching is a very difficult task in mountainous areas due to the presence of speckle noise and dissimilar geometric/radiometric distortions. In this article, an adaptive-window least squares matching (AW-LSM) approach with an enhanced epipolar geometric constraint is proposed to robustly identify homologous points after compensation for radiometric discrepancies and geometric distortions. The matching procedure consists of two stages. In the first stage, the right image is re-projected into the left image space to generate epipolar images using rigorous imaging geometries enhanced with elevation information extracted from the prior DEM data e.g. SRTM DEM instead of the mean height of the mapped area. Consequently, the dissimilarities in geometric distortions between the left and right images are largely reduced, and the residual disparity corresponds to the height difference between true ground surface and the prior DEM. In the second stage, massive per-pixel matching between StereoSAR epipolar images identifies the residual disparity. To ensure the reliability and accuracy of the matching results, we develop an iterative matching scheme in which the classic cross correlation matching is used to obtain initial results, followed by the least squares matching (LSM) to refine the matching results. An adaptively resizing search window strategy is adopted during the dense matching step to help find right matching points. The feasibility and effectiveness of the proposed approach is demonstrated using Stripmap and Spotlight mode TerraSAR-X stereo data pairs covering Mount Song in central China. Experimental results show that the proposed method can provide a robust and effective matching tool for radargrammetry in mountainous areas.

Automatic Matching of Multi-Source Satellite Images: A Case Study on ZY-1-02C and ETM+

The ever-growing number of applications for satellites is being compromised by their poor direct positioning precision. Existing orthoimages, such as enhanced thematic mapper (ETM+) orthoimages, can provide georeferences or improve the geo-referencing accuracy of satellite images, such ZY-1-02C images that have unsatisfactory positioning precision, thus enhancing their processing efficiency and application. In this paper, a feasible image matching approach using multi-source satellite images is proposed on the basis of an experiment carried out with ZY-1-02C Level 1 images and ETM+ orthoimages. The proposed approach overcame differences in rotation angle, scale, and translation between images. The rotation and scale variances were evaluated on the basis of rational polynomial coefficients. The translation vectors were generated after blocking the overall phase correlation. Then, normalized cross-correlation and least-squares matching were applied for matching. Finally, the gross errors of the corresponding points were eliminated by local statistic vectors in a TIN structure. Experimental results showed a matching precision of less than two pixels (root-mean-square error), and comparison results indicated that the proposed method outperforms Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Affine-Scale Invariant Feature Transform (A-SIFT) in terms of reliability and efficiency.

Bound-Constrained Multiple-Image Least-Squares Matching for Multiple-Resolution Images

Satellite images from multiple sources with different resolutions are currently able to observe the same region. Reliable image matching between these images is the first step in their integrated use. Image matching of multiple-resolution images is not trivial because of the large geometric differences among the images, which can cause failure of matching and losses of matching accuracy. This paper presents a bound-constrained, multiple-image, least-squares matching (<small>LSM</small>) method that extends the classical <small>LSM</small> in two ways for better performance. First, the a priori metadata of the images, including the geo-referencing and scale information, are used for initial matching and to provide bound constraints in the <small>LSM</small> to improve its stability. Second, multiple images are matched in a single optimization rather than the traditional pairwise matching. This brings additional observations in the least-squares optimization, which makes the matching aware of both larger and local context and improves matching quality even with inaccurate initializations for high resolution images. Experimental analysis using multiple-source satellite images with multiple resolutions collected on Mars and in Hong Kong reveals that the proposed method is capable of obtaining reliable multiple-fold matches effectively, even in challenging cases with resolution differences as much as 20-fold. The method proposed in this paper has significance for the synergistic use of multiple-source satellite images in various applications.

A PHOTOGRAMMETRIC PIPELINE FOR THE 3D RECONSTRUCTION OF CASSIS IMAGES ON BOARD EXOMARS TGO

Abstract. CaSSIS (Colour and Stereo Surface Imaging System) is the stereo imaging system onboard the European Space Agency and ROSCOSMOS ExoMars Trace Gas Orbiter (TGO) that has been launched on 14 March 2016 and entered a Mars elliptical orbit on 19 October 2016. During the first bounded orbits, CaSSIS returned its first multiband images taken on 22 and 26 November 2016. The telescope acquired 11 images, each composed by 30 framelets, of the Martian surface near Hebes Chasma and Noctis Labyrithus regions reaching at closest approach at a distance of 250 km from the surface. Despite of the eccentricity of this first orbit, CaSSIS has provided one stereo pair with a mean ground resolution of 6 m from a mean distance of 520 km. The team at the Astronomical Observatory of Padova (OAPD-INAF) is involved into different stereo oriented missions and it is realizing a software for the generation of Digital Terrain Models from the CaSSIS images. The SW will be then adapted also for other projects involving stereo camera systems. To compute accurate 3D models, several sequential methods and tools have been developed. The preliminary pipeline provides: the generation of rectified images from the CaSSIS framelets, a matching core and post-processing methods. The software includes in particular: an automatic tie points detection by the Speeded Up Robust Features (SURF) operator, an initial search for the correspondences through Normalize Cross Correlation (NCC) algorithm and the Adaptive Least Square Matching (LSM) algorithm in a hierarchical approach. This work will show a preliminary DTM generated by the first CaSSIS stereo images.

Unmanned Aerial Systems and DSM matching for rock glacier monitoring

Among other techniques, aerial and terrestrial photogrammetry have long been used to control the displacements of landslides and glaciers as well as for the detection of terrain morphological changes. Unmanned Aerial Systems (UAS) are today an efficient tool to perform data acquisition in rough or difficult terrain, both safely and quickly, avoiding hazards and risks for the operators while at the same time containing the survey costs. Since 2012 ARPAVdA (the Regional Environmental Protection Agency of Aosta Valley, Italy) periodically surveys with UAS photogrammetry the Gran Sometta rock glacier, the Agency main monitoring site for the climate change impacts on high-mountain areas and related infrastructures.A Digital Surface Model (DSM) and an orthophoto of the rock glacier are produced after each survey flight. In order to accurately georeference them in a stable reference system, a Global Navigation Satellite System (GNSS) campaign is carried out at each epoch, to update the coordinates of signalised Ground Control Points (GCPs), since they partly lay in unstable (moving) areas. In late August 2015 a survey flight has been executed with a senseFly eBee RTK, with differential corrections sent from a ground reference station. The block has been adjusted without GCP using, as control information, only the projection centres coordinates encoded in the images. The RMS of the differences found on twelve Check Points were about 4cm in horizontal and 7cm in elevation, i.e. practically the same accuracy found using GCP. Differences between the DSMs produced at the same epoch with block orientation performed with GCP and with GNSS-determined projection centres were also investigated.To evaluate the rock glacier displacement fields between two epochs, corresponding features were at first manually identified on the orthophotos by a trained operator. To avoid the manual time-consuming procedure and increase the density of displacement information, two automatic procedures, the former using Least Squares Matching (LSM) and the latter a proprietary implementation of Semi-Global Matching (SGM) have been implemented. Both techniques were applied to pairs of orthophotos as well as to pairs of DSMs at different epochs. A discussion of the characteristics of the implemented methods is provided and the results of the comparison of the two methods with manual measurements are illustrated. Overall, results using DSM matching provided higher completeness of the displacement field than orthophoto matching, especially if long-term (year-to-year) comparisons are considered. At the same time, SGM in both cases produced less mismatches and more smooth and reliable displacement fields than LSM.

Point cloud optimization method of low-altitude remote sensing image based on vertical patch-based least square matching

This paper presents a point cloud optimization method of low-altitude remote sensing image based on least square matching (LSM). The proposed method is designed to be especially effective for addressing the conundrum of stereo matching on the discontinuity of architectural structures. To overcome the error matching and blur on building discontinuities in three-dimensional (3-D) reconstruction, a pair of mutually perpendicular patches is set up for every point of object discontinuities instead of a single patch. Then an error equation is built to compute the optimal point according to the LSM method, space geometry relationship, and collinear equation constraint. Compared with the traditional patch-based LSM method, the proposed method can achieve higher accuracy 3-D point cloud data and sharpen the edge. This is because a geometric mean patch in patch-based LSM is the local tangent plane of an object’s surface. Using a pair of mutually perpendicular patches instead of a single patch evades the problem that the local tangent plane on the discontinuity of a building did not exist and highlights the edges of buildings. Comparison studies and experimental results prove the high accuracy of the proposed algorithm in low-altitude remote sensing image point cloud optimization.

Stable least-squares matching for oblique images using bound constrained optimization and a robust loss function

Least-squares matching is a standard procedure in photogrammetric applications for obtaining sub-pixel accuracies of image correspondences. However, least-squares matching has also been criticized for its instability, which is primarily reflected by the requests for the initial correspondence and favorable image quality. In image matching between oblique images, due to the blur, illumination differences and other effects, the image attributes of different views are notably different, which results in a more severe convergence problem. Aiming at improving the convergence rate and robustness of least-squares matching of oblique images, we incorporated prior geometric knowledge in the optimization process, which is reflected as the bounded constraints on the optimizing parameters that constrain the search for a solution to a reasonable region. Furthermore, to be resilient to outliers, we substituted the square loss with a robust loss function. To solve the composite problem, we reformulated the least-squares matching problem as a bound constrained optimization problem, which can be solved with bounds constrained Levenberg–Marquardt solver. Experimental results consisting of images from two different penta-view oblique camera systems confirmed that the proposed method shows guaranteed final convergences in various scenarios compared to the approximately 20–50% convergence rate of classical least-squares matching.

A radial-velocity survey of Ap stars with HARPS

Context. Chemically peculiar Ap stars are main-sequence stars of spectral type B8–F3 which show large overabundances of some chemical elements (e.g., Si, Sr, Cr), especially of rare-earth elements, compared to normal A-type stars. Furthermore, they have strong, global magnetic fields and low rotational velocities. Some Ap stars exhibit high-overtone, low-degree, non-radial p-mode pulsations with periods of 6–24 min and are called rapidly oscillating Ap (roAp) stars. Interestingly, no roAp star is known to be a spectroscopic binary (SB), while many non-oscillating Ap (noAp) stars are found in SB systems. Aims. The goal of this survey was to carry out a systematic search for sub-stellar and stellar companions around Ap stars. Methods. Between 2004 and 2009, we observed 65 chemically peculiar stars with the HARPS spectrograph at the 3.6-m telescope at the European Southern Observatory (La Silla, Chile). The radial-velocity (RV) measurements were obtained using a new software called HARPS-TERRA which is based on a least-squares matching of each individual spectrum to a high signal-to-noise ratio template for each star. Results. We find significant variability with a period of 93.2 days and a semi-amplitude of 11.85 km s −1 in the RVs of HD 42659, a star of our sample that belongs to the group of roAp stars. This variability is caused by a stellar companion with a minimum mass of

Accurate Affine Invariant Image Matching Using Oriented Least Square

AbstractImage matching is a vital process for many photogrammetric and remote sensing applications such as image registration and aerial triangulation. In this paper, an accurate affine invariant image matching approach is presented. The proposed approach consists of three main steps. In the first step, two affine invariant feature detectors, including MSER and Harris-Affine features are applied for feature extraction. In the second step, initial corresponding features are selected using Euclidean distance between feature descriptors, followed by a consistency check process. Finally to overcome low positional accuracy of the local affine feature, an advanced version of the least square matching (LSM) namely, Oriented Least Square Matching (OLSM) is developed. Wellknown LSM method has been widely accepted as one of the most accurate methods to obtain high reliable corresponding points from a stereo image pair. However, it is sensitive to significant geometric distortion and requires very good initial approximation. In the proposed OLSM method, shape and size of the matching window are appropriately approximated using obtained affine shape information of the initial elliptical feature pairs. The proposed method was successfully applied for matching various synthetic and real close range and satellite images. Results demonstrate its accuracy and capability compared to standard LSM method.

Total Least Squares Registration of 3D Surfaces

Co-registration of point clouds of partially scanned
 objects is the first step of the 3D modeling workflow. The aim of
 co-registration is to merge the overlapping point clouds by estimating the
 spatial transformation parameters. In computer vision and photogrammetry domain
 one of the most popular methods is the ICP (Iterative Closest Point) algorithm
 and its variants. There exist the 3D Least Squares (LS) matching methods as
 well (Gruen and Akca, 2005). The
 co-registration methods commonly use the least squares (LS) estimation method
 in which the unknown transformation parameters of the (floating) search surface
 is functionally related to the observation of the (fixed) template surface.
 Here, the stochastic properties of the search surfaces are usually omitted.
 This omission is expected to be minor and does not disturb the solution vector
 significantly. However, the a posteriori covariance matrix will be affected by
 the neglected uncertainty of the function values of the search surface. . This
 causes deterioration in the realistic precision estimates. In order to overcome
 this limitation, we propose a method where the stochastic properties of both
 the observations and the parameters are considered under an errors-in-variables
 (EIV) model. The experiments have been carried out using diverse laser scanning
 data sets and the results of EIV with the ICP and the conventional LS matching
 methods have been compared.

Method for wavelength stabilization of pulsed difference frequency laser at 1572 nm for CO(2) detection lidar.

High-accuracy on-line wavelength stabilization is required for differential absorption lidar (DIAL), which is ideal for precisely measuring atmospheric CO(2) concentration. Using a difference-frequency laser, we developed a ground-based 1.57-μm pulsed DIAL for performing atmospheric CO(2) measurements. Owing to the system complexity, lacking phase, and intensity instability, the stabilization method was divided into two parts-wavelength calibration and locking-based on saturated absorption. After obtaining the on-line laser position, accuracy verification using statistical theory and locking stabilization using a one-dimensional template matching method, namely least-squares matching (LSM), were adopted to achieve wavelength locking. The resulting system is capable of generating a stable wavelength.