Matching Points Research Articles

A contrastive learning based unsupervised multi-view stereo with multi-stage self-training strategy

Recent years, unsupervised multi-view stereo (MVS) methods have achieved excellent success that can produce comparable results to earlier supervised work. However, as unsupervised MVS uses image reconstruction as pretext task, it faces two vital drawbacks: RGB value, which is the measurement of image, is not robust enough across views due to complicated environment like lighting conditions and reconstruction itself cannot reflect quality of depth estimation linearly. These problems cause the actual optimization goal to diverge from the expected optimization goal, thus could impair the training process. To enhance robustness of pretext task, we propose a contrastive learning based constraint. The constraint adds featuremetric consistency across views by forcing the features between matching points similar and the features between unmatched points opposite. To add linear properties to overall training procedure, we propose a multi-stage training strategy that uses pseudo label as supervision after unsupervised training at the beginning. On the other hand, we adopt an iterative optimizer that proven to be quite effective in supervised MVS to accelerate training. Finally, we conduct a series of experiments on the DTU dataset and Tanks and Temples dataset that demonstrate the efficiency and robustness of our method compared with the state-of-the-art methods in terms of accuracy, completeness and speed.

A robust method for multi-view 3D data stitching based on pasted marked points

High-precision three-dimensional (3D) measurement techniques commonly perform 3D point cloud data stitching using pasted marked points. However, pasted marked points exhibit a high degree of randomness and are susceptible to mismatching due to their structural similarity. Therefore, this paper presents a robust method for multi-view 3D data stitching based on pasted marked points (RMSM). First, potential matched marked point pairs (MMPP) are identified according to spatial feature invariance constraints, and a matching strength calculation model is established to find the fiducial point pairs. Then, a quick public MMPP searching algorithm using the congruent triangle search method is proposed. Thus, the problem of mismatching caused by structurally similar marked points can be eliminated, and robust stitching of 3D data from different views can be achieved. In simulation and measurement experiments, the RMSM demonstrates superior accuracy compared to alternative methods and holds promise for application in various multi-view 3D data stitching methodologies.

Design and simulation of three-junction all perovskite tandem solar cells: A path to enhanced photovoltaic performance

Despite numerous publications on two-junction tandems using SCAPS-1D simulation, there is a gap in research regarding three-junction tandems. This paper explores the advantages of three junction all-perovskite tandem solar cells (3J-APTSC) where we introduce a new methodology utilizing two-step filtered spectrum and current matching techniques in SCAPS-1D to design realistic tandem solar cells. Crucial steps involve determining filtered spectrum and current matching points among sub-cells for tandem optimization. Our designed three-terminal tandem solar cell exhibits promising photovoltaic parameters, including VOC: 3.76 V, JSC: 10.70 mA/cm2, FF: 69.00 %, and PCE: 26.24 %, achieved through optimized active layer thicknesses. This thorough examination of 3J-APTSC offers insights that could drive the future development of highly efficient tandem solar cells.

GPU Accelerated Processing Method for Feature Point Extraction and Matching in Satellite SAR Images

This paper addresses the challenge of extracting feature points and image matching in Synthetic Aperture Radar (SAR) satellite images, particularly focusing on large-scale embedding. The widely used Scale Invariant Transform (SIFT) algorithm, successful in computer vision and optical satellite image matching, faces challenges when applied to satellite SAR images due to the presence of speckle noise, leading to increased matching errors. The SAR–SIFT method is explored and analyzed in-depth, considering the unique characteristics of satellite SAR images. To enhance the efficiency of matching identical feature points in two satellite SAR images, the paper proposes a Graphics Processing Unit (GPU) mapping implementation based on the SAR–SIFT algorithm. The paper introduces a multi-GPU collaborative acceleration strategy for SAR image matching. This strategy addresses the challenge of matching feature points in the region and embedding multiple SAR images in large areas. The goal is to achieve efficient matching processing of multiple SAR images in extensive geographical regions. The proposed multi-GPU collaborative acceleration algorithm is validated through experiments involving feature point extraction and matching using 21 GF-3 SAR images. The results demonstrate the feasibility and efficiency of the algorithm in enhancing the processing speed of matching feature points in large-scale satellite SAR images. Overall, the paper contributes to the advancement of SAR image processing techniques, specifically in feature point extraction and matching in large-scale applications.

Remote sensing image feature matching via graph classification with local motion consistency

ABSTRACT Feature matching is a classic challenge in the computer vision field. In this paper, we propose an innovative graph classification method based on neighborhood motion consistency to eliminate erroneous matches. Specifically, we transform the coordinates of feature matching points into vectors on a unified scale. For a given match, we construct a graph centered around the match and incorporating neighboring matches. Node attributes are designed to represent the similarity between the vector of the central node and those of its neighbors. To facilitate this, we develop a lightweight graph attention neural network dedicated to graph property classification, thereby predicting the accuracy of the match under consideration. To effectively train the model, we employ a random cropping strategy to generate a plethora of diverse graphs for classifier training. We evaluate our method on datasets encompassing translational remote sensing data, rotational and scaled remote sensing imagery produced via random cropping, and nonrigid fisheye datasets. Our algorithm demonstrates superior performance to current state-of-the-art methods.

Review of the state of practice in geovisualization in the geosciences

Geosciences modelling and 3D geovisualization is growing and evolving rapidly. Driven by commercial urgency and an increase in data from sensor-based sources, there is an abundance of opportunities to analyze geosciences data in 3D and 4D. Geosciences modelling is developing in GIS based systems, 3D modelling through both game engines and custom programs, and the use of extended reality to further interact with data. The key limitations that are currently prevalent in 3D geovisualization in the geosciences are GIS representations having difficulty displaying 3D data and undergoing translations to pseudo-3D, thus losing fidelity, financial and personnel capital, processing issues with the terabytes worth of data and limited computing, digital occlusion and spatial interpretation challenges with users, and matching and alignment of 3D points. The future of 3D geovisualization lies in its accelerated growth, data management solutions, further interactivity in applications, and more information regarding the benefits and best practices in the field.

Joint Panchromatic and Multispectral Geometric Calibration Method for the DS-1 Satellite

The DS-1 satellite was launched successfully on 3 June 2021 from the Taiyuan Satellite Launch Center. The satellite is equipped with a 1 m panchromatic and a 4 m multispectral sensor, providing high-resolution and wide-field optical remote sensing imaging capabilities. For satellites equipped with panchromatic and multispectral sensors, conventional geometric processing methods in the past involved separate calibration for the panchromatic sensor and the multispectral sensor. This method produced distinct internal and external calibration parameters in the respective bands, and also resulted in nonlinear geometric misalignments between the panchromatic and multispectral images due to satellite chattering and other factors. To better capitalize on the high spatial resolution of panchromatic imagery and the superior spectral resolution of multispectral imagery, it is necessary to perform registration on the calibrated panchromatic and multispectral images. When registering separately calibrated panchromatic and multispectral images, poor consistency between panchromatic and multispectral images leads to a small number of corresponding points, resulting in poor accuracy and registration effects. To address this issue, we propose a joint panchromatic and multispectral calibration method to register the panchromatic and multispectral images. Before geometric calibration, it is necessary to perform corresponding points matching. When matching, the small interval between the panchromatic and multispectral Charge-Coupled Devices (CCDs) results in a small intersection angle of the corresponding points between the panchromatic and multispectral images. As a result of this, the consistency between the spectral bands significantly improves, and the corresponding points match to have a more uniform distribution and a wider coverage. The technique enhances the consistent registration accuracy of both the panchromatic and multispectral bands. Experiments demonstrate that the joint calibration method yields a registration accuracy of panchromatic and multispectral bands exceeding 0.3 pixels.

Local feature matching and tracking optimization of machine VSLAM based on AG Sinkhorn

SummaryAt present, the visual Simultaneous localization and mapping method has a wide range of applications in mobile robots and unmanned driving. However, when the environment changes, the accuracy of the system drops sharply. Aiming at the low matching accuracy of local feature matching and tracking optimization in current visual simultaneous localization and mapping methods, the attention map neural network and Sinkhorn algorithm are used to extract and match the image features. Then, it improves the real‐time performance of the visual simultaneous localization and mapping system by local optimization of feature point matching, and proposes an optimization method for visual simultaneous localization and mapping in view of an end‐to‐end algorithm. The experiment illustrates that in the experimental comparison of HPatches View dataset, the Homography estimation was improved by 0.092, the matching accuracy was improved by 0.087, and the matching recall was improved by 0.038. In the experimental comparison of HPatches Illum dataset, the Homography estimation was improved by 0.015, the matching accuracy is improved by 0.076, and the matching recall was improved by 0.036. The local feature matching and tracking optimization method of the visual Simultaneous localization and mapping system through the attention map neural network and Sinkhorn algorithm can enhance the matching accuracy of local feature matching and tracking optimization in dynamic environment and under changing lighting conditions.

The Psychological Construction of Early Childhood Art Education

The psychological construction of early childhood art education refers to the process of promoting children's mental health, emotional development, and personality shaping through art education activities. Art Education provides a platform for early childhood to express themselves, achieve self-realization, unleash their wisdom, develop own aesthetic standards, learn cooperate and communication skills, achieve psychological balance and establish understanding and respect. This study highlights the strategic shift in the approach to art education in China, with a focus on interdisciplinary education and the integration of art into extracurricular activities. With the issue on knowledge is often imparted in a one-way manner and children is encouraged to passively imitate, does not align with the developmental needs of young children or the goals of comprehensive education, this paper researches the dynamic process that involves teachers, students, and sometimes parents and community members in a cycle of planning, acting, observing, and reflecting to improve educational practices, and outlines the Mission and Vision on Development goals and the level of emphasis placed on families and kindergartens, Professional advantages of promoting mental hygiene through art education and Matching of key points, principles, and strategies.

An adaptive grouping sonar-inertial odometry for underwater navigation

This paper presents a sonar-inertial navigation system with an adaptive grouping optimization framework, which tightly couples inertial measurements with acoustic data from forward-looking sonar (FLS). The system’s front-end uses the sonar-inertial constraint-based feature matching (SICFM) to increase the accuracy and robustness of data association. SICFM is designed by incorporating two novel measures. Firstly, it combines prior motion information to refine the feature search region. Then, a Gaussian-gamma mixture model is formulated to reduce the impact of speckle noise on the selection of matching points. In the back-end, an adaptive grouping factor graph optimization framework is established. A frame manager is designed using the solvability of motion estimation to dynamically group sonar frames, thereby ensuring stable feature tracking for factor graph optimization. Pre-integration-based inertial measurement factor and reprojection-based sonar measurement factor are constructed as constraints in the optimization, which reduces the dependence on prior knowledge of the scene. Furthermore, an elevation angle parameterization is introduced, aiming at the nonlinear projection of sonar features. The evaluation is presented in simulation and real-world experiments, which validate our proposed algorithm in terms of robust localization with high accuracy.

Local topology similarity guided probabilistic sampling for mismatch removal

Feature point matching between two images is a fundamental and important process in machine vision. In many cases, mismatches are inevitable, and removing mismatches is an indispensable task. The existing methods attempt to find comprehensive constraints or sampling model to achieve better performance, which results in the increasingly complexity and may cause the weakness of the generality and scalability. To address this issue, a method called Local Topology similarity guided probabilistic Sampling consensus (LTS) is proposed. It constructs a topological network, then quantifies the mismatch probability in a concise approach based on comparing the topological relationship with neighbourhoods. Then, it detects and removes the mismatches by sampling guided by the mismatch probability. Compared with the state-of-the-art methods, LTS has an excellent performance in accuracy and robustness.

CGAN-Based Forest Scene 3D Reconstruction from a Single Image

Forest scene 3D reconstruction serves as the fundamental basis for crucial applications such as forest resource inventory, forestry 3D visualization, and the perceptual capabilities of intelligent forestry robots in operational environments. However, traditional 3D reconstruction methods like LiDAR present challenges primarily because of their lack of portability. Additionally, they encounter complexities related to feature point extraction and matching within multi-view stereo vision sensors. In this research, we propose a new method that not only reconstructs the forest environment but also performs a more detailed tree reconstruction in the scene using conditional generative adversarial networks (CGANs) based on a single RGB image. Firstly, we introduced a depth estimation network based on a CGAN. This network aims to reconstruct forest scenes from images and has demonstrated remarkable performance in accurately reconstructing intricate outdoor environments. Subsequently, we designed a new tree silhouette depth map to represent the tree’s shape as derived from the tree prediction network. This network aims to accomplish a detailed 3D reconstruction of individual trees masked by instance segmentation. Our approach underwent validation using the Cityscapes and Make3D outdoor datasets and exhibited exceptional performance compared with state-of-the-art methods, such as GCNDepth. It achieved a relative error as low as 8% (with an absolute error of 1.76 cm) in estimating diameter at breast height (DBH). Remarkably, our method outperforms existing approaches for single-image reconstruction. It stands as a cost-effective and user-friendly alternative to conventional forest survey methods like LiDAR and SFM techniques. The significance of our method lies in its contribution to technical support, enabling the efficient and detailed utilization of 3D forest scene reconstruction for various applications.

QUIZ: An arbitrary volumetric point matching method for medical image registration

Rigid pre-registration involving local–global matching or other large deformation scenarios is crucial. Current popular methods rely on unsupervised learning based on grayscale similarity, but under circumstances where different poses lead to varying tissue structures, or where image quality is poor, these methods tend to exhibit instability and inaccuracies. In this study, we propose a novel method for medical image registration based on arbitrary voxel point of interest matching, called query point quizzer (QUIZ). QUIZ focuses on the correspondence between local–global matching points, specifically employing CNN for feature extraction and utilizing the Transformer architecture for global point matching queries, followed by applying average displacement for local image rigid transformation.We have validated this approach on a large deformation dataset of cervical cancer patients, with results indicating substantially smaller deviations compared to state-of-the-art methods. Remarkably, even for cross-modality subjects, it achieves results surpassing the current state-of-the-art.

Point cloud based hand gesture recognition using template matching

Hand gesture recognition is important in human-computer interaction with wide applications in many fields. Different from common hand gesture recognition based on 2D images acquired from RGB camera, the utilization of 3D images provides additional spatial information about the target and attracts more and more attention in hand gesture recognition. However, most 3D images for hand gesture recognition are based on depth maps, which only take the distance information as a channel of 2D images, without taking full use of the 3D information. Besides, greater data volume of 3D images brings challenges to the arithmetic facility of hand gesture recognition. Here, we proposed a point cloud based method for hand gesture recognition. To fully use the 3D information, plane points for template matching were extracted based on their normal distributions, which leads to the average recognition rate over 97%. Pre-classification was implemented to ensure a high-efficient recognition without additional requirements for the computer. The proposed method may provide approach for accurate and efficient hand gesture recognition based on 3D images.

Research on Image Stitching Algorithm Based on Point-Line Consistency and Local Edge Feature Constraints.

Image stitching aims to synthesize a wider and more informative whole image, which has been widely used in various fields. This study focuses on improving the accuracy of image mosaic and proposes an image mosaic method based on local edge contour matching constraints. Because the accuracy and quantity of feature matching have a direct influence on the stitching result, it often leads to wrong image warpage model estimation when feature points are difficult to detect and match errors are easy to occur. To address this issue, the geometric invariance is used to expand the number of feature matching points, thus enriching the matching information. Based on Canny edge detection, significant local edge contour features are constructed through operations such as structure separation and edge contour merging to improve the image registration effect. The method also introduces the spatial variation warping method to ensure the local alignment of the overlapping area, maintains the line structure in the image without bending by the constraints of short and long lines, and eliminates the distortion of the non-overlapping area by the global line-guided warping method. The method proposed in this paper is compared with other research through experimental comparisons on multiple datasets, and excellent stitching results are obtained.

Blowout Fluid Velocity Measurement Method based on High-speed Sequence Images

Given the high-temperature and high-pressure environment after the blowout, the staff usually cannot calculate and analyze various blowout parameters at close range and thus take effective measures to control accidents quickly. In this paper, a blowout fluid velocity measurement method is proposed based on high-speed sequence images, and simulation experiments are conducted. The scene conditions of blowout accidents are simulated and analyzed. On this basis, cameras are adopted to shoot a large number of simulated blowout fluid videos from different angles under outdoor conditions. Afterward, the video is converted into a multi-frame image, and the blowout fluid image feature point detection and extraction model are established based on the orb (oriented fast and rotated brief) algorithm. On the feature points extracted by the model, the gms algorithm is used to match the features of the blowout fluid images in the front and back frames. Additionally, the wrong matching points are eliminated using the ransac (random sample consensus) algorithm, and the obtained feature point pairs are employed into the flow velocity calculation model for flow velocity calculation. The results demonstrate that the measured fluid velocity is more accurate. The method proposed in this study can provide reliable data support for early blowout emergency rescue.

Image moment extraction based aerial photo selection for UAV high-precision geolocation without GPS

Since GPS is susceptible to environmental interference or cannot be used in some specific situations, it is important to study the UAV geolocation under GPS-free condition. How to ensure that the aerial photos used for positioning have high accuracy is a pressing concern. The conventional approach relies on the number of matching feature points between aerial and satellite photos and verifying whether the aerial photo center projection falls within a designated area of the satellite photo. However, this method often yields suboptimal accuracy. This paper introduces a novel method for selecting aerial photos for geolocation based on an image moment extraction module. The image moment extraction module contains two parts: the relative error of the image invariant moment and the area relative error. The former is used to select the aerial photos with good performance in translation, rotation and scaling before and after projection, and the latter is adopted to reject aerial photos with excessive scaling before and after projection. Through experiments, we demonstrate the effectiveness and superiority of this approach. Results show that the image moment extraction module, when used for positioning with aerial photo center projection, significantly reduces the mean absolute error in UAV geolocation from 63.220 m to 9.454 m. The maximum error is reduced from 486.7 m to 47.194 m, with 70.96% of aerial photos suitable for positioning. This research is pivotal for achieving high-precision UAV geolocation in GPS-free environments.

Appearance flow estimation for online virtual clothing warping via optimal feature linear assignment

Clothing warping to spatially align source garments with the corresponding body parts is crucial in clothing media tasks such as virtual try-on and pose-guided person generation. Recent pioneering work has utilized flow fields with additional dimensions of freedom to simulate clothing warping flexibly. However, current methods for estimating appearance flow typically rely on calculating just the local cost volume which contains multiple noisy matching points, potentially leading to a mismatch between clothing and body parts. To address this issue, we propose a novel appearance flow estimation network(Warping-Flow) for clothing warping based on optimal features linear assignment. Specially, two remarkable contributions are made to improve feature matching precision. First, a local context feature aggregation module is proposed to enhance the semantic feature distinction of the source cloth and target pose. Second, Warping-Flow estimates a hard attention mask through the cost volume to filter irrelevant features, followed by the optimal linear assignment algorithm to normalize the cost volume to a discrete permutation matrix that explicitly models the most contributing bipartite matches. Experiments conducted on the VITON and VITON-HD datasets demonstrate that Warping-Flow outperforms existing state-of-the-art algorithms, particularly in cases involving complex clothing deformation. Furthermore, Warping-Flow can serve as a plug-in to improve existing garment media technologies.

Directional region-based feature point matching algorithm based on SURF.

Feature point matching is one of the fundamental tasks in binocular vision. It directly affects the accuracy and quality of 3D reconstruction. This study proposes a directional region-based feature point matching algorithm based on the SURF algorithm to improve the accuracy of feature point matching. First, same-name points are selected as the matching reference points in the left and right images. Then, the SURF algorithm is used to extract feature points and construct the SURF feature point descriptors. During the matching process, the location relationship between the query feature point and the reference point in the left image is directed to determine the corresponding matching region in the right image. Then, the matching is completed within this region based on Euclidean distance. Finally, the grid-based motion statistics algorithm is used to eliminate mismatches. Experimental results show that the proposed algorithm can substantially improve the matching accuracy and the number of valid matched points, particularly in the presence of a large amount of noise and interference. It also exhibits good robustness and stability.

Feature matching based on Gaussian kernel convolution and minimum relative motion

Feature matching is a necessary and important step for remote sensing image registration, intended to establish reliable point correspondences between two sets of features. In this paper, we propose a feature registration model based on local relative motion, which combines Gaussian kernel convolution with relative motion (GRM) vector to obtain better results by removing wrong matches and improving the inlier point accuracy. We first establish putative matching based on the similarity between local descriptors. Then, the preliminary hypothetical matching point set is filtered using consistency with nearest neighbors among the inlier points to obtain a more accurate motion vector, and to fit the real motion vector through the Gaussian convolution kernel. Finally, we find the displacement between the fitted motion vector and the matching generated motion vector. And combine the displacement with the optimization model to find the inlier point set. Experimental results show that our GRM method outperforms related work, achieving better matching results.