Traditional Matching Methods Research Articles

A deep learning semantic template matching framework for remote sensing image registration

We propose a deep learning framework by the probability of the predicting semantic spatial position distribution for remote sensing image registration. Traditional matching methods optimize similarity metrics with pixel-by-pixel searching, which is time consuming and sensitive to radiometric differences. Driven by learning-based methods, we take the reference and template images as inputs and map them to the semantic distribution position of the corresponding reference image. We apply the affine invariant to obtain a correspondence between the overlap of the barycenter position of the semantic template and the center pixel point, which transforms the semantic boundary alignment into a point-to-point matching problem. Additionally, two loss functions are proposed, one for optimizing the subpixel matching position and the other for determining the semantic spatial probability distribution of the matching template. In this work, we explore the influence of the template radius size, the filling form of training labels, and the weighted combination of loss function on the matching accuracy. Our experiments show that the trained model is robust to template deformation while still operating orders of magnitude faster. Furthermore, our proposed method implements high matching accuracy in four large scene images with radiometric differences. This ensures the improved speed of remote sensing image analysis and pipeline processing while facilitating novel directions in learning-based registration. Our code is freely available at https://github.com/liliangzhi110/semantictemplatematching.

An Image Matching Method Based on SIFT Feature Extraction and FLANN Search Algorithm Improvement

In order to solve the problems of less feature information and high mismatching rate in traditional image matching algorithms, this paper proposed to extract and describe features based on the SIFT algorithm. FLANN algorithm was used to pre-match feature points, and RANSAC algorithm was used to optimize the matching results, so as to realize real-time image matching and recognition. Experimental results show that the proposed algorithm has better accuracy and better matching effect than traditional image matching methods.

Lithology Classification Using TASI Thermal Infrared Hyperspectral Data with Convolutional Neural Networks

In recent decades, lithological mapping techniques using hyperspectral remotely sensed imagery have developed rapidly. The processing chains using visible-near infrared (VNIR) and shortwave infrared (SWIR) hyperspectral data are proven to be available in practice. The thermal infrared (TIR) portion of the electromagnetic spectrum has considerable potential for mineral and lithology mapping. In particular, the abovementioned rocks at wavelengths of 8–12 μm were found to be discriminative, which can be seen as a characteristic to apply to lithology classification. Moreover, it was found that most of the lithology mapping and classification for hyperspectral thermal infrared data are still carried out by traditional spectral matching methods, which are not very reliable due to the complex diversity of geological lithology. In recent years, deep learning has made great achievements in hyperspectral imagery classification feature extraction. It usually captures abstract features through a multilayer network, especially convolutional neural networks (CNNs), which have received more attention due to their unique advantages. Hence, in this paper, lithology classification with CNNs was tested on thermal infrared hyperspectral data using a Thermal Airborne Spectrographic Imager (TASI) at three small sites in Liuyuan, Gansu Province, China. Three different CNN algorithms, including one-dimensional CNN (1-D CNN), two-dimensional CNN (2-D CNN) and three-dimensional CNN (3-D CNN), were implemented and compared to the six relevant state-of-the-art methods. At the three sites, the maximum overall accuracy (OA) based on CNNs was 94.70%, 96.47% and 98.56%, representing improvements of 22.58%, 25.93% and 16.88% over the worst OA. Meanwhile, the average accuracy of all classes (AA) and kappa coefficient (kappa) value were consistent with the OA, which confirmed that the focal method effectively improved accuracy and outperformed other methods.

Binocular stereo matching of real scenes based on a convolutional neural network and computer graphics.

The binocular stereo matching method based on deep learning has limited cross-domain generalization ability, and it is a tricky problem to obtain a large amount of data from real scenes for training. The most advanced stereo matching network is difficult to apply to new real scenes. In this paper, we propose a real-scene stereo matching method based on a convolutional neural network and computer graphics. A virtual binocular imaging system is constructed by introducing graphics software, and a high-quality semi-synthetic dataset close to the texture characteristics of the real scene is constructed for training the network. A feature standardization layer is embedded in the feature extraction module of the proposed network to further reduce the feature space difference between semi-synthetic data and real scene data. Three small 4D cost volumes are constructed to replace one large 4D cost volume, which reduces GPU memory consumption and improves the matching performance of the network. The experimental results show that compared with the traditional stereo matching method, the matching accuracy of the proposed method is significantly improved by about 60%. Compared with other learning-based methods, the matching accuracy is increased by about 30%, the matching speed is increased by 38%, and it has good robustness to the interference of defocus blur and Gaussian noise.

Topological distance-constrained feature descriptor learning model for vessel matching in coronary angiographies

BackgroundFeature matching technology is vital to establish the association between virtual and real objects in virtual reality and augmented reality systems. Specifically, it provides them with the ability to match a dynamic scene. Many image matching methods, of which most are deep learning-based, have been proposed over the past few decades. However, vessel fracture, stenosis, artifacts, high background noise, and uneven vessel gray-scale make vessel matching in coronary angiography extremely difficult. Traditional matching methods perform poorly in this regard. MethodsIn this study, a topological distance-constrained feature descriptor learning model is proposed. This model regards the topology of the vasculature as the connection relationship of the centerline. The topological distance combines the geodesic distance between the input patches and constrains the descriptor network by maximizing the feature difference between connected and unconnected patches to obtain more useful potential feature relationships. ResultsMatching patches of different sequences of angiographic images are generated for the experiments. The matching accuracy and stability of the proposed method is superior to those of the existing models. ConclusionsThe proposed method solves the problem of matching coronary angiographies by generating a topological distance-constrained feature descriptor.

Application of the response-surface method and CFD simulations in the matching optimization of spiral sealing for a high-speed cone bit

ABSTRACT The cone bit is an important rock-breaking tool in oil and gas mining. Sealing plays a crucial role in determining the high rotational speed and service life of the bit. The traditional seal structure is passive and wears out rapidly. Under the high-speed rotation of the drill, it has a short sealing service life. This paper describes a spiral-seal active sand discharge structure that incorporates a new type of sealing for the cone bit. Because traditional matching optimization methods are not suitable for this type of sealing, the response-surface method (RSM) was applied to optimize the spiral seal structure for a high-speed cone bit. The optimization significantly improved the service life of the bit. The working conditions and the effect of the spiral-seal structure were studied through computational fluid dynamics (CFD) simulations that revealed the sealing mechanism of this structure. Additionally, the RSM was used to perform matching and optimization of the spiral-seal structure for a high-speed cone bit and to study the dependence of the spiral structure parameters on the sand discharge performance at different rotation speeds. This not only provides more accurate theoretical data for actual production, but also provides a means to optimize other mechanical structures.

A Rotation-Invariant Optical and SAR Image Registration Algorithm Based on Deep and Gaussian Features

Traditional feature matching methods of optical and synthetic aperture radar (SAR) used gradient are sensitive to non-linear radiation distortions (NRD) and the rotation between two images. To address this problem, this study presents a novel approach to solving the rigid body rotation problem by a two-step process. The first step proposes a deep learning neural network named RotNET to predict the rotation relationship between two images. The second step uses a local feature descriptor based on the Gaussian pyramid named Gaussian pyramid features of oriented gradients (GPOG) to match two images. The RotNET uses a neural network to analyze the gradient histogram of the two images to derive the rotation relationship between optical and SAR images. Subsequently, GPOG is depicted a keypoint by using the histogram of Gaussian pyramid to make one-cell block structure which is simpler and more stable than HOG structure-based descriptors. Finally, this paper designs experiments to prove that the gradient histogram of the optical and SAR images can reflect the rotation relationship and the RotNET can correctly predict them. The similarity map test and the image registration results obtained on experiments show that GPOG descriptor is robust to SAR speckle noise and NRD.

Large parallax image matching based on Graph Matching

In the security video surveillance system, image splicing technology faces the difficulty of achieving large parallax and small overlap area. This paper summarizes the traditional stitching technology and analyzes that the image matching algorithm is the key technology to solve the problem of large parallax stitching. Aiming at the problems that traditional matching methods with low matching accuracy cannot adapt to large parallax non-rigid transformatio, a model based on image matching is proposed in this paper for splicing tasks. The model which combines the similarity of the global structure and the local structure can not only divide the feature point on the structural level but also can pair with each other. At the same time, the framework can be trained end-to-end. The training samples composed of simulation data sets are adopted to train the designed model which is tested and verifyed by the test sample sets composed of three kinds of mixed data. Compared with PCA-GM, the model proposed in this paper adapts well to splicing tasks. The matching accuracy can achieve 93.1% and 91.9% on the simulation data set and the small parallax splicing data respectively, which can initially solve the image matching problem in the traditional image splicing scene. However, a further research aiming at the problem of the matching accuracy on the large parallax data set can only achieve 74.5% will be needed.

Non-intrusive load transient identification based on multivariate LSTM neural network and time series data augmentation

Getting the specific energy consumption information of each appliance is an effective way to reduce energy wastage and improve energy efficiency. The non-intrusive load monitoring is a promising solution to achieve this goal. In terms of the event-based non-intrusive load monitoring, the load transient identification models have attracted great attention recently. In this study, a hybrid load transient identification model based on multivariate fusion, time series data augmentation, and deep neural network computation is proposed. In the hybrid model, the multivariate LSTM neural network is utilized to extend the traditional template matching methods to the deep learning field, which is beneficial to improve the identification accuracy and the efficiency in the testing stage. A hybrid time series data augmentation framework is designed to handle the class-imbalance and insufficient sample size, which is often the case in non-intrusive load monitoring. The effectiveness of each component, as well as the whole hybrid model, are evaluated based on the BLUED dataset. The comparison results indicate that (a) multivariate LSTM neural network is effective in modeling the characters of the load transient; (b) the proposed hybrid time series data augmentation framework is effective in improving the overall performance of the multivariate LSTM neural network; (c) taking identification accuracy and the application efficiency in testing stage into consideration, the proposed hybrid model performs better than that of the state of the art Dynamic Time Warping (DTW) based models.

Single-shot 3D shape measurement using an end-to-end stereo matching network for speckle projection profilometry.

Speckle projection profilometry (SPP), which establishes the global correspondences between stereo images by projecting only a single speckle pattern, has the advantage of single-shot 3D reconstruction. Nevertheless, SPP suffers from the low matching accuracy of traditional stereo matching algorithms, which fundamentally limits its 3D measurement accuracy. In this work, we propose a single-shot 3D shape measurement method using an end-to-end stereo matching network for SPP. To build a high-quality SPP dataset for training the network, by combining phase-shifting profilometry (PSP) and temporal phase unwrapping techniques, high-precision absolute phase maps can be obtained to generate accurate and dense disparity maps with high completeness as the ground truth by phase matching. For the architecture of the network, a multi-scale residual subnetwork is first leveraged to synchronously extract compact feature tensors with 1/4 resolution from speckle images for constructing the 4D cost volume. Considering that the cost filtering based on 3D convolution is computationally costly, a lightweight 3D U-net network is proposed to implement efficient 4D cost aggregation. In addition, because the disparity maps in the SPP dataset should have valid values only in the foreground, a simple and fast saliency detection network is integrated to avoid predicting the invalid pixels in the occlusions and background regions, thereby implicitly enhancing the matching accuracy for valid pixels. Experiment results demonstrated that the proposed method improves the matching accuracy by about 50% significantly compared with traditional stereo matching methods. Consequently, our method achieves fast and absolute 3D shape measurement with an accuracy of about 100µm through a single speckle pattern.

A Comparative Study of Camouflage Printing Color Matching Based on Monitor and Paper Card

To improve the accuracy of color reproduction for camouflage printing, a new color matching method, named monitor-paper-fabric, was proposed by importing color management into textiles, which matched the printing pastes according to the color of the paper card printed by the ink-jet printer after color management. Not like the traditional color matching method, it matched the printing pastes according to the color from computer monitor after color correction. Two color matching methods were analyzed by comparing the color difference. It was found that the “monitor-paper-fabric” color matching method could be considered as a convenient and feasible color matching method. Most color differences between the monitor and the fabric in the camouflage pattern were reduced to lower than 4, except for color blocks 5 and 6. In addition, the color differences of six color blocks between the paper and the fabric were all less than 3.5, and were lower than those between the monitor and the fabric. The color consistency between the paper and the fabric was better than that between the monitor and the fabric.

One-dimensional Block Matching Algorithm in Video Image Motion Estimation

<p indent=0mm>Motion estimation is a basic problem in the fields of video image compression and video image restoration. The traditional block matching methods have good search quality, but the search speed is not fast enough. Aiming at the shortcomings of the search speed in the traditional block matching methods, we proposed a fast one-dimensional block matching motion estimation algorithm. Firstly, the motion vector is orthogonally decomposed, and the two-dimensional matching block is reduced by a special weight coefficient matrix to obtain two sets of one-dimensional feature matrices. Then the one-dimensional three step search method is selected as the search strategy. The sum of absolute differences criterion is used as the matching criterion. The two sets of one-dimensional feature matrices are used to search for the two components of the matching motion vector. Finally the two components are formed into a complete motion vector. The results of multiple sets of comparative experiments show that the search speed of motion estimation is significantly improved while the algorithm is guaranteed to quantitatively evaluate the PSNR. The higher the video definition and the larger the pixel size of the matching block, the better the algorithm can improve the search speed of motion estimation.

Derivation of Z-R relationship parameter for Alor Setar radar using Traditional Matching Method (TMM)

The rainfall measurement can be done by using rain gauge and weather radar instruments. However, weather radar does not measure the rainfall depth directly as contrary to rain gauge. Therefore, an empirical relationship between reflectivity (Z) and rainfall rate (R) which is commonly known as reflectivity-rainfall (Z-R) relationship consisting of parameter A and exponential b (Z= A·Rb ), usually used to convert reflectivity data into rainfall rate for a radar. Presently, the Z-R relationship parameters proposed by Marshall and Palmer (1947) used in Malaysia is seem no longer suitable for Malaysia condition. Hence, a new relationship should be developed. The reflectivity data from year 2006 to year 2007 at Alor Setar radar and gauge rainfall data from 14 rain gauge in the Northern of Peninsular Malaysia were calibrated. By using the Traditional Matching Method (TMM), a new parameter was developed for Alor Setar radar which located in Northern Peninsular Malaysia. By minimizing the errors, a Mean Field Bias Correction (MFB) technique was apply to all selected parameter in this study with adjusting the value A and fixed the value b. As a results, new climatological Z-R relationship (Z=14.30R 1.9) was obtained. To justify the new relationship, validation analysis has been performed by using the five statistical measure. It was found that the validation analysis has given the best results of Mean Error (ME), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Bias and Pearson Correlation Coefficient (r) with value of 0.00, 0.00, 7.54, 1.00 and 0.85 respectively. Concluded that, it is suitable for radar rainfall estimation in the Northern Region of Peninsular Malaysia.

YOLOv3-Based Matching Approach for Roof Region Detection from Drone Images

Due to the large data volume, the UAV image stitching and matching suffers from high computational cost. The traditional feature extraction algorithms—such as Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and Oriented FAST Rotated BRIEF (ORB)—require heavy computation to extract and describe features in high-resolution UAV images. To overcome this issue, You Only Look Once version 3 (YOLOv3) combined with the traditional feature point matching algorithms is utilized to extract descriptive features from the drone dataset of residential areas for roof detection. Unlike the traditional feature extraction algorithms, YOLOv3 performs the feature extraction solely on the proposed candidate regions instead of the entire image, thus the complexity of the image matching is reduced significantly. Then, all the extracted features are fed into Structural Similarity Index Measure (SSIM) to identify the corresponding roof region pair between consecutive image sequences. In addition, the candidate corresponding roof pair by our architecture serves as the coarse matching region pair and limits the search range of features matching to only the detected roof region. This further improves the feature matching consistency and reduces the chances of wrong feature matching. Analytical results show that the proposed method is 13× faster than the traditional image matching methods with comparable performance.

Intelligent colour matching method for 3D character animation based on texture features

In order to overcome the problems of low matching degree and long matching time in traditional intelligent colour matching methods of 3D character animation, this paper proposes a colour intelligent matching method based on texture features. Three parameters H, S and V are taken as the main parameters of 3D character animation colour recognition, and the animation colour is compared with the colour quantisation template. According to the comparison results and colour difference formula, the animation colour recognition is realised. On this basis, SIFT algorithm is used to obtain colour features of 3D character animation, and clonal selection algorithm is used to realise colour intelligent matching of 3D character animation. The experimental results show that the accuracy rate of colour feature extraction is always above 96%, the colour matching degree is always above 97%, and the average colour matching time is 0.4 s. The practical application effect is good.

Image tracking and matching algorithm of semi-dense optical flow method

The traditional optical flow method is based on the assumption of spatial consistency of the optical flow field. It is easy to reduce the tracking quality and even lead to the loss of target tracking in the areas of image feature missing, boundary and occlusion. A semi-dense optical flow method is proposed to realise stable tracking of image features. Firstly, the feature points are preserved by calculating the pixel points with large change of pixel gradient in the image; Secondly, according to the principle of grey level invariance, the grey level difference function between the corresponding feature points of adjacent frames is constructed; Finally, the gradient descent principle is used to optimise the grey difference function and realise the accurate matching of feature points of adjacent frames. The results show that compared with the traditional LK optical flow method, this algorithm can effectively improve the feature tracking capability, and at the same time can effectively retain the useful information in the image. Compared with the traditional feature point matching method, the algorithm presented in this paper has an efficient operation rate.

Joint Feature Representation and Classifier Learning Based Unsupervised Domain Adaption ELM

In the problem of unsupervised domain adaption Extreme learning machine (ELM), the output layer parameters need to have both classification and domain adaptation functions, which often cannot be simultaneously fully utilized. In addition, traditional matching method based on data probability distribution cannot find the common subspace of source and target domains under large difference between domains. In order to alleviate the pressure of double functions of classifier parameters, the entire ELM learning process is mainly divided into two stages: feature representation and adaptive classifier learning, thus a joint feature representation and classifier learning based unsupervised domain adaption ELM model is proposed. In the feature representation stage, the source and target domain data are projected to their respective subspace while minimizing the difference in probability distribution between the two domains. In the adaptive classifier learning stage, the smooth manifold regularization term of target domain is used to improve the parameter adaptive ability. Experiments on six different types of datasets show that the proposed model has higher cross-domain classification accuracy.

Destriping and evaluating FY-3D MERSI-2 data with the moment matching method based on synchronous reference image

Medium Resolution Spectral Imager II (MERSI-2) is a payload of the China meteorological satellite FY-3D. The sensor bands 24 (10.3 to 11.3 μm) and 25 (11.5 to 12.5 μm) images, which are most suitable for land surface temperature (LST) retrieval, have higher spatial resolution than that of similar sensors. However, bands 24/25 images with spatial resolution of 250 m (hereafter bands 24/25 250-m images) have considerable horizontal stripe noise caused by the limitation of the sensor imaging system’s multi-detector parallel scanning and the multiple interferences from space. Thus, the application value of LST retrieval using bands 24/25 250-m images is severely affected. At present, existing methods used to remove stripe noise often cannot retain the spectral information of the original image. Our research considers that MERSI-2 bands 24/25 images with spatial resolution of 1000 m (hereafter bands 24/25 1000-m images) and no stripe noise can provide reasonable reference statistics for bands 24/25 250-m images. Thus, the moment matching method considering synchronous reference image is applied to remove the stripe noise of bands 24/25 250-m images in the land area. Results shows that the moment matching method considering the synchronous reference image can effectively remove the stripe noise in the land area. In addition, quantitative evaluation criteria, such as information entropy H, mean-absolute-error (MAE), and peak signal-to-noise ratio, are better than those of the traditional moment matching method. After destriping, the LST values using MERSI-2 data are retrieved with better effect and decrease by 0.339 and 0.504 K using MAE and root-mean-square error, respectively, in contrast to the Moderate Resolution Imaging Spectroradiometer LST product.

PRNU Extraction from Stabilized Video: a Patch Maybe Better than a Bunch

This paper presents an algorithm to solve the problem of Photo-Response Non-Uniformity (PRNU) noise facing stabilized video. The stabilized video undergoes in-camera processing like rolling shutter correction. Thus, misalignment exists between the PRNU noises in the adjacent frames owing to the global and local frame registration performed by the in-camera processing. The misalignment makes the reference PRNU noise and the test PRNU noise unable to extract and match accurately. We design a computing method of maximum likelihood estimation algorithm for extracting the PRNU noise from stabilized video frames. Besides, unlike most prior arts tending to match the PRNU noise in whole frame, we propose a new patch-based matching strategy, aiming at reducing the influence from misalignment of frame the PRNU noise. After extracting the reference PRNU noise and the test PRNU noise, this paper adopts the reference and the test PRNU overlapping patch-based matching. It is different from the traditional matching method. This paper conducts different experiments on 224 stabilized videos taken by 13 smartphones in the VISION database. The area under curve of the algorithm proposed in this paper is 0.841, which is significantly higher than 0.805 of the whole frame matching in the traditional algorithm. Experimental results show good performance and effectiveness the proposed strategy by comparing with the prior arts.

Research on the application of convolutional-deep neural networks in parallel fingerprint minutiae matching

In order to overcome the problem of low throughput and time-consuming of traditional fingerprint minutiae matching methods, a new convolutional-deep neural network is proposed for parallel fingerprint minutiae matching. This method realises the preprocessing of the initial image through four steps: normalisation, image enhancement, parallel thinning and image segmentation. The convolutional-deep neural network is constructed from convolution kernel, convolution layer, pooling layer and full connection layer to extract minutiae of fingerprint image. Through feature minutiae matching, local matching and global matching, the matching results of fingerprint parallel nodes are obtained. The experimental results show that compared with the traditional matching method, the fingerprint matching throughput of convolutional-deep neural network is increased by 25%, and the matching time is saved by about 8 seconds.