Nonlinear Scale Space Research Articles

PDE-Constrained Scale Optimization Selection for Feature Detection in Remote Sensing Image Matching

Feature detection and matching is the key technique for remote sensing image processing and related applications. In this paper, a PDE-constrained optimization model is proposed to determine the scale levels advantageous for feature detection. A variance estimation technique is introduced to treat the observation optical images polluted by additive zero-mean Gaussian noise and determine the parameter of a nonlinear scale space governed by the partial differential equation. Additive Operator Splitting is applied to efficiently solve the PDE constraint, and an iterative algorithm is proposed to approximate the optimal subset of the original scale level set. The selected levels are distributed more uniformly in the total variation sense and helpful for generating more accurate and robust feature points. The experimental results show that the proposed method can achieve about a 30% improvement in the number of correct matches with only a small increase in time cost.

Construction of medical aromatherapy experimental teaching resources based on digital three-dimensional technology

Abstract In this paper, on the basis of the Unity3D engine, image feature points are coarsely extracted, and a quadtree-based division algorithm is applied to filter the optimal FAST corner points. The fast 3D reconstruction technique is constructed through the construction method of nonlinear scale space similar to scale-invariant feature transform (SIFT), which is layer incremented according to the scale level logarithmic layer. The reconstructed 3D virtual teaching laboratory tool is obtained by importing the matched feature points into the model after the diffusion is completed. Finally, the teaching process and teacher-student behavioral data analysis were conducted for TCM and acupuncture students using the medical aromatherapy experimental course “Clinical Experiment of Aromatherapy for Primary Dysmenorrhea” as the teaching objective. The results showed that the maximum value of iFIAS of the experimental group was 100, and the mean value was 79.35, while the maximum value of technical iFIAS of the control group was only 51.32, and the mean value was 21.34, which was a significant difference between the two. It shows that the teaching method improves classroom learning interest and effect. The purpose of this paper is to provide relevant opinions and references for the future development direction of the combination of aromatherapy teaching and 3D virtual reality.

An improved SIFT algorithm for registration between SAR and optical images

Aiming at improving the performance of scale invariant feature transform (SIFT) algorithm during the registration of optical and synthetic aperture radar (SAR) images, a new SIFT algorithm is proposed. Firstly, the nonlinear diffusion scale space of optical and SAR images is constructed by using nonlinear diffusion filtering, the uniform gradient information is calculated by using multi-scale Sobel operator and multi-scale exponential weighted mean ratio operator respectively. Then, after removing the first layer of the scale space with the image blocking strategy, the scale space is partitioned, and Harris feature points are extracted on the basis of consistent gradient information to obtain stable and uniform point features. Descriptors are constructed based on gradient position and direction histogram templates and normalized to overcome nonlinear radiation differences between images. Finally, the correct matching point pairs are obtained by using bilateral fast approximate nearest neighbor (FLANN) search matching method and random sampling consistent (RANSAC) method, and the affine transformation model parameters are obtained. Compared with the other two algorithms, the CMR of this algorithm is improved by 80.53%, 75.61% and 81.74% respectively in the three groups of images, and the RMSE is reduced by 0.6491, 1.0287 and 0.6306 respectively.

Optical and SAR Image Registration Based on Multi-Scale Orientated Map of Phase Congruency

Optical and Synthetic Aperture Radar (SAR) images are highly complementary, and their registrations are a fundamental task for other remote sensing applications. Traditional feature-matching algorithms fail to solve the significant nonlinear radiation difference (NRD) caused by different sensors. To address this problem, a robust registration algorithm with the multi-scale orientated map of phase congruency (MSPCO) is proposed. First, a nonlinear diffusion scale space is established to obtain the scale invariance of feature points. Compared with the linear Gaussian scale space, the nonlinear diffusion scale space can better preserve the edge and texture information. Second, to ensure the quantity and repeatability of features, corner points and edge points are detected on the moment map of phase congruency, respectively, which is the foundation to the next feature matching. Third, the MSPCO descriptor is constructed via the orientation of phase congruency (PCO). PCO is highly robust to NRD, and the different scales of PCOs enhance the robustness of the descriptor. Finally, a feature-matching strategy based on an effective scale ratio is proposed, which reduces the number of comparisons among features and improves computational efficiency. The experimental results show that the proposed method is better than the existing feature-based methods in terms of the number of correct matches and registration accuracy. The registration accuracy is only inferior to that of the most advanced template matching method, and the accuracy difference is within 0.3 pixels, which fully demonstrates the robustness and accuracy of the proposed method in optical and SAR image registration.

Rotation-Invariant Registration Algorithm of Optical and SAR Images Using Fast Bilateral Filters and Fourier-Based Descriptors

Determining precise main orientations for optical and synthetic aperture radar (SAR) image registration is challenging because of serious radiometric differences and multiplicative speckles. In this letter, a rotation-invariant registration algorithm of optical and SAR images is proposed based on fast bilateral filters and Fourier-based descriptors. First, two nonlinear scale spaces are, respectively, designed for optical and SAR images based on fast bilateral filters to preserve structural features of the images while smoothing. The consistent gradients of the images are calculated in the process using the Sobel operator and the ratio of exponentially weighted average operator, respectively. Then, a novel framework is proposed for rotation-invariant optical and SAR image registration. In the framework, Harris detectors are utilized to detect sufficient and reliable feature points in the scale spaces, and downsampled Fourier-based descriptors, achieving a high discriminability by avoiding the process of orientation assignment, are designed for rotation-invariant and fast feature description. Experiments on three pairs of optical and SAR images demonstrate the effectiveness of the proposed algorithm compared to existing state-of-the-art algorithms.

An improved synthetic aperture radar‐scale invariant feature transform algorithm for interferometric imaging radar altimeter image registration

Interferometric imaging radar altimeter (InIRA) integrating synthetic aperture radar (SAR) and synthetic aperture radar interferometry (InSAR) is a new type of radar altimeter. InIRA will be widely adopted in marine scientific research and surface water topography measurements. Owing to the geometric distortion of InIRA images, there are difficulties in feature point extraction and matching, making InIRA registration challenging. Here, an improved SAR-scale invariant feature transform (SIFT, SAR-SIFT) algorithm is proposed to address these difficulties. First, a modified Harris non-linear scale space is constructed using the Harris function and non-linear scale space, and extracted the feature points according to the SAR-SIFT algorithm. Second, the orbital parameters of InIRA are used to calculate the geographic coordinates of each feature point by Hermit interpolation. Subsequently, a geographic-coordinate-based method is proposed to match the feature points. The results show that the proposed algorithm gained more correct matches and improved the registration accuracy by 13% and 16.7% compared with the SIFT and SAR-SIFT algorithms, respectively, and was 18% more time efficient than the SAR-SIFT algorithm. The experiments demonstrate that the proposed algorithm outperforms the SAR-SIFT algorithm for InIRA image registration in terms of accuracy and efficiency.

Feature Matching of Multimodal Images Based on Nonlinear Diffusion and Progressive Filtering

Traditional image feature matching methods cannot obtain satisfactory results for multi-modal images in most cases because different imaging mechanisms bring significant nonlinear radiation distortion differences and geometric distortion. The key to multi-modal image matching is trying to eliminate the nonlinear radiation distortion and extract more robust features. This paper proposes a new robust feature matching method for multi-modal images. Our method starts by detecting feature points on phase congruency maps in nonlinear scale space and then removing mismatches by progressive filtering. Specifically, the phase congruency maps are generated by the Log-Gabor filter. Then, the feature points on phase congruency maps are detected in nonlinear scale space constructed by the nonlinear diffusion filter. Subsequently, the structure descriptor is established by the Log-Gabor filter, and the initial correspondences are constructed by bilateral matching. Finally, an iterative strategy is used to remove mismatches by progressive filtering. We perform comparison experiments on our proposed method with the SIFT, RIFT, VFC, LLT, LPM, and mTopKPR methods using multi-modal images. The algorithms of each method are comprehensively evaluated both qualitatively and quantitatively. Our experimental results indicate the superiority of our method over the other six matching methods.

A computationally efficient affine-invariant feature for image matching with wide viewing angles

This paper proposes an efficient affine-invariant feature image matching method, especially when there are wide viewing angles. In previous studies, all the AKAZE-based algorithms are built on nonlinear scale-space by using Fast Explicit Diffusion (FED) scheme, which has the advantage of high computational efficiency. But all of them do not have affine invariance and cannot favor the image matching when the image views are too different. Unlike AKAZE features, AFREAK features are extracted from the simulated images obtained by sampling the latitude and longitude angles of an original image, which have good robustness for viewpoint changes. Therefore, we combine the idea of simulating an affine transformation with the construction of a nonlinear scale-space to build an affine-invariant nonlinear scale-space. Then, in order to reduce the redundancy information, we optimize the sampling of longitude angles and simplify the FREAK descriptor. Moreover, a coarse-to-fine matching strategy based on Vector Field Consistency (VFC) and Progressive Sampling Consensus (PROSAC) is developed to further improve the rapidity and correct matching rate. Experimental results on various benchmark image datasets demonstrate that the proposed method is computationally efficient for image matching while has the fully affine invariance.

A comparison of an RGB-D cameras performance and a stereo camera in relation to object recognition and spatial position determination

Results of using an RGB-D camera (Kinect sensor) and a stereo camera, separately, in order to determine the 3D real position of characteristic points of a predetermined object in a scene are presented. KAZE algorithm was used to make the recognition, that algorithm exploits the nonlinear scale space through nonlinear diffusion filtering; 3D coordinates of the centroid of a predetermined object were calculated employing the camera calibration information and the depth parameter provided by a Kinect sensor and a stereo camera. Experimental results show it is possible to get the required coordinates with both cameras in order to locate a robot, although a balance in the distance where the sensor is placed must be guaranteed: no fewer than 0.8 m from the object to guarantee the real depth information, it is due to Kinect operating range; 0.5 m to stereo camera, but it must not be 1 m away to have a suitable rate of object recognition, besides, Kinect sensor has more precision with distance measures regarding a stereo camera.

Real-time FPGA-based implementation of the AKAZE algorithm with nonlinear scale space generation using image partitioning

The first step in a scale invariant image matching system is scale space generation. Nonlinear scale space generation algorithms such as AKAZE, reduce noise and distortion in different scales while retaining the borders and key-points of the image. An FPGA-based hardware architecture for AKAZE nonlinear scale space generation is proposed to speed up this algorithm for real-time applications. The three contributions of this work are (1) mapping the two passes of the AKAZE algorithm onto a hardware architecture that realizes parallel processing of multiple sections, (2) multi-scale line buffers which can be used for different scales, and (3) a time-sharing mechanism in the memory management unit to process multiple sections of the image in parallel. We propose a time-sharing mechanism for memory management to prevent artifacts as a result of separating the process of image partitioning. We also use approximations in the algorithm to make hardware implementation more efficient while maintaining the repeatability of the detection. A frame rate of 304 frames per second for a 1280 times 768 image resolution is achieved which is favorably faster in comparison with other work.

Modality-Free Feature Detector and Descriptor for Multimodal Remote Sensing Image Registration

The nonlinear radiation distortions (NRD) among multimodal remote sensing images bring enormous challenges to image registration. The traditional feature-based registration methods commonly use the image intensity or gradient information to detect and describe the features that are sensitive to NRD. However, the nonlinear mapping of the corresponding features of the multimodal images often results in failure of the feature matching, as well as the image registration. In this paper, a modality-free multimodal remote sensing image registration method (SRIFT) is proposed for the registration of multimodal remote sensing images, which is invariant to scale, radiation, and rotation. In SRIFT, the nonlinear diffusion scale (NDS) space is first established to construct a multi-scale space. A local orientation and scale phase congruency (LOSPC) algorithm are then used so that the features of the images with NRD are mapped to establish a one-to-one correspondence, to obtain sufficiently stable key points. In the feature description stage, a rotation-invariant coordinate (RIC) system is adopted to build a descriptor, without requiring estimation of the main direction. The experiments undertaken in this study included one set of simulated data experiments and nine groups of experiments with different types of real multimodal remote sensing images with rotation and scale differences (including synthetic aperture radar (SAR)/optical, digital surface model (DSM)/optical, light detection and ranging (LiDAR) intensity/optical, near-infrared (NIR)/optical, short-wave infrared (SWIR)/optical, classification/optical, and map/optical image pairs), to test the proposed algorithm from both quantitative and qualitative aspects. The experimental results showed that the proposed method has strong robustness to NRD, being invariant to scale, radiation, and rotation, and the achieved registration precision was better than that of the state-of-the-art methods.

Image Stitching using AKAZE Features

Accelerated KAZE (AKAZE) is a multi-scale 2D feature detection and description algorithm in nonlinear scale spaces proposed recently. This paper presents an image stitching algorithm which uses a feature detection and description algorithm; AKAZE and an image blending algorithm; weighted average blending. The whole process is divided into the following steps: First of all, detect feature points in the image and then get feature descriptors of detected points using AKAZE. Next, obtain corresponding matching pairs by using K-NN (K nearest neighbors) algorithm and remove the false matched points by MSAC (M-estimator SAmple Consensus) algorithm. MSAC is a variant of the RANSAC (Random Sample Consensus) algorithm and more accurate than RANSAC. Thereafter, calculate the homography matrix from correct matches. At last, blend the images by using weighted average blending algorithm. Comparison of proposed AKAZE-based algorithm with SIFT-, SURF- and ORB-based algorithms is also presented. According to the experiments, the proposed AKAZE-based image stitching algorithm minimizes stitching seam and generates a perfect stitched image, and also this algorithm is faster than previous algorithms.

A Novel Harris Feature Detection-Based Registration for Remote Sensing Image

In view of the significant intensity difference between remote sensing image pairs, weak robustness, and insufficient key point correspondence, the novel remote sensing image registration method is proposed. Firstly, a nonlinear scale space is established by means of the anisotropic diffusion equation and fast explicit diffusion. Then, an improved gradient calculation method is used to calculate the gradient amplitude of the nonlinear scale-space image to establish the gradient amplitude space of the nonlinear scale space, and the multiscale Harris method is used to detect the feature points in the gradient amplitude space. The experimental results show that this feature extraction method can consider the boundaries and smoothness of objects and reduce the problem of gray-level difference to increase the number of feature points with potential of being correctly matched, and the distribution of feature points is relatively uniform. In addition, the improved gradient calculation method can effectively reduce the impact of nonlinear intensity differences on image registration. Overall, the algorithm can effectively solve the problem of registration difficulties caused by the significant grayscale difference between multisource remote sensing images and enhance the robustness. Compared with other advanced algorithms, this one has higher accuracy and more correct correspondence relations, and the registration performance has been significantly improved.

Temporally Forward Nonlinear Scale Space for High Frame Rate and Ultra-Low Delay A-KAZE Matching System

Temporally Forward Nonlinear Scale Space for High Frame Rate and Ultra-Low Delay A-KAZE Matching System

Improved image registration scheme based on multiscale nonlinear Harris and Gaussian–Krawtchouk moment

Image registration is an indispensable operation in image processing. To solve the problem of nonlinear distortion which easily causes low registration property, a registration scheme including a nonlinear scale space, a new discrete orthogonal Gaussian–Krawtchouk (GK) descriptor vector, and a dual similarity measurement of distance-cosine is proposed. Nonlinear scale space is constructed to preserve a lot of important structures for increasing the quantity of the salient keypoints. Meanwhile, considering both blur stability and simple computation, multiscale Harris algorithm on the basis of nonlinear scale space is applied to extract the keypoints. Moreover, according to different scales, a descriptor vector based on multiscale GK invariant moments is presented and applied to the following image matching. Finally, a dual similarity measurement of the initial Euclidean distance and the fine space cosine is proposed for the purpose of enhancing the matching accuracy. The effectiveness of the proposed algorithm has been tested on the standard images with different types of variations. The experimental results show that the performance of feature extraction and the correct matching rate of the proposed scheme are superior to those of other classical methods. The proposed scheme has a better robustness and a higher registration accuracy.

Feature Extraction from Non-Linear Scale Spaces Using the Guided Filter for Fast Robust Organ Tracking in Water-Filled Laparoendoscopic Surgery

Feature Extraction from Non-Linear Scale Spaces Using the Guided Filter for Fast Robust Organ Tracking in Water-Filled Laparoendoscopic Surgery

Image Forgery Detection using AKAZE Keypoint Feature Extraction and Trie Matching

Image Forgery is an illegal activity in the society as per cyber laws. There are various types of forgeries in which forgery on images is considered as an illegal activity. Image forgery may take place in different ways. One way for doing forgery on images is copy and move forgery which may result in loss of image integrity or authenticity. There are number of popular detection techniques exist such as SIFT, SURF etc., but have high complexity in detection of forgery. Here we have proposed a method to detect the forgery on images which results in loss of integrity or authenticity. In our proposed method we have used descriptor matching using Trie Data Structure The descriptor matching method of implementation using Trie data structure made the complexity of the problem to reduce to O (n log n). Using Key points approach we can verify the integrity of the image. Extracting the features with key points approach is computational expensive task. But there is KAZE method which overcomes this situation. KAZE’s method of using non-linear diffusion filtering requires it to solve a series of PDEs. This cannot be done analytically forcing KAZE to use a numerical method called an AOS scheme to solve the PDEs. However, this process is computationally costly and therefore an accelerated version of KAZE was created. The Accelerated KAZE or AKAZE which creates non-linear scale space through Fast Explicit Diffusion for reduce the complexity in extracting the features.

Calibrating Thresholds Based on Trade-Offs Between Detection Accuracy and FPR for Copy Move Forgery Detection

In this paper, prominent keypoint based features are compared in order to analyze their reliability and efficiency against forgery detection. Four features specifically SURF, KAZE, Harris corner points and BRISK features are used individually on a set of images. The method includes four phases: Image pre-processing, keypoint detection, feature vector description and feature vector matching. In feature matching, MaxRatio has been chosen as a varying parameter for calculating values of false positives and false negatives for each feature. MaxRatio defines the ratio for rejecting ambiguous matches of feature descriptors in the images. The optimal threshold value for MaxRatio is calibrated with the help of trade-off between detection accuracy and false positive ratio. The changes in false negative ratio and false positive ratio are picturized in order to find out optimal threshold for detection accuracy. ROC curves are also plotted for each feature at different values of MaxRatio and area under the ROC curves are calculated. The experiments are performed on two benchmark datasets, namely CASIA version 2.0 and MICC-F600. It has been perceived from experimental outcomes that KAZE features gave best values for all the performance metrics namely accuracy, precision, area under the ROC curve and F1-score with little compromise in time complexity, whereas Harris corner points gave the worst results as compared to rest of the features. Further, in order to improve the execution time, the computation of non-linear scale space process in KAZE can be simplified and GPU programming for real-time performance may also be used.

Design of pedestrian detectors using combinations of scale spaces and classifiers

With the increasing demand for surveillance applications, pedestrian detection has been a topic of interest for many researchers in recent time. The quality of a pedestrian detector is decided in terms of detection accuracy and rate of detection. This paper presents new pedestrian detectors based on two types of classifiers, linear support vector machine and cascade of boosted classifier. These classifiers are trained by using a feature set comprising of the histogram of oriented gradients and dense local difference binary features. Both the image pyramid and non-linear scale space are used to detect pedestrians of various sizes. In order to combine the benefits of the two classifiers, a new two-stage detection scheme is also presented. The detection accuracies of the proposed detectors are studied in terms of miss-rate versus false positive per image and miss-rate versus false positive per window. The performances of the detectors are also compared with the performances of existing detectors of similar type.

A Hybrid 3D Registration Method of Augmented Reality for Intelligent Manufacturing

Presently, core 3D registration technologies for augmented reality have problems like low accuracy and poor tracking stability in natural environments pertaining to mass customization and intelligent manufacturing, resulting in error display or poor visual performance. (1) A non-linear scale space was used to alleviate the problem associated with scale invariance, the relevant calculations and construction methods were studied. The adaptive non-maximal suppression method was examined, which reduced redundancy of ORB (Oriented FAST and Rotated BRIEF) feature points. (2) The method of combining nonlinear local descriptors with LK method is studied to improve the low stability problem against Light change by LK method only, and a forward-backward error detection method was studied to evaluate feature point tracking results. (3) The improved ORB and LK methods are used to track the target and achieve data fusion. Then, the fused data is voted and clustered by means of consistent voting, only the maximum number of clusters within the threshold is left as the final tracking result to realize 3D registration. Finally, the paper validates the proposed method through the natural environment dataset of open source. The dimensions of verification include challenging scales, rotation changes, perspective changes, motion blur, occlusion, and out-of-view object.