Keypoint Matching Algorithm Research Articles

EyeLiner: A Deep Learning Pipeline for Longitudinal Image Registration using Fundus Landmarks

ObjectiveDetecting and measuring changes in longitudinal fundus imaging is key to monitoring disease progression in chronic ophthalmic diseases, such as glaucoma and macular degeneration. Clinicians assess changes in disease status by either independently reviewing or manually juxtaposing longitudinally acquired color fundus photos (CFPs). Distinguishing variations in image acquisition due to camera orientation, zoom, and exposure from true disease-related changes can be challenging. This makes manual image evaluation variable and subjective, potentially impacting clinical decision making. We introduce our deep learning pipeline, “EyeLiner,” for registering, or aligning, two-dimensional CFPs. Improved alignment of longitudinal image pairs may compensate for differences that are due to camera orientation, while preserving pathological changes. DesignEyeLiner registers a “moving” image to a “fixed” image using a deep learning-based keypoint matching algorithm. ParticipantsWe evaluate EyeLiner on three longitudinal datasets: Fundus Image REgistration (FIRE), Sequential Fundus for Glaucoma Forecast (SIGF), and our internal glaucoma dataset from the Colorado Ophthalmology Research Information System (CORIS). MethodsAnatomical keypoints along the retinal blood vessels are detected from the moving and fixed images using a convolutional neural network and subsequently matched using a transformer-based algorithm. Finally, transformation parameters are learned using the corresponding keypoints. Main Outcome MeasuresWe computed the mean distance (MD) between manually annotated keypoints from the fixed and the registered moving image. For comparison to existing state-of-the-art retinal registration approaches, we used the mean area under the curve (mAUC) metric introduced in the FIRE dataset study. ResultsEyeLiner effectively aligns longitudinal image pairs from FIRE, SIGF and CORIS, as qualitatively evaluated through registration checkerboards and flicker animations. Quantitative results show that the MD decreased for this model after alignment from 321.32 to 3.74 pixels for FIRE, 9.86 to 2.03 pixels for CORIS, and 25.23 to 5.94 pixels for SIGF. We also obtain an AUC of 0.85, 0.94 and 0.84 on FIRE, CORIS and SIGF respectively, beating the current state-of-the-art SuperRetina (AUCFIRE=0.76, AUCCORIS=0.83, AUCSIGF=0.74). ConclusionsOur pipeline demonstrates improved alignment of image pairs in comparison to the current state-of-the-art methods on three separate datasets. We envision that this method will enable clinicians to align image pairs, and better visualize changes in disease over time.

Deep Learning Feature Extraction Using Attention-Based DenseNet 121 for Copy Move Forgery Detection

In modern society, digital images can be far-reaching, and the images are manipulated by various software and hardware technologies. The image forgery activities are undertaken by the attackers mainly for damaging the reputation of people or receiving fiscal gain, etc. Taking this into consideration, many techniques are developed to detect the forged images. In this paper, a new deep learning-based approach is introduced for copy-move forgery detection. The input images are segmented into non-overlapping patches using superpixel-based modified dense peak clustering and the features are extracted from the segmented patches by applying deep learning structure of attention-based DenseNet 121 model. Besides, to compare every block, the depth of each pixel is reconstructed, and eventually matching process is carried out using the adaptive chimp patch matching approach, which detects the suspicious forged regions in an image. Finally, the matched keypoints are merged with the segmented patches using the merged keypoint matching algorithm. As a result, the new deep learning approach has detected the forged regions efficiently from the tampered image with less time even the image is compressed, rotated, or scaled. The performance is evaluated in terms of recall, precision, accuracy, F1-score, computational time, and False Positive Rate (FPR). Moreover, the performance is compared with the other existing approaches, and the outcomes showed that the proposed method has achieved higher accuracy of 97%, recall of 99%, precision of 97.84%, F1-score of 98.81%, FPR of 0.022 and less computational time of 2.5 s.

Tensor-based keypoint detection and switching regression model for relative radiometric normalization of bitemporal multispectral images

ABSTRACT In some remote sensing applications, such as unsupervised change detection, bitemporal multispectral images must be first aligned/harmonized radiometrically. For doing so, Many Relative Radiometric Normalization (RRN) algorithms exist; however, most suffer from misregistration problems and can only operate on geo/co-registered image pairs, while unregistered multispectral pairs are required. To tackle this situation, keypoint-based RRN methods were introduced, which can radiometrically calibrate unregistered/registered image pairs using keypoint matching algorithms. However, they ignore the spatial and spectral characteristics of spectral bands of input images, resulting in potential RRN errors. They also employ a linear mapping function for RRN modelling, which can not handle non-linear radiometric distortions. To address these limitations, this paper proposes a robust algorithm for RRN of bitemporal multispectral images, using a new extension of SURF detector for multispectral images, namely the Weighted Spectral Structure Tensor SURF (WSST-SURF), and a flexible Switching Regression (SR) model. Taking advantage of the tensor theory, WSST-SURF efficiently preserves both spatial and spectral information distributed over all bands of multispectral images for keypoint detection, resulting in extracting reliable inliers (or keypoints) for RRN. An adaptive SR model is introduced based on the normalized mutual information, accurately approximating a linear/non-linear relationship between inliers in multispectral images. Six unregistered multispectral image pairs captured by inter/intra remote sensing sensors were employed to validate the efficacy of the proposed method. The results indicate that adopting spectral tensor-based SURF methods in the RRN process exhibits better local and global performance than using the original SURF. Furthermore, the proposed method outperforms the existing conventional RRN methods in terms of accuracy and visual quality, indicating its competence for RRN of bitemporal multispectral images with high illumination, viewpoint, and scale differences.

Use of a DNN-Based Image Translator with Edge Enhancement Technique to Estimate Correspondence between SAR and Optical Images

In this paper, the local correspondence between synthetic aperture radar (SAR) images and optical images is proposed using an image feature-based keypoint-matching algorithm. To achieve accurate matching, common image features were obtained at the corresponding locations. Since the appearance of SAR and optical images is different, it was difficult to find similar features to account for geometric corrections. In this work, an image translator, which was built with a DNN (deep neural network) and trained by conditional generative adversarial networks (cGANs) with edge enhancement, was employed to find the corresponding locations between SAR and optical images. When using conventional cGANs, many blurs appear in the translated images and they degrade keypoint-matching accuracy. Therefore, a novel method applying an edge enhancement filter in the cGANs structure was proposed to find the corresponding points between SAR and optical images to accurately register images from different sensors. The results suggested that the proposed method could accurately estimate the corresponding points between SAR and optical images.

Matching Algorithm for 3D Point Cloud Recognition and Registration Based on Multi-Statistics Histogram Descriptors.

Establishing an effective local feature descriptor and using an accurate key point matching algorithm are two crucial tasks in recognizing and registering on the 3D point cloud. Because the descriptors need to keep enough descriptive ability against the effect of noise, occlusion, and incomplete regions in the point cloud, a suitable key point matching algorithm can get more precise matched pairs. To obtain an effective descriptor, this paper proposes a Multi-Statistics Histogram Descriptor (MSHD) that combines spatial distribution and geometric attributes features. Furthermore, based on deep learning, we developed a new key point matching algorithm that could identify more corresponding point pairs than the existing methods. Our method is evaluated based on Stanford 3D dataset and four real component point cloud dataset from the train bottom. The experimental results demonstrate the superiority of MSHD because its descriptive ability and robustness to noise and mesh resolution are greater than those of carefully selected baselines (e.g., FPFH, SHOT, RoPS, and SpinImage descriptors). Importantly, it has been confirmed that the error of rotation and translation matrix is much smaller based on our key point matching algorithm, and the precise corresponding point pairs can be captured, resulting in enhanced recognition and registration for three-dimensional surface matching.

A novel copy-move forgery detection algorithm via two-stage filtering

With the wide application of simple image editing software, forgery images have become severe social problems with extremely damaging effects. The copy-move forgery technique is commonly used to temper an image by copying some regions of an image and pasting them somewhere in the same image. In this paper, a novel algorithm is proposed for the detection of copy-move forgery. First, the keypoints of the input image are computed using a keypoint extraction algorithm. Second, a keypoint-matching algorithm is used to match similar keypoints as the candidate keypoint pairs. Third, a proposed novel Two-Stage Filtering algorithm, including the Grid-Based Filter and the Clustering-Based Filter, is applied to filter out most of the false matching keypoint pairs. Subsequently, image matting is achieved by the Delaunay triangulation algorithm so that the marked areas indicate the forgery regions. Three copy-move forgery datasets are used to compare the performances of the proposed algorithm with some state-of-the-art algorithms. The experimental results demonstrate that the proposed algorithm's overall performance is superior to other solutions for detecting copy-move forgery images.

A Method to Detect and Track Moving Airplanes from a Satellite Video

In recent years, satellites capable of capturing videos have been developed and launched to provide high definition satellite videos that enable applications far beyond the capabilities of remotely sensed imagery. Moving object detection and moving object tracking are among the most essential and challenging tasks, but existing studies have mainly focused on vehicles. To accurately detect and then track more complex moving objects, specifically airplanes, we need to address the challenges posed by the new data. First, slow-moving airplanes may cause foreground aperture problem during detection. Second, various disturbances, especially parallax motion, may cause false detection. Third, airplanes may perform complex motions, which requires a rotation-invariant and scale-invariant tracking algorithm. To tackle these difficulties, we first develop an Improved Gaussian-based Background Subtractor (IPGBBS) algorithm for moving airplane detection. This algorithm adopts a novel strategy for background and foreground adaptation, which can effectively deal with the foreground aperture problem. Then, the detected moving airplanes are tracked by a Primary Scale Invariant Feature Transform (P-SIFT) keypoint matching algorithm. The P-SIFT keypoint of an airplane exhibits high distinctiveness and repeatability. More importantly, it provides a highly rotation-invariant and scale-invariant feature vector that can be used in the matching process to determine the new locations of the airplane in the frame sequence. The method was tested on a satellite video with eight moving airplanes. Compared with state-of-the-art algorithms, our IPGBBS algorithm achieved the best detection accuracy with the highest F1 score of 0.94 and also demonstrated its superiority on parallax motion suppression. The P-SIFT keypoint matching algorithm could successfully track seven out of the eight airplanes. Based on the tracking results, movement trajectories of the airplanes and their dynamic properties were also estimated.

An Effective Scheme for Shot Boundary Detection and Key Frame Extraction for Video Retrieval

The rapid development of devices for image capture and information sharing has resulted in the availability of huge amounts of online video for various applications such as education, news, entertainment, etc. This leads to problems and difficulties when users query any content-related video. The reason for this scenario is that the presently available techniques of content representation and retrieval are based primarily on annotation. It therefore provides insufficient information for understanding and retrieving the content to match the query of the user. Content Based Video Retrieval (CBVR) is one of the promising new ways for finding content in a large video archive, rather than simply searching terms. The primary steps for indexing, summarizing and retrieving video are shot transition recognition and representative frame extraction. We have proposed a key point matching algorithm for a superior and robust Scale Invariant Feature Transform (SIFT) followed by the collection of representative frames from each segmented shot using the Image Information Entropy method. By using the Rough Set Theory, we can get better the concert of this scheme through removing unnecessary representative frames. All the methods suggested to prove the efficacy were tested on TRECVID datasets and contrasted with state-of - the-art approaches.

A Vocabulary Forest Object Matching Processor With 2.07 M-Vector/s Throughput and 13.3 nJ/Vector Per-Vector Energy for Full-HD 60 fps Video Object Recognition

Approximate nearest neighbor searching has been studied as the keypoint matching algorithm for object recognition systems, and its hardware realization has reduced the external memory access which is the main bottleneck in object recognition process. However, external memory access reduction alone cannot satisfy the ever-increasing memory bandwidth requirement due to the rapid increase of the image resolution and frame rate of many recent applications such as advanced driver assistance system. In this paper, vocabulary forest (VF) processor is proposed that achieves both high accuracy and high speed by integrating on-chip database (DB) to remove external memory access. The area-efficient reusable-vocabulary tree architecture is proposed to reduce area, and the propagate-and-compute-array architecture is proposed to enhance the processing speed of the VF. The proposed VF processor can speed up the object matching stage by 16.4x compared with the state-of-the-art matching processor [Hong et al., Symp. VLSIC, 2013] for high resolution (Full-HD) and real-time (60 fps) video object recognition. It is fabricated using 65 nm CMOS technology and integrated into an object recognition SoC. The proposed VF chip achieves 2.07 M-vector/s throughput and 13.3 nJ/vector per-vector energy with 95.7% matching accuracy for 100 objects.

Fast Feature-Based Template Matching, Based on Efficient Keypoint Extraction

In order to improve the performance of feature-based template matching techniques, several research papers have been published. Real-time applications require the computational complexity of keypoint matching algorithms to be as low as possible. In this paper, we propose a method to improve the keypoint detection stage of feature-based template matching algorithms. Our experiment results show that the proposed method outperforms keypoint matching techniques in terms of speed, keypoint stability and repeatability.

A Divide-And-Rule Scheme For Shot Boundary Detection Based on SIFT

Shot boundary detection is an important fundamental process toward automatic video indexing, retrieval, editing, etc. After a critical review of most approaches seeking to solve this problem, we propose a novel shot boundary detection. To improve the performance of the algorithm and reduce the computational cost, frames that are clearly not shot boundaries are first removed from the original video. After that, a novel SIFT keypoint matching algorithm based on SVM is proposed, which is used to capture the changing statistics of different kinds of shot transitions so as to identify, not only abrupt transitions, but also gradual transitions(fade, dissolve, wipe) accordingly. At last, our system use different algorithms for different kinds of shot transitions to help us to get a better solution for shot boundary detection problem. Numerical experiments in the evaluation of TRECVID and a variety of film videos demonstrate that our method is capable of accurately detecting shot transitions, and could greatly reduce the computational cost.