Keypoint Matching Research Articles

A Novel Copy–Move Forgery Detection Algorithm via Gradient-Hash Matching and Simplified Cluster-Based Filtering

Copy–move forgery is one of the most frequently used methods for producing fake digital images. Current algorithms for copy–move forgery detection (CMFD) cannot combine high accuracy and fast speed. Motivated by the observation, we propose a novel CMFD algorithm whose workflow is as follows. First, we use a keypoint-extraction method with the lowest contrast threshold to extract more keypoints from the input image. Second, a new technique, gradient-hash matching, finds pairs of similar keypoints quickly and effectively using a hash table, where the hash value is computed using gradients of keypoints. Subsequently, a new method called simplified cluster-based filtering exploits the density pattern of keypoints in the copy–move regions to remove false matching keypoint pairs. Finally, image matting is applied to indicate the forgery regions vividly. Extensive experiments show that not only the new algorithm is better than the state-of-the-art algorithms in terms of computation correctness, but also its computation time is drastically less. Commonly only about half time is needed. The relative time saving is even higher when images are larger. Different algorithms modules are compared through experiments to choose the best combination.

Finding Nemo’s Giant Cousin: Keypoint Matching for Robust Re-Identification of Giant Sunfish

The Giant Sunfish (Mola alexandrini) has unique patterns on its body, which allow for individual identification. By continuously gathering and matching images, it is possible to monitor and track individuals across location and time. However, matching images manually is a tedious and time-consuming task. To automate the process, we propose a pipeline based on finding and matching keypoints between image pairs. We evaluate our pipeline with four different keypoint descriptors, namely ORB, SIFT, RootSIFT, and SuperPoint, and demonstrate that the number of matching keypoints between a pair of images is a strong indicator for the likelihood that they contain the same individual. The best results are obtained with RootSIFT, which achieves an mAP of 75.91% on our test dataset (TinyMola+) without training or fine-tuning any parts of the pipeline. Furthermore, we show that the pipeline generalizes to other domains, such as re-identification of seals and cows. Lastly, we discuss the impracticality of a ranking-based output for real-life tasks and propose an alternative approach by viewing re-identification as a binary classification. We show that the pipeline can be easily modified with minimal fine-tuning to provide a binary output with a precision of 98% and recall of 44% on the TinyMola+ dataset, which basically eliminates the need for time-consuming manual verification on nearly half the dataset.

Binary Neighborhood Coordinate Descriptor for Circuit Board Defect Detection

Due to the periodicity of circuit boards, the registration algorithm based on keypoints is less robust in circuit board detection and is prone to misregistration problems. In this paper, the binary neighborhood coordinate descriptor (BNCD) is proposed and applied to circuit board image registration. The BNCD consists of three parts: neighborhood description, coordinate description, and brightness description. The neighborhood description contains the grayscale information of the neighborhood, which is the main part of BNCD. The coordinate description introduces the actual position of the keypoints in the image, which solves the problem of inter-period matching of keypoints. The brightness description introduces the concept of bright and dark points, which improves the distinguishability of BNCD and reduces the calculation amount of matching. Experimental results show that in circuit board image registration, the matching precision rate and recall rate of BNCD is better than that of classic algorithms such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF), and the calculation of descriptors takes less time.

HFNet-SLAM: An Accurate and Real-Time Monocular SLAM System with Deep Features.

Image tracking and retrieval strategies are of vital importance in visual Simultaneous Localization and Mapping (SLAM) systems. For most state-of-the-art systems, hand-crafted features and bag-of-words (BoW) algorithms are the common solutions. Recent research reports the vulnerability of these traditional algorithms in complex environments. To replace these methods, this work proposes HFNet-SLAM, an accurate and real-time monocular SLAM system built on the ORB-SLAM3 framework incorporated with deep convolutional neural networks (CNNs). This work provides a pipeline of feature extraction, keypoint matching, and loop detection fully based on features from CNNs. The performance of this system has been validated on public datasets against other state-of-the-art algorithms. The results reveal that the HFNet-SLAM achieves the lowest errors among systems available in the literature. Notably, the HFNet-SLAM obtains an average accuracy of 2.8 cm in EuRoC dataset in pure visual configuration. Besides, it doubles the accuracy in medium and large environments in TUM-VI dataset compared with ORB-SLAM3. Furthermore, with the optimisation of TensorRT technology, the entire system can run in real-time at 50 FPS.

Research on robotic assembly of gear motors (Stator recognition using keypoint matching and stator insertion using contact position estimation)

Research on robotic assembly of gear motors (Stator recognition using keypoint matching and stator insertion using contact position estimation)

Three-dimensional structural displacement measurement using monocular vision and deep learning based pose estimation

Computer vision-based displacement measurement methods have received increasing attention for the structural health monitoring of buildings and infrastructures owing to their advantages over traditional contact sensors. Meanwhile, surveillance cameras widely equipped in urban areas can record a large number of images and videos of buildings and infrastructure, which have the potential to support structural analysis in structural health monitoring or engineering investigations. The three-dimensional (3D) displacement of structures is important for structural analysis. It is challenging for the existing vision-based measurement methods to obtain all the 3D displacement components because they require either multi-view camera systems or additional specially designed targets, which makes it difficult to meet the requirements of measurement applications based on urban surveillance cameras. Therefore, this study proposes a 3D structural displacement measurement method using monocular vision and deep learning based pose estimation. The method uses virtual rendering to synthesize the training set based on the 3D models of the target objects, then trains the deep learning model DPOD (Dense Pose Object Detector) to estimate the poses of the target object, and finally measures the 3D translation of the structures based on the original and destination poses or the original pose and keypoint matching. The effectiveness of the proposed method was validated through static and dynamic experiments. The results showed that the method can meet the needs of obtaining 3D structural displacement and has good accuracy in identifying the principal frequencies of the dynamic responses. The proposed method can support the 3D displacement measurements of buildings and infrastructure based on urban surveillance cameras.

Reliable and stable fundus image registration based on brain-inspired spatially-varying adaptive pyramid context aggregation network.

The task of fundus image registration aims to find matching keypoints between an image pair. Traditional methods detect the keypoint by hand-designed features, which fail to cope with complex application scenarios. Due to the strong feature learning ability of deep neural network, current image registration methods based on deep learning directly learn to align the geometric transformation between the reference image and test image in an end-to-end manner. Another mainstream of this task aims to learn the displacement vector field between the image pair. In this way, the image registration has achieved significant advances. However, due to the complicated vascular morphology of retinal image, such as texture and shape, current widely used image registration methods based on deep learning fail to achieve reliable and stable keypoint detection and registration results. To this end, in this paper, we aim to bridge this gap. Concretely, since the vessel crossing and branching points can reliably and stably characterize the key components of fundus image, we propose to learn to detect and match all the crossing and branching points of the input images based on a single deep neural network. Moreover, in order to accurately locate the keypoints and learn discriminative feature embedding, a brain-inspired spatially-varying adaptive pyramid context aggregation network is proposed to incorporate the contextual cues under the supervision of structured triplet ranking loss. Experimental results show that the proposed method achieves more accurate registration results with significant speed advantage.

CDFSL: Image Registration for Spaceborne Hyperspectral and Multispectral Data Having Large Spatial-Resolution Difference

Image registration aims to eliminate the geometric deviation between multi-source data with the same range, and to promote the collaborative application of data. In recent years, spaceborne hyperspectral (HS) and multispectral (MS) data have been widely used in Earth observation. However, the difference in the number of bands, spatial resolution, and spectral resolution puts forward higher requirements on the registration algorithm. The key to HS and MS image registration is to extract more common key points, weaken and eliminate the difference of radiation and spatial texture information to build superior descriptors, and achieve high-precision matching of key points. This paper introduces a new robust HS and MS registration method based on common deep feature subspaces. We first construct the common deep feature subspaces extraction network to extract consistent edge features and common subspace images of the image pair. Then, Harris algorithm is used to extract key points from consistent edge features between images, which reduces the impact of spatial resolution differences between images. Besides, the SIFT descriptor and subspace images are used to describe key points, which reduces the impact of radiation differences between images. Finally, Euclidean distance is used for the initial matching of key points, and the affine matrix is calculated after the outliers are eliminated, and image registration is performed. We perform experiments on spaceborne HS and MS datasets of different spatial resolutions and comparisons with state-of-the-art methods. Experimental results show that our method can obtain satisfactory registration results and is robust.

A Detector-Oblivious Multi-Arm Network for Keypoint Matching.

This paper presents a matching network to establish point correspondence between images. We propose a Multi-Arm Network (MAN) capable of learning region overlap and depth, which can greatly improve keypoint matching robustness while bringing an extra 50% of computational time during the inference stage. By adopting a different design from the state-of-the-art learning based pipeline SuperGlue framework, which requires retraining when a different keypoint detector is adopted, our network can directly work with different keypoint detectors without time-consuming retraining processes. Comprehensive experiments conducted on four public benchmarks involving both outdoor and indoor scenarios demonstrate that our proposed MAN outperforms state-of-the-art methods.

Lightweight Semantic-Aided Localization With Spinning LiDAR Sensor

Autonomous driving demands robust and precise vehicle localization in complex environments with limited on-board computational resources. Incorporating reliable semantic information with localization algorithms can increase accuracy remarkably, however, the process of extracting semantic information from LiDAR point clouds and matching it to semantic maps is computationally intensive. Moreover, pure semantic localization cannot achieve the robustness requirements for safe self-driving as the necessary quantity of semantic landmarks cannot be guaranteed under extreme conditions. In this paper, we present a lightweight semantic-aided localization method that improves upon traditional techniques in two ways. First, we propose a highly efficient pipeline to extract three semantic classes from a LiDAR scan. Second, instead of semantic 3D point cloud registration, map matching is performed through 2D key point matching. We then integrate these two functions into a dynamic semantic aided localization framework. Our on-road experiments demonstrate that the proposed method achieves both the high accuracy of semantic localization and the robustness of non-semantic localization. With our algorithm consuming under 10% of CPU resources, we observe reduced positioning error, especially peak error, when comparing to non-semantic counterparts.

A Novel Image Copy–Move Forgery Detection Algorithm Using the Characteristics of Local Descriptors

The image copy–move forgery can achieve the realistic effects without leaving any obvious manipulation traces because the copied source and pasted target belong to the homogenous sources. Image copy–move forgery detection (CMFD) is still a serious challenge issue for researchers, e.g. computational efficiency, occlusive (smoothing) object detection, and partial object detection, under various post-processing and geometrical transformations. This paper proposes a novel CMFD scheme using the characteristics of local descriptors. First, the proposed method deeply analyzes the structure and mines the inherent characteristics of the local descriptor (SURF) for feature extraction in the coarse and smooth regions. Subsequently, the proposed method takes the kernel features as coarse feature matching to decrease matching costs. Then, the found candidate keypoints with a relatively small quantity are used with complete features to perform fine keypoint matching for getting suspicious candidate keypoint pairs. Furthermore, based on the deep mining of the inherent local descriptor characteristics, the proposed method can indicate the forgery region localizations through keypoint pairs, even the forgeries suffering from various geometrical transformations. Then, the proposed method proposes adaptive dual-filtering algorithms of [Formula: see text]-means relying on keypoint characteristics, and spatial distance for hierarchical progress of feature filtering. Finally, Delaunay Triangulation relies on the true-positive keypoint pairs to perform the forgery region matting effectively and efficiently. The experimental results demonstrate that the proposed method can achieve the best F1 scores compared to the state-of-the-art methods in most cases, especially in anti-scaling transformations.

Robust Estimation and Optimized Transmission of 3D Feature Points for Computer Vision on Mobile Communication Network.

Due to the amount of transmitted data and the security of personal or private information in wireless communication, there are cases where the information for a multimedia service should be directly transferred from the user's device to the cloud server without the captured original images. This paper proposes a new method to generate 3D (dimensional) keypoints based on a user's mobile device with a commercial RGB camera in a distributed computing environment such as a cloud server. The images are captured with a moving camera and 2D keypoints are extracted from them. After executing feature extraction between continuous frames, disparities are calculated between frames using the relationships between matched keypoints. The physical distance of the baseline is estimated by using the motion information of the camera, and the actual distance is calculated by using the calculated disparity and the estimated baseline. Finally, 3D keypoints are generated by adding the extracted 2D keypoints to the calculated distance. A keypoint-based scene change method is proposed as well. Due to the existing similarity between continuous frames captured from a camera, not all 3D keypoints are transferred and stored, only the new ones. Compared with the ground truth of the TUM dataset, the average error of the estimated 3D keypoints was measured as 5.98 mm, which shows that the proposed method has relatively good performance considering that it uses a commercial RGB camera on a mobile device. Furthermore, the transferred 3D keypoints were decreased to about 73.6%.

Target-free 3D tiny structural vibration measurement based on deep learning and motion magnification

• A novel target-free tiny vibration displacement measurement approach is proposed. • It is based on deep learning and motion magnification techniques. • Binocular vision system is used for measuring 3D vibration measurement. • Deep learning techniques are used for key-points detection, matching and tracking. • Experimental and in-field studies are conducted to validate the accuracy. In the field of structural health monitoring (SHM), computer vision based methods have been usually developed for 2-dimensional (2D) vibration displacement measurement and crack identification of civil engineering structures. However, the accurate measurement of tiny 3-dimensional (3D) vibrations for real civil engineering structures remains a very difficult task. To overcome this challenge, this paper proposes a target-free full-field 3D tiny vibration measurement approach for civil engineering structures by using a binocular vision system. The proposed approach is based on deep learning and motion magnification. A phase-based video motion magnification algorithm is employed to achieve a high measurement accuracy of tiny vibrations at the submillimeter level. The advanced key point detection, matching and tracking algorithms via deep learning techniques are employed to achieve target-free tiny vibration displacement measurement. The accuracy and performance of the proposed approach are evaluated through experimental tests on a steel cantilever beam in the laboratory. In-field experimental tests are conducted on a pedestrian bridge on a university campus to investigate the accuracy of the proposed approach in practical applications. The measured 3D tiny vibration displacement from the proposed approach is compared with those measured by laser displacement sensors, and the derived acceleration responses are compared with those measured from the installed accelerometers on the testing structures. The results demonstrate that the 3D tiny vibration measurements are obtained accurately by the proposed approach.

A Comparative Study on The Performance Analysis of Feature Extractors Used in Augmented Reality Applications Under Various Image Conditions

Until today, various approaches have been proposed in order to create Augmented Reality (AR) environment where virtual-real integration takes place. One of these approaches is vision-based model and it is divided into two branches as Marker-Based Augmented Reality (MBAR) and Markerless Augmented Reality (MAR). In the use of MBAR model, a reference image is introduced to the system before and when this image enters the camera view, an AR environment is created. However, in MAR model, no image is introduced to the system before. Instead, it uses natural characteristics present in the image such as edges, corners and geometrical shapes to create an AR environment. In order to use MAR model, it is necessary to use algorithms which require high processing power and memory capacity. Within the scope of this study, MAR model was chosen as reference and an evaluation on combinations of descriptive extractors (such as ORB, SIFT and SURF) and matchers (such as Bruteforce, Bruteforce-Hamming and Flannbase) was presented. In this context, it was aimed to obtain knowledge about i) the number of keypoints and detection time with use of different descriptor extractors, ii) the number of matching keypoints and the amount of positional deviation of a virtual object placed on a real world scene with use of different matchers. In line with this goal, analyzes were made using different image scales and brightness levels on both PC and mobile platforms. Results showed that, for both platforms, combinations using ORB method could work faster with less deviation than the combinations using other methods in all conditions. In addition, RANSAC algorithm was also used to reduce the total mean deviation ratio and it was seen that the rate could be reduced from 70% to 4.5%.

Deep images enhancement for turbid underwater images based on unsupervised learning

In agriculture, aquaculture technologies such as precise feeding, fish identification and fishing based on underwater machine vision all rely on the analysis of underwater images. However, due to the scatting and attenuation of the illumination in the real-world underwater environment, turbid underwater images are inevitably degraded, limiting their applicability in many vision tasks. In this paper, we present an unsupervised deep learning framework, called Underwater Loop Enhancement Network (ULENet), to improve the quality of turbid underwater images. We first propose an underwater dataset construction scheme and construct the dataset on which the network proposed above is trained. The underwater dataset contains images of three different scenes: lake and reservoir scene data (no label), pool scene data (weakly correlated label), and laboratory scene data (strongly correlated label). Then we propose a loop enhancement structure that uses the approximate candidates as labels and improves the visual quality of the image through the iterative training process. We formulate a new underwater visual perception loss function that evaluates the perceptual image quality based on its color, contrast, saturation and clarity. During the training process, a more realistic, higher-contrast, and clearer underwater image is gradually generated. Qualitative and quantitative evaluations show that the proposed method can effectively enhance image clarity. Moreover, the enhanced images are applied to several vision tasks to achieve better results, such as edge detection, key point matching, fish target detection and saliency prediction etc.

Unsupervised Histological Image Registration Using Structural Feature Guided Convolutional Neural Network.

Registration of multiple stained images is a fundamental task in histological image analysis. In supervised methods, obtaining ground-truth data with known correspondences is laborious and time-consuming. Thus, unsupervised methods are expected. Unsupervised methods ease the burden of manual annotation but often at the cost of inferior results. In addition, registration of histological images suffers from appearance variance due to multiple staining, repetitive texture, and section missing during making tissue sections. To deal with these challenges, we propose an unsupervised structural feature guided convolutional neural network (SFG). Structural features are robust to multiple staining. The combination of low-resolution rough structural features and high-resolution fine structural features can overcome repetitive texture and section missing, respectively. SFG consists of two components of structural consistency constraints according to the formations of structural features, i.e., dense structural component and sparse structural component. The dense structural component uses structural feature maps of the whole image as structural consistency constraints, which represent local contextual information. The sparse structural component utilizes the distance of automatically obtained matched key points as structural consistency constraints because the matched key points in an image pair emphasize the matching of significant structures, which imply global information. In addition, a multi-scale strategy is used in both dense and sparse structural components to make full use of the structural information at low resolution and high resolution to overcome repetitive texture and section missing. The proposed method was evaluated on a public histological dataset (ANHIR) and ranked first as of Jan 18th, 2022.

SMDAF: A novel keypoint based method for copy‐move forgery detection

Copy–move forgery poses a significant threat to social life and has aroused much attention in recent years. Although many copy-move forgery detection (CMFD) methods have been proposed, the most existing CMFD methods are short of adaptability in detecting images, which leads to the limitation on detection effects. To solve this problem, the paper proposes a novel keypoint-based CMFD method: second-keypoint matching and double adaptive filtering (SMDAF). Motivated by image matching based on keypoint, the second-keypoint matching method is designed to match keypoints extracted from copy–move forgery images, which can be used for both the single-CMFD and the multiple-CMFD. Then, a double adaptive filter (DAF) based on the AdaLAM algorithm and the KANN-DBSCAN clustering algorithm to filter wrong keypoint matches adaptively are proposed, according to the distinct distribution of keypoints in each image. Finally, the forgery regions are presented by finding their convex hulls and padding them. Compared with existing methods, extensive experiments show that the SMDAF method significantly provides more efficiency in detecting images under simulated real-world conditions, has better robustness when facing images with different post-treatment attacks, and is more effective in distinguishing images that look copy–move forged but are real.

Application of Sparse-Channel Lidar Sensors on Viewpoint-Invariant Loop Closing Task

Loop closing is an essential task for SLAM system, which can reduce the pose drifts and map distortion. However, challenges from viewpoint changes and sparse-channel lidar are still open problems. In this paper, a novel efficient and viewpoint-invariant loop closing method is presented, which well fits sparse-channel lidars. To keep the metric information of 3D point clouds, elevation maps are explored to represent sparse point clouds, based on which a novel and viewpoint-invariant key point feature detection method is proposed. To improve efficiency, Bag of Words technique is applied to accelerate the process of scene description and matching. In addition, a strict geometric consistency check of matched key point pairs is proposed to reject false positive detections, which makes our method robust to scenes with similar layouts in urban environments. Extensive experiments on challenging public datasets verify the validity and superiority of the proposed method.

La llegenda contemporània de la violació venjada i el seu tractament a <i>Estudi en lila</i> i en altres obres de la literatura catalana

Summary: In this article we compare the contemporary legend known as «revenge rape» with the literary versions present in several works of Catalan literature, such as the famous novel Estudi en lila by Maria-Antònia Oliver, and the short narrative that is at its origin, «Fils trencats»; the play Anna, o la venjança by Manuel Crespo Serrat, the short novel by Inés Vidal Farré El metge i un gos d’atura, and the story «Bèsties carnivores» by Carlota Gurt. We also mention some other cultural reflections of this urban legend, such as the Greek mythology, the rape and revenge films, and some specific contemporary films. Finally, the matching key points are extracted and a common pattern is formed.Keywords: urban legend, Catalan literature, Maria-Antònia Oliver, Estudi en lila, rapeand revenge, castration

Deep Learning Feature Extraction Using Attention-Based DenseNet 121 for Copy Move Forgery Detection

In modern society, digital images can be far-reaching, and the images are manipulated by various software and hardware technologies. The image forgery activities are undertaken by the attackers mainly for damaging the reputation of people or receiving fiscal gain, etc. Taking this into consideration, many techniques are developed to detect the forged images. In this paper, a new deep learning-based approach is introduced for copy-move forgery detection. The input images are segmented into non-overlapping patches using superpixel-based modified dense peak clustering and the features are extracted from the segmented patches by applying deep learning structure of attention-based DenseNet 121 model. Besides, to compare every block, the depth of each pixel is reconstructed, and eventually matching process is carried out using the adaptive chimp patch matching approach, which detects the suspicious forged regions in an image. Finally, the matched keypoints are merged with the segmented patches using the merged keypoint matching algorithm. As a result, the new deep learning approach has detected the forged regions efficiently from the tampered image with less time even the image is compressed, rotated, or scaled. The performance is evaluated in terms of recall, precision, accuracy, F1-score, computational time, and False Positive Rate (FPR). Moreover, the performance is compared with the other existing approaches, and the outcomes showed that the proposed method has achieved higher accuracy of 97%, recall of 99%, precision of 97.84%, F1-score of 98.81%, FPR of 0.022 and less computational time of 2.5 s.