Hierarchical Matching Strategy Research Articles

Injecting Text Clues for Improving Anomalous Event Detection From Weakly Labeled Videos.

Video anomaly detection (VAD) aims at localizing the snippets containing anomalous events in long unconstrained videos. The weakly supervised (WS) setting, where solely video-level labels are available during training, has attracted considerable attention, owing to its satisfactory trade-off between the detection performance and annotation cost. However, due to lack of snippet-level dense labels, the existing WS-VAD methods still get easily stuck on the detection errors, caused by false alarms and incomplete localization. To address this dilemma, in this paper, we propose to inject text clues of anomaly-event categories for improving WS-VAD, via a dedicated dual-branch framework. For suppressing the response of confusing normal contexts, we first present a text-guided anomaly discovering (TAG) branch based on a hierarchical matching scheme, which utilizes the label-text queries to search the discriminative anomalous snippets in a global-to-local fashion. To facilitate the completeness of anomaly-instance localization, an anomaly-conditioned text completion (ATC) branch is further designed to perform an auxiliary generative task, which intrinsically forces the model to gather sufficient event semantics from all the relevant anomalous snippets for completely reconstructing the masked description sentence. Furthermore, to encourage the cross-branch knowledge sharing, a mutual learning strategy is introduced by imposing a consistency constraint on the anomaly scores of these two branches. Extensive experimental results on two public benchmarks validate that the proposed method achieves superior performance over the competing methods.

Exploiting Deep Matching and Underwater Terrain Images to Improve Underwater Localization Accuracy

Underwater terrain image (UTI) matching provides a new idea for terrain-aided localization technology. However, the diversity of UTIs makes it very difficult to accurately match real-time terrain images (RTTIs) with reference terrain images (RTIs). To improve the localization accuracy of UTIs, this letter describes how UTI matching can be modeled as a deep learning problem based on supervised algorithms. Specifically, we propose a hierarchical matching strategy for the diverse features of UTIs by combining the deep and shallow feature characteristics of a convolutional neural network (CNN) to improve the localization accuracy. We also propose a sample-generation strategy based on candidate area quality index classification, and develop a data enhancement method by simulating the underwater terrain environment to overcome the lack of network training data. Simulations prove that the proposed method achieves accurate and stable localization results.

Fast and effective Keypoint-based image copy-move forgery detection using complex-valued moment invariants

Copy-move forgery is one of the most common image tampering schemes, with the potential use for misleading the opinion of the general public. Keypoint-based detection methods exhibit remarkable performance in terms of computational cost and robustness. However, these methods are difficult to effectively deal with the cases when 1) forgery only involves small or smooth regions, 2) multiple clones are conducted or 3) duplicated regions undergo geometric transformations or signal corruptions. To overcome such limitations, we propose a fast and accurate copy-move forgery detection algorithm, based on complex-valued invariant features. First, dense and uniform keypoints are extracted from the whole image, even in small and smooth regions. Then, these keypoints are represented by robust and discriminative moment invariants, where a novel fast algorithm is designed especially for the computation of dense keypoint features. Next, an effective magnitude-phase hierarchical matching strategy is proposed for fast matching a massive number of keypoints while maintaining the accuracy. Finally, a reliable post-processing algorithm is developed, which can simultaneously reduce false negative rate and false positive rate. Extensive experimental results demonstrate the superior performance of our proposed scheme compared with existing state-of-the-art algorithms, with average pixel-level F-measure of 94.54% and average CPU-time of 36.25 s on four publicly available datasets.

Hierarchical Point Matching Method Based on Triangulation Constraint and Propagation

Reliable image matching is the basis of image-based three-dimensional (3D) reconstruction. This study presents a quasi-dense matching method based on triangulation constraint and propagation as applied to different types of close-range image matching, such as illumination change, large viewpoint, and scale change. The method begins from a set of sparse matched points that are used to construct an initial Delaunay triangulation. Edge-to-edge matching propagation is then conducted for the point matching. Two types of matching primitives from the edges of triangles with areas larger than a given threshold in the reference image, that is, the midpoints of edges and the intersections between the edges and extracted line segments, are used for the matching. A hierarchical matching strategy is adopted for the above-mentioned primitive matching. The points that cannot be matched in the first stage, specifically those that failed in a gradient orientation descriptor similarity constraint, are further matched in the second stage. The second stage combines the descriptor and the Mahalanobis distance constraints, and the optimal matching subpixel is determined according to an overall similarity score defined for the multiple constraints with different weights. Subsequently, the triangulation is updated using the newly matched points, and the aforementioned matching is repeated iteratively until no new matching points are generated. Twelve sets of close-range images are considered for the experiment. Results reveal that the proposed method has high robustness for different images and can obtain reliable matching results.

A Novel Loop Closure Detection Approach Using Simplified Structure for Low-Cost LiDAR.

Reducing the cumulative error is a crucial task in simultaneous localization and mapping (SLAM). Usually, Loop Closure Detection (LCD) is exploited to accomplish this work for SLAM and robot navigation. With a fast and accurate loop detection, it can significantly improve global localization stability and reduce mapping errors. However, the LCD task based on point cloud still has some problems, such as over-reliance on high-resolution sensors, and poor detection efficiency and accuracy. Therefore, in this paper, we propose a novel and fast global LCD method using a low-cost 16 beam Lidar based on “Simplified Structure”. Firstly, we extract the “Simplified Structure” from the indoor point cloud, classify them into two levels, and manage the “Simplified Structure” hierarchically according to its structure salience. The “Simplified Structure” has simple feature geometry and can be exploited to capture the indoor stable structures. Secondly, we analyze the point cloud registration suitability with a pre-match, and present a hierarchical matching strategy with multiple geometric constraints in Euclidean Space to match two scans. Finally, we construct a multi-state loop evaluation model for a multi-level structure to determine whether the two candidate scans are a loop. In fact, our method also provides a transformation for point cloud registration with “Simplified Structure” when a loop is detected successfully. Experiments are carried out on three types of indoor environment. A 16 beam Lidar is used to collect data. The experimental results demonstrate that our method can detect global loop closures efficiently and accurately. The average global LCD precision, accuracy and negative are approximately 0.90, 0.96, and 0.97, respectively.

Hierarchical-Matching-Based Online and Real-Time Multi-Object Tracking with Deep Appearance Features

Based on tracking-by-detection, we propose a hierarchical-matching-based online and real-time multi-object tracking approach with deep appearance features, which can effectively reduce the false positives (FP) in tracking. For the purpose of increasing the accuracy rate of data association, we define the trajectory confidence using its position information, appearance information, and the information of historical relevant detections, after which we can classify the trajectories into different levels. In order to obtain discriminative appearance features, we developed a deep convolutional neural network to extract the appearance features of objects and trained it on a large-scale pedestrian re-identification dataset. Last but not least, we used the proposed diverse and hierarchical matching strategy to associate detection and trajectory sets. Experimental results on the MOT benchmark dataset show that our proposed approach performs well against other online methods, especially for the metrics of FP and frames per second (FPS).

Fast and Effective Image Copy-Move Forgery Detection via Hierarchical Feature Point Matching

Copy-move forgery is one of the most commonly used manipulations for tampering digital images. Keypoint-based detection methods have been reported to be very effective in revealing copy-move evidence due to their robustness against various attacks, such as large-scale geometric transformations. However, these methods fail to handle the cases when copy-move forgeries only involve small or smooth regions, where the number of keypoints is very limited. To tackle this challenge, we propose a fast and effective copy-move forgery detection algorithm through hierarchical feature point matching. We first show that it is possible to generate a sufficient number of keypoints that exist even in small or smooth regions by lowering the contrast threshold and rescaling the input image. We then develop a novel hierarchical matching strategy to solve the keypoint matching problems over a massive number of keypoints. To reduce the false alarm rate and accurately localize the tampered regions, we further propose a novel iterative localization technique by exploiting the robustness properties (including the dominant orientation and the scale information) and the color information of each keypoint. Extensive experimental results are provided to demonstrate the superior performance of our proposed scheme in terms of both efficiency and accuracy.

An efficient photogrammetric stereo matching method for high-resolution images

Stereo matching of high-resolution images is a great challenge in photogrammetry. The main difficulty is the enormous processing workload that involves substantial computing time and memory consumption. In recent years, the semi-global matching (SGM) method has been a promising approach for solving stereo problems in different data sets. However, the time complexity and memory demand of SGM are proportional to the scale of the images involved, which leads to very high consumption when dealing with large images. To solve it, this paper presents an efficient hierarchical matching strategy based on the SGM algorithm using single instruction multiple data instructions and structured parallelism in the central processing unit. The proposed method can significantly reduce the computational time and memory required for large scale stereo matching. The three-dimensional (3D) surface is reconstructed by triangulating and fusing redundant reconstruction information from multi-view matching results. Finally, three high-resolution aerial date sets are used to evaluate our improvement. Furthermore, precise airborne laser scanner data of one data set is used to measure the accuracy of our reconstruction. Experimental results demonstrate that our method remarkably outperforms in terms of time and memory savings while maintaining the density and precision of the 3D cloud points derived.

A NOVEL METHOD FOR ESTIMATION OF GLACIER SURFACE MOTION IN 1960s FROM ARGON KH-5 OPTICAL IMAGERY

The Antarctic ice sheet response to the global climate change, specifically the ice flow speed change of the glaciers, has been investigated by many researchers. However, most research results cover the period since 1970s or after the operation of the LANDSAT series. The availability of the film-based ARGON KH-5 data makes it possible to quantify the changes of the Antarctic ice sheet in 1960s. To meet the challenges of processing the low quality film-based ARGON images, a novel method was developed to allow estimating the ice sheet surface motion and reconstructing the surface model simultaneously from ARGON stereo images by decomposing the total parallaxes to terrain and motion based components. A photogrammetric approach was developed to distinguish stable ice surface features from those on motion and use them for recovering the camera orientation information. Several existing Antarctic mapping products were used to establish the ground control. The ice flow speed field is reconstructed using a hierarchical image matching strategy. Firstly, epipolar images are generated via a fundamental matrix derived from correspondences used in the geometric modelling process, and then an image pyramid is built. Second, the normalized cross-correlation (NCC) technique is conducted on each layer of the pyramid to match the extracted features. Since the images were taken at different times, during which the glacier motion occurred, the measured total parallaxes are decomposed to terrain and motion parallaxes according to given ice flow directions which are derived from the iteratively produced DTM or images. Finally, a speed map and a DTM can be generated at each level of the image pyramid. This process repeats itself. At the bottom of the pyramid the final speed map and DTM are produced at a resolution of about 60m and represent the ice flow field of 1963. This approach was tested using two ARGON stereo-pairs in Rayner glacier in East Antarctica. Both the ice flow speed map and DTM were generated, and their difference with recent products is briefly discussed.

A combined image matching method for Chinese optical satellite imagery

ABSTRACTImage matching is one of the key technologies for digital Earth. This paper presents a combined image matching method for Chinese satellite images. This method includes the following four steps: (1) a modified Wallis-type filter is proposed to determine parameters adaptively while avoiding over-enhancement; (2) a mismatch detection procedure based on a global-local strategy is introduced to remove outliers generated by the Scale-invariant feature transform algorithm, and geometric orientation with bundle block adjustment is employed to compensate for the systematic errors of the position and attitude observations; (3) we design a novel similarity measure (distance, angle and the Normalized Cross-Correlation similarities, DANCC) which considers geometric similarity and textural similarity; and (4) we introduce a hierarchical matching strategy to refine the matching result level by level. Four typical image pairs acquired from Mapping Satellite-1, ZY-1 02C, ZY-3 and GeoEye-1, respectively, are used for experimental analysis. A comparison with the two current main matching algorithms for satellite imagery confirms that the proposed method is capable of producing reliable and accurate matching results on different terrains from not only Chinese satellite images, but also foreign satellite images.

Hierarchical stereo matching with image bit-plane slicing

We propose a new stereo matching framework based on image bit-plane slicing. A pair of image sequences with various intensity quantization levels constructed by taking different bit-rate of the images is used for hierarchical stereo matching. The basic idea is to use the low bit-rate image pairs to compute rough disparity maps. The hierarchical matching strategy is then carried out iteratively to update the low confident disparities with the information provided by extra image bit-planes. It is shown that, depending on the stereo matching algorithms, even the image pairs with low intensity quantization are able to produce fairly good disparity results. Consequently, variate bit-rate matching is performed only regionally in the images for each iteration, and the average image bit-rate for disparity computation is reduced. Our method provides a hierarchical matching framework and can be combined with the existing stereo matching algorithms. Experiments on Middlebury datasets show that the proposed technique gives good results compared to the conventional full bit-rate matching.

Automatic orientation and 3D modelling from markerless rock art imagery

This paper investigates the use of two detectors and descriptors on image pyramids for automatic image orientation and generation of 3D models. The detectors and descriptors replace manual measurements and are used to detect, extract and match features across multiple imagery. The Scale-Invariant Feature Transform (SIFT) and the Speeded Up Robust Features (SURF) will be assessed based on speed, number of features, matched features, and precision in image and object space depending on the adopted hierarchical matching scheme. The influence of applying in addition Area Based Matching (ABM) with normalised cross-correlation (NCC) and least squares matching (LSM) is also investigated. The pipeline makes use of photogrammetric and computer vision algorithms aiming minimum interaction and maximum accuracy from a calibrated camera. Both the exterior orientation parameters and the 3D coordinates in object space are sequentially estimated combining relative orientation, single space resection and bundle adjustment. The fully automatic image-based pipeline presented herein to automate the image orientation step of a sequence of terrestrial markerless imagery is compared with manual bundle block adjustment and terrestrial laser scanning (TLS) which serves as ground truth. The benefits of applying ABM after FBM will be assessed both in image and object space for the 3D modelling of a complex rock art shelter.

A Region-Based Image Matching Combining Global and Local Features

A new approach is presented to match two images in presenting large scale changes. The novelty of our algorithm is a hierarchical matching strategy for global region features and local descriptors, which combines the descriptive power of global features and the discriminative power of local descriptors. To predict the likely location and scale of an object, global features extracted from the segmentation regions is used in the first stage for an efficient region matching. This initial matching can be ambiguous due to the instability and unreliability of global region feature, and therefore in the later stage local descriptors are matched within each region pair to discard false positives and the final matches are filtered by RANSAC. Experiments show the effectiveness and superiority of the proposed method in comparing to other approaches.

Partially occluded object recognition

Partially occluded object recognition is considered as one of the most difficult problems in machine vision; it has significant importance in industrial environment. In this paper, a 2-D object recognition algorithm applicable for both stand-alone and partially occluded objects is presented. The main contributions are the development of a scale and partial occlusion invariant boundary partition algorithm and a multi-resolution feature extraction algorithm using wavelet. We also implemented a hierarchical matching strategy for feature matching to reduce computational load, but with higher matching accuracy. Experiment results show that the proposed recognition algorithm is robust to similarity transformation and partial occlusion.

Challenges in Wide-area Structure-from-motion

The topic of this paper is wide area structure from motion. We first describe recent progress in obtaining large-scale 3D visual models from images. Our approach consists of a multi-stage processing pipeline, which can process a recorded video stream in real-time on standard PC hardware by leveraging the computational power of the graphics processor. The output of this pipeline is a detailed textured 3D model of the recorded area. The approach is demonstrated on video data recorded in Chapel Hill containing more than a million frames. While for these results GPS and inertial sensor data was used, we further explore the possibility to extract the necessary information for consistent 3D mapping over larger areas from images only. In particular, we discuss our recent work focusing on estimating the absolute scale of motion from images as well as finding intersections where the camera path crosses itself to effectively close loops in the mapping process. For this purpose we introduce viewpoint-invariant patches (VIP) as a new 3D feature that we extract from 3D models locally computed from the video sequence. These 3D features have important advantages with respect to traditional 2D SIFT features such as much stronger viewpoint-invariance, a relative pose hypothesis from a single match and a hierarchical matching scheme naturally robust to repetitive structures. In addition, we also briefly discuss some additional work related to absolute scale estimation and multi-camera calibration.

A trifocal sensor system for 3D reconstruction of architectural models

This article describes a fully automated video sensor system for the 3D reconstruction of architectural models from image sequences. The hybrid system unifies triangulation methods of three high-resolution cameras (spatial stereo) with feature-tracking methods over time (temporal stereo). Because standard tracking techniques suffer from outliers between subsequent images, we improve the outlier detection using temporal as well as spatial trifocal information. Furthermore, these spatio-temporal constraints allow the system to perform a guided matching that increases the number of tracked features and is robust to partial occlusions. The fully automatic and reliable calculation of the camera path from those tie points is still a challenging task. The use of multiple calibrated cameras that are fixed on a rig leads to additional constraints, which significantly stabilize the pose estimation process. To achieve a dense surface reconstruction, we propose an efficient spatial image-matching algorithm, which is based on trifocal image rectification and semi-global optimization using mutual information. Our improvements include a symmetric and hierarchical matching strategy with sub-pixel accuracy.