Invariant Image Research Articles

AIFD Based 2D Image Registration to Multi-View Stereo Mapped 3D Models

Multi-view stereo (MVS) map based 3D range reconstruction is to generate 3D ranges by analyzing the surrounding snapshots from different perspectives. Different to the traditional method which employing the expensive and difficult maintaining laser range devices to calibrate the range of the real 3D objects, MVS has achieved its success by seeking the geometrical correlations between the correspondences from the snapshot of different perspectives. The concerning of MVS keeps rising thanks to the fast development of digital maps and 3D printing. Several algorithms with regard to MVS has been well developed and achieved their success with regard to reconstruction of 3D ranges. Meanwhile, most of the algorithms were mainly focusing on the fusion and merging of different scenes and surface refinement. Less capability of the feature matching algorithms on the affine invariant images renders the current MVS algorithms need huge amount of images with tiny perspective differences. In this paper, we will propose a new MVS algorithm, deploying our previous published Affine Invariant Feature Descriptor (AIFD) to detect and match the correspondences from different perspectives and applying Homograph matrix and segmentation to define the planes of the objects. Thanks to the AIFD and Homograph based projection model, our proposed MVS algorithm outperform other MVS algorithms in terms of speed and efficiency.

Non-unit mapped radial moments platform for robust, geometric invariant image watermarking and reversible data hiding

In this paper to adapt radial moment based watermarking for reversible cases, we have developed non-unit mapped radial moments. They have been used to extract the image invariant features to produce robust geometric invariant watermarking and reversible data hiding. This aim has been achieved through using two other newly introduced methods called reversible interval phase modulation and diagonal rotation estimation. In this paper, at first a mathematical procedure to compute these newly modified radial moments has been proposed. Then a watermarking platform has been suggested which can be used for either robust or reversible data hiding fields. The proposed robust watermarking provides robustness against signal processing and both local and global geometric attacks while at the same time the achieved robust watermarked image can remain integer as the original one. Also, the proposed radial moment computation procedure has been used to produce reversible data hiding. Reversible interval phase modulation and diagonal rotation estimation methods have been proposed to avoid any type of normalization-like degradation in the embedding and extraction. Various aspects of the proposed watermarking scheme have been thoroughly examined through several experiments and the results are highly promising.

Semantic invariant cross-domain image generation with generative adversarial networks

Abstract Recently, thanks to the state-of-the-art techniques in Generative Adversarial Networks, a lot of work achieves remarkable performance on learning the mapping between an input image and an output image without any paired relation. However, traditional methods on image-to-image translation merely consider the visual appearance properties, they fail to maintain the true semantics of an image during the transfer learning procedure from source to target domain. We propose a new approach that utilizes GAN to translate unpaired images between domains and remain high level semantic abstraction aligned. Our model controls the hierarchical semantics of images by processing semantic information on label level and spatial level respectively by constructing label and attention consistent losses. The experimental results on several benchmark datasets show that generated samples are both visually similar with target images and semantically consistent with their source counterparts. Furthermore, the experiment also suggests that our method can effectively improve the classification performance in unsupervised domain adaptation problem.

Robust color image watermarking using invariant quaternion Legendre-Fourier moments

In this paper, a geometrically invariant color image watermarking method using Quaternion Legendre-Fourier moments (QLFMs) is presented. A highly accurate, fast and numerically stable method is proposed to compute the QLFMs in polar coordinates. The proposed watermarking method consists of three main steps. First, the Arnold scrambling algorithm is applied to a binary watermark image. Second, the QLFMs of the original host color image are computed. Third, the binary digital watermark is embedding by performing the quantization of selected QLFMs. Two different groups of attacks are considered. The first group includes geometric attacks such as rotation, scaling and translation while the second group includes the common signal processing attacks such as image compression and noise. Experiments are performed where the performance of proposed method is compared with the existing moment-based watermarking methods. The proposed method is superior over all existing quaternion moment-based watermarking in terms of visual imperceptibility capability and robustness to different attacks.

3D skeleton based action recognition by video-domain translation-scale invariant mapping and multi-scale dilated CNN

In this paper, we present an image classification approach to action recognition with 3D skeleton videos. First, we propose a video domain translation-scale invariant image mapping, which transforms the 3D skeleton videos to color images, namely skeleton images. Second, a multi-scale dilated convolutional neural network (CNN) is designed for the classification of the skeleton images. Our multi-scale dilated CNN model could effectively improve the frequency adaptiveness and exploit the discriminative temporal-spatial cues for the skeleton images. Even though the skeleton images are very different from natural images, we show that the fine-tuning strategy still works well. Furthermore, we propose different kinds of data augmentation strategies to improve the generalization and robustness of our method. Experimental results on popular benchmark datasets such as NTU RGB + D, UTD-MHAD, MSRC-12 and G3D demonstrate the superiority of our approach, which outperforms the state-of-the-art methods by a large margin.

Image fusion based on shift invariant shearlet transform and stacked sparse autoencoder

Stacked sparse autoencoder is an efficient unsupervised feature extraction method, which has excellent ability in representation of complex data. Besides, shift invariant shearlet transform is a state-of-the-art multiscale decomposition tool, which is superior to traditional tools in many aspects. Motivated by the advantages mentioned above, a novel stacked sparse autoencoder and shift invariant shearlet transform-based image fusion method is proposed. First, the source images are decomposed into low- and high-frequency subbands by shift invariant shearlet transform; second, a two-layer stacked sparse autoencoder is adopted as a feature extraction method to get deep and sparse representation of high-frequency subbands; third, a stacked sparse autoencoder feature-based choose-max fusion rule is proposed to fuse the high-frequency subband coefficients; then, a weighted average fusion rule is adopted to merge the low-frequency subband coefficients; finally, the fused image is obtained by inverse shift invariant shearlet transform. Experimental results show the proposed method is superior to the conventional methods both in terms of subjective and objective evaluations.

Road area detection method based on DBNN for robot navigation using single camera in outdoor environments

PurposeThe purpose of this study is to develop a cost-effective autonomous wheelchair robot navigation method that assists the aging population.Design/methodology/approachNavigation in outdoor environments is still a challenging task for an autonomous mobile robot because of the highly unstructured and different characteristics of outdoor environments. This study examines a complete vision guided real-time approach for robot navigation in urban roads based on drivable road area detection by using deep learning. During navigation, the camera takes a snapshot of the road, and the captured image is then converted into an illuminant invariant image. Subsequently, a deep belief neural network considers this image as an input. It extracts additional discriminative abstract features by using general purpose learning procedure for detection. During obstacle avoidance, the robot measures the distance from the obstacle position by using estimated parameters of the calibrated camera, and it performs navigation by avoiding obstacles.FindingsThe developed method is implemented on a wheelchair robot, and it is verified by navigating the wheelchair robot on different types of urban curve roads. Navigation in real environments indicates that the wheelchair robot can move safely from one place to another. The navigation performance of the developed method and a comparison with laser range finder (LRF)-based methods were demonstrated through experiments.Originality/valueThis study develops a cost-effective navigation method by using a single camera. Additionally, it utilizes the advantages of deep learning techniques for robust classification of the drivable road area. It performs better in terms of navigation when compared to LRF-based methods in LRF-denied environments.

Covering the Space of Tilts. Application to Affine Invariant Image Comparison

We propose a mathematical method to analyze the numerous algorithms performing Image Matching by Affine Simulation (IMAS). To become affine invariant they apply a discrete set of affine transforms to the images, previous to the comparison of all images by a Scale Invariant Image Matching (SIIM), like SIFT. Obviously this multiplication of images to be compared increases the image matching complexity. Three questions arise: a) what is the best set of affine transforms to apply to each image to gain full practical affine invariance? b) what is the lowest attainable complexity for the resulting method? c) how to choose the underlying SIIM method? We provide an explicit answer and a mathematical proof of quasi-optimality of the solution to the first question. As an answer to b) we find that the near-optimal complexity ratio between full affine matching and scale invariant matching is more than halved, compared to the current IMAS methods. This means that the number of key points necessary for affine matching can be halved, and that the matching complexity is divided by four for exactly the same performance. This also means that an affine invariant set of descriptors can be associated with any image. The price to pay for full affine invariance is that the cardinality of this set is 6.34 times larger than for a SIIM.

Geometrically Invariant Image Watermarking Using Histogram Adjustment

In this article, a blind image watermarking scheme, which is a robust against common image processing and geometric attacks is proposed by adopting the concept of histogram-based embedding. The average filter is employed to low-pass pre-filter the host image. The watermark bits are embedded into the histogram of the low-frequency component and the template bits are embedded in the high-frequency residual. The embedding is performed by adjusting the value of two consecutive histogram bins. Furthermore, a post-quantization is employed after the embedding round to improve robustness. All pixel modifications incurred are based on the human visual system (HVS) characteristics. As a result, a good tradeoff between robustness and imperceptibility is achieved. Experimental results reported the satisfactory performance of the proposed scheme with respect to both common image processing and geometric attacks.

Target Localization Based on Invariant Image Features

Distinctive invariant features from images are perform reliable to image scale and rotation, also applicable to change in 3D viewpoint, so it is widely used in image matching and target locating. This paper analyses the extracting theory and process of invariant features from images in detail, and presents a method that can be used to calculate transforming modulus between two images are according to invariant features, finally accomplishing the target locating, which exist in two images together. Simulation results indicate that the paper method have excellent performance in target locating, it is precision for target locating between two images based on invariant features from images.

Exploring Web Images to Enhance Skin Disease Analysis Under A Computer Vision Framework.

To benefit the skin care, this paper aims to design an automatic and effective visual analysis framework, with the expectation of recognizing the skin disease from a given image conveying the disease affected surface. This task is nontrivial, since it is hard to collect sufficient well-labeled samples. To address such problem, we present a novel transfer learning model, which is able to incorporate external knowledge obtained from the rich and relevant Web images contributed by grassroots. In particular, we first construct a target domain by crawling a small set of images from vertical and professional dermatological websites. We then construct a source domain by collecting a large set of skin disease related images from commercial search engines. To reinforce the learning performance in the target domain, we initially build a learning model in the target domain, and then seamlessly leverage the training samples in the source domain to enhance this learning model. The distribution gap between these two domains are bridged by a linear combination of Gaussian kernels. Instead of training models with low-level features, we resort to deep models to learn the succinct, invariant, and high-level image representations. Different from previous efforts that focus on a few types of skin diseases with a small and confidential set of images generated from hospitals, this paper targets at thousands of commonly seen skin diseases with publicly accessible Web images. Hence the proposed model is easily repeatable by other researchers and extendable to other disease types. Extensive experiments on a real-world dataset have demonstrated the superiority of our proposed method over the state-of-the-art competitors.

2DIGH: a polar invariant local image descriptor based on joint histogram

One of the key challenges of current image matching techniques is how to build a robust local descriptor which is invariant to large variations in scale and rotation. To address this issue, in this work a polar gradient local oriented histogram pattern (PGP) is localized on normalized cropped regions around detected interest points. Then, a new image descriptor named two-dimensional intensity gradient histogram (2DIGH) is introduced using the joint histogram scheme. 2DIGH builds the extracted feature vector by intersecting of gradient and intensity information on subregions of the PGP. The measured distance with K-nearest neighbor represents feature vectors similarity/distance for image matching. The experimental results on Graffiti, Boat, Bark and ZuBud datasets indicate that the performance of the introduced 2DIGH is at least 41% better than other widely applied descriptors.

Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network

AbstractThe CrackNet, an efficient architecture based on the Convolutional Neural Network (CNN), is proposed in this article for automated pavement crack detection on 3D asphalt surfaces with explicit objective of pixel‐perfect accuracy. Unlike the commonly used CNN, CrackNet does not have any pooling layers which downsize the outputs of previous layers. CrackNet fundamentally ensures pixel‐perfect accuracy using the newly developed technique of invariant image width and height through all layers. CrackNet consists of five layers and includes more than one million parameters that are trained in the learning process. The input data of the CrackNet are feature maps generated by the feature extractor using the proposed line filters with various orientations, widths, and lengths. The output of CrackNet is the set of predicted class scores for all pixels. The hidden layers of CrackNet are convolutional layers and fully connected layers. CrackNet is trained with 1,800 3D pavement images and is then demonstrated to be successful in detecting cracks under various conditions using another set of 200 3D pavement images. The experiment using the 200 testing 3D images showed that CrackNet can achieve high Precision (90.13%), Recall (87.63%) and F‐measure (88.86%) simultaneously. Compared with recently developed crack detection methods based on traditional machine learning and imaging algorithms, the CrackNet significantly outperforms the traditional approaches in terms of F‐measure. Using parallel computing techniques, CrackNet is programmed to be efficiently used in conjunction with the data collection software.

Accurate and Efficient Inspection of Speckle and Scratch Defects on Surfaces of Planar Products

We propose a unified framework for detecting defects in planar industrial products or planar surfaces of nonplanar products based on a template-matching strategy. The framework includes three parts: an automatic selection of template image for a given test one, a robust geometric alignment between template and test images based on an approximate maximum clique approach, and an illumination invariant image comparison method for defect detection in the aligned images. Experimental results on challenging image datasets demonstrate the excellent performance of the proposed framework.

Invariant image watermarking using accurate Polar Harmonic Transforms

This paper proposes a geometrically invariant watermarking method based on highly accurate moments of Polar Harmonic Transforms (PHTs). Highly accurate, fast and numerically stable moments of PHTs are computed using a new simplified trigonometric formula. The proposed PHTs-based watermarking process is robust to geometric attacks. The proposed method consists of three main steps. In the first step, the Arnold scrambling algorithm is applied on a binary image. In the second step, accurate PHTs’ moments of the original host image are computed. In the third step, a digital watermark is embedded by performing the quantization of selected PHTs’ moments. The results of the numerical experiments ensure the validity of the proposed watermarking method. The proposed method shows a better visual imperceptibility and a better robustness to both geometric distortions and common signal processing attacks when compared with the existing moment-based watermarking methods.

Affine scale space: an affine invariant image structure to promote the detection of correspondences from stereo images

Calculating the geometry relationship by the positions of the correspondences from stereo images is a fundamental method to obtain the depth information. Such a method was quite widespread and popular thanks to its efficiency and easily accessed implementation. General speaking, the more density of the correspondences are, the more precisely the depth information can be calculated. Theoretically, a sufficient strengthened correspondences match algorithm can be utilized for depth information calculation under any circumstances. Unfortunately, the updated image feature detectors are all sensitive to the view point changes: with the rising of the stereo images' view angle, the number of matched features drastically reduced, resulting in the number of matched features not adequate to cover every details of the stereo images. This disadvantages of feature detections in practice hampers its application for the depth calibration. To tackle the sensitive of view point to stereo images, in this paper, we will propose an affine invariant affine scale space structure, which is more robust to detect the correspondences from stereo images. The purpose of affine scale space is to create a more general approach to the affine invariant image scale representation by modifying the corresponding Gaussian filters in order to cope with the specific change of view point. The affine adaptation of the scale space is to retain a linear relationship with the transiting of the view point. With linear relationship, the affine scale space can be established as a more general approach for the detection of correspondences from stereo images. With a better correspondences detection, a more precise depth information can be made.

Direct Visual Odometry in Low Light Using Binary Descriptors

Feature descriptors are powerful tools for photometrically and geometrically invariant image matching. To date, however, their use has been tied to sparse interest point detection, which is susceptible to noise under adverse imaging conditions. In this letter, we propose to use binary feature descriptors in a direct tracking framework without relying on sparse interest points. This novel combination of feature descriptors and direct tracking is shown to achieve robust and efficient visual odometry with applications to poorly lit subterranean environments.

A matching algorithm for large viewpoint changes images

Images with large viewpoint variation are hard to match by traditional image matching methods. To overcome the disadvantages of traditional perspective invariant image matching method, a perspective scale invariant feature transform (PSIFT) method based on improved artificial bee colony (IABC) for large viewpoint is proposed in this paper, which called IABC-PSIFT. The viewpoint variation of camera is described by perspective transformation model. According to the characteristic of the perspective transformation model, the colony choosing equation and probability equation of artificial bee colony algorithm are improved to obtain the optimal simulate viewpoint efficiently. The parameters of the proposed method are ascertained through experimental analysis. The experiment of image with large viewpoint is compared by using traditional affine invariant methods and the proposed method respectively. The results show that the proposed method can obtain more matching points while maintaining the same accuracy to that of the other affine invariant methods. Moreover, the efficiency of proposed method is improved compared with ASIFT.

Think locally, fit globally: Robust and fast 3D shape matching via adaptive algebraic fitting

In this paper, we propose a novel 3D free form surface matching method based on a novel key-point detector and a novel feature descriptor. The proposed detector is based on algebraic surface fitting. By global smooth fitting, our detector achieved high computational efficiency and robustness against non-rigid deformations. For the feature descriptor, we provide algorithms to compute 3D critical net which generates a meaningful structure on standalone local key-points. The scale invariant and deformation robust Dual Spin Image descriptor is provided based on the 3D critical net. Our method is proved by solid mathematics. Intensive quantitative experiments demonstrate the robustness, efficiency and accuracy of the proposed method.

Classifier ensembles for image identification using multi-objective Pareto features

In this paper we propose classifier ensembles that use multiple Pareto image features for invariant image identification. Different from traditional ensembles that focus on enhancing diversity by generating diverse base classifiers, the proposed method takes advantage of the diversity inherent in the Pareto features extracted using a multi-objective evolutionary Trace transform algorithm. Two variants of the proposed approach have been implemented, one using multilayer perceptron neural networks as base classifiers and the other k-Nearest Neighbor. Empirical results on a large number of images from the Fish-94 and COIL-20 datasets show that on average, the proposed ensembles using multiple Pareto features perform much better than both, the traditional classifier ensembles of single Pareto features with data randomization, and the well-known Random Forest ensemble. The better classification performance of the proposed ensemble is further supported by diversity analysis using a number of measures, indicating that the proposed ensemble consistently produces a higher degree of diversity than traditional ones. Our experimental results demonstrate that the proposed classifier ensembles are robust to various geometric transformations in images such as rotation, scale and translation, and to additive noise.