Scale-invariant Image Research Articles

Elimination of optical phase sensitivity in a shift, scale, and rotation invariant hybrid opto-electronic correlator via off-axis operation.

The hybrid opto-electronic correlator (HOC) uses a combination of optics and electronics to perform target recognition. Achieving a stable output from this architecture has previously presented a significant challenge due to a high sensitivity to optical phase variations, limiting the real-world feasibility of the device. Here we present a modification to the architecture that essentially eliminates the dependence on optical phases, and demonstrate verification of the proposed approach. Experimental results are shown to agree with the theory and simulations, for scale, rotation and shift invariant image recognition. This approach represents a major innovation in making the HOC viable for real-world applications.

Motor Fault Diagnosis Based on Scale Invariant Image Features

Traditional fault diagnosis methods are easy to be affected by different working conditions. This article proposed a motor fault diagnosis method based on visual knowledge, to reduce the impact of changes in working conditions and improve the feature extraction ability. The mapping relationship between actual faults and image intuitive features by symmetrized dot pattern and scale-invariant feature transform is established in this article. The fault state is obtained by statistics of the matching point with the dictionary templates generated from signals of normal and unnormal motors. Compared with other machine learning algorithms, this method does not need too much data training and learning. The efficiency of this method is validated by experiments, and the data image processing technology has great industrial application value in the field of motor fault detection or monitoring in the age of intelligence.

Classification of Low-Resolution Satellite Images Using Fractal Augmented Descriptors

Satellite imagery consists of highly complex spatial features that make it difficult for traditional image processing techniques to use them for classification tasks. In this paper, we propose a novel method to use these hidden fractal information that naturally exist in these satellite images. We have designed a fractal-based descriptor which generates a scale invariant fractal image for easier fractal-based pattern extraction and uses it as an added feature vector that is combined with the original image and fed into a VGG-16 deep learning architecture which successfully classifies even low-resolution satellite images with an f1-score of 0.78.

Real-time FPGA-based implementation of the AKAZE algorithm with nonlinear scale space generation using image partitioning

The first step in a scale invariant image matching system is scale space generation. Nonlinear scale space generation algorithms such as AKAZE, reduce noise and distortion in different scales while retaining the borders and key-points of the image. An FPGA-based hardware architecture for AKAZE nonlinear scale space generation is proposed to speed up this algorithm for real-time applications. The three contributions of this work are (1) mapping the two passes of the AKAZE algorithm onto a hardware architecture that realizes parallel processing of multiple sections, (2) multi-scale line buffers which can be used for different scales, and (3) a time-sharing mechanism in the memory management unit to process multiple sections of the image in parallel. We propose a time-sharing mechanism for memory management to prevent artifacts as a result of separating the process of image partitioning. We also use approximations in the algorithm to make hardware implementation more efficient while maintaining the repeatability of the detection. A frame rate of 304 frames per second for a 1280 times 768 image resolution is achieved which is favorably faster in comparison with other work.

Scale-Invariant Feature Transform-Based Heterogeneous Image Registration Method

Scale-Invariant Feature Transform-Based Heterogeneous Image Registration Method

Feature Keypoint-Based Image Compression Technique Using a Well-Posed Nonlinear Fourth-Order PDE-Based Model

A digital image compression framework based on nonlinear partial differential equations (PDEs) is proposed in this research article. First, a feature keypoint-based sparsification algorithm is proposed for the image coding stage. The interest keypoints corresponding to various scale-invariant image feature descriptors, such as SIFT, SURF, MSER, ORB, and BRIEF, are extracted, and the points from their neighborhoods are then used as sparse pixels and coded using a lossless encoding scheme. An effective nonlinear fourth-order PDE-based scattered data interpolation is proposed for solving the decompression task. A rigorous mathematical investigation of the considered PDE model is also performed, with the well-posedness of this model being demonstrated. It is then solved numerically by applying a consistent finite difference method-based numerical approximation algorithm that is next successfully applied in the image compression and decompression experiments, which are also discussed in this work.

Scale invariant digital image encryption using 3D modular chaotic map

Cryptography techniques are used to preserve the privacy and integrity of image media. Most proposed cryptographic images methods based on chaotic maps are dependent upon image sizes and most of them worked on square images. Tackling this problem, a scale invariant gray level image encryption method is proposed by means of 3D modular chaotic map. Initially, a 2D image with desirable size M × N is converted into a 3D image with m sub-images by size n × n. Optimizing the calculation, the parameters m and n should be appropriately selected. Having diffusion and confusion properties, two steps of 3D substitution and a 3D permutation step are used. In the 3D substitution steps, the pixels value of sub-images will be changed using the XOR operators and the circular shift. In the permutation step the pixels position of sub-images will be changed using 3D modular chaotic map, In this case, the m sub-images are divided into $$ k=\left\lceil \frac{m}{n}\right\rceil $$ equal size windows which each window has n sub-images with size n × n, and the 3D modular chaotic map is performed on any window with independent parameters. Depending upon the value of m, the number of windows is greater than or equal to 1; for k > 1, the last two windows may overlap to perform 3D chaotic map operation. Accelerating the proposed encryption method, the 3D chaotic map operations on windows can be implemented in parallel. The proposed method, in comparison with other methods of image encryption, improves the statistical parameters of cryptographic analysis, key space, and speed.

Investigation of scale-invariant image classification mechanisms

This study investigates the markers of operation of the classification mechanism that is invariant to scale transformations of images of test objects. Observers were asked to classify images according to the criterion of animate/inanimate object. Two series of studies were conducted using the method of cognitive evoked potentials. The angular sizes of the image of the objects presented to the observer were 3° in one series and 0.4° in another. It was established that the classification of the images of the stimuli of various angular sizes according to semantic features (animate/inanimate) causes a difference in the amplitudes of the P200 components (in the leads F7 and F8 of the frontal area). The role of the P200 components of the evoked potentials in the frontal areas of the brain as a marker of the classification process was investigated. It is observed that neural networks of the prefrontal cortex use the invariant description of images implemented in the previous stages of their processing to perform the classification of objects.

Scale-invariant structure saliency selection for fast image fusion

In this paper, we present a fast yet effective method for pixel-level scale-invariant image fusion in spatial domain based on the scale space theory. Specifically, we propose a scale-invariant structure saliency selection scheme based on the difference-of-Gaussian (DoG) scale space pyramid of images to build the weights or activity map. Due to the scale-invariant structure saliency selection, our method can keep both details of small size objects and the integrity information of large size objects in images. In addition, our method is very efficient since there are no complex operations involved and easy to be implemented. Experimental results demonstrate that compared to state-of-the-art image fusion methods, the proposed method yields competitive or even better results in terms of both visual quality and objective metrics, but it is much faster and therefore is appropriate to high-resolution and high-throughput image fusion tasks. Code is available at https://github.com/yiqingmy/Fusion.

Efficient misalignment-robust multi-focus microscopical images fusion

In this paper we propose a very efficient method to fuse the unregistered multi-focus microscopical images based on the speed-up robust features (SURF). Our method follows the standard pipeline of first registration and then fusion. However, instead of treating the registration and fusion as two completely independent stages, we propose to reuse the determinant of the approximate Hessian generated in SURF detection stage as the corresponding salient response for the final image fusion, thus it enables nearly cost-free saliency map generation. In addition, due to the adoption of SURF scale space representation, our method can generate scale-invariant saliency map which is desired for scale-invariant image fusion. We present an extensive evaluation on the dataset consisting of several groups of unregistered multi-focus 4K ultra HD microscopic images. Compared with the state-of-the-art multi-focus image fusion methods, our method is much faster and gains competitive or even better performance both in terms of the visual quality and objective metrics. Our method provides an efficient way to integrate complementary and redundant information from multiple multi-focus high-resolution unregistered images into a fused image that contains better description than any of the individual input images.

Rotation and scale invariant image watermarking based on polar harmonic transforms

Geometric attacks such as rotation and scale can destroy the synchronization of watermarking, designing a geometric-distortion resilient watermarking is currently a challenging work. In this paper, we propose a robust image watermarking technique based on polar harmonic transforms (PHTs) which are defined in a circle domain. The watermarking embedding process includes three principal steps: Firstly, we explicitly exploit invariant properties of PHTs’ moments to compute orthogonal moment features. Next, according to the order and repetition of PHTs’ moments, steady moments those are not sensitive to geometric attacks are picked out. Finally, watermark embedding is performed through quantization modulation on the selected steady moments, and the watermarked image is reconstructed by combining the original and the compensating image acquired from unmodulated and modulated PHTs’ moments, respectively. Experimental results show that the proposed watermarking approach is robust to both geometric distortions (rotation and scaling) and general signal processing attacks (Gaussian filtering, median filtering, average filtering, Gaussian noise, salt & pepper noise, speckle noise, JPEG compression, etc.) and outperforms state-of-the-art watermarking methods.

An empirical evaluation of recent texture features for the classification of natural images

ABSTRACTFeature extraction and classification are considered to be major tasks in image processing applications. This paper presents the performance of three feature extraction algorithms: the Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and Dominant Rotation Local Binary Pattern (DRLBP). The SIFT is robust to occlusion and clutter since it considers local features. The SURF is used to generate features for small objects and the extraction of interest points is also faster. The DRLBP is computationally efficient, given that it considers dominant and frequently occurring patterns. The texture performance of all these descriptors is measured in terms of accuracy on rotation invariant, scale invariant, and illumination invariant images taken from the Brodatz and Outex texture image databases. The texture images are classified using the K-Nearest Neighbor (KNN) and Naïve Bayes classifiers. The experimental result shows that the DRLBP with the KNN classifier provides better classification accuracy against various challenges.

Efficient contour match kernel

We propose a novel concept of asymmetric feature maps (AFM), which allows to evaluate multiple kernels between a query and database entries without increasing the memory requirements. To demonstrate the advantages of the AFM method, we derive an efficient contour match kernel – short vector image representation that, due to asymmetric feature maps, supports efficient scale and translation invariant sketch-based image retrieval. Unlike most of the short-code based retrieval systems, the proposed method provides the query localization in the retrieved image. The efficiency of the search is boosted by approximating a 2D translation search via trigonometric polynomial of scores by 1D projections. The projections are a special case of AFM. An order of magnitude speed-up is achieved compared to traditional trigonometric polynomials. The results are boosted by an image-based average query expansion approach and, without any learning, significantly outperform the state-of-the-art hand-crafted descriptors on standard benchmarks. Our method competes well with recent CNN-based approaches that require large amounts of labeled sketches, images and sketch-image pairs.

A scale-invariant change detection method for land use/cover change research

Land Use/Cover Change (LUCC) detection relies increasingly on comparing remote sensing images with different spatial and spectral scales. Based on scale-invariant image analysis algorithms in computer vision, we propose a scale-invariant LUCC detection method to identify changes from scale heterogeneous images. This method is composed of an entropy-based spatial decomposition, two scale-invariant feature extraction methods, Maximally Stable Extremal Region (MSER) and Scale-Invariant Feature Transformation (SIFT) algorithms, a spatial regression voting method to integrate MSER and SIFT results, a Markov Random Field-based smoothing method, and a support vector machine classification method to assign LUCC labels. We test the scale invariance of our new method with a LUCC case study in Montreal, Canada, 2005–2012. We found that the scale-invariant LUCC detection method provides similar accuracy compared with the resampling-based approach but this method avoids the LUCC distortion incurred by resampling.

Covering the Space of Tilts. Application to Affine Invariant Image Comparison

We propose a mathematical method to analyze the numerous algorithms performing Image Matching by Affine Simulation (IMAS). To become affine invariant they apply a discrete set of affine transforms to the images, previous to the comparison of all images by a Scale Invariant Image Matching (SIIM), like SIFT. Obviously this multiplication of images to be compared increases the image matching complexity. Three questions arise: a) what is the best set of affine transforms to apply to each image to gain full practical affine invariance? b) what is the lowest attainable complexity for the resulting method? c) how to choose the underlying SIIM method? We provide an explicit answer and a mathematical proof of quasi-optimality of the solution to the first question. As an answer to b) we find that the near-optimal complexity ratio between full affine matching and scale invariant matching is more than halved, compared to the current IMAS methods. This means that the number of key points necessary for affine matching can be halved, and that the matching complexity is divided by four for exactly the same performance. This also means that an affine invariant set of descriptors can be associated with any image. The price to pay for full affine invariance is that the cardinality of this set is 6.34 times larger than for a SIIM.

Scale coding bag of deep features for human attribute and action recognition

Most approaches to human attribute and action recognition in still images are based on image representation in which multi-scale local features are pooled across scale into a single, scale-invariant encoding. Both in bag-of-words and the recently popular representations based on convolutional neural networks, local features are computed at multiple scales. However, these multi-scale convolutional features are pooled into a single scale-invariant representation. We argue that entirely scale-invariant image representations are sub-optimal and investigate approaches to scale coding within a bag of deep features framework. Our approach encodes multi-scale information explicitly during the image encoding stage. We propose two strategies to encode multi-scale information explicitly in the final image representation. We validate our two scale coding techniques on five datasets: Willow, PASCAL VOC 2010, PASCAL VOC 2012, Stanford-40 and Human Attributes (HAT-27). On all datasets, the proposed scale coding approaches outperform both the scale-invariant method and the standard deep features of the same network. Further, combining our scale coding approaches with standard deep features leads to consistent improvement over the state of the art.

Think locally, fit globally: Robust and fast 3D shape matching via adaptive algebraic fitting

In this paper, we propose a novel 3D free form surface matching method based on a novel key-point detector and a novel feature descriptor. The proposed detector is based on algebraic surface fitting. By global smooth fitting, our detector achieved high computational efficiency and robustness against non-rigid deformations. For the feature descriptor, we provide algorithms to compute 3D critical net which generates a meaningful structure on standalone local key-points. The scale invariant and deformation robust Dual Spin Image descriptor is provided based on the 3D critical net. Our method is proved by solid mathematics. Intensive quantitative experiments demonstrate the robustness, efficiency and accuracy of the proposed method.

Explicit Krawtchouk moment invariants for invariant image recognition

The existing Krawtchouk moment invariants are derived by a linear combination of geometric moment invariants. This indirect method cannot achieve perfect performance in rotation, scale, and translation (RST) invariant image recognition since the derivation of these invariants are not built on Krawtchouk polynomials. A direct method to derive RST invariants from Krawtchouk moments, named explicit Krawtchouk moment invariants, is proposed. The proposed method drives Krawtchouk moment invariants by algebraically eliminating the distorted (i.e., rotated, scaled, and translated) factor contained in the Krawtchouk moments of distorted image. Experimental results show that, compared with the indirect methods, the proposed approach can significantly improve the performance in terms of recognition accuracy and noise robustness.

Computer Vision System (CVS) for In-Line Monitoring of Visual Texture Kinetics During Shrimp (Penaeus Spp.) Drying

In this study, the effects of drying medium temperature and velocity were surveyed on the image texture features of shrimp (Penaeus spp.) batches in a dryer equipped with a perpendicular dual-view computer vision system (CVS). This was carried out by applying an innovative rotation- and scale-invariant image texture processing approach with the capability of eliminating the effects of sample shrinkage on the visual textural features. Moreover, the variations in image texture parameters were investigated with moisture ratio, color, and geometrical characteristics of the shrimp samples. Drying experiments were conducted at hot air drying (HAD) temperatures of 50–90°C and superheated steam drying (SSD) temperatures of 110–120°C with drying medium velocities of 1–2 m/s. Several configurations of a multilayer perceptron artificial neural network (MLP-ANN) were also used to predict the moisture ratio and the geometrical characteristics of the shrimp batch using the image texture parameters. Generally, the image texture features were significantly affected by drying medium temperatures (p < 0.01), and the effects of drying medium velocities on the textural properties were nonsignificant (p > 0.05). Additionally, the higher drying temperatures generated products with uniform and regular texture patterns. The SSD produced samples with somewhat nonuniform and irregular texture patterns compared with HAD at 90°C. Finally, selected MLP-ANN topologies successfully predicted the moisture ratio and the geometrical characteristics of the shrimp batch using the textural properties with correlation coefficients higher than 0.99.

Image Matching Using Generalized Scale-Space Interest Points

The performance of matching and object recognition methods based on interest points depends on both the properties of the underlying interest points and the choice of associated image descriptors. This paper demonstrates advantages of using generalized scale-space interest point detectors in this context for selecting a sparse set of points for computing image descriptors for image-based matching. For detecting interest points at any given scale, we make use of the Laplacian $$\nabla ^2_{norm} L$$?norm2L, the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL and four new unsigned or signed Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL, $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL, $${\mathcal {D}}_{2,norm} L$$D2,normL and $$\tilde{\mathcal {D}}_{2,norm} L$$D~2,normL, which are defined by generalizing the definitions of the Harris and Shi-and-Tomasi operators from the second moment matrix to the Hessian matrix. Then, feature selection over different scales is performed either by scale selection from local extrema over scale of scale-normalized derivates or by linking features over scale into feature trajectories and computing a significance measure from an integrated measure of normalized feature strength over scale. A theoretical analysis is presented of the robustness of the differential entities underlying these interest points under image deformations, in terms of invariance properties under affine image deformations or approximations thereof. Disregarding the effect of the rotationally symmetric scale-space smoothing operation, the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL is a truly affine covariant differential entity and the Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL and $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL have a major contribution from the affine covariant determinant of the Hessian, implying that local extrema of these differential entities will be more robust under affine image deformations than local extrema of the Laplacian operator or the Hessian feature strength measures $${\mathcal {D}}_{2,norm} L$$D2,normL, $$\tilde{\mathcal {D}}_{2,norm} L$$D~2,normL. It is shown how these generalized scale-space interest points allow for a higher ratio of correct matches and a lower ratio of false matches compared to previously known interest point detectors within the same class. The best results are obtained using interest points computed with scale linking and with the new Hessian feature strength measures $${\mathcal {D}}_{1,norm} L$$D1,normL, $$\tilde{\mathcal {D}}_{1,norm} L$$D~1,normL and the determinant of the Hessian $$\det {\mathcal {H}}_{norm} L$$detHnormL being the differential entities that lead to the best matching performance under perspective image transformations with significant foreshortening, and better than the more commonly used Laplacian operator, its difference-of-Gaussians approximation or the Harris---Laplace operator. We propose that these generalized scale-space interest points, when accompanied by associated local scale-invariant image descriptors, should allow for better performance of interest point based methods for image-based matching, object recognition and related visual tasks.