Invariant Image Representation Research Articles

Rotation invariant Gabor convolutional neural network for image classification

• We propose a rotation invariant Gabor convolutional neural network (RIGCN). • We learn Gabor-guided convolutional features in the Siamese network architecture . • We explore a convolutional fusion operator to obtain rotation invariant features. • We show the effectiveness of RIGCN for rotation invariant image classification . Gabor filters have been recently integrated with deep convolutional neural networks to learn better features with fewer model parameters. However, during feature learning the rotation invariance is not well addressed. In this paper, we propose a rotation invariant Gabor convolutional neural network (RIGCN) for image classification. First, we transform each input image to generate multiple rotated image instances and feed them into a weight-sharing Siamese network architecture to learn Gabor-guided deep convolutional features. Then, we compute the maximum and average feature responses from all the rotated instances of the same input image and send them into a convolutional fusion module to obtain a rotation invariant image representation. Finally, we use the cross-entropy loss for image classification. In our method, the use of Siamese architecture enables us to obtain rotation invariant features from rotated image instances. The use of convolutional fusion operator enables us to obtain richer statistical features with high efficiency. Experimental results on several benchmark datasets demonstrate the effectiveness of RIGCN for rotation invariant image classification.

DGCU–Net: A new dual gradient-color deep convolutional neural network for efficient skin lesion segmentation

Computer aided diagnosis (CAD) systems for Melanoma detection require an efficient skin lesion segmentation from dermoscopy images. However, irregular lesion shape and the large variations of colors have a significant impact on segmentation results. Therefore, methods developed for segmentation process have to be concerned with extracting boundaries and texture details of the lesion images with high accuracy. Recent deep learning convolutional neural networks such as U-Net achieves a significant progress in skin lesion segmentation. However, the segmentation results of original U-Net are not satisfactory to precisely find the borders of skin lesion. This paper presents a new Dual Gradient-Color U-Net (DGCU–Net) model which contains two encoders and a single decoder paths. The proposed DGCU–Net model integrates features extracted from an invariant color image representation with the gradient information of the input image to strengthen the borders of skin lesion. Gradient information of the input image is employed to calculate the attention maps used to strengthen color features in the first color encoder path. In addition, Atrous spatial pyramid pooling (ASPP) block is utilized in the bridge stage between encoder and decoder sub-networks. Spatial and channel attention network is also utilized to strengthen the connections between encoder and decoder blocks. The proposed network architecture significantly improves skin lesion segmentation results of the baseline U-Net model. Proposed model is evaluated using three benchmark datasets: ISIC 2017, ISIC 2018, and PH2 datasets. The comparison with state-of-the-art methods verified the superiority of our proposed DGCU–Net in various evaluation metrics.

Detection of COVID-19 With CT Images Using Hybrid Complex Shearlet Scattering Networks.

With the ongoing worldwide coronavirus disease 2019 (COVID-19) pandemic, it is desirable to develop effective algorithms to automatically detect COVID-19 with chest computed tomography (CT) images. Recently, a considerable number of methods based on deep learning have indeed been proposed. However, training an accurate deep learning model requires a large-scale chest CT dataset, which is hard to collect due to the high contagiousness of COVID-19. To achieve improved detection performance, this paper proposes a hybrid framework that fuses the complex shearlet scattering transform (CSST) and a suitable convolutional neural network into a single model. The introduced CSST cascades complex shearlet transforms with modulus nonlinearities and low-pass filter convolutions to compute a sparse and locally invariant image representation. The features computed from the input chest CT images are discriminative for COVID-19 detection. Furthermore, a wide residual network with a redesigned residual block (WR2N) is developed to learn more granular multiscale representations by applying it to scattering features. The combination of model-based CSST and data-driven WR2N leads to a more convenient neural network for image representation, where the idea is to learn only the image parts that the CSST cannot handle instead of all parts. Experiments on two public datasets demonstrate the superiority of our method. We can obtain more accurate results than several state-of-the-art COVID-19 classification methods in terms of measures such as accuracy, the F1-score, and the area under the receiver operating characteristic curve.

Deep Wavelet Neural Network based Robust Text Recognition for Overlapping Characters

This paper presents a deep learning based intelligent text recognition system with touching and overlapped characters. The robustness and effectiveness in the proposed model are enhanced through the modified configuration of neural network known as Deep Wavelet Neural Network (DWNN). The capability of deep learning networks to learn efficiently from an unlabeled dataset has attracted the attention of many researchers over the last decade. However, the performance of these networks is subject to the quality of the dataset and invariant image representation. Numerous optical character recognition techniques have also been presented in the recent years, but the overlapped and touching characters have not been addressed much. The nonlinear and uncertain representation of image data in case of overlapped text adds severe complexity in the process of feature extraction and respective learning. The proposed architecture of DWNN uses fast decaying wavelet functions as activation function in place of conventional sigmoid function to cope up with the uncertainties and nonlinearity of the data representation in overlapped text images. It comprises of cascaded layered architecture of translated and dilated versions of wavelets as activation functions for the training and feature extraction at multiple levels. The local transformation and deformation variation in the visual data has also been taken care efficiently through the modified architecture of DWNN. Comprehensive experimental analysis has been performed over various test images to verify the effectiveness of the proposed text recognition system. The performance of the proposed method is assessed with the help of the metrics, namely, estimation error, cost function and accuracy. The proposed approach will be implemented in MATLAB.

Deep Convolutional Self-Organizing Map Network for Robust Handwritten Digit Recognition

Deep Convolutional Neural Networks (DCNN) are currently the predominant technique commonly used to learn visual features from images. However, the complex structure of most recent DCNNs impose two major requirements namely, huge labeled dataset and high computational resources. In this paper, we develop a new efficient deep unsupervised network to learn invariant image representation from unlabeled visual data. The proposed Deep Convolutional Self-organizing Maps (DCSOM) network comprises a cascade of convolutional SOM layers trained sequentially to represent multiple levels of features. The 2D SOM grid is commonly used for either data visualization or feature extraction. However, this work employs high dimensional map size to create a new deep network. The N-Dimensional SOM (ND-SOM) grid is trained to extract abstract visual features using its classical competitive learning algorithm. The topological order of the features learned from ND-SOM helps to absorb local transformation and deformation variations exhibited in the visual data. The input image is divided into an overlapped local patches where each local patch is represented by the N-coordinates of the winner neuron in the ND-SOM grid. Each dimension of the ND-SOM can be considered as a non-linear principal component and hence it can be exploited to represent the input image using N-Feature Index Image (FII) bank. Multiple convolutional SOM layers can be cascaded to create a deep network structure. The output layer of the DCSOM network computes local histograms of each FII bank in the final convolutional SOM layer. A set of experiments using MNIST handwritten digit database and all its variants are conducted to evaluate the robust representation of the proposed DCSOM network. Experimental results reveal that the performance of DCSOM outperforms state-of-the-art methods for noisy digits and achieve a comparable performance with other complex deep learning architecture for other image variations.

Modeling global geometric spatial information for rotation invariant classification of satellite images.

The classification of high-resolution satellite images is an open research problem for computer vision research community. In last few decades, the Bag of Visual Word (BoVW) model has been used for the classification of satellite images. In BoVW model, an orderless histogram of visual words without any spatial information is used as image signature. The performance of BoVW model suffers due to this orderless nature and addition of spatial clues are reported beneficial for scene and geographical classification of images. Most of the image representations that can compute image spatial information as are not invariant to rotations. A rotation invariant image representation is considered as one of the main requirement for satellite image classification. This paper presents a novel approach that computes the spatial clues for the histograms of BoVW model that is robust to the image rotations. The spatial clues are calculated by computing the histograms of orthogonal vectors. This is achieved by calculating the magnitude of orthogonal vectors between Pairs of Identical Visual Words (PIVW) relative to the geometric center of an image. The comparative analysis is performed with recently proposed research to obtain the best spatial feature representation for the satellite imagery. We evaluated the proposed research for image classification using three standard image benchmarks of remote sensing. The results and comparisons conducted to evaluate this research show that the proposed approach performs better in terms of classification accuracy for a variety of datasets based on satellite images.

Invariant color images representation using accurate quaternion Legendre–Fourier moments

Color image representation using quaternion moments has gained more interest during the last few years. In this paper, invariant color image representation using quaternion Legendre–Fourier moments (QLFMs) is presented. A new method for exact, fast and stable computation of QLFMs in polar coordinates is proposed. Explicit rotation, scaling and translation (RST) invariants of quaternion Legendre–Fourier are derived. Numerical experiments are performed to compare the performance of QLFMs with the existing quaternion moments. The comparison clearly shows the superiority of the proposed method over all existing quaternion moments in terms of image reconstruction capability, RST invariances and numerical stability.

Exploring Web Images to Enhance Skin Disease Analysis Under A Computer Vision Framework.

To benefit the skin care, this paper aims to design an automatic and effective visual analysis framework, with the expectation of recognizing the skin disease from a given image conveying the disease affected surface. This task is nontrivial, since it is hard to collect sufficient well-labeled samples. To address such problem, we present a novel transfer learning model, which is able to incorporate external knowledge obtained from the rich and relevant Web images contributed by grassroots. In particular, we first construct a target domain by crawling a small set of images from vertical and professional dermatological websites. We then construct a source domain by collecting a large set of skin disease related images from commercial search engines. To reinforce the learning performance in the target domain, we initially build a learning model in the target domain, and then seamlessly leverage the training samples in the source domain to enhance this learning model. The distribution gap between these two domains are bridged by a linear combination of Gaussian kernels. Instead of training models with low-level features, we resort to deep models to learn the succinct, invariant, and high-level image representations. Different from previous efforts that focus on a few types of skin diseases with a small and confidential set of images generated from hospitals, this paper targets at thousands of commonly seen skin diseases with publicly accessible Web images. Hence the proposed model is easily repeatable by other researchers and extendable to other disease types. Extensive experiments on a real-world dataset have demonstrated the superiority of our proposed method over the state-of-the-art competitors.

An Analysis of the SURF Method

The SURF method (Speeded Up Robust Features) is a fast and robust algorithm for local, similarity invariant representation and comparison of images. Similarly to many other local descriptor-based approaches, interest points of a given image are defined as salient features from a scale-invariant representation. Such a multiple-scale analysis is provided by the convolution of the initial image with discrete kernels at several scales (box filters). The second step consists in building orientation invariant descriptors, by using local gradient statistics (intensity and orientation). The main interest of the SURF approach lies in its fast computation of operators using box filters, thus enabling real-time applications such as tracking and object recognition. The SURF framework described in this paper is based on the PhD thesis of H. Bay [ETH Zurich, 2009], and more specifically on the paper co-written by H. Bay, A. Ess, T. Tuytelaars and L. Van Gool [Computer Vision and Image Understanding, 110 (2008), pp. 346–359]. An implementation is proposed and used to illustrate the approach for image matching. A short comparison with a state-of-the-art approach is also presented, the SIFT algorithm of D. Lowe [International Journal of Computer Vision, 60 (2004), pp. 91–110], with which SURF shares a lot in common.

Multi-modal robust inverse-consistent linear registration.

Registration performance can significantly deteriorate when image regions do not comply with model assumptions. Robust estimation improves registration accuracy by reducing or ignoring the contribution of voxels with large intensity differences, but existing approaches are limited to monomodal registration. In this work, we propose a robust and inverse-consistent technique for cross-modal, affine image registration. The algorithm is derived from a contextual framework of image registration. The key idea is to use a modality invariant representation of images based on local entropy estimation, and to incorporate a heteroskedastic noise model. This noise model allows us to draw the analogy to iteratively reweighted least squares estimation and to leverage existing weighting functions to account for differences in local information content in multimodal registration. Furthermore, we use the nonparametric windows density estimator to reliably calculate entropy of small image patches. Finally, we derive the Gauss-Newton update and show that it is equivalent to the efficient second-order minimization for the fully symmetric registration approach. We illustrate excellent performance of the proposed methods on datasets containing outliers for alignment of brain tumor, full head, and histology images.

Image segmentation using optimal and hierarchical piecewise-constant approximations

The problem of approaching optimal image approximations using quasi-optimal hierarchical approximations, which are close to the optimal ones in terms of the total squared error, is considered. The problem is solved using algorithms for merging and dividing pixel clusters. The algorithms are based on elementary formulas. Information encoded in the image is proposed to be described, based on quasi-optimal approximations, using the invariant isomorphic image representation. The definition of the integer quantity of information is given, and the comparison of the integer estimate with the classical estimates in terms of R. Hartley and K. Shannon is performed. The experimental study of the properties of quasi-optimal hierarchical approximations, which are characteristic and uncharacteristic of nonhierarchical optimal image approximations, is carried out.

Generic polar harmonic transforms for invariant image representation

This paper introduces four classes of rotation-invariant orthogonal moments by generalizing four existing moments that use harmonic functions in their radial kernels. Members of these classes share beneficial properties for image representation and pattern recognition like orthogonality and rotation-invariance. The kernel sets of these generic harmonic function-based moments are complete in the Hilbert space of square-integrable continuous complex-valued functions. Due to their resemble definition, the computation of these kernels maintains the simplicity and numerical stability of existing harmonic function-based moments. In addition, each member of one of these classes has distinctive properties that depend on the value of a parameter, making it more suitable for some particular applications. Comparison with existing orthogonal moments defined based on Jacobi polynomials and eigenfunctions has been carried out and experimental results show the effectiveness of these classes of moments in terms of representation capability and discrimination power.

Document protection technique based on a model of digital memory inherent in signal

In this paper, we develop an algebraic description for previously proposed model of image attributed with its own digital hardware–independent memory (named virtual memory) which is similar to a memory of computer. Virtual memory contains the sequence of invariant image representations for different intensity resolutions computed independently of shifting, stretching and other standard transformations of intensity scale. The codes of image representation in virtual memory are divided into fixed and modifiable. The former store the information of source image and the latter contain the variable codes of the message embedded into the virtual memory to protect the image. By the exam-ple of banknotes the technique of steganography protection of usual and electronic documents is dis-cussed.

Rotationally invariant image representation for viewing direction classification in cryo-EM

We introduce a new rotationally invariant viewing angle classification method for identifying, among a large number of cryo-EM projection images, similar views without prior knowledge of the molecule. Our rotationally invariant features are based on the bispectrum. Each image is denoised and compressed using steerable principal component analysis (PCA) such that rotating an image is equivalent to phase shifting the expansion coefficients. Thus we are able to extend the theory of bispectrum of 1D periodic signals to 2D images. The randomized PCA algorithm is then used to efficiently reduce the dimensionality of the bispectrum coefficients, enabling fast computation of the similarity between any pair of images. The nearest neighbors provide an initial classification of similar viewing angles. In this way, rotational alignment is only performed for images with their nearest neighbors. The initial nearest neighbor classification and alignment are further improved by a new classification method called vector diffusion maps. Our pipeline for viewing angle classification and alignment is experimentally shown to be faster and more accurate than reference-free alignment with rotationally invariant K-means clustering, MSA/MRA 2D classification, and their modern approximations.

Invariant Scattering Convolution Networks

A wavelet scattering network computes a translation invariant image representation which is stable to deformations and preserves high-frequency information for classification. It cascades wavelet transform convolutions with nonlinear modulus and averaging operators. The first network layer outputs SIFT-type descriptors, whereas the next layers provide complementary invariant information that improves classification. The mathematical analysis of wavelet scattering networks explains important properties of deep convolution networks for classification. A scattering representation of stationary processes incorporates higher order moments and can thus discriminate textures having the same Fourier power spectrum. State-of-the-art classification results are obtained for handwritten digits and texture discrimination, with a Gaussian kernel SVM and a generative PCA classifier.

Trainable program for recognizing remote-probing data

This article discusses the use of an invariant image representation to create a composite system of segmentation algorithms. The representation is called invariant since it is independent of scale transformations of the image brightness that maintain the number and order of the histogram bins. The operation of the system in the classification and clustering regime is considered. In the first case, an expert incrementally trains the system, improving the composition of the training selection on the basis of the feedback formed by the system. In the training process, the system automatically eliminates from the initial set of segmentation algorithms a subset of algorithms that cannot be used to distinguish the pixels of the training selection. The structure of the software implementation is described. An example is given of the processing of remote-probing data.

Two-Dimensional Polar Harmonic Transforms for Invariant Image Representation

This paper introduces a set of 2D transforms, based on a set of orthogonal projection bases, to generate a set of features which are invariant to rotation. We call these transforms Polar Harmonic Transforms (PHTs). Unlike the well-known Zernike and pseudo-Zernike moments, the kernel computation of PHTs is extremely simple and has no numerical stability issue whatsoever. This implies that PHTs encompass the orthogonality and invariance advantages of Zernike and pseudo-Zernike moments, but are free from their inherent limitations. This also means that PHTs are well suited for application where maximal discriminant information is needed. Furthermore, PHTs make available a large set of features for further feature selection in the process of seeking for the best discriminative or representative features for a particular application.

Algebraic description of data embedding basing on idempotent image transformations

In this paper, we develop an algebraic description for previously proposed model of image containing the invariant image representation and embedded data in its own digital hardware-independent memory (named virtual memory) independently of prescribed image transformations. The properties of idempotent image transformations used for generation of invariant representation and also for data embedding are established. The statements are illustrated with image processing examples.

Illuminant and device invariant colour using histogram equalisation

Colour can potentially provide useful information for a variety of computer vision tasks such as image segmentation, image retrieval, object recognition and tracking. However, for it to be helpful in practice, colour must relate directly to the intrinsic properties of the imaged objects and be independent of imaging conditions such as scene illumination and the imaging device. To this end many invariant colour representations have been proposed in the literature. Unfortunately, recent work (Second Workshop on Content-based Multimedia Indexing) has shown that none of them provides good enough practical performance. In this paper we propose a new colour invariant image representation based on an existing grey-scale image enhancement technique: histogram equalisation. We show that provided the rank ordering of sensor responses are preserved across a change in imaging conditions (lighting or device) a histogram equalisation of each channel of a colour image renders it invariant to these conditions. We set out theoretical conditions under which rank ordering of sensor responses is preserved and we present empirical evidence which demonstrates that rank ordering is maintained in practice for a wide range of illuminants and imaging devices. Finally, we apply the method to an image indexing application and show that the method out performs all previous invariant representations, giving close to perfect illumination invariance and very good performance across a change in device.

Illuminant and device invariant colour using histogram equalisation

Colour can potentially provide useful information for a variety of computer vision tasks such as image segmentation, image retrieval, object recognition and tracking. However, for it to be helpful in practice, colour must relate directly to the intrinsic properties of the imaged objects and be independent of imaging conditions such as scene illumination and the imaging device. To this end many invariant colour representations have been proposed in the literature. Unfortunately, recent work (Second Workshop on Content-based Multimedia Indexing) has shown that none of them provides good enough practical performance. In this paper we propose a new colour invariant image representation based on an existing grey-scale image enhancement technique: histogram equalisation. We show that provided the rank ordering of sensor responses are preserved across a change in imaging conditions (lighting or device) a histogram equalisation of each channel of a colour image renders it invariant to these conditions. We set out theoretical conditions under which rank ordering of sensor responses is preserved and we present empirical evidence which demonstrates that rank ordering is maintained in practice for a wide range of illuminants and imaging devices. Finally, we apply the method to an image indexing application and show that the method out performs all previous invariant representations, giving close to perfect illumination invariance and very good performance across a change in device.