Image Sparse Representation Research Articles

Learning to Hash With Optimized Anchor Embedding for Scalable Retrieval.

Sparse representation and image hashing are powerful tools for data representation and image retrieval respectively. The combinations of these two tools for scalable image retrieval, i.e., sparse hashing (SH) methods, have been proposed in recent years and the preliminary results are promising. The core of those methods is a scheme that can efficiently embed the (high-dimensional) image features into a low-dimensional Hamming space, while preserving the similarity between features. Existing SH methods mostly focus on finding better sparse representations of images in the hash space. We argue that the anchor set utilized in sparse representation is also crucial, which was unfortunately underestimated by the prior art. To this end, we propose a novel SH method that optimizes the integration of the anchors, such that the features can be better embedded and binarized, termed as Sparse Hashing with Optimized Anchor Embedding. The central idea is to push the anchors far from the axis while preserving their relative positions so as to generate similar hashcodes for neighboring features. We formulate this idea as an orthogonality constrained maximization problem and an efficient and novel optimization framework is systematically exploited. Extensive experiments on five benchmark image data sets demonstrate that our method outperforms several state-of-the-art related methods.

Light Field Reconstruction Using Shearlet Transform.

In this article we develop an image based rendering technique based on light field reconstruction from a limited set of perspective views acquired by cameras. Our approach utilizes sparse representation of epipolar-plane images (EPI) in shearlet transform domain. The shearlet transform has been specifically modified to handle the straight lines characteristic for EPI. The devised iterative regularization algorithm based on adaptive thresholding provides high-quality reconstruction results for relatively big disparities between neighboring views. The generated densely sampled light field of a given 3D scene is thus suitable for all applications which require light field reconstruction. The proposed algorithm compares favorably against state of the art depth image based rendering techniques and shows superior performance specifically in reconstructing scenes containing semi-transparent objects.

Joint learning sparsifying linear transformation for low-resolution image synthesis and recognition

Many computer vision problems involve exploring the synthesis and classification models that map images from the observed source space to a target space. Recently, one popular and effective method is to transform images from both source and target space into a shared single sparse domain, in which a synthesis model is established. Motivated by such a technique, this research attempts to explore an effective and robust linear function that maps the sparse representatio ns of images from the source space to the target space, and simultaneously develop a linear classifier on such a coupled space with both supervised and semi-supervised learning. In order to capture the sparse structure shared by each class, we represent this mapping using a linear transformation with the constraint of sparsity. The performance of our proposed method is evaluated on several benchmark image datasets for low-resolution faces/digits classification and super-resolution, and the experimental results verify the effectiveness of the proposed method.

Discriminant WSRC for Large-Scale Plant Species Recognition.

In sparse representation based classification (SRC) and weighted SRC (WSRC), it is time-consuming to solve the global sparse representation problem. A discriminant WSRC (DWSRC) is proposed for large-scale plant species recognition, including two stages. Firstly, several subdictionaries are constructed by dividing the dataset into several similar classes, and a subdictionary is chosen by the maximum similarity between the test sample and the typical sample of each similar class. Secondly, the weighted sparse representation of the test image is calculated with respect to the chosen subdictionary, and then the leaf category is assigned through the minimum reconstruction error. Different from the traditional SRC and its improved approaches, we sparsely represent the test sample on a subdictionary whose base elements are the training samples of the selected similar class, instead of using the generic overcomplete dictionary on the entire training samples. Thus, the complexity to solving the sparse representation problem is reduced. Moreover, DWSRC is adapted to newly added leaf species without rebuilding the dictionary. Experimental results on the ICL plant leaf database show that the method has low computational complexity and high recognition rate and can be clearly interpreted.

Compressed Sensing MRI With Phase Noise Disturbance Based on Adaptive Tight Frame and Total Variation

Magnetic resonance imaging (MRI) has been widely employed in medical diagnosis, since it enables superior visualization of anatomical structure with noninvasive and nonionizing radiation nature. However, during the data acquisition process of MRI, patients’ translational motion usually leads to phase changes of the observed data; moreover, the amplitudes of the observed data are usually contaminated by noises. In this paper, we assume that the phase and amplitude noises, respectively, cause the phase and amplitude changes of the observed data. Therefore, how to reconstruct high-quality magnetic resonance (MR) images via highly undersampled K-space data with noises is a challenge. To address this issue, a novel MR image reconstruction model, named the adaptive tight frame and total variation MR image reconstruction model (TFTV-MRI), is proposed based on the compressed sensing (CS) theory. TFTV-MRI fuses the adaptive tight frame (ATF) learning and total variation (TV) into the image reconstruction model. The sparse representations of MR images in tight frame domain can adapt to the MR image by itself, simultaneously, the advantage of TV is better edge preserving property and MR images are sparse in gradient domain. Differing from the $l_{0}$ -norm or $l_{1}$ -norm utilized in traditional AFT learning, we exploit the logarithm penalty term to measure the sparsity of MR images in TFTV-MRI. The alternating iterative minimization algorithm is utilized to tackle the optimization problem of TFTV-MRI, including ATF learning step and MR image reconstruction step. In MR image reconstruction step, the inertial proximal algorithm for nonconvex optimization is employed. The experiments verified that the proposed model achieved the superior performance for dislodging the phase noises caused by the translational motion and removing the amplitude noises of the observed data, and reconstructed MR images nicely in different sampling schemes. Compared with the existing methods, the proposed approach can achieve higher accurate image reconstruction quality, faster convergence speed, and better robustness to noises.

Fusion of Panchromatic and Multispectral Images via Coupled Sparse Non-Negative Matrix Factorization

In this paper, we construct a new coupled sparse non-negative matrix factorization (CSNMF) model for the fusion of panchromatic (PAN) and multispectral (MS) images. Two CSNMFs are developed for a joint sparse representation of MS and PAN images. Moreover, a sequential iterative algorithm is proposed to simultaneously find the solution to CSNMF. Because learned dictionaries can reveal the latent structure of images in spatial and spectral domains, the fused high-resolution MS images can be calculated by multiplying the dictionary of PAN image and the sparse coefficients of MS images. Some experiments are taken on simulated and real QuickBird data, and the results show that CSNMF outperforms its counterparts in both visual quality and numerical guidelines.

Boosted Dictionary Learning for Image Compression

Sparse representations over redundant dictionaries have shown to produce high-quality results in various signal and image processing tasks. Recent advancements in learning-based dictionary design have made image compression using data-adaptive learned dictionaries a promising field. In this paper, we present a boosted dictionary learning framework to construct an ensemble of complementary specialized dictionaries for sparse image representation. Boosted dictionaries along with a competitive sparse coding form our ensemble model which can provide us with more efficient sparse representations. The constituent dictionaries of the ensemble are obtained using a coherence regularized dictionary learning model for which two novel dictionary optimization algorithms are proposed. These algorithms improve the generalization properties of the trained dictionary compared with several incoherent dictionary learning methods. Based on the proposed ensemble model, we then develop a new image compression algorithm using boosted multi-scale dictionaries learned in the wavelet domain. Our algorithm is evaluated for the compression of natural images. Experimental results demonstrate that the proposed algorithm has better rate–distortion performance as compared with several competing compression methods, including analytic and learned dictionary schemes.

Image Inpainting Based on Exemplar and Sparse Representation

We propose a novel image inpainting approach in which the exemplar and the sparse representation are combined together skillfully. In the process of image inpainting, often there will be such a situation: although the sum of squared differences (SSD) of exemplar patch is the smallest among all the candidate patches, there may be a noticeable visual discontinuity in the recovered image when using the exemplar patch to replace the target patch. In this case, we cleverly use the sparse representation of image over a redundant dictionary to recover the target patch, instead of using the exemplar patch to replace it, so that we can promptly prevent the occurrence and accumulation of errors, and obtain satisfied results. Experiments on a number of real and synthetic images demonstrate the effectiveness of proposed algorithm, and the recovered images can better meet the requirements of human vision.

Study on compressed sensing reconstruction algorithm of medical image based on curvelet transform of image block

Traditional MRI technology may easily generate artifact due to slow imaging speed, therefore, MRI has low imagining quality and over-long sampling duration. Since wavelet transform cannot achieve the best approximation, image block theory is introduced in compressed sensing image reconstruction. In combination of the advantage of curvelet transform – it is suitable for expressing edge detail information and curve information, curvelet transform is utilized to conduct sparse representation of MRI image and proposed compressed sensing reconstruction algorithm of MRI image based on curvelet transform of image block. Signal to Noise Ratio (SNR), Relative L2 norm error (RLNE) and matching degree served as the evaluation indexes, and 4 groups of experiments about the influence of noise-free image, noised image, different sampling frequencies and different regularization parameters on the quality of reconstructed image were done. The results show that during image reconstruction, the algorithm proposed in this paper is superior to SIDCT and PBDCT in terms of three evaluation indexes. Besides, the algorithm owns strong ability to resist noise and good effects on keeping image detail and edge.

SAR Images Change Detection Based on Spatial Coding and Nonlocal Similarity Pooling

Accurate detection of the changed areas and effective speckle suppression are the main difficulties in synthetic aperture radar (SAR) image change detection (CD). The available feature extraction techniques for CD always ignore the spatial context correlation and are not robust to speckle noise. To overcome these drawbacks, we present a novel feature extraction technique that takes full advantage of sparse representation (SR) and nonlocal similarity of SAR images. First, each pixel in the difference image is represented by a feature vector, which is extracted using the sparse coding with a constructed robust discriminative dictionary. Next, a group of related feature vectors for each pixel can be generated according to the nonlocal similarity of SAR image. Finally, the discriminative change feature is obtained by means of the pooling, which can extract significant change information from the feature group. This method not only suppresses the speckle noise effectively but also improves the discrimination of the extracted features. The experimental results verify the superior performance of the proposed method on several real SAR image data sets and simulated image pairs.

A discriminative and sparse topic model for image classification and annotation

Image classification is to assign a category of an image and image annotation is to describe individual components of an image by using some annotation terms. These two learning tasks are strongly related. The main contribution of this paper is to propose a new discriminative and sparse topic model (DSTM) for image classification and annotation by combining visual, annotation and label information from a set of training images. The essential features of DSTM different from existing approaches are that (i) the label information is enforced in the generation of both visual words and annotation terms such that each generative latent topic corresponds to a category; (ii) the zero-mean Laplace distribution is employed to give a sparse representation of images in visual words and annotation terms such that relevant words and terms are associated with latent topics. Experimental results demonstrate that the proposed method provides the discrimination ability in classification and annotation, and its performance is better than the other testing methods (sLDA-ann, abc-corr-LDA, SupDocNADE, SAGE and MedSTC) for LabelMe, UIUC, NUS-WIDE and PascalVOC07 images.

A pan-sharpening method based on NSCT and formulated as compressive sensing problem

In this paper, a pan-sharpening method, using non-subsampled contourlet transform (NSCT) and the theory of compressive sensing (CS), is proposed. The NSCT is used for sparse image representation to perform a multiscale and directional decomposition of source images in order to express their detail and express the sparsity of their high frequency. The CS is used to merge the multispectral (MS) and panchromatic (pan) images from partial random measurements. Two different fusion rules are then applied. The final pan-sharpened image is obtained by inverse NSCT. Experimental results show the efficiency of the proposed method, compared with pan sharpening based on standard NSCT, in terms of visual quality and objective assessment. Moreover, the proposed technique is very effective when the storage and transmission bandwidth are limited.

Hierarchical semantic model and scattering mechanism based PolSAR image classification

The hybrid terrain type, such as the urban area, is generated by the backscattering of the similar ground objects aggregated together. It is a challenge to classify the hybrid terrain types as semantic homogeneous regions for unsupervised polarimetric SAR (PolSAR) image classification, since there are sharp bright-dark variations in intensity. In this paper, a hierarchical semantic model (HSM) is proposed for PolSAR images, which overcomes the above shortcoming by constructing the primal-level and middle-level semantics. The primal-level semantic is a polarimetric sketch map consisting of sketch segments, which is the sparse representation of a PolSAR image. The middle-level semantic is a region map which can extract hybrid terrain types as semantic homogeneous regions from the sketch map. Mapped by the region map, a complex PolSAR scene is partitioned into hybrid, linear and homogeneous subspaces. Then, according to the characteristics of the three subspaces, different methods are adopted, and furthermore polarimetric information is involved to improve the classification result. Experimental results on PolSAR data sets with different bands and sensors demonstrate that the proposed method is superior to the state-of-the-art methods in region homogeneity especially for hybrid terrain types.

Image fusion via nonlocal sparse K-SVD dictionary learning.

Image fusion aims to merge two or more images captured via various sensors of the same scene to construct a more informative image by integrating their details. Generally, such integration is achieved through the manipulation of the representations of the images concerned. Sparse representation plays an important role in the effective description of images, offering a great potential in a variety of image processing tasks, including image fusion. Supported by sparse representation, in this paper, an approach for image fusion by the use of a novel dictionary learning scheme is proposed. The nonlocal self-similarity property of the images is exploited, not only at the stage of learning the underlying description dictionary but during the process of image fusion. In particular, the property of nonlocal self-similarity is combined with the traditional sparse dictionary. This results in an improved learned dictionary, hereafter referred to as the nonlocal sparse K-SVD dictionary (where K-SVD stands for the K times singular value decomposition that is commonly used in the literature), and abbreviated to NL_SK_SVD. The performance of the NL_SK_SVD dictionary is applied for image fusion using simultaneous orthogonal matching pursuit. The proposed approach is evaluated with different types of images, and compared with a number of alternative image fusion techniques. The resultant superior fused images using the present approach demonstrates the efficacy of the NL_SK_SVD dictionary in sparse image representation.

Robust data transmission and recovery of images by compressed sensing for structural health diagnosis

Digital cameras are cost-effective vision sensors and able to directly provide two-dimensional information of structural condition in monitoring and assessment applications. For example, digital cameras are essential components of unmanned aerial vehicles (UAVs) and robotic agents for mobile sensing and inspection of pipelines, buildings, transportation infrastructure, etc, especially in post-natural disaster and man-made extreme events assessment. Additionally, while surveillance cameras have been widely used for transportation systems (e.g., traffic monitoring), if appropriately mounted on the large-scale structures such as the bridges, they can continuously monitor the structural condition under operational loads and hazards, complementing the regular visual inspection and assessment conducted by experts. In these or other applications, efficiently and reliably transferring the structural images or videos, which are as such large-scale, are important and challenging, especially in wireless platform that is either required (e.g., UAVs and robotic agents) or more suitable (e.g., camera monitoring networks) with only limited power and communication resources. This paper studies the computational algorithms for efficient and reliable transmission of the structural monitoring images; in particular, the compressed sensing (CS) technique is explored for robust data transmission and recovery. The sparse representation or data structure of the structural images is exploited, leading to the CS based central strategy: on some sparse domain, randomly encode large-scale image data into few relevant coefficients, which are then transferred (robust to random data loss) and recovered (in base station) for subsequent structural health diagnosis. Image data of bench scale pipe structure, concrete structure and full scale stay cable are employed for validation of the CS based method. Its performance is also compared with traditional transform coding and low-dimensional encoding (sampling), and their advantages and drawbacks are discussed. Copyright © 2016 John Wiley & Sons, Ltd.

Guest Editorial: Emerging Visual Information Processing Technologies for Multimedia Applications

Visual information becomes an indispensable part of our life through diversified multimedia applications, such as smart phones, IPAD, high definition TV, three dimensional TV and free viewpoint TV, to name a few. To provide more interactive and realistic multimedia experience, a new set of visual information processing technologies are emerging. These involve several research fields, such as visual information representation, compression, transmission and display technologies for variousmultimedia applications. The goal of this special issue is to present the latest research achievements of the emerging visual information processing technologies for multimedia applications. This special issue includes 16 papers, which will be briefly discussed as follows. Visual object detection, extraction and analysis are very meaningful and have various applications. The paper entitled BComic Storyboard Extraction via Edge Segment Analysis^ (10.1007/s11042-015-2680-8) aims to decompose the comic image into several frames, which plays a key role in producing the digital comic documents suitable for mobile reading. The authors propose new validation rules to reject the false storyboard detections; therefore, their algorithm achieves much better performance than the existing methods. In touch screen manufacturing, the defect inspection is one of the most important processes. The paper entitled BTouch Screen Defect Inspection based on Sparse Representation in Low Resolution Images^ (10.1007/s11042-015-2559-8) proposes a novel defect inspection approach based on sparse representation of low resolution image. Based on their simulation results, the proposed method achieves high inspection accuracy for different defect types. In the paper entitled BHierarchical Salient Object Detection Model using Contrast-based Saliency and Color Spatial Distribution^ (10.1007/s11042-015-2570-0), the authors aim to accurately interpret object saliency in image, in which the distinctive features are investigated from a global perspective. The effectiveness of this proposed model has been validated on MSRA database. Multimed Tools Appl (2016) 75:2633–2636 DOI 10.1007/s11042-016-3314-5

Enhancing the Prediction of Transmembrane β-Barrel Segments with Chain Learning and Feature Sparse Representation.

Transmembrane β-barrels (TMBs) are one important class of membrane proteins that play crucial functions in the cell. Membrane proteins are difficult wet-lab targets of structural biology, which call for accurate computational prediction approaches. Here, we developed a novel method named MemBrain-TMB to predict the spanning segments of transmembrane β-barrel from amino acid sequence. MemBrain-TMB is a statistical machine learning-based model, which is constructed using a new chain learning algorithm with input features encoded by the image sparse representation approach. We considered the relative status information between neighboring residues for enhancing the performance, and the matrix of features was translated into feature image by sparse coding algorithm for noise and dimension reduction. To deal with the diverse loop length problem, we applied a dynamic threshold method, which is particularly useful for enhancing the recognition of short loops and tight turns. Our experiments demonstrate that the new protocol designed in MemBrain-TMB effectively helps improve prediction performance.

Automatic target recognition with joint sparse representation of heterogeneous multi-view SAR images over a locally adaptive dictionary

Automatic target recognition (ATR) performance of synthetic aperture radar (SAR) is highly dependent on the sensitivity of SAR images to observing angle. Hence, jointly using of multi-view images of the same target is an efficient way to improve ATR accuracy, since multi-view images carry more correlated information than single-view image. Taking into account heterogeneous multi-views with random not uniform observing interval, an ATR approach with joint sparse representation over a locally adaptive dictionary is investigated in this paper. The first step is to establish a locally adaptive dictionary using sparse representation (SR) after training samples dimension reduction process by Independent and Identically Distributed (IID) Gaussian random project matrix. The locally adaptive dictionary is able to alleviate the limitation of target pose by adjusting the use of information in images and between images with the interval changing. Then heterogeneous multi-view test samples are re-presented by selecting atoms from the locally adaptive dictionary using joint sparse representation (JSR). In such way, high recognition accuracy is guaranteed by combination of more target information and adjustment of the inter-correlation information guarantee. Experiments based on the Moving and Stationary Target Acquisition and Recognition database verify the performance of the proposed algorithm.

On the selection of 2D Krawtchouk moments for face recognition

Sparse representation of images using orthogonal two-dimensional Krawtchouk moments (2D KCMs) for face recognition is motivated by their ability to capture region-based higher-order hidden nonlinear structures from discrete coordinates of finitely supported images and the invariance of affine transformations of these moments to common geometric distortions. This paper presents the effectiveness of selecting the discriminatory set of KCMs as the global and local face features as opposed to traditional features obtained from heuristic choice of fixed-order moments or projection of the moments for recognizing an identity. The selection of significantly sparse 2D KCM-based features according to the proposed approach results in highly efficient face recognition method as compared to the other methods that use orthogonal moments such as the 2D Zernike, 2D Tchebichef or 2D Gaussian-Hermite. Experiments on challenging databases (viz., FRGC and CK-AUC) and comparisons with the well established projection, texture, and moment-based methods indicate superior recognition performance in terms of mean accuracy and robustness of the proposed holistic- or hybrid-type discriminative KCM-based method, especially when sample sizes are small and the intraclass faces have significant variations due to expressions.

Multiplicative noise removal via adaptive learned dictionaries and TV regularization

Multiplicative noise removal is a key issue in image processing problem. While a large amount of literature on this subject are total variation (TV)-based and wavelet-based methods, recently sparse representation of images has shown to be efficient approach for image restoration. TV regularization is efficient to restore cartoon images while dictionaries are well adapted to textures and some tricky structures. Following this idea, in this paper, we propose an approach that combines the advantages of sparse representation over dictionary learning and TV regularization method. The method is proposed to solve multiplicative noise removal problem by minimizing the energy functional, which is composed of the data-fidelity term, a sparse representation prior over adaptive learned dictionaries, and TV regularization term. The optimization problem can be efficiently solved by the split Bregman algorithm. Experimental results validate that the proposed model has a superior performance than many recent methods, in terms of peak signal-to-noise ratio, mean absolute-deviation error, mean structure similarity, and subjective visual quality.