Sparse Coefficients Research Articles

Pansharpening through an improved sparsity-based fusion method

ABSTRACTIn this paper, a patch-wise manner based on the sparsity is proposed to fuse a panchromatic (PAN) image and a low resolution multispectral (LMS) image. In the sparsity-based pansharpening methods, improving the training process of the dictionaries and sparse coefficients of the fused image, which is the main goal of this paper, have a significant impact on the fused results. In this paper, the fused image is obtained by minimizing the cost function which is obtained from incorporating a Markov random field (MRF)-based prior model into the maximum a posteriori (MAP) estimation. The contribution of this paper is twofold derived from our proposed prior model. 1) The prior model only involves the parts of the PAN information related to a considered band of the high-resolution multispectral image in the training process of the dictionary of the considered band. Not only does it improve the training of the dictionaries, but it also leads to finding more accurate sparse coefficients. 2) The high-frequency information of the PAN image is also involved in the training process as a separate term. This term decreases the spectral distortion by relieving the adverse effect of dissimilarity of the grey levels between the PAN and multispectral images on the fused image. The visual and quantitative comparison between the performance of the proposed method and eight well-known fusion methods on the Pleiades, QuickBird, and DEIMOS-2 data demonstrate the superiority of the proposed method.

Phase harmonic correlations and convolutional neural networks

Abstract A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters that are well localized in the frequency domain, with different phases. We show that a rectifier then acts as a filter on the phase of the resulting coefficients. It computes signal descriptors that are local in space, frequency and phase. The nonlinear phase filter becomes a multiplicative operator over phase harmonics computed with a Fourier transform along the phase. We prove that it defines a bi-Lipschitz and invertible representation. The correlations of phase harmonics coefficients characterize coherent structures from their phase dependence across frequencies. For wavelet filters, we show numerically that signals having sparse wavelet coefficients can be recovered from few phase harmonic correlations, which provide a compressive representation.

Improved Wavelet Denoising by Penalty Function and Regularization Parameter

Wavelet-based convex optimization in sparse signal processing has attracted extensive research interest in recent years. Research demonstrates that the penalty function promotes the sparsity and improves the accuracy better than L1 norm method in dealing with convex and sparse signal reconstruction issues. This paper presents an improved denoising model which utilizes a penalty function to induce stronger sparsity in wavelet domain. In this paper, we have redefined the relationship model between signal noise levels and the regularization parameter to enhance sparsity caused by penalty functions in a wavelet domain. And three penalty functions are employed for wavelet coefficients to induce strong sparse wavelet coefficients under the wavelet domain. We then apply the improved wavelet denoising algorithm for image denoising to get the values of the PSNR are analysed under different noise level conditions. Experimental results clearly show that the improved wavelet denoising model is strength in terms of both quantitative measure and structural similarity quality, and outperforms many widely used denoising algorithms.

Single image super resolution using dictionary learning and sparse coding with multi-scale and multi-directional Gabor feature representation

Recently, sparse representation theory has shown the state-of-art performance in image super resolution (SR) field. The sparse representation based single image SR methods learn dictionaries to discover the co-occurence relationship between low-resolution (LR) and high-resolution (HR) image feature spaces to generate satisfactory SR images. However, most existing learning based methods focus on Gradient and Laplacian filters to achieve accurate feature representation in SR task. In this paper, a novel multi-scale and multi-directional feature descriptor approach is proposed to improve SR quality. In the proposed approach, Gabor filter is first used as feature representation in learning based super resolution to extract image features at different scales and orientations. Therefore, the difficulty of capturing the complex local structures in all scale and directions using traditional 1-D filters is resolved by the proposed Gabor filter approach. Then the efficient mapping between LR and HR images is achieved by searching the sparse representation coefficients over the LR and HR dictionaries in dictionary learning phase. Finally, the learned relationship is applied to validation LR input in order to achieve accurate SR image in reconstruction phase. The experimental results show that the proposed method outperforms other state-of-art methods in terms of both quantity and quality.

Structured dictionary learning using mixed-norms and group-sparsity constraint

Recently, sparse representation and dictionary learning have shown significant performance in machine vision. In particular, several supervised dictionary learning methods have been proposed for classification aim and increasing its accuracy. Among them, structured dictionary learning is an interesting approach which captures the discriminative properties of each class and common features among all classes in class-specific sub-dictionaries and a distinct shared sub-dictionary, respectively. It extracts the structural information that exists in samples of each class to increase the classification accuracy. Therefore, in this paper, a group-based structured dictionary learning method is proposed that captures structural information in each class and learns class-specific and shared sub-dictionaries based on mixed $$l_{2,1}$$ norm. Also, mixed $$l_{2,1}$$ norm is used for acquiring the sparse coefficients of data samples based on the learned sub-dictionaries. Then, classification is done by finding the class with (1) minimum reconstruction error or (2) maximum number of nonzero groups based on $$l_{1,0}$$ norm. The proposed method is evaluated by conducting experiments on Extended YaleB, AR and CMU-PIE face databases and the USPS handwritten digits database. The experimental results demonstrate the effectiveness of the proposed method in data representation and classification.

Clustering K-SVD for sparse representation of images

K-singular value decomposition (K-SVD) is a frequently used dictionary learning (DL) algorithm that iteratively works between sparse coding and dictionary updating. The sparse coding process generates sparse coefficients for each training sample, and the sparse coefficients induce clustering features. In the applications like image processing, the features of different clusters vary dramatically. However, all the atoms of dictionary jointly represent the features, regardless of clusters. This would reduce the accuracy of sparse representation. To address this problem, in this study, we develop the clustering K-SVD (CK-SVD) algorithm for DL and the corresponding greedy algorithm for sparse representation. The atoms are divided into a set of groups, and each group of atoms is employed to represent the image features of a specific cluster. Hence, the features of all clusters can be utilized and the number of redundant atoms are reduced. Additionally, two practical extensions of the CK-SVD are provided. Experimental results demonstrate that the proposed methods could provide more accurate sparse representation of images, compared to the conventional K-SVD and its existing extended methods. The proposed clustering DL model also has the potential to be applied to the online DL cases.

Convolutional Sparse Representation and Local Density Peak Clustering for Medical Image Fusion

Aiming at the problem of insufficient detail retention in multimodal medical image fusion (MMIF) based on sparse representation (SR), an MMIF method based on density peak clustering and convolution sparse representation (CSR-DPC) is proposed. First, the base layer is obtained based on the registered input image by the averaging filter, and the original image minus the base layer to obtain the detail layer. Second, for retaining the details of the fused image, the detail layer image is fused by CSR to obtain the fused detail layer image, then the base layer image is segmented into several image blocks, and the blocks are clustered by using DPC to obtain some clusters, and each class cluster is trained to obtain a sub-dictionary, and all the sub-dictionaries are fused to obtain an adaptive dictionary. The sparse coefficient is fused through the learned adaptive dictionary, and the fused base layer image is obtained through reconstruction. Finally, fusing the detail layer and the base layer and reconstructing them forms the ultimate fused image. Experiments show that compared to the state-of-the-art two multi-scale transformation methods and five SR methods, the proposed method(CSR-DPC) outperforms the other methods in terms of the image details, the visual quality and the objective evaluation index, which can be helpful for clinical diagnosis and adjuvant treatment.

Compressed sensing for high-noise astronomical image recovery

In deep exploration, we often use high-noise and high-resolution astronomical images because of long distance transmission, cosmic noise interference, or other disturbances. Designing an effective method for high-noise astronomical image denoising plays a crucial role in this field. The famous compressed sensing (CS) has shown to be a successful technique for high-resolution image reconstruction. CS is employed to solve the denoising problem in high-noise astronomical images, and a CS reconstruction model and a CS curvelet soft thresholding (NCCST) reconstruction algorithm are proposed simultaneously. In image denoising, the <italic>l</italic>₁-norm reconstruction model based on CS cannot fully consider the internal structure of the image, thus affecting the reconstruction quality. Combined with the <italic>l</italic>₁-norm reconstruction model and group sparse total variation method, the CS reconstruction model is first established. In this framework, the NCCST algorithm uses an adaptive curvelet adjustment operator proposed to select the sparse coefficients in the stage of image sparsity transform, and then a descending VisuShrink threshold is adopted to pick out these reconstructed astronomical image coefficients in each iteration. The experimental results demonstrate that the algorithm proposed can reconstruct clear images from high-noise and high-resolution astronomical images. Meanwhile, it can preserve more astronomical image details, edges, and textures. Even at a low-compression sampling ratio, this algorithm can achieve superior reconstruction performance. The proposed algorithm can be applied in the satellite receiving station to quickly and accurately reconstruct high-resolution astronomical images with high quality.

Microseismic Denoising and Reconstruction by Unsupervised Machine Learning

Microseismic data reconstruction is a procedure to compensate for acquisition deficiencies and to improve the data quality, which is important for subsequent processing steps such as event location. The performance of most reconstruction methods depends on 1) their parameter settings and 2) degrades greatly in case of strong noise interference. We propose an unsupervised machine learning algorithm to realize the incomplete noisy data reconstruction, using the Indian Buffet Process (IBP) as a prior to learning an appropriate dictionary from the noisy data. An approximation to the full posterior is obtained via Gibbs sampling, yielding an ensemble of dictionary and sparse coefficients. Finally, the signal of interest is reconstructed by the product of the dictionary and sparse coefficients. Tests on synthetic and real microseismic data demonstrate that the proposed method works very well for low signal-to-noise ratio data with missing traces.

Exploiting Block-sparsity for Hyperspectral Kronecker Compressive Sensing: a Tensor-based Bayesian Method.

Bayesian methods are attracting increasing attention in the field of compressive sensing (CS), as they are applicable to recover signals from random measurements. However, these methods have limited use in many tensor-based cases such as hyperspectral Kronecker compressive sensing (HKCS), because they exploit the sparsity in only one dimension. In this paper, we propose a novel Bayesian model for HKCS in an attempt to overcome the above limitation. The model exploits multi-dimensional block-sparsity such that the information redundancies in all dimensions are eliminated. Laplace prior distributions are employed for sparse coefficients in each dimension, and their coupling is consistent with the multi-dimensional block-sparsity model. Based on the proposed model, we develop a tensor-based Bayesian reconstruction algorithm, which decouples the hyperparameters for each dimension via a low-complexity technique. Experimental results demonstrate that the proposed method is able to provide more accurate reconstruction than existing Bayesian methods at a satisfactory speed. Additionally, the proposed method can not only be used for HKCS, it also has the potential to be extended to other multi-dimensional CS applications and to multi-dimensional block-sparse-based data recovery.

Sub-Linear Time Support Recovery for Compressed Sensing Using Sparse-Graph Codes

We study the support recovery problem for compressed sensing, where the goal is to reconstruct the a high-dimensional $K$-sparse signal $\mathbf{x}\in\mathbb{R}^N$, from low-dimensional linear measurements with and without noise. Our key contribution is a new compressed sensing framework through a new family of carefully designed sparse measurement matrices associated with minimal measurement costs and a low-complexity recovery algorithm. The measurement matrix in our framework is designed based on the well-crafted sparsification through capacity-approaching sparse-graph codes, where the sparse coefficients can be recovered efficiently in a few iterations by performing simple error decoding over the observations. We formally connect this general recovery problem with sparse-graph decoding in packet communication systems, and analyze our framework in terms of the measurement cost, time complexity and recovery performance. In the noiseless setting, our framework can recover any arbitrary $K$-sparse signal in $O(K)$ time using $2K$ measurements asymptotically with high probability. In the noisy setting, when the sparse coefficients take values in a finite and quantized alphabet, our framework can achieve the same goal in time $O(K\log(N/K))$ using $O(K\log(N/K))$ measurements obtained from measurement matrix with elements $\{-1,0,1\}$. When the sparsity $K$ is sub-linear in the signal dimension $K=O(N^\delta)$ for some $0<\delta<1$, our results are order-optimal in terms of measurement costs and run-time, both of which are sub-linear in the signal dimension $N$. The sub-linear measurement cost and run-time can also be achieved with continuous-valued sparse coefficients, with a slight increment in the logarithmic factors. This offers the desired scalability of our framework that can potentially enable real-time or near-real-time processing for massive datasets featuring sparsity.

Convolutional Sparse Representation of Injected Details for Pansharpening

In this letter, we address the pansharpening problem, which focuses on constructing a high-resolution (HR) multispectral (MS) image from a low-resolution (LR) MS and an HR panchromatic (Pan) image. The accuracy of pansharpening method based on sparse representation (SR) mainly depends on the construction of dictionary and the learning of sparse coefficients, while the details injection (DI)-based pansharpening method sharpens the MS bands by adding the proper spatial details from Pan. The combination of SR and DI has been put forward as the pansharpening method based on SR of injected details (SR-D). However, limited to the patch-based manner, pansharpening with traditional SR model faces two disadvantages, i.e., limited ability in detail preservation and high sensitivity to misregistration. In this letter, we replace the traditional SR model with convolutional SR (CSR) as a global SR model in the SR-D method and propose a new pansharpening method called CSR of injected details (CSR-D) to overcome the above-mentioned two drawbacks. Experimental results on the IKONOS and WorldView2 data sets show that the proposed method can achieve remarkable spectral and spatial quality on both reduced scale and full scale.

Spectral–Spatial Hyperspectral Image Classification Using a Multiscale Conservative Smoothing Scheme and Adaptive Sparse Representation

Spatial information has been demonstrated to be useful for hyperspectral images (HSIs) classification. The challenge is that spatial properties are often present at various spatial scales instead of a single fixed scale. A multiscale conservative smoothing algorithm is proposed in this paper to reduce noise and extract spatial structure information from coarse to fine levels. Over-smoothing is prevented automatically by imposing a weighting scheme on the neighboring pixels used for smoothing, where dissimilar neighbors' contributions are suppressed. Motived by multitask learning, an adaptive sparse representation is introduced to integrate different characteristics from the series of enhanced HSIs. The sparse coefficients of a given unknown pixel can be obtained from this representation and then used for classification. Experiments conducted on three benchmark data sets demonstrate that the proposed methodology leads to superior classification performance when compared to several well-known classifiers.

Image Denoising Algorithm based on Improved NCSR Model

The Nonlocally Centralized Sparse Representation (NCSR) algorithm aims to reduce sparse coding noise and utilizes the Nonlocal Self-similarity of the image to improve the effectiveness and accuracy of the image denoising algorithm, which has achieved good denoising effect. However, in the local dictionary learning and sparse coefficient estimation, the algorithm directly uses the Euclidean distance of the noise image to measure the similarity, which will have a certain impact on the final denoising effect. Aiming at this deficiency, it is proposed to use the Gaussian Mixed Model (GMM) to train nonlocal self-similar image patch of the external clean image in the local dictionary learning, and use the learned GMM to guide the internal image clustering; when performing the sparse coefficient estimation, the structural similarity index(SSIM) is introduced into the Euclidean distance and the improved Euclidean distance is introduced into the weight calculation. The experimental results show that the improved algorithm proposed in this paper has improved the PSNR and the visual effect of the image.

Light Field Video Compression and Real Time Rendering

AbstractLight field imaging is rapidly becoming an established method for generating flexible image based description of scene appearances. Compared to classical 2D imaging techniques, the angular information included in light fields enables effects such as post‐capture refocusing and the exploration of the scene from different vantage points. In this paper, we describe a novel GPU pipeline for compression and real‐time rendering of light field videos with full parallax. To achieve this, we employ a dictionary learning approach and train an ensemble of dictionaries capable of efficiently representing light field video data using highly sparse coefficient sets. A novel, key element in our representation is that we simultaneously compress both image data (pixel colors) and the auxiliary information (depth, disparity, or optical flow) required for view interpolation. During playback, the coefficients are streamed to the GPU where the light field and the auxiliary information are reconstructed using the dictionary ensemble and view interpolation is performed. In order to realize the pipeline we present several technical contributions including a denoising scheme enhancing the sparsity in the dataset which enables higher compression ratios, and a novel pruning strategy which reduces the size of the dictionary ensemble and leads to significant reductions in computational complexity during the encoding of a light field. Our approach is independent of the light field parameterization and can be used with data from any light field video capture system. To demonstrate the usefulness of our pipeline, we utilize various publicly available light field video datasets and discuss the medical application of documenting heart surgery.

A Robust Pansharpening Algorithm Based on Convolutional Sparse Coding for Spatial Enhancement

Pansharpening (PS) is a prominent remote sensing image fusion technique. It yields high-resolution multispectral (HRMS) images, which are imperative for the applications, such as recognition and detection. The PS methods based on conventional sparse representation induce blurring effects and are unable to preserve the essential spatial details in the fused outcome. In this article, to overcome these drawbacks, a robust fusion scheme is proposed based on convolutional sparse coding (CSC). The source images are decomposed into its constituent texture and cartoon components. The sparse coefficient maps are acquired from texture components by adapting CSC. Texture components are fused using activity level measurement, whereas averaging mechanism is used to fuse the cartoon components. The HRMS image is reconstructed by combining the fused components in proportion to the gradient information. Impact of number of filters on quality metrics estimation is analyzed. Comprehensive experiments are performed on the images acquired from distinct sensors. The proposed method is evaluated in terms of visual analysis and the quantitative metrics with reduced-scale and full-scale experiments. Extensive evaluations manifest the capability of the proposed method of maintaining the balanced tradeoff and retaining the desired spatial and spectral details.

Video Anomaly Detection with Sparse Coding Inspired Deep Neural Networks.

This paper presents an anomaly detection method that is based on a sparse coding inspired Deep Neural Networks (DNN). Specifically, in light of the success of sparse coding based anomaly detection, we propose a Temporally-coherent Sparse Coding (TSC), where a temporally-coherent term is used to preserve the similarity between two similar frames. The optimization of sparse coefficients in TSC with the Sequential Iterative Soft-Thresholding Algorithm (SIATA) is equivalent to a special stacked Recurrent Neural Networks (sRNN) architecture. Further, to reduce the computational cost in alternatively updating the dictionary and sparse coefficients in TSC optimization and to alleviate hyperparameters selection in TSC, we stack one more layer on top of the TSC-inspired sRNN to reconstruct the inputs, and arrive at an sRNN-AE. We further improve sRNN-AE in the following aspects: i) rather than using a predefined similarity measurement between two frames, we propose to learn a data-dependent similarity measurement between neighboring frames in sRNN-AE to make it more suitable for anomaly detection; ii) to reduce computational costs in the inference stage, we reduce the depth of the sRNN in sRNN-AE and, consequently, our framework achieves real-time anomaly detection; iii) to improve computational efficiency, we conduct temporal pooling over the appearance features of several consecutive frames for summarizing information temporally, then we feed appearance features and temporally summarized features into a separate sRNN-AE for more robust anomaly detection. To facilitate anomaly detection evaluation, we also build a large-scale anomaly detection dataset which is even larger than the summation of all existing datasets for anomaly detection in terms of both the volume of data and the diversity of scenes. Extensive experiments on both a toy dataset under controlled settings and real datasets demonstrate that our method significantly outperforms existing methods, which validates the effectiveness of our sRNN-AE method for anomaly detection. Codes and data have been released at https://github.com/StevenLiuWen/sRNN_TSC_Anomaly_Detection.

Fuzzy weighted sparse reconstruction error-steered semi-supervised learning for face recognition

Since the number of labeled data is limited in the semi-supervised learning settings, we propose a fuzzy weighted sparse reconstruction error-steered semi-supervised learning method for face recognition. The fuzzy membership functions are introduced to the reconstruction error calculation for the unlabeled data. A weight function is utilized to capture the locality property of data when learning the sparse coefficients. The fuzzy weighted sparse reconstruction error-steered semi-supervised learning not only inherits the advantages of sparse representation classification techniques and neighborhood methods, but also steers the reconstruction errors of unlabeled data. Experimental studies on well-known face image datasets demonstrate that the proposed method outperforms the comparative approaches.

An Entropy-Based Algorithm with Nonlocal Residual Learning for Image Compressive Sensing Recovery

Image recovery from compressive sensing (CS) measurement data, especially noisy data has always been challenging due to its implicit ill-posed nature, thus, to seek a domain where a signal can exhibit a high degree of sparsity and to design an effective algorithm have drawn increasingly more attention. Among various sparsity-based models, structured or group sparsity often leads to more powerful signal reconstruction techniques. In this paper, we propose a novel entropy-based algorithm for CS recovery to enhance image sparsity through learning the group sparsity of residual. To reduce the residual of similar packed patches, the group sparsity of residual is described by a Laplacian scale mixture (LSM) model, therefore, each singular value of the residual of similar packed patches is modeled as a Laplacian distribution with a variable scale parameter, to exploit the benefits of high-order dependency among sparse coefficients. Due to the latent variables, the maximum a posteriori (MAP) estimation of the sparse coefficients cannot be obtained, thus, we design a loss function for expectation–maximization (EM) method based on relative entropy. In the frame of EM iteration, the sparse coefficients can be estimated with the denoising-based approximate message passing (D-AMP) algorithm. Experimental results have shown that the proposed algorithm can significantly outperform existing CS techniques for image recovery.

Domain-Transformable Sparse Representation for Anomaly Detection in Moving-Camera Videos.

This paper presents a special matrix factorization based on sparse representation that detects anomalies in video sequences generated with moving cameras. Such representation is made by associating the frames of the target video, that is a sequence to be tested for the presence of anomalies, with the frames of an anomaly-free reference video, which is a previously validated sequence. This factorization is done by a sparse coefficient matrix, and any target-video anomaly is encapsulated into a residue term. In order to cope with camera trepidations, domaintransformations are incorporated into the sparse representation process. Approximations of the transformed-domain optimization problem are introduced to turn it into a feasible iterative process. Results obtained from a comprehensive video database acquired with moving cameras on a visually cluttered environment indicate that the proposed algorithm provides a better geometric registration between reference and target videos, greatly improving the overall performance of the anomaly-detection system.