Overcomplete Dictionary Research Articles

Harmonization of diffusion MRI data sets with adaptive dictionary learning.

Diffusion magnetic resonance imaging can indirectly infer the microstructure of tissues and provide metrics subject to normal variability in a population. Potentially abnormal values may yield essential information to support analysis of controls and patients cohorts, but subtle confounds could be mistaken for purely biologically driven variations amongst subjects. In this work, we propose a new harmonization algorithm based on adaptive dictionary learning to mitigate the unwanted variability caused by different scanner hardware while preserving the natural biological variability of the data. Our harmonization algorithm does not require paired training data sets, nor spatial registration or matching spatial resolution. Overcomplete dictionaries are learned iteratively from all data sets at the same time with an adaptive regularization criterion, removing variability attributable to the scanners in the process. The obtained mapping is applied directly in the native space of each subject toward a scanner‐space. The method is evaluated with a public database which consists of two different protocols acquired on three different scanners. Results show that the effect size of the four studied diffusion metrics is preserved while removing variability attributable to the scanner. Experiments with alterations using a free water compartment, which is not simulated in the training data, shows that the modifications applied to the diffusion weighted images are preserved in the diffusion metrics after harmonization, while still reducing global variability at the same time. The algorithm could help multicenter studies pooling their data by removing scanner specific confounds, and increase statistical power in the process.

Simultaneous coded aperture and dictionary optimization in compressive spectral imaging via coherence minimization.

Coded aperture snapshot spectral imaging (CASSI) reconstructs a hyperspectral image from several two-dimensional (2D) projections via compressive sensing. The reconstruction quality and the sampling efficiency of CASSI can be effectively improved by decreasing the coherence of the underlying sensing matrix. Efforts have been made to minimize the coherence with individual optimization on coded aperture or sparse basis. In this paper, a simultaneous optimization on the system projection and the over-complete dictionary is introduced to minimize the Frobenius norm coherence. The dual-disperser structure and the RGB image sensor are adopted for the lowest coherence in terms of system configuration. The coded aperture and the dictionary are optimized with genetic algorithm and gradient descent respectively, and simultaneous optimization is conducted iteratively. Low coherence of sensing matrix is acquired after the simultaneous optimization, with both reconstruction quality and sampling efficiency significantly improved. Compared to the non-optimized system and state-of-the-art systems with individually optimized coded aperture or dictionary, the simultaneous optimization promotes the peak signal-to-noise ratio by more than 5dB. The coherence minimization via simultaneous optimization on the system matrix and the sparse representation basis may open opportunities for further development of other compressive-sensing-based computational imaging systems.

Ambiguity and sensitivity in imprecise dictionaries for compressed sensing

An imprecise dictionary is commonly used in compressed sensing in cases where the dictionary is derived via a learning process or is based on incomplete prior knowledge. This paper investigates the issue of ambiguity in dictionary estimation and the sensitivity of a true sparse vector and signal (i.e., signals represented by true sparse vectors with respect to the true dictionary) to dictionary perturbations in compressed sensing. We first demonstrate that the inherent ambiguity in dictionary estimation cannot be resolved, even when the desired sparse vector can be obtained. By imposing conditions on the perturbation of the true dictionary, we then demonstrate that the sparse vector and signal can both be stably estimated within the bounds of the dictionary perturbation. Our analysis results are based on the condition that AD holds the restricted isometry property (RIP), where A is a precise sensing matrix and D is a dictionary mismatch. The RIP of AD cannot be efficiently determined for a general over-complete dictionary; therefore, we numerically verified our analytical results by conducting experiments under the specific but commonly encountered situation in which D (an orthonormal matrix) and A (a Gaussian sensing matrix), in which the RIP of AD holds with a high degree of probability.

Sparse Representation based Super-Resolution of MRI Images with Non-Local Total Variation Regularization

Diffusion-weighted (DW) and spectroscopic MR (MRS) images are found to be very helpful for diagnostic purposes as they provide complementary information to that provided by conventional MRI. These images are also acquired at a faster rate, but with low signal-to-noise ratio. This limitation can be overcome by applying image super-resolution techniques. In this paper, we propose sparse representation over a learned overcomplete dictionary based single-image super-resolution (SISR) technique for DW and MRS images. The proposed SISR method incorporates patch-wise sparsity constraint based on external HR information together with the non-local total variation (NLTV) as internal information to make the regularization problem more robust. Experiments are conducted for both DW and MRS test images and results are compared with some of the recent methods. Results indicate the potential of the proposed method for clinical MRI applications.

A novel hyper-chaotic image encryption with sparse-representation based compression

This digital era uses lots of images in communication, which are confidential and large in volume. The transmission channel thus arises the question of security as well as transmission load. To date, several works are there to solve this issue. However, they fail to provide a single unit that gives these two facilities in a unified way. This paper presents a novel technique of unification of image compression and encryption in a single module via sparse-representation of image frames and hyper-chaotic encryption techniques. In this work, we have proposed a method to estimates the sparse vectors of a given image using a regularized trained over-complete dictionary and encrypt the non-zero coefficients of the sparse vectors using key-streams generated by a hyper-chaotic system. This sparse coding based encryption technique provides a higher compression ratio (CR) compared to some recently proposed techniques on one side and increases security level on the other side. Moreover, the security is strengthened by using this key-sequence in different steps in the encryption scheme. Thus the compressed-encrypted outputs are stronger than simple chaotic encryption against any intruder. The efficiency and authenticity of the proposed algorithm are verified through several quality-index e.g. entropy, CR, etc. The resistivity of the proposed algorithm toward the known or chosen-plaintext attack is also analyzed. The results of the key sensitivity test, and cropping attack test also ensure the authors’ claim. The comparison of the proposed technique with some recently published works justifies the reliability of this work.

Tensor Least Angle Regression for Sparse Representations of Multidimensional Signals.

Sparse signal representations have gained much interest recently in both signal processing and statistical communities. Compared to orthogonal matching pursuit (OMP) and basis pursuit, which solve the and constrained sparse least-squares problems, respectively, least angle regression (LARS) is a computationally efficient method to solve both problems for all critical values of the regularization parameter . However, all of these methods are not suitable for solving large multidimensional sparse least-squares problems, as they would require extensive computational power and memory. An earlier generalization of OMP, known as Kronecker-OMP, was developed to solve the problem for large multidimensional sparse least-squares problems. However, its memory usage and computation time increase quickly with the number of problem dimensions and iterations. In this letter, we develop a generalization of LARS, tensor least angle regression (T-LARS) that could efficiently solve either large or large constrained multidimensional, sparse, least-squares problems (underdetermined or overdetermined) for all critical values of the regularization parameter and with lower computational complexity and memory usage than Kronecker-OMP. To demonstrate the validity and performance of our T-LARS algorithm, we used it to successfully obtain different sparse representations of two relatively large 3D brain images, using fixed and learned separable overcomplete dictionaries, by solving both and constrained sparse least-squares problems. Our numerical experiments demonstrate that our T-LARS algorithm is significantly faster (46 to 70 times) than Kronecker-OMP in obtaining -sparse solutions for multilinear leastsquares problems. However, the -sparse solutions obtained using Kronecker-OMP always have a slightly lower residual error (1.55% to 2.25%) than ones obtained by T-LARS. Therefore, T-LARS could be an important tool for numerous multidimensional biomedical signal processing applications.

Data-Driven Human Modeling by Sparse Representation

Data-driven methods for modeling the realistic shapeof 3D human bodies need to access datasets that contain a large amount of 3D human models. We develop a method based on sparse representation in this paper to represent 3D human models as signals of patches. Unlike the general mesh compression approaches, all mesh models used in a data-driven human modeling framework have the same mesh connectivity. By using this property, we segment a human model into patches containing the same number of vertices. L0-learning algorithm is selected to train an overcomplete dictionary matrix, which in turn introduces sparse representation of the dataset. Patch signals of individual human models can then be extracted by using the dictionary matrix. With the ease of balance control between sparsity and accuracy that is featured by the chosen learning algorithm, a representation with high compression ratio and low shape-approximation error can be determined. The results have been compared with the widely used statistic representation based on principal component analysis (PCA) to verify the effectiveness of our approach. Moreover, the method for using sparse representation in the regression-based statistical modeling of 3D human models has been presented at the end of the paper.

Neural networks-based regularization for large-scale medical image reconstruction

In this paper we present a generalized Deep Learning-based approach for solving ill-posed large-scale inverse problems occuring in medical image reconstruction. Recently, Deep Learning methods using iterative neural networks (NNs) and cascaded NNs have been reported to achieve state-of-the-art results with respect to various quantitative quality measures as PSNR, NRMSE and SSIM across different imaging modalities. However, the fact that these approaches employ the application of the forward and adjoint operators repeatedly in the network architecture requires the network to process the whole images or volumes at once, which for some applications is computationally infeasible. In this work, we follow a different reconstruction strategy by strictly separating the application of the NN, the regularization of the solution and the consistency with the measured data. The regularization is given in the form of an image prior obtained by the output of a previously trained NN which is used in a Tikhonov regularization framework. By doing so, more complex and sophisticated network architectures can be used for the removal of the artefacts or noise than it is usually the case in iterative NNs. Due to the large scale of the considered problems and the resulting computational complexity of the employed networks, the priors are obtained by processing the images or volumes as patches or slices. We evaluated the method for the cases of 3D cone-beam low dose CT and undersampled 2D radial cine MRI and compared it to a total variation-minimization-based reconstruction algorithm as well as to a method with regularization based on learned overcomplete dictionaries. The proposed method outperformed all the reported methods with respect to all chosen quantitative measures and further accelerates the regularization step in the reconstruction by several orders of magnitude.

Sparse representation-based detail-injection method for pan-sharpening

Pan-sharpening is an important image preprocessing technique for remote sensing that aims to enhance spatial resolution of multispectral (MS) images under the guidance of panchromatic (PAN) image while preserving spectral properties. The existing pan-sharpening methods usually adopt the globally consistent detail-injection models, neglecting the detail differences between spectral channels, which leads to imprecise spatial details and distorted spectral properties of pan-sharpening results. We propose a sparse representation-based detail-injection model for pan-sharpening that utilizes the structure similarity and detail differences between PAN and low-resolution multispectral (LRM) images at each channel, to improve the performance of pan-sharpening. Specifically, to better express the inherent detail properties of the MS image, the overcomplete dictionary of each channel is constructed from synthesized high-resolution multispectral (HRM) images. Moreover, the most proposed methods require that the spectral responses of the PAN image and the MS image cover the same wavelength range; nevertheless, most sensors cannot match this condition. To address this problem, we propose constructing coupled low-resolution and high-resolution dictionaries from LRM and synthesized HRM images so that the structure similarities can be used for detail injection. The qualitative and quantitative experimental results on various data sets demonstrate the superiority of our proposed method over the state-of-the-art methods.

SDM3d: shape decomposition of multiple geometric priors for 3D pose estimation

Recovering the 3D human pose from a single image with 2D joints is a challenging task in computer vision applications. The sparse representation (SR) model has been successfully adopted in 3D pose estimation approaches. However, since existing available training 3D data are often collected in a constrained environment (i.e., indoor) with limited diversity of subjects and actions, most SR-based approaches would have a lower generalization to real-world scenarios that may contain more complex cases. To alleviate this issue, this paper proposes SDM3d, a novel shape decomposition using multiple geometric priors for 3D pose estimation. SDM3d makes a new attempt by separating a 3D pose into the global structure and body deformations that are encoded explicitly via different priors constraints. Furthermore, a joint learning strategy is designed to learn two over-complete dictionaries from training data to capture more geometric priors information. We have evaluated SDM3d on four well-recognized benchmarks, i.e., Human3.6M, HumanEva-I, CMU MoCap, and MPII. The experiment results show the effectiveness of SDM3d.

Compressive Sampling with Multiple Bit Spread Spectrum-Based Data Hiding

We propose a novel data hiding method in an audio host with a compressive sampling technique. An over-complete dictionary represents a group of watermarks. Each row of the dictionary is a Hadamard sequence representing multiple bits of the watermark. Then, the singular values of the segment-based host audio in a diagonal matrix are multiplied by the over-complete dictionary, producing a lower size matrix. At the same time, we embed the watermark into the compressed audio. In the detector, we detect the watermark and reconstruct the audio. This proposed method offers not only hiding the information, but also compressing the audio host. The application of the proposed method is broadcast monitoring and biomedical signal recording. We can mark and secure the signal content by hiding the watermark inside the signal while we compress the signal for memory efficiency. We evaluate the performance in terms of payload, compression ratio, audio quality, and watermark quality. The proposed method can hide the data imperceptibly, in the range of 729–5292 bps, with a compression ratio 1.47–4.84, and a perfectly detected watermark.

Single-Spectrum-Driven Binary-Class Sparse Representation Target Detector for Hyperspectral Imagery

In this article, a single-spectrum-driven binary-class sparse representation target detector (SSBSTD) via target and background dictionary construction (BDC) is proposed. The SSBSTD leans upon the binary-class sparse representation (BSR) model. Due to the fact that a background spectrum usually consists in background samples composed low-dimensional subspace and a target spectrum also consists in target samples composed low-dimensional subspace, only background samples should be used for sparsely representing the test pixel under the target absent hypothesis and the samples from target-only dictionary for target present hypothesis. To alleviate the problem that there are insufficient available target samples in the sparse representation model, this article proposed a predetection method to construct the target dictionary utilizing the given target spectrum. With regard to the BDC, we proposed an approach based on the classification to generate a global over-complete background dictionary. The detection output is composed of the residual difference between the BSR. Extensive experiments were made on four benchmark hyperspectral images and the experimental results indicate that our SSBSTD algorithm demonstrates superior detection performances.

Special Object Recognition Based on Sparse Representation in Multisource Data Fusion Samples

Wireless sensor networks (WSNs) suffer from limited power and large amounts of redundant data. This paper describes a multisource data fusion method for WSNs that can be combined with the characteristics of a profile detection system. First, principal component analysis is used to extract sample features and eliminate redundant information. Feature samples from different sources are then fused using a method of superposition to reduce the amount of data transmitted by the network. Finally, a mathematical model is proposed. On the basis of this model, a novel method of special object recognition based on sparse representation is developed for multisource data fusion samples according to the distribution of nonzero coefficients under an overcomplete dictionary. The experimental results from numerical simulations show that the proposed recognition method can effectively identify special objects in the fusion samples, and the overall performance is better than that of traditional methods.

Diffusion-weighted and spectroscopic MRI super-resolution using sparse representations

Diffusion-weighted magnetic resonance imaging (DW-MRI) and spectroscopic MRI (MRSI) are powerful diagnostic imaging tools as they provide complementary information over conventional MRI. Imaging is done at a low-resolution (LR) as the scanning time for high-resolution (HR) MR images would be very long and not practical besides being expensive for imaging. In this paper, we propose a novel single image super-resolution (SISR) scheme to improve spatial resolution of DW and MRS images. It is based on patch-wise sparse reconstruction of HR patches from LR feature patches utilizing a pair of learned overcomplete dictionaries. Reconstruction not only exploits the sparsity of MR image but also utilize the non-local self-similarity of patches of the input LR image as prior knowledge. Experiments are done using magnitude images of DW-MRI and MRSI along with a synthetic image. Performance evaluations based on different matrices besides visual analysis are carried out to validate and compare the obtained results with the state-of-the-art. It is observed that the proposed method clearly outperforms recent methods in terms of both quantitative and visual analysis. Finally, the proposed algorithm is also implemented using the GP-GPU based parallel hardware along with sequential implementations in order to showcase its potential for real clinical applications.

When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition With Limited Data

We present a new Deep Dictionary Learning and Coding Network (DDLCN) for image recognition tasks with limited data. The proposed DDLCN has most of the standard deep learning layers (e.g., input/output, pooling, fully connected, etc.), but the fundamental convolutional layers are replaced by our proposed compound dictionary learning and coding layers. The dictionary learning learns an over-complete dictionary for input training data. At the deep coding layer, a locality constraint is added to guarantee that the activated dictionary bases are close to each other. Then the activated dictionary atoms are assembled and passed to the compound dictionary learning and coding layers. In this way, the activated atoms in the first layer can be represented by the deeper atoms in the second dictionary. Intuitively, the second dictionary is designed to learn the fine-grained components shared among the input dictionary atoms, thus a more informative and discriminative low-level representation of the dictionary atoms can be obtained. We empirically compare DDLCN with several leading dictionary learning methods and deep learning models. Experimental results on five popular datasets show that DDLCN achieves competitive results compared with state-of-the-art methods when the training data is limited. Code is available at https://github.com/Ha0Tang/DDLCN.

Dictionary learning with low mutual coherence constraint

This paper presents efficient algorithms for learning low-coherence dictionaries. First, a new algorithm based on proximal methods is proposed to solve the dictionary learning (DL) problem regularized with the mutual coherence of dictionary. This is unlike the previous approaches that solve a regularized problem where an approximate incoherence promoting term, instead of the mutual coherence, is used to encourage low-coherency. Then, a new solver is proposed for constrained low-coherence DL problem, i.e., a DL problem with an explicit constraint on the mutual coherence of the dictionary. As opposed to current methods, which follow a suboptimal two-step approach, the new algorithm directly solves the associated DL problem. Using previous studies, convergence of the new schemes to critical points of the associated cost functions is also provided. Furthermore, it is shown that the proposed algorithms have lower iteration complexity than existing algorithms. Our simulation results on learning low-coherence dictionaries for natural image patches as well as image classification based on discriminative over-complete dictionary learning demonstrate the superiority of the proposed algorithms compared with the state-of-the-art method.

Adaptive dictionary learning based on local configuration pattern for face recognition

Sparse representation based on classification and collaborative representation based classification with regularized least square has been successfully used in face recognition. The over-completed dictionary is crucial for the approaches based on sparse representation or collaborative representation because it directly determines recognition accuracy and recognition time. In this paper, we proposed an algorithm of adaptive dictionary learning according to the inputting testing image. First, nearest neighbors of the testing image are labeled in local configuration pattern (LCP) subspace employing statistical similarity and configuration similarity defined in this paper. Then the face images labeled as nearest neighbors are used as atoms to build the adaptive representation dictionary, which means all atoms of this dictionary are nearest neighbors and they are more similar to the testing image in structure. Finally, the testing image is collaboratively represented and classified class by class with this proposed adaptive over-completed compact dictionary. Nearest neighbors are labeled by local binary pattern and microscopic feature in the very low dimension LCP subspace, so the labeling is very fast. The number of nearest neighbors is changeable for the different testing samples and is much less than that of all training samples generally, which significantly reduces the computational cost. In addition, atoms of this proposed dictionary are these high dimension face image vectors but not lower dimension LCP feature vectors, which ensures not only that the information included in face image is not lost but also that the atoms are more similar to the testing image in structure, which greatly increases the recognition accuracy. We also use the Fisher ratio to assess the robustness of this proposed dictionary. The extensive experiments on representative face databases with variations of lighting, expression, pose, and occlusion demonstrate that the proposed approach is superior both in recognition time and in accuracy.

Limited-Angle X-Ray CT Reconstruction Using Image Gradient ℓ₀-Norm With Dictionary Learning

Limited-angle X-ray computed tomography (CT) reconstruction is a typical ill-posed problem. To recover satisfied reconstructed images with limited-angle CT projections, prior information is usually introduced into image reconstruction, such as the piece-wise constant, nonlocal image similarity, and so on. To further improve the image quality for limited-angle CT reconstruction, the dictionary learning (DL) and image gradient ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> -norm are combined into image reconstruction model, it can be called as ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> DL reconstruction technique. The advantages of ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> DL can be divided into two aspects. On one hand, the proposed ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> DL method can inherit the advantages of DL in image details preservation and features recovery by exploring an over-complete dictionary. On the other hand, the image gradient ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> -norm minimization can further protect image edges and reduce shadow artifact. Both numerical simulation and preclinical mouse experiments are performed to validate and evaluate the outperformances of proposed ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> DL method by comparing with other state-of-the-art methods, such as total variation (TV) minimization and TV with low rank (TV + LR).

A new K-means singular value decomposition method based on self-adaptive matching pursuit and its application in fault diagnosis of rolling bearing weak fault

Sparse decomposition has excellent adaptability and high flexibility in describing arbitrary complex signals based on redundant and over-complete dictionary, thus having the advantage of being free from the limitations of traditional signal processing methods such as wavelet and fast Fourier transform being imposed by orthogonal basis. Sparse decomposition provides an effective approach for feature extraction of intricate vibration signals collected from rotating machinery. Self-learning over-complete dictionary and pre-defined over-complete dictionary are the two dictionary construction modes of sparse decomposition. Normally, the former mode owns the virtues of much more adaptive and flexible than the latter one, and several kinds of classical self-learning over-complete dictionary methods have been arising in recent years. K-means singular value decomposition is a classical self-learning over-complete dictionary method and has been used in image processing, speech processing, and vibration signal processing. However, K-means singular value decomposition has relative low reconstruction accuracy and poor stability to enhance the desired features. To overcome the above-mentioned shortcomings of K-means singular value decomposition, a new K-means singular value decomposition sparse representation method based on traditional K-means singular value decomposition method was proposed in this article, which uses the sparse adaptive matching pursuit algorithm and an iterative method based on the minimum similarity of atomic structure. The effectiveness and advantage of the proposed method were verified through simulation and experiment.

Mixed image denoising using weighted coding and non-local similarity

Denoising an image is a heuristic and objective process. Still, underlying noise that is predominant in the images reduces the quality. Additive white Gaussian noise (AWGN) and impulse noise are the most exploited types of noise. For a specified amount of density, a combination of AWGN and impulse noise may distract the entire signal causing a loss in the magnitude. This paper presents a denoising model by exploiting such a combination that uses an overcomplete dictionary by sparse based denoising scheme with suitable regularization terms. A weight matrix is defined to optimize the operation at specific locations of the image. Finally, the use of non-local similarity features improves the quality of reconstructed images. The weight matrix maps the regions where the effect of multiple noise sources is present. The results proved the superiority of the proposed technique. Simulation of the proposed technique on many images with different quantities of noise produced an improvement of up to 2 dB when the noise effect is more when compared to the state-of-the-art techniques.