Sparse Representation Framework Research Articles

Sparse Variational Student-t Processes

The theory of Bayesian learning incorporates the use of Student-t Processes to model heavy-tailed distributions and datasets with outliers. However, despite Student-t Processes having a similar computational complexity as Gaussian Processes, there has been limited emphasis on the sparse representation of this model. This is mainly due to the increased difficulty in modeling and computation compared to previous sparse Gaussian Processes. Our motivation is to address the need for a sparse representation framework that reduces computational complexity, allowing Student-t Processes to be more flexible for real-world datasets. To achieve this, we leverage the conditional distribution of Student-t Processes to introduce sparse inducing points. Bayesian methods and variational inference are then utilized to derive a well-defined lower bound, facilitating more efficient optimization of our model through stochastic gradient descent. We propose two methods for computing the variational lower bound, one utilizing Monte Carlo sampling and the other employing Jensen's inequality to compute the KL regularization term in the loss function. We propose adopting these approaches as viable alternatives to Gaussian processes when the data might contain outliers or exhibit heavy-tailed behavior, and we provide specific recommendations for their applicability. We evaluate the two proposed approaches on various synthetic and real-world datasets from UCI and Kaggle, demonstrating their effectiveness compared to baseline methods in terms of computational complexity and accuracy, as well as their robustness to outliers.

Enhanced Micro-Doppler Feature Extraction Using Adaptive Short-Time Kernel-Based Sparse Time-Frequency Distribution

The extraction of the micro-Doppler (m-D) feature based on time-frequency distribution (TFD) is of great significance for target detection and identification. To improve the feature extraction performance, numerous TFDs have been developed, with the majority falling under Cohen’s class. Nevertheless, these TFDs basically face a trade-off between artifact suppression and energy concentration. The main reason is that each Cohen’s class TFD is constructed by applying the two-dimensional Fourier transform to a kerneled ambiguity function directly, while existing kernels generally attenuate artifacts at the expense of losing valuable information. In this paper, a TFD reconstruction method employing an adaptive short-time kernel (ASTK) is developed in the framework of sparse representation (SR) theory to overcome this trade-off and enhance the m-D feature. Firstly, the task of the optimal kernel is explained from the viewpoint of the instantaneous auto-correlation function (IAF). Secondly, based on the quasi-linear frequency modulation feature of most m-D signals during short-time periods, the distribution rule of the short-time IAF (STIAF) in the ambiguity plane is concluded. Guided by this rule, an ASTK that can effectively remove unwanted artifacts with the least information loss is designed. Finally, an SR-based reconstruction procedure is conducted on the kerneled STIAF to generate an artifact-free TFD with high energy concentration, which can effectively enhance the m-D feature. Experiments using both simulated and real-world m-D signals demonstrate the effectiveness of the proposed method.

A unified kernel sparse representation framework for supervised learning problems

A unified kernel sparse representation framework for supervised learning problems

Compressive nonstationary near-field acoustic holography for reconstructing the instantaneous sound field

A compressive nonstationary near-field acoustic holography based on time-domain plane wave superposition method is proposed to precisely reconstruct the instantaneous sound field using the fewer measurement points. In the proposed method, by means of an instantaneous propagation kernel, a time-domain convolution superposition formula is established for relating the time-variant pressure of the hologram plane to the pressure time-wavenumber spectra of the virtual source plane. Next, compressed sensing theory and sparse representation framework are introduced into the time-marching solving inverse process of the established formula, and the smoothed l0 norm and revised Newton optimization algorithm are employed to solve the serious cumulative ill-posed problem over time for acquiring the pressure time-wavenumber spectra. The key is to approximately replace the discontinuous ℓ0 norm by using a suitable continuous function, and the revised Newton algorithm is applied to minimize the continuous function for obtaining the optimal solution. Finally, with the aid of a time-domain convolution formula, the solved pressure time-wavenumber spectra are employed to calculate the time-variant pressure on the reconstruction plane. The proposed method not only can evidently suppress the ill-posed problem of time-marching solving inverse process for improving the reconstruction accuracy, but also can greatly reduce the measurement points and microphone number for drastically decreasing the measurement cost on the premise of ensuring the reconstruction accuracy and spatial resolution. A numerical simulation is conducted, and the simulation effects prove that the proposed method can accurately reconstruct the instantaneous sound field with the fewer measurement points in time–space domains. The reconstruction results are also compared to those of time-domain plane wave superposition method with Tikhonov regularization to verify the superiority of the proposed method with fewer measurement points under different parameter configurations. An experiment of an impaction steel plate is implemented for further validating the competence of the proposed method.

A sparse dictionary representation approach for anti-scattering grid artifact removal in X-ray images

The aim of this work is to introduce a new joint estimation approach for anatomical content and artifacts in digital radiography. Herein, we apply this approach to the anti-scattering grid suppression problem. These grids are generally used in low-dose radiography to improve the image contrast by absorbing the scattered radiation. However, they introduce an irregular stripe pattern in the clinical images which can hinder the diagnosis made by the radiologist.The problem is tackled in a sparse dictionary representation framework with an observation and prior models accounting for the physical model of image contents. Thus, the anatomical content is supposed to be sparsely represented by a Fourier or K-SVD dictionary. Furthermore, the artifact is modeled by a frequency dictionary. In addition, the noise is modeled by a locally stationary Gaussian distribution with zero mean and unknown variance. The joint estimation of all the projections is then performed using two methods: Beam-forming (IAA) and Bayesian (SUBLIM). Moreover, patch-wise processing is applied to avoid models’ errors due to spatial variations of the clinical contents and the artifact.IAA and SUBLIM performance is demonstrated for the anti-scattering grid extraction problem using experimental and real clinical images. The proposed methods remove gridline artifacts while maintaining the image details and avoiding ringing artifacts. In addition, they produce better results compared to the conventional filtering approaches, especially when the metallic materials are present in or on the patients. SUBLIM outperforms IAA in those regions thanks to a better clinical content model and physical noise prior.

MRI Image Fusion Based on Sparse Representation with Measurement of Patch-Based Multiple Salient Features

Multimodal medical image fusion is a fundamental, but challenging, problem in the fields of brain science research and brain disease diagnosis, as it is challenging for sparse representation (SR)-based fusion to characterize activity levels with a single measurement and not lose effective information. In this study, the Kronecker-criterion-based SR framework was applied for medical image fusion with a patch-based activity level, integrating salient features of multiple domains. Inspired by the formation process of vision systems, the spatial saliency was characterized by textural contrast (TC), composed of luminance and orientation contrasts, to promote the participation of more highlighted textural information in the fusion process. As a substitute for the conventional l1-norm-based sparse saliency, the sum of sparse salient features (SSSF) was used as a metric for promoting the participation of more significant coefficients in the composition of the activity level measurement. The designed activity level measurement was verified to be more conducive to maintaining the integrity and sharpness of detailed information. Various experiments on multiple groups of clinical medical images verified the effectiveness of the proposed fusion method in terms of both visual quality and objective assessment. Furthermore, this study will be helpful for the further detection and segmentation of medical images.

An iteratively reweighted time-domain acoustic method for reconstructing the transient acoustic field

An improved iterative reweighted least-squares based on real-time near-field acoustic hologram method (RTNAH) is proposed to reconstruct the transient acoustic field. First, time-domain convolution relation formula between the pressure time-wavenumber spectra on the hologram and reconstruction planes is first established by an impulse response function, and the corresponding matrix equation is obtained. Then, for solving the pressure spectra on the reconstruction plane, an improved iterative reweighted least-squares algorithm is applied for solving the minimization problem of the sum of 2-norm residual and 2-norm weighted solution under the weighted and sparse representation framework. By controlling the iteratively updated weighting matrix, the sparsity of the solution is promoted. Finally, the pressure spectra of the reconstruction plane are estimated, and the corresponding time-varying pressures are acquired by the two-dimensional inverse Fourier transform. A numerical simulation is conducted to observe the reconstruction capacity of the proposed method. The simulation results demonstrate that the proposed method can reconstruct the transient acoustic field in time-space domains very well. Meanwhile, several vital parameters are discussed, and the superiorities of the proposed method are testified by comparing to RTNAH with Tikhonov regularization and YALL1 model. An experiment is implemented for further validating the effectivity of the proposed method.

Deep Multiple Instance Learning Approach for Classification in Clinical Decision Support Systems

To get around the drawbacks of conventional classification algorithms that required manual feature extraction and the high computational cost of neural networks, this paper introduces a deep convolutional neural network with multiple instance learning approaches, namely dynamic max pooling and sparse representation. For the categorization of tuberculosis lung illness, this model combines deep convolutional neural networks and multiple instance learning. The design was composed of four phases: pre-processing, instance production, feature extraction, and classification. To perform feature extraction, a model based on a customized version of the VGG16 architecture was trained from scratch. Multiple instance learning techniques such as Diverse Density (DD) and the Maximum pattern bag formulation of the Support Vector Machine were used to evaluate how well the proposed classification algorithm performed in comparison (SVM).The numerical findings demonstrated that the new method offered a higher level of accuracy than the methods that had been used in the past. When evaluating the efficacy of the current method, accuracy, specificity, sensitivity, and error rate were all taken into consideration. The accuracy of the max-pooling based framework and the sparse representation framework was found to be greater than that of the other multiple instance strategies, coming in at 91.51% and 89.84%, respectively, when compared to that of the other methods. The improved accuracy of the present system that makes use of deep neural networks is mostly attributable to the contributions made by features such as transfer learning and automatic feature extraction.

Consensus Sparsity: Multi-context Sparse Image Representation via L∞-induced Matrix Variate.

The sparsity is an attractive property that has been widely and intensively utilized in various image processing fields (e.g., robust image representation, image compression, image analysis, etc.). Its actual success owes to the exhaustive mining of the intrinsic (or homogenous) information from the whole data carrying redundant information. From the perspective of image representation, the sparsity can successfully find an underlying homogenous subspace from a collection of training data to represent a given test sample. The famous sparse representation (SR) and its variants embed the sparsity by representing the test sample using a linear combination of training samples with L0-norm regularization and L1-norm regularization. However, although these state-of-the-art methods achieve powerful and robust performances, the sparsity is not fully exploited on the image representation in the following three aspects: 1) the within-sample sparsity, 2) the between-sample sparsity, and 3) the image structural sparsity. In this paper, to make the above-mentioned multi-context sparsity properties agree and simultaneously learned in one model, we propose the concept of consensus sparsity (Con-sparsity) and correspondingly build a multi-context sparse image representation (MCSIR) framework to realize this. We theoretically prove that the consensus sparsity can be achieved by the L∞-induced matrix variate based on the Bayesian inference. Extensive experiments and comparisons with the state-of-the-art methods (including deep learning) are performed to demonstrate the promising performance and property of the proposed consensus sparsity.

Depth Map Super-Resolution via Joint Local Gradient and Nonlocal Structural Regularizations

Depth maps have been widely used in many real world applications, such as human-computer interaction and virtual reality. However, due to the limitation of current depth sensing technology, the captured depth maps usually suffer from low resolution and insufficient quality. In this paper, we propose a depth map super-resolution method via joint local gradient and nonlocal structural regularizations. Depth maps contain mainly smooth areas separated by textures which demonstrate distinct geometry direction characteristic. Motivated by this, we classify depth map patches according to their geometrical directions and learn a compact online dictionary in each class. We further introduce two regularization terms into the sparse representation framework. Firstly, a multi-directional total variation model is proposed to characterize the local patterns in the gradient domain. Secondly, a nonlocal autoregressive model is introduced to provide nonlocal constraint to the local structures, which can effectively restore image details and suppress noise. Quantitative and qualitative evaluations compared with state-of-the-art methods demonstrate that the proposed method achieves superior performance for various configurations of magnification factors and datasets.

SRNet: Sparse representation-based network for image denoising

Image denoising is a foundational but important task in image processing. Various traditional methods have been proposed to remove noise from noisy images. Sparse representation (SR) is a representative image denoising method. Although with good denoising effects, sparse representation suffers from performance bottlenecks and large time consumption. Recently, deep learning has demonstrated excellent ability in image denoising. Therefore, we consider combining the sparse representation with deep learning to make this traditional model more effective and efficient. The sparse representation-based network (SRNet) is proposed by embedding the convolutional neural network (CNN) into the sparse representation framework, in which all the parameters are learned by training. The iterative optimization process of sparse coding is unrolled into a network with several similar phases. In each phase, two subnetworks are designed with the multiscale residual block (MSR-block) to model the updating of sparse coefficient and image, respectively. The experimental results show that compared with the traditional sparse representation denoising methods, the proposed method can significantly reduce time consumption and improve denoising performance, especially in terms of reconstructing textures. The PyTorch code and models of the proposed SRNet can be released at https://github.com/JiechaoSheng/SRNet.

A sparse representation strategy to eliminate pseudo-HFO events from intracranial EEG for seizure onset zone localization

Objective. High-frequency oscillations (HFOs) are considered a biomarker of the epileptogenic zone in intracranial EEG recordings. However, automated HFO detectors confound true oscillations with spurious events caused by the presence of artifacts. Approach. We hypothesized that, unlike pseudo-HFOs with sharp transients or arbitrary shapes, real HFOs have a signal characteristic that can be represented using a small number of oscillatory bases. Based on this hypothesis using a sparse representation framework, this study introduces a new classification approach to distinguish true HFOs from the pseudo-events that mislead seizure onset zone (SOZ) localization. Moreover, we further classified the HFOs into ripples and fast ripples by introducing an adaptive reconstruction scheme using sparse representation. By visualizing the raw waveforms and time-frequency representation of events recorded from 16 patients, three experts labeled 6400 candidate events that passed an initial amplitude-threshold-based HFO detector. We formed a redundant analytical multiscale dictionary built from smooth oscillatory Gabor atoms and represented each event with orthogonal matching pursuit by using a small number of dictionary elements. We used the approximation error and residual signal at each iteration to extract features that can distinguish the HFOs from any type of artifact regardless of their corresponding source. We validated our model on sixteen subjects with thirty minutes of continuous interictal intracranial EEG recording from each. Main results. We showed that the accuracy of SOZ detection after applying our method was significantly improved. In particular, we achieved a 96.65% classification accuracy in labeled events and a 17.57% improvement in SOZ detection on continuous data. Our sparse representation framework can also distinguish between ripples and fast ripples. Significance. We show that by using a sparse representation approach we can remove the pseudo-HFOs from the pool of events and improve the reliability of detected HFOs in large data sets and minimize manual artifact elimination.

The high‐resolution seismic deconvolution method based on joint sparse representation using logging–seismic data

ABSTRACTSeismic high‐resolution processing is an essential part of seismic processing. Sparse‐spike deconvolution is a widely used method for improving the resolution of seismic data. However, the stratigraphic reflection coefficients do not fully satisfy the hypothesis of sparse‐spike deconvolution, and this method does not make full use of prior information, such as well‐logging data. In this paper, we have developed a high‐resolution processing method based on joint sparse representation using logging and seismic data. This method can extract stratigraphic information from well‐logging reflection coefficients and observational seismic data at the same location by joint dictionary learning. Through joint sparse representation, the relationship between observed seismic data and the reflection coefficient is established. Under the framework of joint sparse representation, the deconvolution of seismic data is realized. The synthetic data and field data tests show that our method can reveal thin layers and can invert reflection coefficients from strongly noisy seismic data accurately. Moreover, the deconvolution results of our method match well with the well‐logging data. The tests demonstrate that the improvement of accuracy of deconvolution results with our method, compared to sparse‐spike deconvolution.

Elimination of pseudo-HFOs in iEEG using sparse representation and Random Forest classifier.

High-Frequency Oscillation (HFO) is a promising biomarker of the epileptogenic zone. However, sharp artifacts might easily pass the conventional HFO detectors as real HFOs and reduce the seizure onset zone (SOZ) localization. We hypothesize that, unlike pseudo-HFOs, which originates from artifacts with sharp changes or arbitrary waveform characteristic, real HFOs could be represented by a limited number of oscillatory waveforms. Accordingly, to distinguish true ones from pseudo-HFOs, we established a new classification method based on sparse representation of candidate events that passed an initial detector with high sensitivity but low specificity. Specifically, using the Orthogonal Matching Pursuit (OMP) and a redundant Gabor dictionary, each event was represented sparsely in an iterative fashion. The approximation error was estimated over 30 iterations which were concatenated to form a 30-dimensional feature vector and fed to a random forest classifier. Based on the selected dictionary elements, our method can further classify HFOs into Ripples (R) and Fast Ripples (FR). In this scheme, two experts visually inspected 2075 events captured in iEEG recordings from 5 different subjects and labeled them as true-HFO or Pseudo-HFO. We reached 90.22% classification accuracy in labeled events and a 21.16% SOZ localization improvement compared to the conventional amplitude-threshold-based detector. Our sparse representation framework also classified the detected HFOs into R and FR subcategories. We reached 91.24% SOZ accuracy with the detected [Formula: see text] events. Clinical Relevance---This sparse representation framework establishes a new approach to distinguish real from pseudo-HFOs in prolonged iEEG recordings. It also provides reliable SOZ identification without the selection of artifact-free segments.

Denoising Marine Controlled Source Electromagnetic Data Based on Dictionary Learning

Marine controlled source electromagnetic (CSEM) is an efficient method to explore ocean resources. The amplitudes of marine CSEM signals decay rapidly with the measuring offsets. The signal is easily contaminated by various kinds of noise when the offset is large. These noise include instrument internal noise, dipole vibration noise, seawater motion noise and environmental noise Suppressing noise is the key to improve data quality and interpretation accuracy. Sparse representation based denoising method has been used for denoising for a long time. provides a new way to remove noise. Under the framework of sparse representation, the denoising effect is closely related to the chosen transform matrix. This matrix is called dictionary and its column named atom. In general, the stronger the correlation between signal and dictionary is, the sparser representation will be, and further the better the denoising effect will be. In this article, a new method based on dictionary learning is proposed for marine CSEM denoising. Firstly, the signal segments suffering little from noise are captured to compose the training set. Then the learned dictionary is trained from the training set via K-singular value decomposition (K-SVD) algorithm. Finally, the learned dictionary is used to sparsely represent the contaminated signal and reconstruct the filtered one. The effectiveness of the proposed approach is verified by a synthetic data denoising experiment, in which windowed-Fourier-transform (WFT) and wavelet-transform (WT) denoising methods and three dictionaries (discrete-sine-transform (DST) dictionary, DST-wavelet merged dictionary and the learned dictionary) under a sparse representation framework are tested. The results demonstrate the superiority of the proposed dictionary-learning-based denoising method. Finally, the proposed approach is applied to field data denoising process, coupled with DST and DST-wavelet dictionaries based denoising methods. The outcomes further proves that the propsoed approach is effective and superior for marine CSEM data denoising.

Localisation and classification of mixed far‐field and near‐field sources with sparse reconstruction

A sparse reconstruction algorithm for the localisation of mixed near-field and far-field sources (MFNS) based on four-order statistics is proposed in this study. First, utilising the structural characteristics of a uniform symmetric linear array, a fourth-order cumulant (FOC) matrix is constructed, which decouples the angular information from the range parameters. Based on the sparse representation framework, a weighted l1-norm minimisation algorithm is developed to obtain the direction of arrivals (DOAs) of the MFNS. However, the existing selection strategy of the tuning factor is not adaptive to different observation scenarios. So a closed-form expression of the tuning factor based on the FOC estimation error is presented. Then, another FOC matrix is constructed, which includes both the DOA and range information of the MFNS. With the DOA estimates, the two-dimensional spatial dictionary can be reduced into a one-dimensional dictionary, which only depends on the range parameters. Using the similar sparse reconstruction method, the range estimates of the MFNS can be obtained, and the types of the sources can be distinguished according to their range parameters. According to numerical simulations, the estimation performance of the proposed algorithm approaches the CRB in the high signal-to-noise ratio region, which successfully circumvents the saturation problem due to the fixed tuning factor.

Unit-norm tight frame-based sparse representation with application to speech inpainting

This paper deals with the unit-norm tight frames (UNTFs) that better represent and solve sparse representation problems. Under this sparse representation framework, a novel model termed tight dictionary learning (TDL) is proposed. Unlike those dictionaries that only focus on the signals of interest, TDL intends to learn a more comprehensive dictionary that better represents the signals with the UNTF properties to facilitate sparse recovery. To better describe the physical meanings, the normalization constraints on the dictionary columns are imposed. A gradient-based approach is developed to obtain the tight dictionary where the normalization constraints are embedded into the cost function by a parametrization method. The learned tight dictionary is applied to the speech inpainting problem to restore speech signals corrupted by data missing. In order to compensate the degradation effect of the data missing, a UNTF-based preconditioning technique is developed to reform the frame properties of the degraded system. Parametrization and gradient-based approaches similar to those applied in the TDL are adopted to design a tight preconditioner. Extensive experiments on speech denoising and speech segment restoration are conducted to demonstrate the superiority of the proposed schemes.

Sparse Representation-Based Discriminative Metric Learning for Brain MRI Image Retrieval.

Magnetic resonance imaging (MRI) can have a good diagnostic function for important organs and parts of the body. MRI technology has become a common and important disease detection technology. At the same time, medical imaging data is increasing at an explosive rate. Retrieving similar medical images from a huge database is of great significance to doctors’ auxiliary diagnosis and treatment. In this paper, combining the advantages of sparse representation and metric learning, a sparse representation-based discriminative metric learning (SRDML) approach is proposed for medical image retrieval of brain MRI. The SRDML approach uses a sparse representation framework to learn robust feature representation of brain MRI, and uses metric learning to project new features into the metric space with matching discrimination. In such a metric space, the optimal similarity measure is obtained by using the local constraints of atoms and the pairwise constraints of coding coefficients, so that the distance between similar images is less than the given threshold, and the distance between dissimilar images is greater than another given threshold. The experiments are designed and tested on the brain MRI dataset created by Chang. Experimental results show that the SRDML approach can obtain satisfactory retrieval performance and achieve accurate brain MRI image retrieval.

Incremental Dictionary Learning for Multiframe Satellite Image Representation via Gradual Optimization

Dictionary learning has been widely used in image representation under the framework of sparse theory. But most of the current dictionary learning strategies can only be used for single-frame image separately, which are insufficient from the perspective of incremental information acquisition and global optimization for the sequential or multiframe satellite images. To this end, this paper proposes an incremental dictionary learning method for multiframe satellite images representation in the spectral domain. The incremental dictionary learning is formulated analytically in the framework of sparse representation with low-rank constraint, as a frame-by-frame gradual optimization process of global and local dictionaries, and their corresponding sparse coefficients with the sequence. Specifically, the global dictionary representing the common spectral information of the sequential frames, is optimized by two adjacent frames gradually. Meanwhile, the local dictionary representing the specific spectral information of each frame, is optimized by the newly added frame itself. In addition, an activity ratio for separating the global dictionary from the local dictionaries, an outlier detection method for initializing the local dictionary are also given, and the alternating direction methods of multipliers (ADMM) is employed to implement the above optimization. Comparison results with the related state-of-the-art methods on different datasets demonstrate that, our proposed method achieves the best representation performance in both spatial and spectral domains, and also helps to improve the performance of dictionary-based tasks using sequential satellite images, such as sea surface anomaly detection.

Functional Subdivisions of the Cerebellum in Naturalistic Paradigm Functional Magnetic Resonance Imaging.

Compelling evidence has suggested that the human cerebellum is engaged in a wide range of cognitive tasks besides traditional opinions of motor control, and it is organized into a set of distinct functional subregions. The existing model-driven cerebellum parcellations through resting-state functional MRI (rsfMRI) and task-fMRI are relatively coarse, introducing challenges in resolving the functions of the cerebellum especially when the brain is exposed to naturalistic environments. The current study took the advantages of the naturalistic paradigm (i.e., movie viewing) fMRI (nfMRI) to derive fine parcellations via a data-driven dual-regression-like sparse representation framework. The parcellations were quantitatively evaluated by functional homogeneity, and global and local boundary confidence. In addition, the differences of cerebellum–cerebrum functional connectivities between rsfMRI and nfMRI for some exemplar parcellations were compared to provide qualitatively functional validations. Our experimental results demonstrated that the proposed study successfully identified distinct subregions of the cerebellum. This fine parcellation may serve as a complementary solution to existing cerebellum parcellations, providing an alternative template for exploring neural activities of the cerebellum in naturalistic environments.