Sparse Representation Research Articles

Learning a more compact representation for low-rank tensor completion

Transform-based tensor nuclear norm (TNN) methods have gained considerable attention for their effectiveness in addressing tensor recovery challenges. The integration of deep neural networks as nonlinear transforms has been shown to significantly enhance their performance. Minimizing transform-based TNN is equivalent to minimizing the ℓ1 norm of singular values in the transformed domain, which can be interpreted as finding a sparse representation with respect to the bases supported by singular vectors. This work aims to advance deep transform-based TNN methods by identifying a more compact representation through learnable bases, ultimately improving recovery accuracy. We specifically employ convolutional kernels as these learnable bases, demonstrating their capability to generate more compact representation, i.e., sparser coefficients of real-world tensor data compared to singular vectors. Our proposed model consists of two key components: a transform component, implemented through fully connected neural networks (FCNs), and a convolutional component that replaces traditional singular matrices. Then, this model is optimized using the ADAM algorithm directly on the incomplete tensor in a zero-shot manner, meaning all learnable parameters within the FCNs and convolution kernels are inferred solely from the observed data. Experimental results indicate that our method, with this straightforward yet effective modification, outperforms state-of-the-art approaches on video and multispectral image recovery tasks.

Adaptive sparsity detection-based evolutionary algorithm for large-scale sparse multi-objective optimization problems

Adaptive sparsity detection-based evolutionary algorithm for large-scale sparse multi-objective optimization problems

Sparse joint representation for massive MIMO satellite uplink and downlink based on dictionary learning

AbstractWe address the challenge of jointly representing uplink (UL) and downlink (DL) channels for a massive multiple‐input multiple‐output satellite system. We employ dictionary learning for sparse representation with the goal of minimizing the number of UL/DL pilots and improving accuracy. Additionally, by considering the angular reciprocity, a common dictionary support can be established to enhance the performance. However, what type of dictionary model is suited for UL/DL channel representation remains an unknown field. Previous research has utilized predefined dictionaries, such as DFT or ODFT bases, which are unable to adapt to dynamic scenarios. Training dictionaries have demonstrated the potential to significantly improve accuracy; however, a lack of analysis regarding dictionary constraints exists. To address this issue, we analyze the conditional constraints of the dictionary for joint UL/DL channel representation, aiming to quantify the maximum boundary while proposing a constrained dictionary learning algorithm with singular value decomposition to obtain an effective representation and conduct an adaptability analysis in dynamic satellite communication scenarios.

A multi-dictionary fusion method based on improved dynamic time warping for moveable rail damage localization

Abstract Traditional structural health monitoring (SHM) of rails relies on a fixed single sensor, limited by detection range and noise interference. Therefore, a multi-dictionary fusion method for movable rail damage localization is proposed based on improved dynamic time warping (DTW). The approach combines onboard acoustic emission sensors with peak detection frames to measure the moving distance of the inspection wheels and monitor a wide range of rails. Aiming to enhance the damage information, an innovative DTW-based multi-dictionary sparse representation algorithm is presented for data fusion. The second-order difference of the Mahalanobis distance is employed to optimize the fusion weights from the global property. A two-feature adaptive threshold is designed to precisely detect and localize damage on rails. The effectiveness of this method is verified at laboratory testing speeds less than 0.75 m s−1. The results demonstrate that it can accurately detect 2 mm deep strip and square damage, providing new inspiration for rail SHM.

On the Efficacy of Sparse Representation Approaches for Determining Nonlinear Structural System Equations of Motion

Abstract A sparsity-based optimization approach is presented for determining the equations of motion of stochastically excited nonlinear structural systems. This is done by utilizing measured excitation-response realizations in the formulation of the related optimization problem, and by considering a library of candidate functions for representing the system governing dynamics. Note that a novel aspect of the approach relates to treating, also, systems endowed with fractional derivative elements. Clearly, this is of significant importance to a multitude of diverse applications in engineering mechanics taking into account the enhanced modeling capabilities of fractional calculus. Further, the fundamental theoretical and computational aspects of various representative, state-of-the-art, numerical schemes for solving the derived sparsity-based optimization problem are reviewed and discussed. A Bayesian compressive sampling approach that exhibits the additional advantage of quantifying the uncertainty of the estimates is considered as well. Furthermore, comparisons and a critical assessment of the employed numerical schemes are provided with respect to their efficacy in determining the nonlinear structural system equations of motion. In this regard, two illustrative numerical examples are considered pertaining to a nonlinear tuned mass-damper–inerter vibration control system and to a nonlinear electromechanical energy harvester, both endowed with fractional derivative elements.

A Deep Shrinkage Network for Direction-of-Arrival Estimation with Sparse Prior

Direction-of-arrival (DOA) estimation of underwater multipath signals plays a indispensable role in both military and civilian underwater applications. Despite its importance, accurately estimating DOA under multipath conditions is challenging due to the proximity of paths in the spatial domain. Current methods struggle with this problem in passive detection scenarios. To address these limitations, this study proposes a deep learning (DL)-based DOA estimation framework leveraging sparse representation. First, the approach models the array covariance matrix as an undersampled linear measurement of the spatial spectrum. Then, a super-resolution deep shrinkage reconstruction network (SDSR-Net) is designed to map the sparse representation of the covariance matrix directly to the DOA. The network integrates a shrinkage module as nonlinear transformation layers, promoting sparsity and enhancing the discrimination of features. Simulations and experimental evaluations validated the effectiveness of the proposed method, showing that the DOA estimation accuracy was significantly improved and able to achieve a resolution of 0.2° in the spatial spectrum. Compared with existing methods, SDSR-Net achieved superior performance by effectively utilizing a sparsity prior, maintaining a high-resolution performance at signal-to-noise ratios higher than −10 dB. This work contributes a robust and efficient solution to DOA estimation challenges in underwater environments.

Are Bayesian regularization methods a must for multilevel dynamic latent variables models?

Due to the increased availability of intensive longitudinal data, researchers have been able to specify increasingly complex dynamic latent variable models. However, these models present challenges related to overfitting, hierarchical features, non-linearity, and sample size requirements. There are further limitations to be addressed regarding the finite sample performance of priors, including bias, accuracy, and type I error inflation. Bayesian estimation provides the flexibility to treat these issues simultaneously through the use of regularizing priors. In this paper, we aim to compare several Bayesian regularizing priors (ridge, Bayesian Lasso, adaptive spike-and-slab Lasso, and regularized horseshoe). To achieve this, we introduce a multilevel dynamic latent variable model. We then conduct two simulation studies and a prior sensitivity analysis using empirical data. The results show that the ridge prior is able to provide sparse estimation while avoiding overshrinkage of relevant signals, in comparison to other Bayesian regularization priors. In addition, we find that the Lasso and heavy-tailed regularizing priors do not perform well compared to light-tailed priors for the logistic model. In the context of multilevel dynamic latent variable modeling, it is often attractive to diversify the choice of priors. However, we instead suggest prioritizing the choice of ridge priors without extreme shrinkage, which we show can handle the trade-off between informativeness and generality, compared to other priors with high concentration around zero and/or heavy tails.

Comparison of Sparse Representation Methods for Complex Data Based on the Smoothed L0 Norm and Modified Minimum Fuel Neural Network

This article outlines the results of comparison methods for representing complex data based on a redundant basis using the L0 norm and analyses the method of a modified MFNN (minimum fuel neural network) and the sparse representation method for the complex-data SL0 (smoothed L0 norm), based on the smoothed L0 norm. The example of numerical modeling for determining the direction of arrival (DOA) of sources received by an equidistant antenna array (ULA—Uniform Linear Array) shows that the SL0 method ensures a high convergence rate. However, unlike the MFNN-like neural network method, it does not guarantee convergence to the correct solution.

QoS-Aware cloud security using lightweight EfficientNet with Adaptive Sparse Bayesian Optimization

QoS-Aware cloud security using lightweight EfficientNet with Adaptive Sparse Bayesian Optimization

Sparse Wasserstein Barycenters and Application to Reduced Order Modeling

We develop a general theoretical and algorithmic framework for sparse approximation and structured prediction in P2(Ω) with Wasserstein barycenters. The barycenters are sparse in the sense that they are computed from an available dictionary of measures, but the approximations only involve a reduced number of atoms. We show that the best reconstruction from the class of sparse barycenters is characterized by a notion of best n-term barycenter which we introduce, and which can be understood as a natural extension of the classical concept of best n-term approximation in Banach spaces. We show that the best n-term barycenter is the minimizer of a highly non-convex, bi-level optimization problem, and we develop algorithmic strategies for practical numerical computation. We next leverage this approximation tool to build interpolation strategies that involve a reduced computational cost, and that can be used for structured prediction, and metamodeling of parametrized families of measures. We illustrate the potential of the method through the specific problem of Model Order Reduction (MOR) of parametrized PDEs. Since our approach is sparse, adaptive and preserves mass by construction, it has potential to overcome known bottlenecks of classical linear methods in hyperbolic conservation laws transporting discontinuities. It also paves the way towards MOR for measure-valued PDE problems such as gradient flows.

Iterative mix thresholding algorithm with continuation technique for mix sparse optimization and application

Iterative mix thresholding algorithm with continuation technique for mix sparse optimization and application

Contrastive independent subspace analysis network for multi-view spatial information extraction.

Multi-view classification integrates features from different views to optimize classification performance. Most of the existing works typically utilize semantic information to achieve view fusion but neglect the spatial information of data itself, which accommodates data representation with correlation information and is proven to be an essential aspect. Thus robust independent subspace analysis network, optimized by sparse and soft orthogonal optimization, is first proposed to extract the latent spatial information of multi-view data with subspace bases. Building on this, a novel contrastive independent subspace analysis framework for multi-view classification is developed to further optimize from spatial perspective. Specifically, contrastive subspace optimization separates the subspaces, thereby enhancing their representational capacity. Whilst contrastive fusion optimization aims at building cross-view subspace correlations and forms a non overlapping data representation. In k-fold validation experiments, MvCISA achieved state-of-the-art accuracies of 76.95%, 98.50%, 93.33% and 88.24% on four benchmark multi-view datasets, significantly outperforming the second-best method by 8.57%, 0.25%, 1.66% and 5.96% in accuracy. And visualization experiments demonstrate the effectiveness of the subspace and feature space optimization, also indicating their promising potential for other downstream tasks. Our code is available at https://github.com/raRn0y/MvCISA.

Near-Field Clutter Mitigation in Speckle Tracking Echocardiography.

Near-field (NF) clutter filters are critical for unveiling true myocardial structure and dynamics. Randomized singular value decomposition (rSVD) stands out for its proven computational efficiency and robustness. This study investigates the effect of rSVD-based NF clutter filtering on myocardial motion estimation. In silico, material points and their displacements in a homogeneous medium under uniaxial compressions (0.5% - 9% axial strains at 0.5% increments) were simulated in finite-element models. They were exported to the k-Wave toolbox for simulations of pre- and post-deformed ultrasound images with/ without a realistic phase aberrating layer in a high-contrast diverging wave compounding scheme. In vivo, echocardiograms of 20 normal human hearts were acquired using a coded diverging wave compounding imaging method at 3200 frames/second in the transthoracic apical four-chamber view. Morphological component analysis (MCA), which is also a sparse representation method but computationally intensive, was used for comparison with rSVD. Both rSVD- and MCA-based filters were applied to beamformed ultrasound radio-frequency (RF) data before cross-correlation-based speckle tracking. Contrast-to-noise ratios (CNRs) and root-mean-square deviations (RMSDs) were computed from regions of interest to evaluate NF clutter filtering performance of rSVD and MCA. In silico, 2-D displacements estimated from rSVD-based clutter-reduced image data showed strong agreement with ground truth (R2 of 0.95). In vivo, CNR improvements ranged from 1.02 dB to 17.68 dB, consistently enhancing image quality across all subjects. An improvement of ∼4.9 dB in the apical segments was observed in 80% of cases. Mean RMSDs were below 5.0% for all rSVD-based NF clutter-reduced data. While both rSVD and MCA effectively filtered NF clutter, rSVD was significantly more practical. Our findings confirm the reliability, accuracy, and efficiency of rSVD-based clutter filtering in speckle tracking echocardiography. This underscores the feasibility of matrix decomposition-based methods, exemplified by rSVD, in NF clutter filtering for myocardial motion estimation.

Machine Learning for Sparse Nonlinear Modeling and Control

Machine learning is rapidly advancing nearly every field of science and engineering, and control theory is no exception. In particular, it has shown incredible promise for handling several of the main challenges facing modern dynamics and control, including complexity, unmodeled dynamics, strong nonlinearity, and hidden variables. However, machine learning models are often expensive to train and deploy, fail to generalize beyond the training data, and suffer from a lack of explainability, interpretability, and guarantees, all of which limit their use in real-world and safety-critical control applications. Sparse nonlinear modeling and control techniques are a powerful class of machine learning that promote parsimony through sparse optimization, providing data-efficient models that are more interpretable and generalizable and have proven effective for control. In this review, we explore the use of sparse optimization in the context of machine learning to develop compact models and controllers that are easy to train, require significantly less data, and make low-latency predictions. In particular, we focus on applications in model predictive control and reinforcement learning, two of the foundational algorithms in control theory.

Reconstruction of Water Entry Cavity by OECT through combining Field Transformation and Group-based Sparse Representation

Abstract Due to the fact that a water entry cavity directly affects the hydrodynamic characteristics of underwater vehicles, its monitoring is crucial for controlling the motion stability of the vehicles. However, there is limited research on the visualization and measurement of cavities during vehicle motion. Based on Open-Electrical Capacitance Tomography (OECT), this article presents an image reconstruction method to measure the water entry cavity by combining field transformation and group-based sparse representation (FT-GSR). To address the openness of the cavity measurement domain, a field transformation method is proposed that unifies open- and closed-domain image reconstruction while reducing the dimensionality of the unknowns to be resolved and the computational complexity of the forward problem. To achieve high-resolution reconstruction, a group-based sparse representation method is proposed that utilizes global sparsity and neighborhood self-similarity in the reconstructed image, which shows good performance in simulation. Finally, water entry experiments were conducted to validate the performacne of the proposed method. Images captured by a high-speed camera were used as references to assess the accuracy of the calculated equivalent cavity diameter. The experimental results indicate that the proposed method can outperform typical reconstruction methods while maintaining computational cost. Based on the aforementioned performance, the proposed method is anticipated to be integrated into system-on-chip technology to enable real-time, high-precision monitoring of the water entry cavity.

Minimized Mainlobe Width Beamforming Based on Sparse Optimization

Minimized Mainlobe Width Beamforming Based on Sparse Optimization

Simple Linear Loops: Algebraic Invariants and Applications

The automatic generation of loop invariants is a fundamental challenge in software verification. While this task is undecidable in general, it is decidable for certain restricted classes of programs. This work focuses on invariant generation for (branching-free) loops with a single linear update. Our primary contribution is a polynomial-space algorithm that computes the strongest algebraic invariant for simple linear loops, generating all polynomial equations that hold among program variables across all reachable states. The key to achieving our complexity bounds lies in mitigating the blow-up associated with variable elimination and Gröbner basis computation, as seen in prior works. Our procedure runs in polynomial time when the number of program variables is fixed. We examine various applications of our results on invariant generation, focusing on invariant verification and loop synthesis. The invariant verification problem investigates whether a polynomial ideal defining an algebraic set serves as an invariant for a given linear loop. We show that this problem is coNP-complete and lies in PSPACE when the input ideal is given in dense or sparse representations, respectively. In the context of loop synthesis, we aim to construct a loop with an infinite set of reachable states that upholds a specified algebraic property as an invariant. The strong synthesis variant of this problem requires the construction of loops for which the given property is the strongest invariant. In terms of hardness, synthesising loops over integers (or rationals) is as hard as Hilbert's Tenth problem (or its analogue over the rationals). When the constants of the output are constrained to bit-bounded rational numbers, we demonstrate that loop synthesis and its strong variant are both decidable in PSPACE, and in NP when the number of program variables is fixed.

Sparse representation for restoring images by exploiting topological structure of graph of patches

AbstractImage restoration poses a significant challenge, aiming to accurately recover damaged images by delving into their inherent characteristics. Various models and algorithms have been explored by researchers to address different types of image distortions, including sparse representation, grouped sparse representation, and low‐rank self‐representation. The grouped sparse representation algorithm leverages the prior knowledge of non‐local self‐similarity and imposes sparsity constraints to maintain texture information within images. To further exploit the intrinsic properties of images, this study proposes a novel low‐rank representation‐guided grouped sparse representation image restoration algorithm. This algorithm integrates self‐representation models and trace optimization techniques to effectively preserve the original image structure, thereby enhancing image restoration performance while retaining the original texture and structural information. The proposed method was evaluated on image denoising and deblocking tasks across several datasets, demonstrating promising results.

Infrared and Visible Image Fusion via Sparse Representation and Adaptive Dual-Channel PCNN Model Based on Co-Occurrence Analysis Shearlet Transform

Infrared and Visible Image Fusion via Sparse Representation and Adaptive Dual-Channel PCNN Model Based on Co-Occurrence Analysis Shearlet Transform

Scattering Power Decomposition Based on PolInSAR Images and Sparse Representation

Scattering Power Decomposition Based on PolInSAR Images and Sparse Representation