Sparsity Constraint Research Articles

Comparing regularized Kelvinlet functions and the finite element method for registration of medical images to sparse organ data

Image-guided surgery collocates patient-specific data with the physical environment to facilitate surgical decision making. Unfortunately, these guidance systems commonly become compromised by intraoperative soft-tissue deformations. Nonrigid image-to-physical registration methods have been proposed to compensate for deformations, but clinical utility requires compatibility of these techniques with data sparsity and temporal constraints in the operating room. While finite element models can be effective in sparse data scenarios, computation time remains a limitation to widespread deployment. This paper proposes a registration algorithm that uses regularized Kelvinlets, which are analytical solutions to linear elasticity in an infinite domain, to overcome these barriers. This algorithm is demonstrated and compared to finite element-based registration on two datasets: a phantom liver deformation dataset and an in vivo breast deformation dataset. The regularized Kelvinlets algorithm resulted in a significant reduction in computation time compared to the finite element method. Accuracy as evaluated by target registration error was comparable between methods. Average target registration errors were 4.6 ± 1.0 and 3.2 ± 0.8 mm on the liver dataset and 5.4 ± 1.4 and 6.4 ± 1.5 mm on the breast dataset for the regularized Kelvinlets and finite element method, respectively. Limitations of regularized Kelvinlets include the lack of organ-specific geometry and the assumptions of linear elasticity and infinitesimal strain. Despite limitations, this work demonstrates the generalizability of regularized Kelvinlets registration on two soft-tissue elastic organs. This method may improve and accelerate registration for image-guided surgery, and it shows the potential of using regularized Kelvinlets on medical imaging data.

Waveform inversion with structural regularizing constraint based on gradient decomposition

Abstract Full waveform inversion (FWI) can simultaneously update low-to-medium wavenumber velocity components and high-wavenumber velocity components. However, if seismic data lack large-offset data and effective low-frequency components, FWI updates will be dominated by high-wavenumber velocity perturbation. Meanwhile, providing that the initial model is inaccurate, inversion will have the problem of local minima. In this study, FWI is developed with structural regularizing constraint based on gradient decomposition (RGDFWI). By correlating the separated forward wavefield and backward wavefield with specific propagating direction, FWI gradient is decomposed into tomography-mode gradient and migration-mode gradient. We propose an optimized strategy taking full advantage of the two modes of FWI gradient. On the one hand, we use tomography-mode gradient to enhance low-to-medium wavenumber updates. On the other hand, we use migration-mode gradient to apply structural regularizing constraint by estimating structure dip and adding sparsity constraint in Seislet domain. During the inversion process, high-wavenumber structural information constrains and guides low-wavenumber model updates. The results of two numerical tests, Marmousi model test and Overthrust model test, validate the optimized strategy, which can produce a better initial velocity model for FWI. The inversion finally generates a high-precision and high-resolution velocity model.

Text to image synthesis with multi-granularity feature aware enhancement Generative Adversarial Networks

Synthesizing complex images from text presents challenging. Compared to autoregressive and diffusion model-based methods, Generative Adversarial Network-based methods have significant advantages in terms of computational cost and generation efficiency yet remain two limitations: first, these methods often refine all features output from the previous stage indiscriminately, without considering these features are initialized gradually during the generation process; second, the sparse semantic constraints provided by the text description are typically ineffective for refining fine-grained features. These issues complicate the balance between generation quality, computational cost and inference speed. To address these issues, we propose a Multi-granularity Feature Aware Enhancement GAN (MFAE-GAN), which allows the refinement process to match the order of different granularity features being initialized. Specifically, MFAE-GAN (1) samples category-related coarse-grained features and instance-level detail-related fine-grained features at different generation stages based on different attention mechanisms in Coarse-grained Feature Enhancement (CFE) and Fine-grained Feature Enhancement (FFE) to guide the generation process spatially, (2) provides denser semantic constraints than textual semantic information through Multi-granularity Features Adaptive Batch Normalization (MFA-BN) in the process of refining fine-grained features, and (3) adopts a Global Semantics Preservation (GSP) to avoid the loss of global semantics when sampling features continuously. Extensive experimental results demonstrate that our MFAE-GAN is competitive in terms of both image generation quality and efficiency.

Regularisation constrained denoising discriminant least squares regression for image classification

Discriminative Least Squares Regression (DLSR) is an algorithm that utilizes ε-draggings to expand the distance between classes to better solve multi-classification problems. However, this method increases the differences in regression targets within classes during the classification process, and noise in the samples adversely affects its performance. To address the above problems, we propose a novel method called Regularisation Constrained Denoising Discriminant Least Squares Regression (RCDDLSR). Firstly, the idea of matrix decomposition is introduced and sparse constraints are imposed on the noise matrix for noise elimination, thus helping to maintain the manifold structure of the raw data. Secondly, the neighbourhood relationship between samples of the same category is ensured through graph regularisation to avoid overfitting. In addition, we introduce low-rank constraints in the labelling space to better capture the local structure of the regression target within the class. Finally, our experiments show that the method performs better in various types of image datasets compared to other different algorithms.

M-band wavelet network for machine anomaly detection from a frequency perspective

The autoencoder (AE) is widely utilized in deep anomaly detection, but it lacks explainability due to the complexity of nonlinear mapping. One approach to address this issue is incorporating wavelet theory, which shares similarities in decomposition and reconstruction procedures. However, the perfect reconstruction property of wavelet theory conflicts with AE-based anomaly detection. To tackle this problem, we introduce a novel deep anomaly detection method from a frequency perspective. A learnable M-band wavelet network (MWNet) is designed to offer a flexible frequency band structure for signal representation. Subsequently, with the aid of sparsity constraint, MWNet dynamically focuses on key components within each frequency band. A learnable hard threshold function with a threshold maximization constraint is proposed to retain the essential frequency band of normal signals. After training, the MWNet is exclusively capable of well reconstructing normal signals, thereby producing a noticeable reconstruction error difference between normal and abnormal signals. Extensive experiments on both simulated and experimental datasets validate the effectiveness of the proposed method. The corresponding Python codes are available at https://github.com/albertszg/MbandWaveletNet.

A mechanics-informed neural network method for structural modal identification

Modal identification is one of the core topics within the realm of structural health monitoring (SHM). In this study, we summarize four modal mechanical properties and propose a mechanics-informed neural network (MINN) method for structural modal identification. The proposed MINN method incorporates the sparsity of the data in the time–frequency domain and cross-correlation minimization in the time domain into the neural network to obtain modal parameters, which uses sparsity constraint and cross-correlation minimization constraint to obtain the accurate modal responses and mode shapes. Subsequently, modal frequencies and damping ratios can be derived from the modal responses. The proposed MINN method is verified by numerical simulations and two actual suspension bridges. Compared with traditional methods, the proposed MINN method has two major advantages. Firstly, the proposed MINN method presents explicit mathematical equations to distinguish the modes and the spurious modes, which obviates the necessity for priori information such as model order or time-consuming manual intervention to distinguish the modes and the spurious modes. Therefore, it can be implemented adaptively to determine the modal order and obtain the modal parameters. Secondly, the proposed MINN method can obtain a greater number of accurate modal parameters than traditional methods and achieves an increase of 102.6%, 43.4%, and 31.5% in the number of accurate results when compared to covariance-driven stochastic subspace identification (SSI-COV), data-driven stochastic subspace identification (SSI-DATA) and the natural excitation technique and the eigensystem realization algorithm (NExT-ERA), respectively. Therefore, the proposed MINN method provides an adaptively modal identification method that has clear modal mechanical properties to distinguish the modes and the spurious modes and can obtain a greater number of accurate results.

Automatical Spike Sorting With Low-Rank and Sparse Representation.

Spikesorting is crucial in studying neural individually and synergistically encoding and decoding behaviors. However, existent spike sorting algorithms perform unsatisfactorily in real scenarios where heavy noises and overlapping samples are commonly in the spikes, and the spikes from different neurons are similar. To address such challenging scenarios, we propose an automatic spike sporting method in this paper, which integrally combines low-rank and sparse representation (LRSR) into a unified model. In particular, LRSR models spikes through low-rank optimization, uncovering global data structure for handling similar and overlapped samples. To eliminate the influence of the embedded noises, LRSR uses a sparse constraint, effectively separating spikes from noise. The optimization is solved using alternate augmented Lagrange multipliers methods. Moreover, we conclude with an automatic spike-sorting framework that employs the spectral clustering theorem to estimate the number of neurons. Extensive experiments over various simulated and real-world datasets demonstrate that our proposed method, LRSR, can handle spike sorting effectively and efficiently.

Variational Attention-Based Interpretable Transformer Network for Rotary Machine Fault Diagnosis.

Deep learning technology provides a promising approach for rotary machine fault diagnosis (RMFD), where vibration signals are commonly utilized as input of a deep network model to reveal the internal state of machinery. However, most existing methods fail to mine association relationships within signals. Unlike deep neural networks, transformer networks are capable of capturing association relationships through the global self-attention mechanism to enhance feature representations from vibration signals. Despite this, transformer networks cannot explicitly establish the causal association between signal patterns and fault types, resulting in poor interpretability. To tackle these problems, an interpretable deep learning model named the variational attention-based transformer network (VATN) is proposed for RMFD. VATN is improved from transformer encoder to mine the association relationships within signals. To embed the prior knowledge of the fault type, which can be recognized based on several key features of vibration signals, a sparse constraint is designed for attention weights. Variational inference is employed to force attention weights to samples from Dirichlet distributions, and Laplace approximation is applied to realize reparameterization. Finally, two experimental studies conducted on bevel gear and bearing datasets demonstrate the effectiveness of VATN to other comparison methods, and the heat map of attention weights illustrates the causal association between fault types and signal patterns.

Temperature scaling unmixing framework based on convolutional autoencoder

Hyperspectral unmixing is a key technology in the development of remote sensing applications. However, since both endmembers and abundances are unknown, unmixing is a non-convex problem with a large solution space. To solve this, existing methods usually impose the same strength of sparsity constraint. However, this often does not hold in practice. Because the abundances of purer regions are generally sparse, while the abundances distribution of more mixed regions should be smoother. Temperature scaling is a technique of introducing a temperature parameter T into softmax activation function to adjust the sparsity of the output. Inspired by this, we propose a temperature scaling unmixing (TSU) framework based on convolutional autoencoder (CAE). In this framework, sparse constraints of different intensities are applied to diverse regions by considering spatial similarity of ground objects distribution while preserving the ability of CAE to extract spatial features. What is more, equal-frequency binning is adopted to guide the division of regions by similarity matrix to realize the automatic temperature parameter setting. In addition, a CAE network is designed under the TSU framework in this paper, called TSUCAE. The TSUCAE method exhibits superior accuracy compared to state-of-the-art approaches, as demonstrated through extensive comparative experiments. Furthermore, the TSU framework can be transferred to other CAE-based unmixing methods directly while keeping the network structure of these methods unchanged. Sufficient ablation experiments also prove that the transfer of framework can improve the performance of unmixing. The code is publicly available at https://github.com/UPCGIT/TSUCAE.

3D high-resolution radon transform based on a strong sparseLp–1norm and its applications

AbstractMultiple reflections are among the most challenging noises to suppress in seismic data, as they differ from effective waves only in terms of apparent velocity. Also, radon transform, an essential technique for attenuating multiple reflections, has been widely incorporated into various commercial software packages. Thus, this study introduces a 3D radon transform method based on the ${L_{p - 1}}$ norm to enhance sparsity-constraining capability in the transform domain, leveraging high-resolution radon transform techniques. Specifically, an iteratively reweighted least-squares algorithm is employed to obtain the transformed data in the radon domain. Given that the ${L_{p - 1}}$ norm was first applied to the Radon transform in seismic data processing methods, this paper theoretically demonstrates its powerful sparsity-constraining capability. Indeed, the proposed strategy enhances energy concentration in the radon transform domain, better separating primaries from multiples, and ultimately suppressing the multiples. Both model tests and real data indicate that the 3D radon transform constrained by the ${L_{p - 1}}$ norm outperforms existing high-resolution radon transform methods with sparsity constraints regarding energy concentration and effectiveness in multiple reflection attenuation.

Tensor robust principal component analysis via dual lp quasi-norm sparse constraints

As a pioneering method, tensor robust principal component analysis (TRPCA) can separate an underlying low-rank component and a sparse component from the original data by minimizing a convex objective function composed of tensor nuclear norm and l1-norm. However, it has two limitations. One is that tensor nuclear norm, as a constraint on the low-rank component, treats all singular values uniformly, ignoring the differences among singular values. In essence, this constraint is a sparse constraint that is imposed on singular values of the low-rank component by l1-norm. The other is that l1-norm is used as a constraint for the sparse component. However, l1-norm is a loose constraint, leading to the solutions of TRPCA deviating from the authentic ones. To alleviate these issues, we propose a TPRCA model called p-TRPCA that utilizes the lp-norm to impose sparse constraints on both the singular values and the sparse component simultaneously. The lp quasi-norm (0<p<1) is a tighter constraint than l1-norm, which enhances the low-rankness and sparsity of the proposed model. To solve p-TRPCA, we present an effective algorithm based on the alternating direction method of multipliers and also analyze its convergence. Extensive experiments are performed on data simulation, image recovery, and background modeling. Experimental results show that our p-TRPCA outperforms TRPCA and its variants.

Cross-dataset discriminant subspace learning algorithm for apple leaf diseases identification

Apple has a large planting area and high yield in China, but it is easily affected by diseases. Artificial intelligence technology has achieved good results in apple leaf disease identification. However, the training and testing data in complex environments often come from different collection scenarios. There are significant differences in the training and testing datasets, which leads to a significant decline in the identification performance. To improve the generalization ability of apple leaf disease identification, we develop a cross-dataset discriminant subspace learning (CDDSL) algorithm by utilizing the idea of transfer learning, low-rank sparse representation, and maximum margin neighborhood preserving embedding (NPE) to cross-dataset scenarios. Firstly, based on the transfer learning and low-rank sparse representation, each target sample can be represented by a linear combination of source domain samples. The sparse constraint on the noise matrix weakens the influence of noise in the sample data. Then, CDDSL progressively projects the subspace features into the semantic space and establishes a discriminant classifier. Meanwhile, CDDSL utilizes the maximal margin NPE to improve the intraclass compactness and interclass separability of projection features, which can enhance the image identification performance. The experiments conducted on real-world apple leaf datasets verify the effectiveness of the CDDSL algorithm in cross-dataset scenario.

Imaging Left-Lateral and Reverse Near-Surface Slip of the 2020 Mw 5.1 Sparta, North Carolina, Earthquake

ABSTRACT Coseismic deformation of the 9 August 2020 Mw 5.1 Sparta, North Carolina, earthquake was captured by an Advanced Land Observing Satellite 2 interferogram, a Sentinel 1A/B interferogram, and differential light detection and ranging (lidar). Owing to the moderate surface deformation compared to the inherent uncertainties of interferometric data, the interferograms suffered from low signal-to-noise ratios, exacerbated by spurious signals arising from the discrepancy in resolution between the interferograms and digital elevation model used during processing. Nevertheless, coseismic slip can be imaged using graph-guided fused LASSO, where LASSO is the least-absolute shrinkage and selection operator—an inverse method that combines sparsity constraints and smoothness regularization—to infer signal from noisy and/or incomplete data. We develop five coseismic slip models, one determined only from the Sentinel 1A/B interferogram, one determined from both interferograms, one determined from all the Interferometric Synthetic Aperture Radar (InSAR) data masking potentially spurious data corresponding to industrial parks, one from lidar 3D displacements only, and one constrained by the masked InSAR and the lidar data. All five models are coseismic slip on a single fault plane, trending northwest–southeast with dips ranging from 41° to 60° to the southwest. Coseismic slip is imaged to be shallow, with predominantly oblique reverse slip extending to about 2 km depth. We find that masking the interferometric data corresponding to the industrial parks yields a coseismic slip model in better concordance with ancillary information about the Sparta earthquake and that the lidar data yield additional constraints on near-surface coseismic slip. We show that our preferred coseismic slip model is consistent with a stress regime that does not depart from an Andersonian thrust regime to a large degree, constraining that the background stress is larger than 20–30 MPa.

Direction-of-Arrival Estimation via Sparse Bayesian Learning Exploiting Hierarchical Priors with Low Complexity.

For direction-of-arrival (DOA) estimation problems in a sparse domain, sparse Bayesian learning (SBL) is highly favored by researchers owing to its excellent estimation performance. However, traditional SBL-based methods always assign Gaussian priors to parameters to be solved, leading to moderate sparse signal recovery (SSR) effects. The reason is Gaussian priors play a similar role to l2 regularization in sparsity constraint. Therefore, numerous methods are developed by adopting hierarchical priors that are used to perform better than Gaussian priors. However, these methods are in straitened circumstances when multiple measurement vector (MMV) data are adopted. On this basis, a block-sparse SBL method (named BSBL) is developed to handle DOA estimation problems in MMV models. The novelty of BSBL is the combination of hierarchical priors and block-sparse model originating from MMV data. Therefore, on the one hand, BSBL transfers the MMV model to a block-sparse model by vectorization so that Bayesian learning is directly performed, regardless of the prior independent assumption of different measurement vectors and the inconvenience caused by the solution of matrix form. On the other hand, BSBL inherited the advantage of hierarchical priors for better SSR ability. Despite the benefit, BSBL still has the disadvantage of relatively large computation complexity caused by high dimensional matrix operations. In view of this, two operations are implemented for low complexity. One is reducing the matrix dimension of BSBL by approximation, generating a method named BSBL-APPR, and the other is embedding the generalized approximate message passing (GAMB) technique into BSBL so as to decompose matrix operations into vector or scale operations, named BSBL-GAMP. Moreover, BSBL is able to suppress temporal correlation and handle wideband sources easily. Extensive simulation results are presented to prove the superiority of BSBL over other state-of-the-art algorithms.

Matrix Pontryagin principle approach to controllability metrics maximization under sparsity constraints

This study considers optimization problems for time-varying network topologies that maximize the controllability of linear network systems under sparsity constraints. The major theoretical differences from the existing findings on control node scheduling problems is that our method is based on the matrix-valued Pontryagin maximum principle applied to the controllability Lyapunov differential equation. This allows us to handle various controllability metrics that can only be approximated by existing methods. Finally, we perform a numerical evaluation of our proposed method and compare results with those of our previous approximation method.

Multi-constraint non-negative matrix factorization for community detection: orthogonal regular sparse constraint non-negative matrix factorization

Community detection is an important method to analyze the characteristics and structure of community networks, which can excavate the potential links between nodes and further discover subgroups from complex networks. However, most of the existing methods only unilaterally consider the direct link topology without comprehensively considering the internal and external characteristics of the community as well as the result itself, which fails to maximize the access to the network information, thus affecting the effectiveness of community detection. To compensate for this deficiency, we propose a new community detection method based on multi-constraint non-negative matrix factorization, named orthogonal regular sparse constraint non-negative matrix factorization (ORSNMF). Based on the network topology, the ORSNMF algorithm models the differences of the outside of the community, the similarities of the nodes inside the community, and the sparseness of the community membership matrices at the same time, which together guides the iterative learning process to better reflect the underlying information and inherent attributes of the community structure in order to improve the correct rate of dividing subgroups. An algorithm with convergence guarantee is also proposed to solve the model, and finally a large number of comparative experiments are conducted, and the results show that the algorithm has good results.

The ℓ1−2-norm-regularized basis pursuit seismic inversion based on the exact Zoeppritz equation

Sparsity constraints have been widely adopted in the regularization of ill-posed problems to obtain subsurface properties with sparseness features. However, the target parameters are generally not sparsely distributed, and sparsity constraints lead to results that are missing information. Smooth constraints (e.g., [Formula: see text] norm) lead to the insufficient resolution of the inversion results. To overcome this issue, an effective solution is to convert the target parameters to a sparse representation, which can then be solved with sparsity constraints. For the estimation of elastic parameters, a high-resolution and reliable seismic basis pursuit inversion is developed based on the exact Zoeppritz equation. Furthermore, the [Formula: see text] norm is developed as a constraint, wherein a regularized function is minimized with the alternating direction method of the multipliers algorithm. Numerical examples and real data applications demonstrate that our method can not only improve the accuracy of the inversion results, especially the S-wave velocity and density information but also increase the resolution of the inversion results. Furthermore, the [Formula: see text]-norm constraint has better noise suppression demonstrating great potential in practical applications.

Orthogonal graph regularized non-negative matrix factorization under sparse constraints for clustering

The standard NMF algorithm is not suitable for sampling data from low-dimensional manifolds embedded in high-dimensional environmental spaces, as the geometric information hidden in feature manifolds and sample manifolds is rarely learned. In order to obtain better clustering performance based on NMF, manifold and orthogonal constraint, a new type of model named Orthogonal Graph regularized Non-negative Matrix Factorization model under Sparse Constraints (OGNMFSC) is proposed. Firstly, this type of model constructs a nearest neighbor graph to encode the geometric information of the data space, in order to obtain more discriminative ability by preserving the structure of the graph. Secondly, this type of model adds orthogonal constraints to achieve better local representation and significantly reduce the inconsistency between the original matrix and the basis vectors. Thirdly, by adding sparse constraints to obtain a sparser representation matrix, the clustering performance of the model can be improved. The main conclusion of this paper is that two effective algorithms have been generated to solve the model, which not only provides theoretical convergence proof for these two algorithms, but also demonstrates significant clustering performance in experiments compared to classical models such as K-means, PCA, NMF, Semi-NMF, NMFSC, ONMF, GNMF, NeNMF.

Analysis of the Relationship Between Cross Capital Flows and Stock Exchange Index with Machine Learning

This paper investigates forecasting the BIST100 stock index using cross-capital flow analysis. It employs feature engineering and the Orthogonal Matching Pursuit (OMP) model to navigate the intricacies of financial time series prediction. The study meticulously selects features such as lagged values, moving averages, and volatility metrics, normalized to ensure unbiased model impact. The OMP model is carefully optimized to handle the dimensionality of financial data, avoiding overfitting through a sparsity constraint. This approach yields an R-squared score of 0.88, indicating a solid capability to capture index variance. Visual comparisons between actual and predicted values further validate the model's accuracy. The paper highlights the importance of methodological precision in developing models capable of discerning complex patterns, offering valuable insights for investment strategies. Implications of the study show that cross-capital movements and macroeconomic variables are a good fit with ML to predict the Stock Market despite the complexity of financial markets.

A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems

In this paper, we explore a specific optimization problem that involves the combination of a differentiable nonconvex function and a nondifferentiable function. The differentiable component lacks a global Lipschitz continuous gradient, posing challenges for optimization. To address this issue and accelerate the convergence, we propose a Bregman proximal stochastic gradient method with extrapolation (BPSGE), which only requires smooth adaptivity of the differentiable part. Under variance reduction framework, we not only analyze the subsequential and global convergence of the proposed algorithm under certain conditions, but also analyze the sublinear convergence rate of the subsequence, and the complexity of the algorithm, revealing that the BPSGE algorithm requires at most O(epsilon\^\,(-2)) iterations in expectation to attain an epsilon-stationary point. To validate the effectiveness of our proposed algorithm, we conduct numerical experiments on three real-world applications: graph regularized nonnegative matrix factorization (NMF), matrix factorization with weakly-convex regularization, and NMF with nonconvex sparsity constraints. These experiments demonstrate that BPSGE is faster than the baselines without extrapolation. The code is available at: https://github.com/nothing2wang/BPSGE-Algorithm.