Sparse Basis Research Articles

Partial Singular Value Assignment for Large-Scale Systems

The partial singular value assignment problem stems from the development of observers for discrete-time descriptor systems and the resolution of ordinary differential equations. Conventional techniques mostly utilize singular value decomposition, which is unfeasible for large-scale systems owing to their relatively high complexity. By calculating the sparse basis of the null space associated with some orthogonal projections, the existence of the matrix in partial singular value assignment is proven and an algorithm is subsequently proposed for implementation, effectively avoiding the full singular value decomposition of the existing methods. Numerical examples exhibit the efficiency of the presented method.

Sparse adaptive basis set methods for solution of the time dependent Schrodinger equation

Scalable numerical solutions to the time dependent Schrodinger equation remain an outstanding goal in theoretical chemistry. Here we present a method which utilises recent breakthroughs in signal processing to consistently adapt a dictionary of basis functions to the dynamics of the system. We show that for two low-dimensional model problems the size of the basis set does not grow quickly with time and appears only weakly dependent on dimensionality. The generality of this finding remains to be seen. The method primarily uses energies and gradients of the potential, opening the possibility for its use in on-the-fly ab initio quantum wavepacket dynamics.

Vector Quantized Variational Autoencoder-Based Compressive Sampling Method for Time Series in Structural Health Monitoring

The theory of compressive sampling (CS) has revolutionized data compression technology by capitalizing on the inherent sparsity of a signal to enable signal recovery from significantly far fewer samples than what is required by the Nyquist–Shannon sampling theorem. Recent advancement in deep generative models, which can represent high-dimension data in a low-dimension latent space efficiently when trained with big data, has been used to further reduce the sample size for image data compressive sampling. However, compressive sampling for 1D time series data has not significantly benefited from this technological progress. In this study, we investigate the application of different architectures of deep neural networks suitable for time series data compression and propose an efficient method to solve the compressive sampling problem on one-dimensional (1D) structural health monitoring (SHM) data, based on block CS and the vector quantized–variational autoencoder model with a naïve multitask paradigm (VQ-VAE-M). The proposed method utilizes VQ-VAE-M to learn the data characteristics of the signal, replaces the “hard constraint” of sparsity to realize the compressive sampling signal reconstruction and thereby does not need to select the appropriate sparse basis for the signal. A comparative analysis against various CS methods and other deep neural network models was performed in both synthetic data and real-world data from two real bridges in China. The results have demonstrated the superiority of the proposed method, with achieving the smallest reconstruction error of 0.038, 0.034 and 0.021, and the highest reconstruction accuracy of 0.882, 0.892 and 0.936 for compression ratios of 4.0, 2.66, and 2.0, respectively.

Sound Field Reconstruction Using Prolate Spheroidal Wave Functions and Sparse Regularization.

Near-field acoustic holography (NAH) based on compressing sensing (CS) theory enables accurate reconstruction of sound fields using a limited number of sampling points. However, the successful implementation of this technique depends on two crucial factors: (1) the appropriate selection or construction of the spatial basis and (2) an effective sparse regularization process. To enhance reconstruction performance for elongated sound sources, this paper proposes a novel sound field reconstruction method that combines prolate spheroidal wave functions (PSWFs) with the orthogonal matching pursuit (OMP) algorithm. In this method, PSWFs serve as a sparse spatial basis for representing the radiated sound field. The sparse coefficients are determined by the OMP algorithm in a linear subspace composed of basic functions that best match the residual error. The OMP algorithm effectively identifies significant components before potentially selecting incorrect ones by setting an appropriate stopping rule. Numerical simulations are conducted using a line-array source model. The results show that the proposed method can accurately reconstruct the sound pressures of the elongated source model using a relatively small number of samplings. In addition, the proposed method exhibits robustness across a wide frequency range, diverse array configurations and various sampling numbers. The experimental results further validate the feasibility and reliability of the proposed method.

Inferring the probability distribution over strain tensors in polycrystals from diffraction based measurements

Polycrystals illuminated by high-energy X-rays or neutrons produce diffraction patterns in which the measured diffraction peaks encode the individual single crystal strain states. While state of the art X-ray and neutron diffraction approaches can be used to routinely recover per grain mean strain tensors, less work has been produced on the recovery of higher order statistics of the strain distributions across the individual grains. In the setting of small deformations, we consider the problem of estimating the crystal elastic strain tensor probability distribution from diffraction data. For the special case of multivariate Gaussian strain tensor probability distributions, we show that while the mean of the distribution is well defined from measurements, the covariance of strain has a null-space. We show that there exist exactly 6 orthogonal perturbations to this covariance matrix under which the measured strain signal is invariant. In particular, we provide analytical parametrisations of these perturbations together with the set of possible maximum-likelihood estimates for a multivariate Gaussian fit to data. The parametric description of the null-space provides insights into the strain PDF modes that cannot be accurately estimated from the diffraction data. Understanding these modes prevents erroneous conclusions from being drawn based on the data. Beyond Gaussian strain tensor probability densities, we derive an iterative radial basis regression scheme in which the strain tensor probability density is estimated by a sparse finite basis expansion. This is made possible by showing that the operator mapping the strain tensor probability density onto the measured histograms of directional strain is linear, without approximation. The utility of the proposed algorithm is demonstrated by numerical simulations in the setting of single crystal monochromatic X-ray scattering. The proposed regression methods were found to robustly reject outliers and accurately predict the strain tensor probability distributions in the presence of Gaussian measurement noise.

ASB-CS: Adaptive sparse basis compressive sensing model and its application to medical image encryption

Recent advances in intelligent wearable devices have brought tremendous chances for the development of healthcare monitoring system. However, the data collected by various sensors in it are user-privacy-related information. Once the individuals’ privacy is subjected to attacks, it can potentially cause serious hazards. For this reason, a feasible solution built upon the compression-encryption architecture is proposed. In this scheme, we design an Adaptive Sparse Basis Compressive Sensing (ASB-CS) model by leveraging Singular Value Decomposition (SVD) manipulation, while performing a rigorous proof of its effectiveness. Additionally, incorporating the Parametric Deformed Exponential Rectified Linear Unit (PDE-ReLU) memristor, a new fractional-order Hopfield neural network model is introduced as a pseudo-random number generator for the proposed cryptosystem, which has demonstrated superior properties in many aspects, such as hyperchaotic dynamics and multistability. To be specific, a plain medical image is subjected to the ASB-CS model and bidirectional diffusion manipulation under the guidance of the key-controlled cipher flows to yield the corresponding cipher image without visual semantic features. Ultimately, the simulation results and analysis demonstrate that the proposed scheme is capable of withstanding multiple security attacks and possesses balanced performance in terms of compressibility and robustness.

Solving multiscale elliptic problems by sparse radial basis function neural networks

Machine learning has been successfully applied to various fields of scientific computing in recent years. For problems with multiscale features such as flows in porous media and mechanical properties of composite materials, however, it remains difficult for machine learning methods to resolve such multiscale features, especially when the small scale information is present. In this work, we propose a sparse radial basis function neural network method to solve elliptic partial differential equations (PDEs) with multiscale coefficients. Inspired by the deep mixed residual method, we rewrite the second-order problem into a first-order system and employ multiple radial basis function neural networks (RBFNNs) to approximate unknown functions in the system. To avoid the overfitting and improve the performance of RBFNN, an additional regularization is introduced in the loss function. Thus the loss function contains two parts: the L2 loss for the residual of the first-order system and boundary conditions, and the ℓ1 regularization term for the weights of radial basis functions (RBFs). An algorithm for optimizing the specific loss function is introduced to accelerate the training process. The accuracy and effectiveness of the proposed method are demonstrated through a collection of multiscale problems with scale separation, discontinuity and multiple scales from one to three dimensions. Notably, the ℓ1 regularization can achieve the goal of representing the solution by fewer RBFs. As a consequence, the total number of RBFs scales like O(ε−nτ), where ε is the smallest scale, n is the dimensionality, and τ is typically smaller than 1. It is worth mentioning that the proposed method not only has the numerical convergence and thus provides a reliable numerical solution in three dimensions when a classical method is typically not affordable, but also outperforms most other available machine learning methods in terms of accuracy and robustness. Executable files are available at https://github.com/wangzhiwensuda/SRBFNN-for-multiscale-elliptic-problems. The program was implemented in PyTorch and was run on Linux.

Nonsparse SAR Scene Imaging Network Based on Sparse Representation and Approximate Observations

Sparse-representation-based synthetic aperture radar (SAR) imaging technology has shown superior potential in the reconstruction of nonsparse scenes. However, many existing compressed sensing (CS) methods with sparse representation cannot obtain an optimal sparse basis and only apply to the sensing matrix obtained by exact observation, resulting in a low image quality occupying more storage space. To reduce the computational cost and improve the imaging performance of nonsparse scenes, we formulate a deep learning SAR imaging method based on sparse representation and approximated observation deduced from the chirp-scaling algorithm (CSA). First, we incorporate the CSA-derived approximated observation model and a nonlinear transform function within a sparse reconstruction framework. Second, an iterative shrinkage threshold algorithm is adopted to solve this framework, and the solving process is unfolded as a deep SAR imaging network. Third, a dual-path convolutional neural network (CNN) block is designed in the network to achieve the nonlinear transform, dramatically improving the sparse representation capability over conventional transform-domain-based CS methods. Last, we improve the CNN block to develop an enhanced version of the deep SAR imaging network, in which all the parameters are layer-varied and trained by supervised learning. The experiments demonstrate that our proposed two imaging networks outperform conventional CS-driven and deep-learning-based methods in terms of computing efficiency and reconstruction performance of nonsparse scenes.

Sound source identification using the boundary element method in a sparse framework

Traditional near-field acoustic holography (NAH) based on the boundary element method (BEM) exhibits good performance for sound source identification. However, good reconstruction results can only be guaranteed if a sufficient number of sampling points are used. In this paper, the unitary matrix is used as the sparse basis matrix of source surface velocity and is obtained from the transfer matrix between the source surface sound pressure and vibration velocity by singular value decomposition (SVD). Based on this sparse basis, a sparse Bayesian learning (SBL) algorithm is introduced into BEM-based NAH to address the case of few sampling points. The use of a standard SBL algorithm in BEM-based NAH leads to hyperparameters that are overly sparse due to the large number of iterations, which results in large reconstruction errors. To solve this problem, a modified SBL algorithm is proposed. Good reconstruction results are maintained by selecting appropriate hyperparameters and revising the results using the steepest descent method. A simulated cuboid shell is used to explore the influence of the (i) number of sampling points and (ii) signal-to-noise ratio (SNR) on the reconstruction. Numerical results show that the proposed method can achieve effective reconstruction when the number of sampling points is approximately one-tenth of the number of discrete points on the source surface. The validity of the proposed method is verified by a comparison with experimental results.

Recovery performance improvement of image compressive sensing using complex‐valued Vandermonde matrix

AbstractHere, a novel image‐based quantized compressive sensing (QCS) framework based on complex‐valued Vandermonde (Vander) matrix is proposed. In the proposed QCS framework, a discrete wavelet transform (DWT) serves as a sparse basis and a partial complex‐valued Vander matrix serves as a measurement matrix. The theoretical analysis based on mutual coherence metric of compressive sensing (CS) theory shows that the proposed Vander measurement matrix has the best reconstruction performance among other conventional measurement matrices. The simulation results also show that the recovery quality using the proposed measurement matrix can be greatly improved compared with the other existing real‐valued measurement matrices. In particular, the experiment results also show that under the same measurement matrix, the reconstruction performance of Smoothed l0 norm (SL0) algorithm is better than that of Orthogonal Matching Pursuit (OMP) algorithm, sparsity adaptive matching pursuit (SAMP) algorithm and approximate message passing (AMP) algorithm. In addition, a sparse measurement matrix scheme is further proposed to achieve a trade‐off between recovery performance and computational complexity. The theoretical analysis and simulation results both show the proposed image‐based QCS is efficient.

Accurate analysis of bistatic scattering problems using compressive sensing with a new sparse basis

Compressive sensing (CS) used as a sparse basis can enhance the characteristic basis functions (CBFs), which helps in analyzing the bistatic scattering problems of a three-dimensional (3D) object. However, a sparse basis constructed by the CBFs lacks current information under a single plane wave, leading to a low computation accuracy when analyzing complex objects. To resolve this issue, a new sparse basis construction method is presented in this work, which considers the current information under multilevel plane waves with different incident angles and different polarization modes. As this new sparse basis contains more comprehensive current information, the accuracy is considerably improved when analyzing the complex targets. Furthermore, the accuracy and efficiency of the proposed method are verified by extensive numerical simulations.

An efficient optimization of measurement matrix for compressive sensing

Compressive sensing (CS) is a new paradigm for signal acquisition and reconstruction, which can reconstruct the signal at less than the Nyquist sampling rate. The sampling of the signal occurs through a measurement matrix (MM); thus, MM generation is significant in the context of the CS framework. In this paper, an optimization algorithm is introduced for the generation of the MM of CS based on Restricted Isometric Property (RIP) mandates that eigenvalues of the sensing matrix fall within an interval also minimizes the mutual coherence of the sensing matrix (i.e. the product of the MM and sparsifying matrix). A novel gradient-based iterative optimization method is used to reduce the eigenvalues of the sensing matrix by SVD decomposition. Meanwhile, the proposed algorithm can also reduce the operational complexity. Experimental results and analysis prove that the optimized MM reduces the maximum mutual and average mutual coherence between the MM and the sparsifying basis, which shows the effectiveness of the proposed algorithm over some state-of-art works.

Adaptive LiDAR Reconstruction by Convolutional Compressive Sensing Network and Multivariate Empirical Mode Decomposition

With the recent advancement of LiDAR data processing pipelines integrated with signal processing techniques, it is now possible to rapidly acquire, process, and infer multidimensional information of 3D surroundings. This work proposes a novel compressive sensing (CS) framework (named MEMD-CSNet), based on Multivariate Empirical Mode Decomposition (MEMD) for adaptive 3D reconstruction of airborne urban LiDAR point clouds. EMD is a data-driven transform for decomposing the input data into Intrinsic Mode Functions (IMFs), capturing the implicit features in terms of variation in spatial frequency. Our approach generates a new filtered 3D signal constituting an optimal subset of IMFs based on the understanding of their statistical significance. Further, this filtered signal hierarchically extracts the overall shape in the last IMF (Residual function). Our proposed MEMD-CSNet is a deep convolutional network that (1) computes a multivariate EMD on input raw 3D LiDAR point clouds for generating data-dependent feature-rich filtered signal and (2) additionally augments this signal with convolutional features, thus learning a data-centric representation function (Ψ) for transforming the input into a low-dimensional sparse basis. We present extensive qualitative and quantitative analysis using the proposed MEMD-CSNet for a set of chosen CS measurement ratios. MEMD-CSNet achieves a maximum reconstruction PSNR of 108.22 dB (approx. 30 dB/1.36x better than without using MEMD) for reconstruction fidelity on selected urban LiDAR scenes. We envisage that our proposed MEMD-CSNet has a high potential (as a data-centric framework) for generating data-driven sparse feature maps and reconstructing entire 3D LiDAR point clouds from a few sparse measurements.

Prediction of MAG Welding Formation Based on Fuzzy Interpolation Reasoning

Welding is an important production and processing method in industrial production. Due to the uncertainty of its process and the complexity of its mechanism, it is challenging to construct an accurate welding process parameter model to describe it. Fuzzy interpolation reasoning is a valuable tool in the realm of intelligent systems. It helps to reduce the complexity of fuzzy models and can be used for reasoning in fuzzy systems with sparse rule bases. This paper provides an interpolation reasoning method that combines fuzzy set distance and similarity. The proposed method is applied to welding forming predictions, and welding penetration is predicted by the given welding current and speed. The results show that the algorithm has high calculation accuracy.

Augmented GBM Nonlinear Model to Address Spectral Variability for Hyperspectral Unmixing

Spectral unmixing (SU) is a significant preprocessing task for handling hyperspectral images (HSI), but its process is affected by nonlinearity and spectral variability (SV). Currently, SV is considered within the framework of linear mixing models (LMM), which ignores the nonlinear effects in the scene. To address that issue, we consider the effects of SV on SU while investigating the nonlinear effects of hyperspectral images. Furthermore, an augmented generalized bilinear model is proposed to address spectral variability (abbreviated AGBM-SV). First, AGBM-SV adopts a generalized bilinear model (GBM) as the basic framework to address the nonlinear effects caused by second-order scattering. Secondly, scaling factors and spectral variability dictionaries are introduced to model the variability issues caused by the illumination conditions, material intrinsic variability, and other environmental factors. Then, a data-driven learning strategy is employed to set sparse and orthogonal bases for the abundance and spectral variability dictionaries according to the distribution characteristics of real materials. Finally, the alternating direction method of multipliers (ADMM) optimization method is used to split and solve the objective function, enabling the AGBM-SV algorithm to estimate the abundance and learn the spectral variability dictionary more effectively. The experimental results demonstrate the comparative superiority of the AGBM-SV method in both qualitative and quantitative perspectives, which can effectively solve the problem of spectral variability in nonlinear mixing scenes and to improve unmixing accuracy.

Optimal Deep Learning Architecture for Automated Segmentation of Cysts in OCT Images Using X-Let Transforms.

The retina is a thin, light-sensitive membrane with a multilayered structure found in the back of the eyeball. There are many types of retinal disorders. The two most prevalent retinal illnesses are Age-Related Macular Degeneration (AMD) and Diabetic Macular Edema (DME). Optical Coherence Tomography (OCT) is a vital retinal imaging technology. X-lets (such as curvelet, DTCWT, contourlet, etc.) have several benefits in image processing and analysis. They can capture both local and non-local features of an image simultaneously. The aim of this paper is to propose an optimal deep learning architecture based on sparse basis functions for the automated segmentation of cystic areas in OCT images. Different X-let transforms were used to produce different network inputs, including curvelet, Dual-Tree Complex Wavelet Transform (DTCWT), circlet, and contourlet. Additionally, three different combinations of these transforms are suggested to achieve more accurate segmentation results. Various metrics, including Dice coefficient, sensitivity, false positive ratio, Jaccard index, and qualitative results, were evaluated to find the optimal networks and combinations of the X-let's sub-bands. The proposed network was tested on both original and noisy datasets. The results show the following facts: (1) contourlet achieves the optimal results between different combinations; (2) the five-channel decomposition using high-pass sub-bands of contourlet transform achieves the best performance; and (3) the five-channel decomposition using high-pass sub-bands formations out-performs the state-of-the-art methods, especially in the noisy dataset. The proposed method has the potential to improve the accuracy and speed of the segmentation process in clinical settings, facilitating the diagnosis and treatment of retinal diseases.

Data-Drive Site Characterization for Benchmark Examples: Sparse Bayesian Learning versus Gaussian Process Regression

In this paper, two data-drive site characterization methods, the sparse Bayesian learning (SBL) method and the Gaussian process regression (GPR) method, are benchmarked by a set of virtual ground examples and a real ground example of cone penetration test (CPT) data. The two methods both assume a zero-mean prior Gaussian random field model for the spatial trend, but the strategies of maintaining model simplicity are different. The SBL method produces a simple trend model by adopting sparse basis functions, whereas the GPR method produces a simple trend model by adopting a kernel function governed by few hyperparameters. The accuracy of the two methods in predicting the cone tip resistance (qt) of CPT was quantified by the root-mean square prediction error (RMSE), whereas the accuracy in identifying soil layers was quantified by the identification rate (IR). It was found that the GPR method in general outperforms the SBL method. Further accuracy improvement for the GPR method can be obtained if a clustering analysis based on the Robertson’s soil behavior index (Ic) is conducted.

A divide-and-conquer method for compression and reconstruction of smart meter data

As smart grid sensors, smart meters generate abundant valuable data, laying the foundation for data-driven applications. However, the data collection brings huge communication pressure to electric utilities. In this context, considering that different types of devices have different power consumption patterns, and different types of data compression methods have their own applicable scenarios, we propose a divide-and-conquer method for compression and reconstruction of smart meter data. First, based on algorithm of voice activity detection (VAD), a load power fluctuation segment location method is proposed, which is combined with load event detection method to divide the load data into the event segments, fluctuation segments, and steady-state segments. Then, for the fluctuation segments, a cloud-device collaboration adaptive strategy based on the compressive sensing (CS) theory is designed, in which the sparse basis and measurement matrix are updated accordingly to ensure the high reconstruction accuracy in different scenarios. For the steady-state segments, a data compression method based on the improved symbolic aggregation approximation (SAX) is established, in which the dividing rectangle (DIRECT) algorithm and the irregular time partitioning method are combined to reduce the data volume for transmission without losing important information. For the event segments, the original data values are retained since the event power curves are relatively more complex and short duration. Finally, the received compressed data are reconstructed into the original power time series data in the master station on cloud to support advanced data analytics. Comparative experiments are conducted on the private and public datasets of 12 households in North America and China. The results show that our method has higher data reconstruction accuracy and compression efficiency compared to the existing methods.

Incremental Cycle Bases for Cycle-Based Pose Graph Optimization

Pose graph optimization is a special case of the simultaneous localization and mapping problem where the only variables to be estimated are pose variables and the only measurements are inter-pose constraints. The vast majority of pose graph optimization techniques are vertex based (variables are robot poses), but recent work has parameterized the pose graph optimization problem in a relative fashion (variables are the transformations between poses) that utilizes a minimum cycle basis to maximize the sparsity of the problem. We explore the construction of a cycle basis in an incremental manner while maximizing the sparsity. We validate an algorithm that constructs a sparse cycle basis incrementally and compare its performance with a minimum cycle basis. Additionally, we present an algorithm to approximate the minimum cycle basis of two graphs that are sparsely connected as is common in multi-agent scenarios. Lastly, the relative parameterization of pose graph optimization has been limited to using rigid body transforms on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$SE(2)$</tex-math></inline-formula> or <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$SE(3)$</tex-math></inline-formula> as the constraints between poses. We introduce a methodology to allow for the use of lower-degree-of-freedom measurements in the relative pose graph optimization problem. We provide extensive validation of our algorithms on standard benchmarks, simulated datasets, and custom hardware.

Novel compressive sensing computing model used for analyzing electromagnetic scattering characteristics of three-dimensional electrically large objects

The method of moments is one of the most effective algorithms for solving electromagnetic scattering problems. However, the high computational complexity limits its application to electrically large problems. As an improved algorithm, the compressive sensing-based method of moments introduces the compressive sensing technique into the algorithmic structure, which avoids the inverse of the matrix equation and improves the computational efficiency. Using the underdetermined equation-based calculation model, this scheme is utilized to efficiently analyze the bistatic scattering of the objects. In this technique, the extracted few rows from the impedance matrix are used to construct the measurement matrix. Nevertheless, the results are unstable due to the random extraction utilized to build the measurement matrix. Furthermore, it is challenging task to provide an appropriate sparse basis for the induced currents of three-dimensional objects discretized by the Rao-Wilton-Glisson basis functions. To address the aforementioned issues, a novel compressive sensing calculation model that enhances measurement matrix building, sparse basis construction, and recovery is provided in this paper. First, several rows of the impedance matrix are uniformly extracted to produce consistent computation results, which is opposed to the randomly constructed measurement matrix in the conventional technique. The number of rows to be extracted is typically set to be 3–5 times the number of basis functions for high accuracy. Then, the characteristic basis functions based on the Foldy-Lax equation are employed to construct the sparse basis. Considering the prior knowledge that the lower order characteristic basis functions are dominant, the columns of the recovery matrix corresponding to some low-order characteristic functions are determined in advance as the columns that will be identified by the recovery algorithm, thus simplifying the recovery algorithm to a least-squares operation. Obviously, the matrix equation is reduced to an overdetermined equation instead of the underdetermined equation since only a few low-order basis functions are to be computed. Compared with the conventional compressive sensing-based method of moments with characteristic basis functions, the computation time is significantly reduced in terms of constructing the basis functions and recovering the current coefficients, while the accuracy is improved. Finally, both the suggested method and the conventional compressive sensing-based method of moments are used to simulate the bistatic radar cross sections of the perfect electrical conductor cube, cylinder, and missile model. The efficiency and accuracy of the proposed method are verified by the numerical results.