Restricted Isometry Research Articles

Analyses of the tail-[formula omitted] minimization for fast and enhanced sparse selections

We investigate the effectiveness and efficiency of the iterative tail-ℓ2 minimization (tail-ℓ2-min) technique for its sparse selection capabilities. We conduct profile analyses on the tail-ℓ2-min, establishing the equivalence of the tail-ℓ2-min problem to a two-stage profile ℓ2 formulation, both featuring analytical solutions. The tail null space property (NSP) of sensing matrix A is shown to be equivalent to the NSP of the newly defined profile matrix Ã. Besides the error bound analysis for the tail-ℓ2-min under the typical tail-NSP condition, a novel error bound of the tail-ℓ2-min formulation is also established without relying on NSP or restricted isometry property (RIP) assumptions. It merely contains tractable coefficients of A, and offers insights into successful recovery, with the observation of the convergent iterative procedure. Numerical studies and the applications to image reconstruction demonstrate the superiority and fast convergence of the tail-ℓ2 sparse solution over state-of-the-art sparse selection methodologies. The sparsity level of a signal that the tail-ℓ2 profile algorithm guarantees the recovery is around 41% higher than that of the basis pursuit algorithm. The analytical solutions of the tail-ℓ2 method at each iteration also ensure that the tail-ℓ2 sparse recovery process is notably fast, especially for high dimensions and high sparsity levels.

Guaranteed matrix recovery using weighted nuclear norm plus weighted total variation minimization

This work presents a general framework regarding the recovery of matrices equipped with hybrid low-rank and local-smooth properties from just a few measurements consisting of linear combinations of the matrix entries. Concretely, we consider the problem of robust low-rank matrix recovery using Weighted Nuclear Norm plus Weight Total Variation (WNNWTV) minimization. First of all, based on a new restricted isometry property, we prove that the WNNWTV method possesses an error bound consisting of a low-rank approximation term, a total variation approximation term, and an observation error term. It should be noted that although there are many models considering both properties, there are very few recoverable error theories about such models. Specifically, the theoretical error bound provides an automatic mechanism to reducing regularization parameters with no need for cross-validation while keeping almost the same selection result with commonly used cross-validation technique. Subsequently, the proposed method is reformulated into a regularized unconstrained problem, and we study its optimization aspects in detail based on the Alternating Direction Method of Multipliers (ADMM). Extensive experiments on synthetic data and two applications, i.e. hyperspectral image recovery and dynamic magnetic resonance imaging recovery verified our theories and proposed algorithms.

An Underwater Target Location Algorithm Combined with Schmidt Orthogonalization

Abstract In underwater target localization research, where challenges such as low signal-to-noise ratio and multiple interferences exist in conventional matched field processing, this paper presents a novel approach combining compressed sensing with Schmidt orthogonalization. Initially, an orthogonalization procedure is applied to the observation matrix, yielding a new matrix with reduced column coherence and maximal adherence to the Restricted Isometry Property (RIP) constraint. Subsequently, the Sparse Adaptive Matching Pursuit (SAMP) algorithm from compressed sensing is employed to reconstruct the signal, leading to precise underwater target localization. Simulation experiments substantiate that this combined approach outperforms the conventional matched field algorithm by yielding a higher signal-to-noise ratio for target acoustic source localization and markedly reducing the count of interference virtual sources. Consequently, the performance of localization for targets is substantially enhanced.

Joint sparse optimization: lower-order regularization method and application in cell fate conversion

Multiple measurement signals are commonly collected in practical applications, and joint sparse optimization adopts the synchronous effect within multiple measurement signals to improve model analysis and sparse recovery capability. In this paper, we investigate the joint sparse optimization problem via ℓp,q regularization ( 0⩽q⩽1⩽p ) in three aspects: theory, algorithm and application. In the theoretical aspect, we introduce a weak notion of joint restricted Frobenius norm condition associated with the ℓp,q regularization, and apply it to establish an oracle property and a recovery bound for the ℓp,q regularization of joint sparse optimization problem. In the algorithmic aspect, we apply the well-known proximal gradient algorithm to solve the ℓp,q regularization problems, provide analytical formulas for proximal subproblems of certain specific ℓp,q regularizations, and establish the global convergence and linear convergence rate of the proximal gradient algorithm under some mild conditions. More importantly, we propose two types of proximal gradient algorithms with the truncation technique and the continuation technique, respectively, and establish their convergence to the ground true joint sparse solution within a tolerance relevant to the noise level and the recovery bound under the assumption of restricted isometry property. In the aspect of application, we develop a novel method, based on joint sparse optimization with lower-order regularization and proximal gradient algorithm, to infer the master transcription factors for cell fate conversion, which is a powerful tool in developmental biology and regenerative medicine. Numerical results indicate that the novel method facilitates fast identification of master transcription factors, give raise to the possibility of higher successful conversion rate and in the hope of reducing biological experimental cost.

Iteratively reweighted least squares for block sparse signal recovery with unconstrained l2,p minimization

In this paper, we study an unconstrained [Formula: see text] minimization and its associated iteratively reweighted least squares algorithm (UBIRLS) for recovering block sparse signals. Wang et al. [Y. Wang, J. Wang and Z. Xu, On recovery of block-sparse signals via mixed [Formula: see text] [Formula: see text] norm minimization, EURASIP J. Adv. Signal Process. 2013(76) (2013) 76] have used numerical experiments to show the remarkable performance of UBIRLS algorithm for recovering a block sparse signal, but no theoretical analysis such as convergence and convergence rate analysis of UBIRLS algorithm was given. We focus on providing convergence and convergence rate analysis of UBIRLS algorithm for block sparse recovery problem. First, the convergence of UBIRLS is proved strictly. Second, based on the block restricted isometry property (block RIP) of linear measurement matrix [Formula: see text], we give the error bound analysis of the UBIRLS algorithm. Lastly, we also characterize the local convergence behavior of the UBIRLS algorithm. The simplicity of UBIRLS algorithm, along with the theoretical guarantees provided in this paper, will make a compelling case for its adoption as a standard tool for block sparse recovery.

AdaCS: Adaptive Compressive Sensing With Restricted Isometry Property-Based Error-Clamping.

Scene-dependent adaptive compressive sensing (CS) has been a long pursuing goal that has huge potential to significantly improve the performance of CS. However, with no access to the ground truth, how to design the scene-dependent adaptive strategy is still an open problem. In this paper, a restricted isometry property (RIP) condition-based error-clamping is proposed, which could directly predict the reconstruction error, i.e., the difference between the current-stage reconstructed image and the ground truth image, and adaptively allocate more samples to regions with larger reconstruction error at the next sampling stage. Furthermore, we propose a CS reconstruction network composed of Progressively inverse transform and Alternating Bi-directional Multi-grid Network, named PiABM-Net, that could efficiently utilize the multi-scale information for reconstructing the target image. The effectiveness of the proposed adaptive and cascaded CS method is demonstrated with extensive quantitative and qualitative experiments, compared with the state-of-the-art CS algorithms.

Uniform RIP analysis for the [formula omitted]-[formula omitted] minimization

This paper presents a uniform restricted isometry property (RIP) analysis of the ℓp−ωℓq minimization method for the recovery of sparse signals in both noiseless and noisy scenarios. The analysis is primarily based on a crucial sparse representation for any given signals. Moreover, we propose an effective algorithm for solving the ℓp−ωℓq minimization problem. The performance of this algorithm is validated through extensive numerical experiments on synthetic data, ECG signals, and grayscale images.

Seismic wavelet shape-oriented reflectivity inversion method

Abstract Reflectivity inversion plays a pivotal role in reservoir prediction. Conventional sparse-spike deconvolution assumes that the reflectivity (reflection coefficient) is sparse, which is solved based on the l1 norm. However, the restricted isometry property (RIP) of wavelet matrix and seismic effective bandwidth limits the accuracy of the sparse-spike reflectivity inversion. Consequently, we investigate the connection between seismic amplitude shape and reflectivity. When the reflectivity contains more non-zero values, the wavelet bandwidth within the effective seismic data bandwidth approaches a limit corresponding to the Sinc wavelet, where the main-lobe amplitude closely approximates the reflectivity. Conversely, when the reflectivity has fewer non-zero values, a wavelet with a smaller sidelobe provides a more accurate approximation of the reflectivity. In this paper, we propose a high-resolution inversion optimization method based on joint l2 norm and l1 norm constraints. By parameter tuning, we construct the Sinc wavelet or the wavelet with a weak-sidelobe corresponding to the seismic spectrum. Subsequently, we determine the extremum to approximate the reflectivity. To mitigate the RIP condition's constraints, we employ the l2 norm to balance the l1 norm (joint constraint) by introducing l2 norm with low-pass filtering characteristics. This approach yields more accurate reflectivity estimates. By taking the extremum, this approach yields more accurate reflectivity estimates. The synthetic test demonstrates that our method achieves better reflectivity inversion accuracy compared to sparse-spike inversion with l1–l2 norm constraint. Furthermore, field tests indicate that the proposed reflectivity inversion method not only can better match the well curve, but also exhibits excellent resolution.

A nonconvex sparse recovery method for DOA estimation based on the trimmed lasso

Sparse direction-of-arrival (DOA) estimation methods can be formulated as a group-sparse optimization problem. Meanwhile, sparse recovery methods based on nonconvex penalty terms have been a hot topic in recent years due to their several appealing properties. Herein, this paper studies a new nonconvex regularized approach called the trimmed lasso for DOA estimation. We define the penalty term of the trimmed lasso in the multiple measurement vector model by ℓ2,1-norm. First, we use the smooth approximation function to change the nonconvex objective function to the convex weighted problem. Next, we derive sparse recovery guarantees based on the extended Restricted Isometry Property and regularization parameter for the trimmed lasso in the multiple measurement vector problem. Our proposed method can control the desired level of sparsity of estimators exactly and give a more precise solution to the DOA estimation problem. Numerical simulations show that our proposed method overperforms traditional approaches, which is more close to the Cramer-Rao bound.

Sparse Signal Recovery via Rescaled Matching Pursuit

We propose the Rescaled Matching Pursuit (RMP) algorithm to recover sparse signals in high-dimensional Euclidean spaces. The RMP algorithm has less computational complexity than other greedy-type algorithms, such as Orthogonal Matching Pursuit (OMP). We show that if the restricted isometry property is satisfied, then the upper bound of the error between the original signal and its approximation can be derived. Furthermore, we prove that the RMP algorithm can find the correct support of sparse signals from random measurements with a high probability. Our numerical experiments also verify this conclusion and show that RMP is stable with the noise. So, the RMP algorithm is a suitable method for recovering sparse signals.

Sparse learning model with embedded RIP conditions for turbulence super-resolution reconstruction

In practical engineering scenarios, constraints arising from sensor placement, quantity, and the limitations of current testing technologies often lead to turbulence data characterized by low resolution and irregular structures. Turbulence super-resolution reconstruction is crucial for extracting finer details from irregularly structured, low-resolution measurement data, thereby facilitating comprehensive flow field analyses. This article introduces a sparse learning model (Embedding Restricted Isometry Property Autoencoder (RIP-AE)) for achieving super-resolution reconstruction of flow fields by obtaining compressed representations and sparse transform domain information. In this model, we embed the Restricted Isometry Property (RIP) condition to ensure the effectiveness of compressed representations and sparse transform domains, thereby enhancing the super-resolution reconstruction accuracy of the model. To assess the performance of the RIP-AE model, we utilized data sets of flow around a cylinder at different Reynolds numbers generated by unsteady Reynolds-Averaged Navier–Stokes simulations. We sequentially validate the effectiveness of the RIP condition at different Reynolds numbers (ranging from 1,000 to 500,000) and compare the reconstruction results of the RIP-AE model with those of the downsampled skip-connection/multi-scale (DSC/MS) and MS-AE models under different sampling ratios. The results indicate that the RIP-AE model excels in terms of L2 relative error and demonstrates the capability to achieve high-precision flow field reconstruction even under high sampling ratios.

Constructing low-rank Tucker tensor approximations using generalized completion

Abstract The projected gradient method for matrix completion is generalized towards the higher-dimensional case of low-rank Tucker tensors. It is shown that an operation order rearrangement in the common projected gradient approach provides a complexity improvement. An even better algorithm complexity can be obtained by replacing the completion operator by a general operator that satisfies restricted isometry property; however, such a replacement transforms the completion algorithm into an approximation algorithm.

Coherence-based sufficient condition for support recovery using block generalized orthogonal matching pursuit

Challenge is to find the support vectors of the unknown block sparse vector with compressed measurements in an underdetermined system where the number of unknowns is more than that of measurements. To recover unknown block sparse vector, restricted isometry property (RIP) is a sufficient condition need to be satisfied. Finding the restricted isometric constant is a non-polynomial hard problem for large values of n. In this paper coherence-based recovery guarantee has been proposed to recover the support vectors using block generalized orthogonal matching pursuit (BGOMP). It is proved that BGOMP can able to recover the support vectors with lesser number of iteration than block orthogonal matching pursuit (BOMP) by selecting multiple block support elements per iteration. Simulation results show detection performance of BGOMP is better than BOMP, block subspace pursuit (BSP) and block compressive sampling matching pursuit (BCoSaMP) for different block sparsity and block length. In most of the cases for different block sparsity and block length computation time for BGOMP is lesser than BCoSaMP, BSP and BOMP due to the multiple selection of elements in each iteration.

A theory of optimal convex regularization for low-dimensional recovery

Abstract We consider the problem of recovering elements of a low-dimensional model from under-determined linear measurements. To perform recovery, we consider the minimization of a convex regularizer subject to a data fit constraint. Given a model, we ask ourselves what is the ‘best’ convex regularizer to perform its recovery. To answer this question, we define an optimal regularizer as a function that maximizes a compliance measure with respect to the model. We introduce and study several notions of compliance. We give analytical expressions for compliance measures based on the best-known recovery guarantees with the restricted isometry property. These expressions permit to show the optimality of the $\ell ^{1}$-norm for sparse recovery and of the nuclear norm for low-rank matrix recovery for these compliance measures. We also investigate the construction of an optimal convex regularizer using the examples of sparsity in levels and of sparse plus low-rank models.

Conditioning of random Fourier feature matrices: double descent and generalization error

Abstract We provide high-probability bounds on the condition number of random feature matrices. In particular, we show that if the complexity ratio $N/m$, where $N$ is the number of neurons and $m$ is the number of data samples, scales like $\log ^{-1}(N)$ or $\log (m)$, then the random feature matrix is well-conditioned. This result holds without the need of regularization and relies on establishing various concentration bounds between dependent components of the random feature matrix. Additionally, we derive bounds on the restricted isometry constant of the random feature matrix. We also derive an upper bound for the risk associated with regression problems using a random feature matrix. This upper bound exhibits the double descent phenomenon and indicates that this is an effect of the double descent behaviour of the condition number. The risk bounds include the underparameterized setting using the least squares problem and the overparameterized setting where using either the minimum norm interpolation problem or a sparse regression problem. For the noiseless least squares or sparse regression cases, we show that the risk decreases as $m$ and $N$ increase. The risk bound matches the optimal scaling in the literature and the constants in our results are explicit and independent of the dimension of the data.

Analysis of T-Orthogonal greedy algorithm in noisy measurement

We define the Tikhonov Orthogonal Greedy algorithm (T-OGA), a variant of the orthogonal greedy algorithm, to study the recovery of sparse signals from noisy measurements. We establish sufficient conditions for T-OGA to recover sparse signals via restricted isometry property (RIP) inherited from frames. We introduce various concepts such as match matrix, match vector, and residue vector to execute T-OGA. Various technical lemmas have been established to prove our main theorem. In the execution of T-OGA, we employ Tikhonov regularization with regularization parameter λ to solve the minimization problem for the N-sparse solution. In our main result, we proved that if a frame satisfies the RIP of order N + 1 with isometry constant τ and (τ+λ)<13N , then T-OGA can recover every N-sparse signal in the atmost N iterations.

A novel non-convex penalty function-based STAP algorithm for airborne passive radar systems

In conventional sparse Space-Time Adaptive Processing (STAP) algorithms, the l0-norm is often used instead of the l1-norm to relax convexity constraints. The approach presented in this paper can be mathematically reformulated as a linear problem, which can be efficiently solved using various convex optimization techniques. While this method effectively circumvents the NP-Hard complexity associated with the l0-norm, it does come with certain potential challenges. For example, the observation matrix must meet the criteria of the Restricted Isometry Property (RIP) and other stringent conditions. In this research, a novel STAP algorithm is introduced, which is based on a non-convex penalty function. This innovative algorithm substitutes the traditional l0-norm with a custom-designed non-convex penalty function and is applied to the Direct Filter Processor (DFP), solved using the recursive least squares (RLS) algorithm. The resulting algorithm, known as gp-RLS, is demonstrated through Monte Carlo simulations to outperform other l1-based and reduced-rank STAP algorithms in terms of faster convergence, improved signal-to-clutter-plus-noise ratio (SCNR), and enhanced target detection performance.

Sufficient condition based on nearly optimal order RIC for IHT algorithm

Iterative hard thresholding (IHT) is one class of sparse signal recovery algorithms widely used in compressed sensing. The well-known restricted isometry constant (RIC) is one of the most popular frameworks used to ensure the convergence of recovery algorithms. In previous literature, sufficient conditions to guarantee the recovery performance of the IHT algorithm have usually been established based on RICs with orders 3 s and 2 s , while no such condition in terms of RIC with order less than 2 s has been reported yet. In this study, based on the theoretical optimal step-length, we develop the RIC condition γ < ( 5 − 1 ) / 8 ≈ 0.1545 for successful recovery via the IHT algorithm. Here, γ = δ s if s is an even number, otherwise γ = δ s + 1 . To the best of our knowledge, this is the first result based on RIC with nearly optimal order for the IHT algorithm.

A continuous-time neurodynamic approach in matrix form for rank minimization

This article proposes a continuous-time neurodynamic approach for solving the rank minimization under affine constraints. As opposed to the traditional neurodynamic approach, the proposed neurodynamic approach extends the form of the variables from the vector form to the matrix form. First, a continuous-time neurodynamic approach with variables in matrix form is developed by combining the optimal rank r projection and the gradient. Then, the optimality of the proposed neurodynamic approach is rigorously analyzed by demonstrating that the objective function satisfies the functional property which is called as (2r,4r)-restricted strong convexity and smoothness ((2r,4r)-RSCS). Furthermore, the convergence and stability analysis of the proposed neurodynamic approach is rigorously conducted by establishing appropriate Lyapunov functions and considering the relevant restricted isometry property (RIP) condition associated with the affine transformation. Finally, through experiments involving low-rank matrix recovery under affine transformations and the completion of low-rank real image, the effectiveness of this approach has been demonstrated, along with its superiority compared to the vector-based approach.

The Perturbation Analysis of Nonconvex Low-Rank Matrix Robust Recovery.

In this article, we bring forward a completely perturbed nonconvex Schatten p -minimization to address a model of completely perturbed low-rank matrix recovery (LRMR). This article based on the restricted isometry property (RIP) and the Schatten- p null space property (NSP) generalizes the investigation to a complete perturbation model thinking over not only noise but also perturbation, and it gives the RIP condition and the Schatten- p NSP assumption that guarantee the recovery of low-rank matrix and the corresponding reconstruction error bounds. In particular, the analysis of the result reveals that in the case that p decreases 0 and for the complete perturbation and low-rank matrix, the condition is the optimal sufficient condition (Recht et al., 2010). In addition, we study the connection between RIP and Schatten- p NSP and discern that Schatten- p NSP can be inferred from the RIP. The numerical experiments are conducted to show better performance and provide outperformance of the nonconvex Schatten p -minimization method comparing with the convex nuclear norm minimization approach in the completely perturbed scenario.