Underdetermined System Research Articles

Iterative re-weighted least squares algorithm for lp-minimization with tight frame and 0 < p ≤ 1

In this paper, we recover signals that are sparse with a tight frame from an underdetermined system via lp-minimization with p∈(0,1]. We prove that if the measurement matrix A satisfies null space property of order s with a tight frame in lp-norm (p-D-NSP), then a signal that is s-sparse with this tight frame can be exactly recovered via lp-minimization. An iterative re-weighted least squares algorithm is presented to solve lp-minimization problems, and we prove the convergence and error bounds of this algorithm under a p-D-NSP condition. The numerical results show that this algorithm performs effectively for signal reconstruction of both sparse and approximately sparse signals with a tight frame. The algorithm converges faster if p<1 (nonconvex scenario) than if p=1 (convex scenario).

A New Smoothed L0 Regularization Approach for Sparse Signal Recovery

Sparse signal reconstruction, as the main link of compressive sensing (CS) theory, has attracted extensive attention in recent years. The essence of sparse signal reconstruction is how to recover the original signal accurately and effectively from an underdetermined linear system equation (ULSE). For this problem, we propose a new algorithm called regularization reweighted smoothed L0 norm minimization algorithm, which is simply called RRSL0 algorithm. Three innovations are made under the framework of this method: (1) a new smoothed function called compound inverse proportional function (CIPF) is proposed; (2) a new reweighted function is proposed; and (3) a mixed conjugate gradient (MCG) method is proposed. In this algorithm, the reweighted function and the new smoothed function are combined as the sparsity promoting objective, and the constraint condition y-Φx22 is taken as a deviation term. Both of them constitute an unconstrained optimization problem under the Tikhonov regularization criterion and the MCG method constructed is used to optimize the problem and realize high-precision reconstruction of sparse signals under noise conditions. Sparse signal recovery experiments on both the simulated and real data show the proposed RRSL0 algorithm performs better than other popular approaches and achieves state-of-the-art performances in signal and image processing.

TADA: phylogenetic augmentation of microbiome samples enhances phenotype classification.

MotivationLearning associations of traits with the microbial composition of a set of samples is a fundamental goal in microbiome studies. Recently, machine learning methods have been explored for this goal, with some promise. However, in comparison to other fields, microbiome data are high-dimensional and not abundant; leading to a high-dimensional low-sample-size under-determined system. Moreover, microbiome data are often unbalanced and biased. Given such training data, machine learning methods often fail to perform a classification task with sufficient accuracy. Lack of signal is especially problematic when classes are represented in an unbalanced way in the training data; with some classes under-represented. The presence of inter-correlations among subsets of observations further compounds these issues. As a result, machine learning methods have had only limited success in predicting many traits from microbiome. Data augmentation consists of building synthetic samples and adding them to the training data and is a technique that has proved helpful for many machine learning tasks.ResultsIn this paper, we propose a new data augmentation technique for classifying phenotypes based on the microbiome. Our algorithm, called TADA, uses available data and a statistical generative model to create new samples augmenting existing ones, addressing issues of low-sample-size. In generating new samples, TADA takes into account phylogenetic relationships between microbial species. On two real datasets, we show that adding these synthetic samples to the training set improves the accuracy of downstream classification, especially when the training data have an unbalanced representation of classes.Availability and implementationTADA is available at https://github.com/tada-alg/TADA.Supplementary information Supplementary data are available at Bioinformatics online.

Estimation of Sparse VAR Models with Artificial Neural Networks for the Analysis of Biosignals.

Vector autoregressive models (VAR models) are often used to model and to analyze multivariate time series, especially to provide short-term forecasts. A common method of estimating coefficients of these VAR models is solving the Yule- Walker equations. This work introduces and investigates a method to set up "sparse" VAR models, in order to obtain a comparable prognosis quality with significantly fewer coefficients. For this purpose, an artificial neural network was programmed in Python with TensorFlow. Sparsity arises from the implementation of regularization algorithms.Based on simulated data and an ECG, we show that a comparable prognosis quality can be achieved with significantly fewer coefficients. In addition, sparse VAR models can also be determined if the data would actually lead to an underdetermined system of equations. Thus, sparse VAR models may help to classify short epochs of biosignals, e.g. P-waves or QRS-complexes.

Multidimensional Gains for Stochastic Approximation.

This paper deals with iterative Jacobian-based recursion technique for the root-finding problem of the vector-valued function, whose evaluations are contaminated by noise. Instead of a scalar step size, we use an iterate-dependent matrix gain to effectively weigh the different elements associated with the noisy observations. The analytical development of the matrix gain is built on an iterative-dependent linear function interfered by additive zero-mean white noise, where the dimension of the function is M ≥ 1 and the dimension of the unknown variable is N ≥ 1 . Necessary and sufficient conditions for M ≥ N algorithms are presented pertaining to algorithm stability and convergence of the estimate error covariance matrix. Two algorithms are proposed: one for the case where M ≥ N and the second one for the antithesis. The two algorithms assume full knowledge of the Jacobian. The recursive algorithms are proposed for generating the optimal iterative-dependent matrix gain. The proposed algorithms here aim for per-iteration minimization of the mean square estimate error. We show that the proposed algorithm satisfies the presented conditions for stability and convergence of the covariance. In addition, the convergence rate of the estimation error covariance is shown to be inversely proportional to the number of iterations. For the antithesis , contraction of the error covariance is guaranteed. This underdetermined system of equations can be helpful in training neural networks. Numerical examples are presented to illustrate the performance capabilities of the proposed multidimensional gain while considering nonlinear functions.

Uncertainty Quantification for Ti-7Al Alloy Microstructure with an Inverse Analytical Model (AUQLin).

The present study addresses an inverse problem for observing the microstructural stochasticity given the variations in the macro-scale material properties by developing an analytical uncertainty quantification (UQ) model called AUQLin. The uncertainty in the material property is modeled with the analytical algorithm, and then the uncertainty propagation to the microstructure is solved with an inverse problem that utilizes the transformation of random variables principle. The inverse problem leads to an underdetermined linear system, and thus produces multiple solutions to the statistical features of the microstructure. The final solution is decided by solving an optimization problem which aims to minimize the difference between the computed and experimental statistical parameters of the microstructure. The final result for the computed microstructural uncertainty is found to provide a good match to the experimental microstructure information.

The Shuey-Radon transform

The traditional hyperbolic Radon transform (RT) decomposes seismic data into a sum of constant amplitude basis functions. This limits the performance of the transform when dealing with real data in which the reflection amplitudes include the amplitude variation with offset (AVO) variations. We adopted the Shuey-Radon transform as a combination of the RT and Shuey’s approximation of reflectivity to accurately model reflections including AVO effects. The new transform splits the seismic gather into three Radon panels: The first models the reflections at zero offset, and the other two panels add capability to model the AVO gradient and curvature. There are two main advantages of the Shuey-Radon transform over similar algorithms, which are based on a polynomial expansion of the AVO response. (1) It is able to model reflections more accurately. This leads to more focused coefficients in the transform domain and hence provides more accurate processing results. (2) Unlike polynomial-based approaches, the coefficients of the Shuey-Radon transform are directly connected to the classic AVO parameters (intercept, gradient, and curvature). Therefore, the resulting coefficients can further be used for interpretation purposes. The solution of the new transform is defined via an underdetermined linear system of equations. It is formulated as a sparsity-promoting optimization, and it is solved efficiently using an orthogonal matching pursuit algorithm. Applications to different numerical experiments indicate that the Shuey-Radon transform outperforms the polynomial and conventional RTs.

On the Computation of Sparse Solutions to the Controllability Problem for Discrete-Time Linear Systems

In this work, we address the fundamental problem of steering the state of a discrete-time linear system to the origin after a given (finite) number of stages by means of the sparsest possible control sequence, that is, the sequence of inputs comprised of the maximum possible number of null elements. In our approach, the latter controllability problem is associated with the problem of finding either the minimum 1-norm solution or the minimum p-norm, with p taking values greater than zero and less than one, solution of an under-determined system of linear equations, which are both known to exhibit good sparsity properties under certain technical assumptions. Motivated by practical considerations, we compute approximate solutions to the latter optimization problems by utilizing the class of iteratively weighted least squares algorithms from the literature of compressive (or compressed) sensing. This particular choice of algorithms is motivated by (1) their straightforward implementation, which makes them appealing to the non-expert and (2) the fact that some of the most costly operations involved in their implementation can be carried out recursively by leveraging well-known properties of the controllability Grammian of a discrete-time linear system. Finally, we apply the proposed approach to a spacecraft proximity operation problem and in particular, a linearized impulsive fixed-time minimum-fuel rendezvous problem in which the 1-norm serves as a proxy to the fuel consumption at a given time interval.

Improved Reconstruction of MR Scanned Images by Using a Dictionary Learning Scheme.

The application of compressed sensing (CS) to biomedical imaging is sensational since it permits a rationally accurate reconstruction of images by exploiting the image sparsity. The quality of CS reconstruction methods largely depends on the use of various sparsifying transforms, such as wavelets, curvelets or total variation (TV), to recover MR images. As per recently developed mathematical concepts of CS, the biomedical images with sparse representation can be recovered from randomly undersampled data, provided that an appropriate nonlinear recovery method is used. Due to high under-sampling, the reconstructed images have noise like artifacts because of aliasing. Reconstruction of images from CS involves two steps, one for dictionary learning and the other for sparse coding. In this novel framework, we choose Simultaneous code word optimization (SimCO) patch-based dictionary learning that updates the atoms simultaneously, whereas Focal underdetermined system solver (FOCUSS) is used for sparse representation because of a soft constraint on sparsity of an image. Combining SimCO and FOCUSS, we propose a new scheme called SiFo. Our proposed alternating reconstruction scheme learns the dictionary, uses it to eliminate aliasing and noise in one stage, and afterwards restores and fills in the k-space data in the second stage. Experiments were performed using different sampling schemes with noisy and noiseless cases of both phantom and real brain images. Based on various performance parameters, it has been shown that our designed technique outperforms the conventional techniques, like K-SVD with OMP, used in dictionary learning based MRI (DLMRI) reconstruction.

New Recurrent Neural Network for Online Solution of Time-Dependent Underdetermined Linear System With Bound Constraint

Recurrent neural network (RNN) has recently been viewed as a significant alternative to online mathematical problem solving. This paper offers important improvements by proposing the first RNN model to solve the time-dependent underdetermined linear system with bound constraint. In particular, by introducing a time-dependent nonnegative vector, the bound-constrained underdetermined linear system is initially transformed into a time-dependent system that comprises linear and nonlinear equations. The newly constructed RNN model can thus zero in on the time-dependent equations. Then, the model is theoretically proven to have convergence properties, and the simulation results further substantiate the efficacy of the proposed RNN model to solve the time-dependent underdetermined linear system with bound constraint. Finally, the proposed RNN model is applied to physically constrained redundant robot manipulators, thereby indicating the applicability of the proposed model.

Exploiting the Underdetermined System in Multichannel Active Noise Control for Open Windows

Active noise control (ANC) is a re-emerging technique to mitigate noise pollution. To reduce the noise power in large spaces, multiple channels are usually required, which complicates the implementation of ANC systems. In this paper, we separate the multichannel ANC problem into two subproblems, where the subproblem of computing the control filter is usually an underdetermined problem. Therefore, we could leverage the underdetermined system to simplify the ANC system without degrading the noise reduction performance. For a single incidence, we compare the conventional fully-coupled (pseudoinverse) multichannel control with the colocated (diagonal) control method and find that they can achieve equivalent performance, but the colocated control method is less computationally intensive. Furthermore, the underdetermined system presents an opportunity to control noise from multiple incidences with one common fixed filter. Both the full-rank and the overdetermined optimal control filters are realized. The performance of these control methods was analyzed numerically with the Finite Element Method (FEM) and the results validate the feasibility of the full-rank and overdetermined optimal control methods, where the latter could even offer more robust performance in more complex noise scenarios.

Efficient Detection Scheme for Physical-Layer Network Coding in Multiway Relay Channels

In this paper, we propose a novel scheme for physical-layer network coding (PNC) in multiway relay channels (MWRC) from the perspective of sequential multiuser detection. We consider an uplink MWRC scenario where N users, each equipped with a single antenna, simultaneously transmit their signal to a relay equipped with K antennas (K <; N). Extraction of the network codes from the superimposed user signals at the relay node is formulated as an under-determined linear system. To solve this problem with low decoding complexity, the proposed method combines successive interference cancellation (SIC) with Babai estimation for regularized integer least squares (ILS). Specifically, SIC decoding is first employed to detect a selected subset of stronger user signals and remove their interfering effects. Babai estimation is then applied to extract the remaining user signals, which is formulated as an ILS problem with reduced dimension. We develop a power allocation scheme to enhance the performance of both SIC and ILS steps, and discuss an optimal user pairing strategy based on the average decoding error probability. Numerical results demonstrate the performance improvement of the proposed method in extracting network codes from multiple superimposed user signals.

A Novel 2D Off-Grid DOA Estimation Method Based on Compressive Sensing and Least Square Optimization

For two-dimensional (2D) direction of arrival (DOA) estimation in a uniform rectangular array (URA), the conventional method converts the 2D problem into a one-dimensional (1D) problem; however, the computational complexity is high, and the accuracy is limited by the grid interval. To address this issue, based on sparse representation theory and a separable observation model (SPM), this paper presents a novel off-grid-framework-based 2D DOA estimation approach by designing a modified 2D off-grid model and a solution for the multisnapshot case in the SPM. The proposed algorithm can be divided into two stages. In the first stage, we use a matching pursuit and focal underdetermined system solver (MFOCUSS) algorithm to quickly identify the candidate or potential areas where the true sources may exist. In the second stage, the candidate areas obtained in the first stage are regarded as the initialization. Then, for a specific source, we regard other sources as interference. By using an alternating descent method, we can obtain accurate DOAs. Moreover, based on the equivalence of time delay and spatial spacing, a 2D off-grid method for multisnapshot cases is proposed in this paper. Numerical simulations demonstrate the effectiveness and efficiency of the proposed algorithm.

A Regularized Weighted Smoothed L₀ Norm Minimization Method for Underdetermined Blind Source Separation.

Compressed sensing (CS) theory has attracted widespread attention in recent years and has been widely used in signal and image processing, such as underdetermined blind source separation (UBSS), magnetic resonance imaging (MRI), etc. As the main link of CS, the goal of sparse signal reconstruction is how to recover accurately and effectively the original signal from an underdetermined linear system of equations (ULSE). For this problem, we propose a new algorithm called the weighted regularized smoothed -norm minimization algorithm (WReSL0). Under the framework of this algorithm, we have done three things: (1) proposed a new smoothed function called the compound inverse proportional function (CIPF); (2) proposed a new weighted function; and (3) a new regularization form is derived and constructed. In this algorithm, the weighted function and the new smoothed function are combined as the sparsity-promoting object, and a new regularization form is derived and constructed to enhance de-noising performance. Performance simulation experiments on both the real signal and real images show that the proposed WReSL0 algorithm outperforms other popular approaches, such as SL0, BPDN, NSL0, and L-RLSand achieves better performances when it is used for UBSS.

A Novel Compressed Sensing Technique for Traffic Matrix Estimation of Software Defined Cloud Networks

Traffic Matrix estimation has always caught attention from researchers for better network management and future planning. With the advent of high traffic loads due to Cloud Computing platforms and Software Defined Networking based tunable routing and traffic management algorithms on the Internet, it is more necessary as ever to be able to predict current and future traffic volumes on the network. For large networks such origin-destination traffic prediction problem takes the form of a large under-constrained and under-determined system of equations with a dynamic measurement matrix. In this work, we present our Compressed Sensing with Dynamic Model Estimation (CS-DME) architecture suitable for modern software defined networks. Our main contributions are: (1) we formulate an approach in which measurement matrix in the compressed sensing scheme can be accurately and dynamically estimated through a reformulation of the problem based on traffic demands. (2) We show that the problem formulation using a dynamic measurement matrix based on instantaneous traffic demands may be used instead of a stationary binary routing matrix which is more suitable to modern Software Defined Networks that are constantly evolving in terms of routing by inspection of its Eigen Spectrum using two real world datasets. (3) We also show that linking this compressed measurement matrix dynamically with the measured parameters can lead to acceptable estimation of Origin Destination (OD) Traffic flows with marginally poor results with other state-of-art schemes relying on fixed measurement matrices. (4) Furthermore, using this compressed reformulated problem, a new strategy for selection of vantage points for most efficient traffic matrix estimation is also presented through a secondary compression technique based on subset of link measurements.

A Bat-Inspired Sparse Recovery Algorithm for Compressed Sensing.

Compressed sensing (CS) is an important research area of signal sampling and compression, and the essence of signal recovery in CS is an optimization problem of solving the underdetermined system of equations. Greedy pursuit algorithms are widely used to solve this problem. They have low computational complexity; however, their recovery performance is limited. In this paper, an intelligence recovery algorithm is proposed by combining the Bat Algorithm (BA) and the pruning technique in subspace pursuit. Experimental results illustrate that the proposed algorithm has better recovery performance than greedy pursuit algorithms. Moreover, applied to the microseismic monitoring system, the BA can recover the signal well.

Nonlinear electrocardiographic imaging using polynomial approximation networks.

Electrocardiography is a valuable tool to aid in medical understanding and treatment of heart-related ailments, specifically atrial fibrillation (AF) and other irregular cardiac behavior. Although signs of AF will manifest in conventional electrocardiogram (ECG) recordings, interpretation and localization of AF sources require significant clinical expertise. In this vein, electrocardiographic imaging has emerged as an important medical imaging modality that provides reconstructions of the heart's electrical activity from non-invasive multi-lead body-surface ECG and anatomical x-ray computed tomography images. In this paper, we present a nonlinear inversion model for computing this mapping to improve upon the reconstruction performance of current methods. While contemporary techniques typically determine an inverse solution by discretizing and inverting an underdetermined linear system of partial differential equations governing the relationship between voltage potentials of the heart and torso, the presented technique re-casts this problem as a task in function approximation and provides a direct parameterization of the inverse operator using a polynomial neural network. That is, the outlined nonlinear inversion technique is a generalization of contemporary reconstruction techniques which allows geometrical and material parameterizations of the forward-model to be optimized using real experimental data collected from patients suffering from AF, as to better represent the inverse operator with respect to reconstruction metrics applicable to electrophysiology. The accuracy of our model is evaluated against a dataset of real-patient recordings to demonstrate its validity, and mathematical analysis is provided to support the polynomial expansion used in our inversion model.

Sparse Vector Coding for Ultra Reliable and Low Latency Communications

Ultra reliable and low latency communication (URLLC) is a newly introduced service category in 5G to support delay-sensitive applications. In order to support this new service category, the 3rd Generation Partnership Project (3GPP) sets an aggressive requirement that a packet should be delivered with 10−5 packet error rate within 1-ms transmission period. Since the current wireless transmission scheme, which is designed to maximize the coding gain by transmitting the capacity achieving long codeblock, is not relevant for this purpose, and a new transmission scheme to support URLLC is required. In this paper, we propose a new approach to support the short packet transmission, called sparse vector coding (SVC). The key idea behind the proposed SVC technique is to transmit the information after the sparse vector transformation. By mapping the information into the position of nonzero elements and then transmitting it after random spreading, we obtain an underdetermined sparse system for which the principle of compressed sensing can be applied. From the numerical evaluations and performance analysis, we demonstrate that the proposed SVC technique is very effective in URLLC transmission and outperforms the 4G LTE and LTE-Advanced scheme.

Random Matching Pursuit for Image Watermarking

The classical solution to an underdetermined system of linear equations mainly has two opposite directions, which lead to either a large $\ell _{2}$ -norm sparse solution or a non-sparse minimum $\ell _{2}$ -norm solution. In this paper, we systematically show that by modifying the well-known basic matching pursuit algorithm originally proposed to identify the sparse solution, an alternative solution between the two classical ones could be obtained. The modified algorithm, termed as random matching pursuit (RMP), is then used to create a novel image watermarking framework. Compared to conventional systems, the security is substantially improved by the use of random over-complete dictionaries and the order parameter of RMP. Capacity can also be increased thanks to the transform with over-complete dictionaries that could expand signal dimension. Meanwhile, imperceptibility and robustness properties of the proposed design framework are not compromised. The classical spread spectrum and improved spread spectrum techniques are applied to the proposed framework for practical implementations. The novelty and effectiveness of the proposed systems are supported by rigorous performance analysis and experimental results using an image data set. This paper reveals the potential of using over-complete dictionaries in multimedia watermarking systems, which theoretically leads to the exploration of alternative candidates among the infinite solutions to underdetermined linear systems other than minimum $\ell _{2}$ -norm and sparse ones.

A preconditioning approach for improved estimation of sparse polynomial chaos expansions

Compressive sampling has been widely used for sparse polynomial chaos (PC) approximation of stochastic functions. The recovery accuracy of compressive sampling highly depends on the incoherence properties of the measurement matrix. In this paper, we consider preconditioning the underdetermined system of equations that is to be solved. Premultiplying a linear equation system by a non-singular matrix results in an equivalent equation system, but it can potentially improve the incoherence properties of the resulting preconditioned measurement matrix and lead to a better recovery accuracy. When measurements are noisy, however, preconditioning can also potentially result in a worse signal-to-noise ratio, thereby deteriorating recovery accuracy. In this work, we propose a preconditioning scheme that improves the incoherence properties of measurement matrix and at the same time prevents undesirable deterioration of signal-to-noise ratio. We provide theoretical motivations and numerical examples that demonstrate the promise of the proposed approach in improving the accuracy of estimated polynomial chaos expansions.