Number Of Measurements Scales Research Articles

Information-Theoretic Lower Bounds for Compressive Sensing With Generative Models

It has recently been shown that for compressive sensing, significantly fewer measurements may be required if the sparsity assumption is replaced by the assumption the unknown vector lies near the range of a suitably-chosen generative model. In particular, in (Bora et al., 2017) it was shown roughly O(k log L) random Gaussian measurements suffice for accurate recovery when the generative model is an L-Lipschitz function with bounded k-dimensional inputs, and O(kdlogw) measurements suffice when the generative model is a k-input ReLU network with depth d and width w. In this paper, we establish corresponding algorithm-independent lower bounds on the sample complexity using tools from minimax statistical analysis. In accordance with the above upper bounds, our results are summarized as follows: (i) We construct an L-Lipschitz generative model capable of generating group-sparse signals, and show that the resulting necessary number of measurements is Ω(k logL); (ii) Using similar ideas, we construct ReLU networks with high depth and/or high depth for which the necessary number of measurements scales as Ω(kd logw/logn) (with output dimension n), and in some cases Ω(kd logw). As a result, we establish that the scaling laws derived in (Bora et al., 2017) are optimal or near-optimal in the absence of further assumptions.

Bispectrum Inversion with Application to Multireference Alignment.

We consider the problem of estimating a signal from noisy circularly-translated versions of itself, called multireference alignment (MRA). One natural approach to MRA could be to estimate the shifts of the observations first, and infer the signal by aligning and averaging the data. In contrast, we consider a method based on estimating the signal directly, using features of the signal that are invariant under translations. Specifically, we estimate the power spectrum and the bispectrum of the signal from the observations. Under mild assumptions, these invariant features contain enough information to infer the signal. In particular, the bispectrum can be used to estimate the Fourier phases. To this end, we propose and analyze a few algorithms. Our main methods consist of non-convex optimization over the smooth manifold of phases. Empirically, in the absence of noise, these non-convex algorithms appear to converge to the target signal with random initialization. The algorithms are also robust to noise. We then suggest three additional methods. These methods are based on frequency marching, semidefinite relaxation and integer programming. The first two methods provably recover the phases exactly in the absence of noise. In the high noise level regime, the invariant features approach for MRA results in stable estimation if the number of measurements scales like the cube of the noise variance, which is the information-theoretic rate. Additionally, it requires only one pass over the data which is important at low signal-to-noise ratio when the number of observations must be large.

Cosmological Distance Scale. Part 4. Calibration Based on SN Ia Supernovae

A number of measurement scales for distance are employed in cosmology: aberration, commoving, photometric, and angular diameter. The photometric distance scale D L based on the absolute magnitude standard M st SN Ia of type SN Ia supernovae has yielded a fundamental result: the expansion of the universe is accelerating. The photometric distance scale is based on the fact that in a binary star system, when a white dwarf reaches the Chandrasekhar mass limit [1] owing to accretion of matter from a satellite or merging, it becomes a supernova with an almost constant absolute stellar magnitude at the luminosity peak. Thus, the importance of a calibration using this type of reference scale for cosmological distances based on red shift, which is inherent in almost all extragalactic objects, is evident. Scales of this type are calibrated by joint measurements. In this case, it is necessary to refi ne the photoelectric distance moduli for type SN Ia supernovae based on light curve templates by introducing corrections to the data on observed stellar magnitudes at the luminosity peak m peak and fi tting the free parameters of the model to the resulting standards:

Observability With Random Observations

Recovery of the initial state of a high-dimensional system can require a large number of measurements. In this paper, we explain how this burden can be significantly reduced when randomized measurement operators are employed. Our work builds upon recent results from Compressive Sensing (CS). In particular, we make the connection to CS analysis for random block diagonal matrices. By deriving Concentration of Measure (CoM) inequalities, we show that the observability matrix satisfies the Restricted Isometry Property (RIP) (a sufficient condition for stable recovery of sparse vectors) under certain conditions on the state transition matrix. For example, we show that if the state transition matrix is unitary, and if independent, randomly-populated measurement matrices are employed, then it is possible to uniquely recover a sparse high-dimensional initial state when the total number of measurements scales linearly in the sparsity level (the number of non-zero entries) of the initial state and logarithmically in the state dimension. We further extend our RIP analysis for scaled unitary and symmetric state transition matrices. We support our analysis with a case study of a diffusion process.

Analogy between electric parameters of inhomogeneous media and geometrical characteristics of the fractal line

Analogy is established between the surface impedance modulus, frequency, and skin depth of inhomogeneous media on the one hand and the fractal line length, measurement scale, and the number of measurement scales required for measuring length, respectively, on the other hand. This analogy makes it possible to obtain frequency characteristics of the skin layer and the surface impedance modulus independently from other methods. The frequency characteristics satisfactorily describe sedimentary and crystalline rocks in a wide frequency range.

Information Theoretic Bounds for Compressed Sensing

In this paper we derive information theoretic performance bounds to sensing and reconstruction of sparse phenomena from noisy projections. We consider two settings: output noise models where the noise enters after the projection and input noise models where the noise enters before the projection. We consider two types of distortion for reconstruction: support errors and mean-squared errors. Our goal is to relate the number of measurements, $m$, and $\snr$, to signal sparsity, $k$, distortion level, $d$, and signal dimension, $n$. We consider support errors in a worst-case setting. We employ different variations of Fano's inequality to derive necessary conditions on the number of measurements and $\snr$ required for exact reconstruction. To derive sufficient conditions we develop new insights on max-likelihood analysis based on a novel superposition property. In particular this property implies that small support errors are the dominant error events. Consequently, our ML analysis does not suffer the conservatism of the union bound and leads to a tighter analysis of max-likelihood. These results provide order-wise tight bounds. For output noise models we show that asymptotically an $\snr$ of $\Theta(\log(n))$ together with $\Theta(k \log(n/k))$ measurements is necessary and sufficient for exact support recovery. Furthermore, if a small fraction of support errors can be tolerated, a constant $\snr$ turns out to be sufficient in the linear sparsity regime. In contrast for input noise models we show that support recovery fails if the number of measurements scales as $o(n\log(n)/SNR)$ implying poor compression performance for such cases. We also consider Bayesian set-up and characterize tradeoffs between mean-squared distortion and the number of measurements using rate-distortion theory.