Band-limited Signals Research Articles

Wave Theory of Information [Book Review

Information theory textbooks and courses largely limit themselves to a purely mathematical treatment of the subject. However, the real world is not just limited by laws of mathematics, but also by laws of physics. This book attempts to fill this gap at least in the context of wave theory. The book weaves beautiful concepts in physics and information theory in a manner that is equally engaging to a student and an expert in information theory. The author is a renowned researcher from UC San Diego, who is responsible for one of the central results characterizing the degrees of freedom of electromagnetic (EM) waves in a scattering environment. Indeed, central to the book is the concept — and the quantification — of the number of degrees of freedom of spatio-temporal signals. A rigorous treatment of degrees of freedom of bandlimited signals is provided in Chapters 2 and 3. Chapters 4 and 5 provide the background on EM waves and their propagation. Fundamental results from information theory are discussed on the way, including those on representation, communication and sensing of signals in space and time. Chapter 7 discusses fundamentals of wireless communication, including an overview of current technologies and strategies for point-to-point and network communication, with an emphasis on attainable degrees of freedom. This builds up to Chapters 8 and 9, where it is shown how, in an arbitrary scattering environment, the radiated field filtered by the Green’s operator becomes spatially bandlimited. Finally, the number of degrees of freedom of bandlimited waves propagating in physical media is characterized through the concept of surface cut-set integral. This establishes limits on amount of information that can radiate from a scattering system, an aspect of communication theory that has been of interest for several decades before significant advances in the 2000s. Ensuing chapters discuss additional in-depth results. At the end of every chapter there is a brief history of how the concepts presented in that chapter developed. I believe that this book is a great success for the following reason: it enables information theorists with interests in physics to understand and contribute to the solution of informational puzzles and paradoxes in physics, which can (and will) enrich both areas. The book is timely: information theory is expanding its horizons and now, more than ever, there is a need to closely tie the information- theoretic understanding with fundamental physics. The book is also timeless: it serves as a great illustration of how, in our attempt to expand these horizons, we still need to embed our understanding in intellectual depth and mathematical rigor and avoid the “scientific bandwagon” that Shannon warned us of in his 1956 article.

The reweighted Landweber scheme for the extrapolation of band-limited signals

The extrapolation of a band-limited signal is in terms of a finite known segment of it in the time interval [−T,T] for T > 0 to determine the unknown part of the signal. Extrapolation in the frequency domain is represented by an integral equation and then discretized as a linear system. If T is small, the theoretical ill-posedness of the extrapolation leads to the linear system having ill-posedness because the condition number of the coefficient matrix is large. The condition number of a matrix equals the maximum singular value of the matrix divided by the minimum singular value. Owing to the large condition number, the solution contains large error, relative to the true solution. We use a reweighting method to improve the condition number for the extrapolation of band-limited signal simulations and propose a reweighted Landweber scheme. Simulations validate the proposed method. Effective reconstructions are obtained from the same number of samples in an appropriately small time interval [−T,T].

Application of signal analysis to the embedding problem of $${\mathbb {Z}}^k$$-actions

We study the problem of embedding arbitrary $${\mathbb {Z}}^k$$ -actions into the shift action on the infinite dimensional cube $$\left( [0,1]^D\right) ^{{\mathbb {Z}}^k}$$ . We prove that if a $${\mathbb {Z}}^k$$ -action X satisfies the marker property (in particular if X is a minimal system without periodic points) and if its mean dimension is smaller than D / 2 then we can embed it in the shift on $$\left( [0,1]^D\right) ^{{\mathbb {Z}}^k}$$ . The value D / 2 here is optimal. The proof goes through signal analysis. We develop the theory of encoding $${\mathbb {Z}}^k$$ -actions into band-limited signals and apply it to proving the above statement. Main technical difficulties come from higher dimensional phenomena in signal analysis. We overcome them by exploring analytic techniques tailored to our dynamical settings. The most important new idea is to encode the information of a tiling of $${\mathbb {R}}^k$$ into a band-limited function which is constructed from another tiling.

Delta-Ramp Encoder for Amplitude Sampling and Its Interpretation as Time Encoding

The theoretical basis for conventional acquisition of bandlimited signals typically relies on uniform time sampling and assumes infinite-precision amplitude values. In this paper, we explore signal representation and recovery based on uniform amplitude sampling with assumed infinite precision timing information. The approach is based on the delta-ramp encoder which consists of applying a one-level level-crossing detector to the result of adding an appropriate sawtooth-like waveform to the input signal. The output samples are the time instants of these level crossings, thus representing a time-encoded version of the input signal. For theoretical purposes, this system can be equivalently analyzed by reversibly transforming through ramp addition a nonmonotonic input signal into a monotonic one which is then uniformly sampled in amplitude. The monotonic function is then represented by the times at which the signal crosses a predefined and equally-spaced set of amplitude values. We refer to this technique as amplitude sampling. The time sequence generated can be interpreted alternatively as nonuniform time sampling of the original source signal. We derive duality and frequency-domain properties for the functions involved in the transformation. Iterative algorithms are proposed and implemented for recovery of the original source signal. As indicated in the simulations, the proposed iterative amplitude-sampling algorithm achieves a faster convergence rate than frame-based reconstruction for nonuniform sampling. The performance can also be improved by appropriate choice of the parameters while maintaining the same sampling density.

On Unlimited Sampling and Reconstruction

Shannon's sampling theorem is one of the cornerstone topics that is well understood and explored, both mathematically and algorithmically. That said, practical realization of this theorem still suffers from a severe bottleneck due to the fundamental assumption that the samples can span an arbitrary range of amplitudes. In practice, the theorem is realized using so-called analog-to-digital converters (ADCs) which clip or saturate whenever the signal amplitude exceeds the maximum recordable ADC voltage thus leading to a significant information loss. In this paper, we develop an alternative paradigm for sensing and recovery, called the Unlimited Sampling Framework. It is based on the observation that when a signal is mapped to an appropriate bounded interval via a modulo operation before entering the ADC, the saturation problem no longer exists, but one rather encounters a different type of information loss due to the modulo operation. Such an alternative setup can be implemented, for example, via so-called folding or self-reset ADCs, as they have been proposed in various contexts in the circuit design literature. The key task that we need to accomplish in order to cope with this new type of information loss is to recover a bandlimited signal from its modulo samples. In this paper we derive conditions when perfect recovery is possible and complement them with a stable recovery algorithm. The sampling density required to guarantee recovery is independent of the maximum recordable ADC voltage and depends on the signal bandwidth only. Our recovery guarantees extend to measurements affected by bounded noise, which includes the case of round-off quantization. Numerical experiments validate our approach. Applications of the unlimited sampling paradigm can be found in a number of fields such as signal processing, communication and imaging.

Invertibility-Driven Interpolation Filter for Video Coding.

Motion compensation with fractional motion vector has been widely utilized in the video coding standards. The fractional samples are usually generated by fractional interpolation filters. Traditional interpolation filters are usually designed based on the signal processing theory with the assumption of band-limited signal, which cannot effectively capture the non-stationary property of video content and cannot adapt to the variety of video quality. In this paper, we reveal an intuitive property of the fractional interpolation problem, named invertibility. That is, the fractional interpolation filters should not only generate fractional samples from integer samples but also recover the integer samples from the fractional samples in an invertible manner. We prove in theory that the invertibility in the spatial domain is equivalent to the constant magnitude in the Fourier transform domain. Driven by the invertibility, we then develop a learning-based method to solve the fractional interpolation problem. Inspired by the advances of convolutional neural network (CNN), we propose to establish an end-to-end scheme using CNN to train invertibility-driven interpolation filter (InvIF). Different from the previous learning-based methods, the proposed training scheme does not need hand-crafted "ground truth" of fractional samples. The proposed InvIF is integrated into high efficiency video coding (HEVC), and extensive experiments are conducted to verify its effectiveness. The experimental results show that the proposed method can achieve on average 4.7% and 3.6% BD-rate reduction compared with the HEVC anchor, under low-delay-B and random-access configurations, respectively.

FFT multichannel interpolation and application to image super-resolution

This paper presents an innovative set of tools to support a methodology for the multichannel interpolation (MCI) of a discrete signal. It is shown that a bandlimited signal f can be exactly reconstructed from finite samples of gk (1 ≤ k ≤ M) which are the responses of M linear systems with input f. The proposed interpolation can also be applied to approximate non-bandlimited signals. Quantitative error is analyzed to ensure its effectiveness in approximating non-bandlimited signals and its Hilbert transform. Based on the FFT technique, a fast algorithm which brings high computational efficiency and reliability for MCI is presented. The standout performance of MCI is illustrated by several simulations. Additionally, the proposed interpolation is applied to the single image super-resolution (SISR). Its superior performance in accuracy and speed of SISR is demonstrated by the experimental studies. Our results are compared qualitatively and quantitatively with the state-of-the-art methods in image upsampling and reconstruction by using the standard measurement criteria.

Orthogonal Full-Field Optical Sampling

Sampling is the first step to convert analogue into digital signals and one of the basic concepts for information handling. All practical sampling systems, however, are accompanied with errors. Bandwidth-limited signals can be seen as a superposition of time-shifted sinc pulses, weighted with the sampling values. Thus, due to orthogonality, bandlimited signals can be perfectly sampled by a corresponding sinc pulse with the correct time shift. But, sinc pulses are just a mathematical construct. Sinc pulse sequences, instead, can simply be generated by a rectangular, phase-locked frequency comb. For a high repetition-time to pulsewidth ratio, or a low duty cycle, the pulses of such a sequence come close to single sinc pulses, and thus, the sampling with them might lead to an almost ideal sampling. Here, we present the full-field optical sampling with a repetition-time to pulsewidth ratio of up to 153, or a duty cycle of around 0.65%. Since it enables amplitude and phase sampling, ultrahigh sampling rates should be possible.

Acoustic modelling of a transient source in shallow water

A desirable requirement for underwater acoustic models is the ability to simulate a transient signal propagating in an ocean waveguide. Relevant broadband signals are generated, for example, by airguns used in seismic exploration, transducers used for underwater communication systems, and vocalizations produced by marine mammals. Single-frequency acoustic models based on ray, mode, wavenumber integration or parabolic equation representations are often used for this purpose by employing a Fourier synthesis method. In this case, the multiple-frequency, bandlimited medium transfer function is filtered by the spectrum of the source and then inverse Fourier transformed to yield the time-domain waveform. In contrast, direct convolution of a source signal with the medium’s impulse response can be carried out in the time domain. In this paper we describe a convolution-based approach using a ray model representation of the propagation whereby each eigenray amplitude is assumed to be independent of frequency but its phase variation is taken into account. As a result, only a single-frequency calculation is required of the ray model at each range. The two methods are subsequently used to simulate a bandlimited transient signal propagating in a shallow-water, lossy-bottom, Pekeris waveguide.

New sampling theorem and multiplicative filtering in the FRFT domain

Having in consideration a fractional convolution associated with the fractional Fourier transform (FRFT), we propose a novel reconstruction formula for bandlimited signals in the FRFT domain without using the classical Shannon theorem. This may be considered the main contribution of this work, and numerical experiments are implemented to demonstrate the effectiveness of the proposed sampling theorem. As a second goal, we also look for the designing of multiplicative filters. Indeed, we also convert the multiplicative filtering in FRFT domain to the time domain, which can be realized by fast Fourier transform. Two concrete examples are included where the use of the present results is illustrated.

On reconstruction of bandlimited signals from purely timing information

Abstract The well known Logan’s theorem asserts that, under some assumptions, a one octave bandpass signal is recoverable, modulo a multiplicative constant, from its zero crossings only. However, such recovery is numerically problematic and the theorem is not applicable to general bandlimited signals. In this paper, we demonstrate that the additional timing information sufficient for recovery of general band limited signals can be provided in the form of the zero crossings of several of its derivatives and propose novel numerically robust algorithms for such timing extraction and for signal reconstruction, both with high fidelity. We tested the proposed algorithms extensively, with both synthetic and audio signals. In particular, we numerically demonstrate that the timing of the zero crossings of a typical speech signal and of its first two derivatives are not sufficient for a numerically robust recovery of such a signal, but that a robust recovery becomes possible by adding the timing of the zero crossings of the third order derivative.

Explicit Wideband Time-Domain Beam Propagation Algorithm Based on Faber Polynomials

A novel explicit algorithm based on a wideband time-domain beam propagation approach for the solution of Maxwell's equation in the time domain is presented. The algorithm utilizes a Faber polynomial expansion to approximate the time-domain propagator. The expansion allows the usage of time steps exceeding the Courant–Friedrichs–Lewy stability condition. The approach enables the conceptual formulation of time stepping schemes featuring a more accurate approximation of the time dependent propagation for band limited signals than the classical finite difference time-domain method at the same computational effort.

Spectral Super-resolution and Band-limited Extrapolation Using Slepian Series

Band-limited signal extrapolation and spectral super-resolutions are closely related. They both can be achieved using Slepian series. This method is often believed to be dependent on computing definite integrals. Instead of using numerical integration, a simple and reliable method, which can be implemented in MATLAB, is presented. This way, it is much easier to investigated the extent to which a signal can be extrapolated from its samples taken in a finite interval. The proposed method is tested with data from a real microphone array.

A Note on Reconstruction of Bandlimited Signals of Several Variables Sampled at Nyquist Rate

A standard problem in certain applications requires one to find a reconstruction of an analogue signal f from a sequence of its samples $f{({t_{k}})_{k}}$. The great success of such a reconstruction consists, under additional assumptions, in the fact that an analogue signal f of a real variable $t\in \mathbb{R}$ can be represented equivalently by a sequence of complex numbers $f{({t_{k}})_{k}}$, i.e. by a digital signal. In the sequel, this digital signal can be processed and filtered very efficiently, for example, on digital computers. The sampling theory is one of the theoretical foundations of the conversion from analog to digital signals. There is a long list of impressive research results in this area starting with the classical work of Shannon. Note that the well known Shannon sampling theory is mainly for one variable signals. In this paper, we concern with bandlimited signals of several variables, whose restriction to Euclidean space ${\mathbb{R}^{n}}$ has finite p-energy. We present sampling series, where signals are sampled at Nyquist rate. These series involve digital samples of signals and also samples of their partial derivatives. It is important that our reconstruction is stable in the sense that sampling series converge absolutely and uniformly on the whole ${\mathbb{R}^{n}}$. Therefore, having a stable reconstruction process, it is possible to bound the approximation error, which is made by using only of the partial sum with finitely many samples.

Calibrated steganalysis of mp3stego in multi-encoder scenario

Experimenting with different mp3 encoders shows subtle differences in their outputs indicating they have been implemented differently. If this characteristic is not addressed properly, it could degrade performance of mp3 steganalysis. Additionally, calibration is a powerful technique which has not found its true potential in mp3 steganalysis. This paper tries to fill these gaps. First, we propose a new set of calibrated features based on quantization step. To that end, we show that quantization steps of mp3 bit stream and steganography-induced noise are respectively band-limited and wide-band signals. We use this observation and apply a low pass filter on quantization steps of mp3 bit stream and then use the result for calibrating of features. Then, we present our analysis on different encoders and show that their statistical properties are quite dissimilar for all fields of mp3 bitstream. We use this observation and propose an encoder classifier that have near-perfect accuracy. This accuracy is achieved after testing a wide range of feature selection methods and finally using genetic algorithm due to its superior performance. Our simulations show that the proposed method achieved accuracy of 95.8% and 95.2% at 3.12% of maximum embedding rate in the single and multi-encoder scenarios, respectively.

The extrapolation of bandlimited signals in the offset linear canonical transform domain

The extrapolation theorem is a fundamental and important theory in signal analysis. As the offset linear canonical transform (OLCT) has proven to be a novel and effective method in signal processing and optics, a variety of properties and theories of the OLCT have been well studied. However, there are still no papers considered the extrapolation theorem for the OLCT bandlimited signals. Therefore in this paper, the extrapolation of the bandlimited signals associated with the OLCT based on the Gerchberg–Papoulis (GP) method has been presented for the first time. First, a useful theorem for the generalized prolate spheroidal wave function (GPSWFs) has been introduced. Then, the extrapolation of bandlimited signals in the OLCT domain has been derived based on this theorem by using the GP algorithm. Moreover, a fast computation of the extrapolation theorem in the OLCT domain also has been attained. Finally, the numerical results have been carried out to show the effective and useful of the proposed results.

Sampling at Unknown Locations: Uniqueness and Reconstruction Under Constraints

Traditional sampling results assume that the sample locations are known. Motivated by simultaneous localization and mapping (SLAM) and structure from motion (SfM), we investigate sampling at unknown locations. Without further constraints, the problem is often hopeless. For example, we recently showed that, for polynomial and bandlimited signals, it is possible to find two signals, arbitrarily far from each other, that fit the measurements. However, we also showed that this can be overcome by adding constraints to the sample positions. In this paper, we show that these constraints lead to a uniform sampling of a composite of functions. Furthermore, the formulation retains the key aspects of the SLAM and SfM problems, whilst providing uniqueness, in many cases. We demonstrate this by studying two simple examples of constrained sampling at unknown locations. In the first, we consider sampling a periodic bandlimited signal composite with an unknown linear function. We derive the sampling requirements for uniqueness and present an algorithm that recovers both the bandlimited signal and the linear warping. Furthermore, we prove that, when the requirements for uniqueness are not met, the cases of multiple solutions have measure zero. For our second example, we consider polynomials sampled such that the sampling positions are constrained by a rational function. We previously proved that, if a specific sampling requirement is met, uniqueness is achieved. In addition, we present an alternate minimization scheme for solving the resulting non-convex optimization problem. Finally, fully reproducible simulation results are provided to support our theoretical analysis.

A Compressed Sensing Framework for Dynamic Sound-Field Measurements

The conventional sampling of sound fields by use of stationary microphones is impractical for large bandwidths. Satisfying the Nyquist–Shannon sampling theorem in three-dimensional space requires a huge number of sampling positions. Dynamic sound-field measurements with moving microphones together with a compressed-sensing recovery allow for weakening the spatial sampling problem. For bandlimited signals, the dynamic samples taken along the microphone trajectory may be related to the room impulse responses on a virtual grid in space via spatial interpolation. The tracking of the microphone positions and the knowledge of the excitation sequence allow for setting up a linear system of equations that can be solved for the room impulse responses on the modeled virtual grid. Nevertheless, there is still the necessity for recovering a huge number of sound-field variables, in order to ensure aliasing-free reconstruction. Thus, for practical applications, random or suboptimally chosen trajectories may be expected to lead to underdetermined sampling problems for a given volume of interest. In this paper, we present a compressed sensing framework that enables us to uniquely solve the dynamic sampling problem despite having underdetermined variables. The spatio-temporal sampling problem is integrated into compressed sensing models that allow for stable and robust sub-Nyquist sampling given incoherent measurements. For a modeled equidistant grid and sparse Fourier representations, the influence of the microphone trajectories on the compressed sensing problem is investigated and a simple expression is derived for evaluating trajectories with regard to compressed-sensing based recovery.

A delayed-feedback filter with negative group delay.

A filter with delay-induced negative group delay is presented. The filter consists of multiple time-delayed feedback terms, which lead to a negative group delay for frequencies in the baseband. It can be used for the real-time prediction of band-limited signals. The filter is universal as it does not rely on a specific model of the signal. Specifically, as long as the signal to be predicted is band-limited with a known cutoff frequency, the filter predicts the signal in real time up to a prediction horizon that depends on the cutoff frequency. How signal prediction arises from the negative group delay of the filter is worked out in detail. Its properties, including stability, are derived analytically and demonstrated by numerical simulations. For chaotic systems, the filter is predictive during phases of high predictability.

On signal reconstruction from FROG measurements

Phase retrieval refers to recovering a signal from its Fourier magnitude. This problem arises naturally in many scientific applications, such as ultra-short laser pulse characterization and diffraction imaging. Unfortunately, phase retrieval is ill-posed for almost all one-dimensional signals. In order to characterize a laser pulse and overcome the ill-posedness, it is common to use a technique called Frequency-Resolved Optical Gating (FROG). In FROG, the measured data, referred to as FROG trace, is the Fourier magnitude of the product of the underlying signal with several translated versions of itself. The FROG trace results in a system of phaseless quartic Fourier measurements. In this paper, we prove that it suffices to consider only three translations of the signal to determine almost all bandlimited signals, up to trivial ambiguities. In practice, one usually also has access to the signal's Fourier magnitude. We show that in this case only two translations suffice. Our results significantly improve upon earlier work.