Information Communication Theory Research Articles

Information theoretic approach in molecular interactions and implications in molecular evolution

Shannon’s information communication theory tells much about the reliable message transmission and has a wide scope of application in communication technologies. The aim of this work is to present the implementation of the information communication theory in the cellular recognition or molecular interaction event models to quantify the message transmission capacities of molecular binding patterns and complexes. It has implications in molecular evolution such that the pattern formation by varied combinations of a possible number of binding events does not necessitate the presence of different types of receptor–ligand complexes. Signals through the same and different types of receptor–ligand complexes seem to have equal information amounts. This is suggesting the possible role of pattern formation in differentiation of distinct conditions, maybe even changing local concentration gradients or so, through sensing the patterns of relevance. Further, recognition of the patterns that are formed up of the same binding partners would be the step preceding the recognition of the patterns with discrete binding partners. All these considerations are valid for cellular networks, wherein the communicating cells are the sources of binding target molecules and are thus imposing diffusion dependent concentration gradients and variations in the probabilities of the binding events.

Compressively Characterizing High-Dimensional Entangled States with Complementary, Random Filtering

The resources needed to conventionally characterize a quantum system are overwhelmingly large for high- dimensional systems. This obstacle may be overcome by abandoning traditional cornerstones of quantum measurement, such as general quantum states, strong projective measurement, and assumption-free characterization. Following this reasoning, we demonstrate an efficient technique for characterizing high-dimensional, spatial entanglement with one set of measurements. We recover sharp distributions with local, random filtering of the same ensemble in momentum followed by position---something the uncertainty principle forbids for projective measurements. Exploiting the expectation that entangled signals are highly correlated, we use fewer than 5,000 measurements to characterize a 65, 536-dimensional state. Finally, we use entropic inequalities to witness entanglement without a density matrix. Our method represents the sea change unfolding in quantum measurement where methods influenced by the information theory and signal-processing communities replace unscalable, brute-force techniques---a progression previously followed by classical sensing.

The Research and Design of a New Electronic Communication Counter for Sensors

With the development of Internet of Things, in addition to the basic counting function, the counter also needs to integrate more functions to meet the application requirements, such as display, communication, charging functions and other functions. Therefore, it is very necessary to design a new electronic counter which has an ability to communicate with outside, display counting information and energy supplement. In this paper, we design a new electronic counter based on the pulse counting theory, information communication theory and flash memory technology. First of all, we will count the pulse signal through the signal input circuit and the signal processing circuit into the control circuit, which can identify the digital pulse signal. Secondly, the control circuit count the number of digital pulse signal, the count number will be real-time displayed on the digital tube. Meanwhile, the counter communicates with the Bluetooth device (smartphone, pad, etc.) via the Bluetooth module. And users can use the Bluetooth device to configure the counter's operating parameters, clear or pre-count values, view the counter operating conditions and historical data.

Toward Transparent Coexistence for Multihop Secondary Cognitive Radio Networks

The dominate spectrum sharing paradigm of today is interference avoidance, where a secondary network can use the spectrum only when such a use is not interfering with the primary network. However, with the advances of physical-layer technologies, the mindset of this paradigm is being challenged. This paper explores a new paradigm called “transparent coexistence” for spectrum sharing between primary and secondary nodes in a multihop network environment. Under this paradigm, the secondary network is allowed to use the same spectrum simultaneously with the primary network as long as their activities are “transparent” (or “invisible”) to the primary network. Such transparency is accomplished through a systematic interference cancelation (IC) by the secondary nodes without any impact on the primary network. Although such a paradigm has been studied in the information theory (IT) and communications (COMM) communities, it is not well understood in the wireless networking community, particularly for multihop networks. This paper offers an in-depth study of this paradigm in a multihop network environment and addresses issues such as scheduling (both in frequency channels and time slots) and IC (to/from primary network and within the secondary network). Through a rigorous modeling and formulation, problem formulation, solution development, and simulation results, we show that transparent coexistence paradigm offers significant improvement in terms of spectrum access and throughput performance as compared to the current prevailing interference avoidance paradigm.

Secrecy Outage Analysis in Cellular Networks

The study of the inherent secrecy capacity of wireless networks of random topologies is currently of great interest to the communication and information theory communities. Indeed, a good amount of work exists on the secrecy analysis of random networks, the majority of which relies on Poisson point processes to model the spatial distribution of devices. It has been recently demonstrated, however, that Matern hard-core point processes—rather than Poisson point processes—are better suited to characterize the random location of users and base stations of cellular systems. In this letter, we therefore offer an analysis of the secrecy outage of random networks under the Matern hard-core point process model, with the objective of shedding some light on the security limitations/capabilities inherent encountered in cellular systems.

On Shannon’s Formula and Hartley’s Rule: Beyond the Mathematical Coincidence

In the information theory community, the following “historical” statements are generally well accepted: (1) Hartley did put forth his rule twenty years before Shannon; (2) Shannon’s formula as a fundamental tradeoff between transmission rate, bandwidth, and signal-to-noise ratio came out unexpected in 1948; (3) Hartley’s rule is inexact while Shannon’s formula is characteristic of the additive white Gaussian noise channel; (4) Hartley’s rule is an imprecise relation that is not an appropriate formula for the capacity of a communication channel. We show that all these four statements are somewhat wrong. In fact, a careful calculation shows that “Hartley’s rule” in fact coincides with Shannon’s formula. We explain this mathematical coincidence by deriving the necessary and sufficient conditions on an additive noise channel such that its capacity is given by Shannon’s formula and construct a sequence of such channels that makes the link between the uniform (Hartley) and Gaussian (Shannon) channels.

Establishing Metrics for Assessing the Performance of Directional Modulation Systems

In this paper, metrics for assessing the performance of directional modulation (DM) physical-layer secure wireless systems are discussed. In the paper DM systems are shown to be categorized as static or dynamic. The behavior of each type of system is discussed for QPSK modulation. Besides EVM-like and bit error rate (BER) metrics, secrecy rate as used in information theory community is also derived for the purpose of this QPSK DM system evaluation.

Ensemble estimators for multivariate entropy estimation.

The problem of estimation of density functionals like entropy and mutual information has received much attention in the statistics and information theory communities. A large class of estimators of functionals of the probability density suffer from the curse of dimensionality, wherein the mean squared error (MSE) decays increasingly slowly as a function of the sample size T as the dimension d of the samples increases. In particular, the rate is often glacially slow of order O(T-γ/d ), where γ > 0 is a rate parameter. Examples of such estimators include kernel density estimators, k-nearest neighbor (k-NN) density estimators, k-NN entropy estimators, intrinsic dimension estimators and other examples. In this paper, we propose a weighted affine combination of an ensemble of such estimators, where optimal weights can be chosen such that the weighted estimator converges at a much faster dimension invariant rate of O(T-1). Furthermore, we show that these optimal weights can be determined by solving a convex optimization problem which can be performed offline and does not require training data. We illustrate the superior performance of our weighted estimator for two important applications: (i) estimating the Panter-Dite distortion-rate factor and (ii) estimating the Shannon entropy for testing the probability distribution of a random sample.

On the relation between kinetically constrained models of glass dynamics and the randomfirst-order transition theory

In this paper we revisit and extend the mapping between two apparently different classes ofmodels. The first class contains the prototypical models described—at the mean-field level—bythe random first-order transition (RFOT) theory of the glass transition, called either the‘random XORSAT problem’ (in the information theory community) or the ‘dilutedp-spin model’ (in the spin glass community), undergoing a single spin-flip Glauber dynamics.The models in the second class are kinetically constrained models (KCM): theirHamiltonian is that of independent spins in a constant magnetic field, hence theirthermodynamics is completely trivial, but the dynamics is such that only groupsof spins can flip together, thus implementing a kinetic constraint that inducesa non-trivial dynamical behavior. A mapping between some representatives ofthese two classes has been known for a long time. Here we formally prove thismapping at the level of the master equation, and we apply it to the particular caseof Bethe lattice models. This allows us to show that an RFOT model can bemapped exactly into a KCM. However, the natural order parameter for the RFOTmodel, namely the spin overlap, is mapped into a very complicated non-localfunction in the KCM. Therefore, if one were to study the KCM without knowingthe mapping onto the RFOT model, one would guess that its physics is quitedifferent from the RFOT one. Our results instead suggest that these two apparentlydifferent descriptions of the glass transition are, at least in some cases, closelyrelated.

Gossip Algorithms for Distributed Signal Processing

Gossip algorithms are attractive for in-network processing in sensor networks because they do not require any specialized routing, there is no bottleneck or single point of failure, and they are robust to unreliable wireless network conditions. Recently, there has been a surge of activity in the computer science, control, signal processing, and information theory communities, developing faster and more robust gossip algorithms and deriving theoretical performance guarantees. This paper presents an overview of recent work in the area. We describe convergence rate results, which are related to the number of transmitted messages and thus the amount of energy consumed in the network for gossiping. We discuss issues related to gossiping over wireless links, including the effects of quantization and noise, and we illustrate the use of gossip algorithms for canonical signal processing tasks including distributed estimation, source localization, and compression.

Some large linear systems in communications: estimation, information and statistical physics

Linear systems are frequently used to model physical channels in communication problems. In such systems, the goal is to recover the information bearing input signal to the linear transformation based on noisy observations of the output. This paper investigates the performance of linear systems randomly selected from some ensembles in the large-dimension limit. Previous analysis of such systems using statistical physics techniques is strengthened, where it is shown that the posterior of each individual input symbol is asymptotically identical to the posterior of a Gaussian channel with the same input. Interestingly, the replica symmetry solution takes an identical form as the rigorous result obtained for an ensemble of sparse linear systems. Some challenges to the information theory and statistical physics communities are highlighted.

The academic and industrial embrace of space-time methods

[Guest Editors introduction to: Special issue on space-time transmission, reception, coding and signal processing] Every episode of the classic 1966–1969 television series Star Trek begins with Captain Kirk’s (played by William Shatner) famous words : “Space: The final frontier….” While space may not be the final frontier for the information and communication theory community, it is proving to be an important and fruitful one. In the information theory community, the notion of space can be broadly defined as the simultaneous use of multiple, possibly coupled, channels. The notions of space–time and multiple-input multiple-output (MIMO) channels are therefore often used interchangeably. The connection between space and MIMO is most transparent when we view the multiple channels as created by two or more spatially separated antennas at a wireless transmitter or receiver. A large component of the current interest in space–time methods can be attributed to discoveries in the late 1980s and early 1990s that a rich wireless scattering environment can be beneficial when multiple antennas are used on a point-to-point link. We now know that adding antennas in a rich environment provides proportional increases in point-to-point data rates, without extra transmitted power or bandwidth.

Statistical mechanics of typical set decoding.

The performance of "typical set (pairs) decoding" for ensembles of Gallager's linear code is investigated using statistical physics. In this decoding method, errors occur, either when the information transmission is corrupted by atypical noise, or when multiple typical sequences satisfy the parity check equation as provided by the received corrupted codeword. We show that the average error rate for the second type of error over a given code ensemble can be accurately evaluated using the replica method, including the sensitivity to message length. Our approach generally improves the existing analysis known in the information theory community, which was recently reintroduced in IEEE Trans. Inf. Theory 45, 399 (1999), and is believed to be the most accurate to date.

Data compression and harmonic analysis

In this paper we review some recent interactions between harmonic analysis and data compression. The story goes back of course to Shannon's R(D) theory in the case of Gaussian stationary processes, which says that transforming into a Fourier basis followed by block coding gives an optimal lossy compression technique; practical developments like transform-based image compression have been inspired by this result. In this paper we also discuss connections perhaps less familiar to the information theory community, growing out of the field of harmonic analysis. Recent harmonic analysis constructions, such as wavelet transforms and Gabor transforms, are essentially optimal transforms for transform coding in certain settings. Some of these transforms are under consideration for future compression standards. We discuss some of the lessons of harmonic analysis in this century. Typically, the problems and achievements of this field have involved goals that were not obviously related to practical data compression, and have used a language not immediately accessible to outsiders. Nevertheless, through an extensive generalization of what Shannon called the "sampling theorem", harmonic analysis has succeeded in developing new forms of functional representation which turn out to have significant data compression interpretations. We explain why harmonic analysis has interacted with data compression, and we describe some interesting recent ideas in the field that may affect data compression in the future.