Information Channel Capacity Research Articles

On Lower Bounding the Information Capacity of Amplify and Forward Wireless Relay Channels with Channel Estimation Errors

We formulate a capacity lower bound for the dual-hop wireless relay channel which employs an amplify-and-forward (AF) protocol at the relay node. In AF relaying, even when the fading channel in both hops is complex Gaussian distributed, the overall dual-hop channel is non-Gaussian. WPe highlight that there is a fundamental difference between Gaussian and non-Gaussian channels in terms of deriving their capacity lower bound. Specifically for non-Gaussian channels, the channel estimation error variance depends on the received pilot signal and is, in general, different from the average error variance. Whereas for Gaussian distributed channels, which have been predominantly studied in the literature, the channel estimation error variance conditioned on the observed pilot signal coincides with the average error variance. We provide an example using the AF dual-hop channel to exhibit the numerical difference between the true capacity lower bound and that obtained by using the average instead of the conditional error variance.

A metric space approach to the information channel capacity of spike trains

A novel method is presented for calculating the information channel capacity of spike trains. This method works by fitting a χ-distribution to the distribution of distances between responses to the same stimulus: the χ-distribution is the length distribution for a vector of Gaussian variables. The dimension of this vector defines an effective dimension for the noise and by rephrasing the problem in terms of distance based quantities, this allows the channel capacity to be calculated. As an example, the capacity is calculated for a data set recorded from auditory neurons in zebra finch.

Control and estimation under information constraints: Toward a unified theory of control, computation and communications

An attempt is made to overview an emerging area of research devoted to analysis and design of control systems under constraints caused by limited information capacity of communication channels. The problem's prehistory dating back to the 1960s---1970s, as well as the new approaches that appeared during the last decade were analyzed. Much attention was paid to various versions of the celebrated data rate theorem. Consideration was given to the problems of control through the communication networks and some results obtained for the nonlinear systems. The basic application areas were listed in brief.

INFORMATIONAL CHARACTERISTICS OF THE DOUBLE-STEP SACCADIC EYE MOVEMENTS

Saccades have traditionally been studied in response to suddenly changing visual stimuli, such as jumping targets. The evaluation of saccadic eye movements based on the analysis of eye jump trajectories as reaction to the jumping targets is complicated. Time delays and errors of eye jumps are related to the target amplitudes and sequence presentation and must be determined in each type of the experiment. In this research, information theory concepts are used to evaluate the control system of the saccadic eye movements which is defined as the information transfer channel. In this case, target jumping on the screen is defined as the input or the source information, the eyesight response trajectory on the screen – as the output information and the difference between them – as the lost information. The amount of information transferred over oculomotor channel is defined as the difference between the input and the lost information rates. Majority of the saccadic eye movements are executed by two-step jumps: primary and corrective saccades. Therefore, the amount of information transferred over oculomotor channel has been measured separately after primary saccades and corrective saccades. We have found that the amount of information obtained during corrective saccades is two times larger than during primary saccades. Primary saccades with bigger amplitudes give more information despite a larger scatter of position errors at the end of jump for them. Theoretical and experimental investigation let us formulate that channel information capacity of the saccadic oculomotor system is in the range of 14 - 15 bits/sec. This value was obtained when the intersaccadic interval was in the range of 0.4 – 0.5 sec.

Capacity limitations in fiber-optic communication systems as a result of polarization-dependent loss

We characterize the effect of polarization dependent loss (PDL) on the information capacity of fiber-optic channels. The reduction in the outage capacity owing to the PDL is quantified as well as the signal-to-noise ratio margin that needs to be allocated for the PDL in order to avoid loss of capacity.

Capacity of a Single Spiking Neuron Channel

Information transfer through a single neuron is a fundamental component of information processing in the brain, and computing the information channel capacity is important to understand this information processing. The problem is difficult since the capacity depends on coding, characteristics of the communication channel, and optimization over input distributions, among other issues. In this letter, we consider two models. The temporal coding model of a neuron as a communication channel assumes the output is tau where tau is a gamma-distributed random variable corresponding to the interspike interval, that is, the time it takes for the neuron to fire once. The rate coding model is similar; the output is the actual rate of firing over a fixed period of time. Theoretical studies prove that the distribution of inputs, which achieves channel capacity, is a discrete distribution with finite mass points for temporal and rate coding under a reasonable assumption. This allows us to compute numerically the capacity of a neuron. Numerical results are in a plausible range based on biological evidence to date.

New Aspects of Electromagnetic Information Theory for Wireless and Antenna Systems

This paper investigates information-theoretic characterization, via Shannon's information capacity and number of degrees of freedom, of wave radiation (antenna) and wireless propagation systems. Specifically, the paper derives, from the fundamental physical point of view of Maxwell's equations describing electromagnetic fields, the Shannon information capacity of space-time wireless channels formed by electromagnetic sources and receivers in a known background medium. The theory is developed first for the case of sources working at a fixed frequency (time-harmonic case) and is expanded later to the more general case of temporally bandlimited systems (time-domain fields). In the bandlimited case we consider separately the two cases of time-limited and essentially bandlimited systems and of purely bandlimited systems. The developments take into account the physical radiated power constraint in addition to a constraint in the source L 2 norm which acts to avoid antenna superdirectivity. Based on such radiated power and current L 2 norm constraints we derive the Shannon information capacity of canonical wireless and antenna systems in free space, for a given additive Gaussian noise level, as well as an associated number of degrees of freedom resulting from such capacity calculations. The derived results also illustrate, from a new information-theoretic point of view, the transition from near to far fields.

Communication system model for information rate evaluation of differential detection over time-varying channels

A communication system model for mutual information performance analysis of multiple-symbol differential M-phase shift keying over time-correlated, time-varying flat-fading communication channels is developed. This model is a finite-state Markov (FSM) equivalent channel representing the cascade of the differential encoder, FSM channel model and differential decoder. A state-space approach is used to model channel phase time correlations. The equivalent model falls in a class that facilitates the use of the forward–backward algorithm, enabling the important information theoretic results to be evaluated. Using such a model, one is able to calculate mutual information for differential detection over time-varying fading channels with an essentially finite time set of correlations, including the Clarke fading channel. Using the equivalent channel, it is proved and corroborated by simulations that multiple-symbol differential detection preserves the channel information capacity when the observation interval approaches infinity.

Optimizing Assisted Communication Devices for Children With Motor Impairments Using a Model of Information Rate and Channel Capacity

For children who depend on devices to communicate, the rate of communication is a primary determinant of success. For children with motor impairments, the rate of communication may be limited by inability to contact buttons or cells rapidly or accurately. It is, therefore, essential to know how to adjust the device interface in order to maximize each child's rate of communication. The optimal rate of communication is determined by the channel capacity, which is the maximum value of the information rate for all possible keyboard button or cell layouts for the communication device. We construct a mathematical model for the information rate based on the relationship between movement time and the number of buttons per screen, the size of the buttons, and the length of a sequence of buttons that must be pressed to communicate each word in the vocabulary. We measure the parameters of the model using a custom-programmed touchscreen interface in 10 children with disorders of arm movement due to cerebral palsy who use a DynaVox communication device. We measure the same parameters in 20 healthy control subjects. We show that the model approximates the measured information rate and that the information rate is lower in children with motor impairments compared with control subjects. The theory predicts that for each child there is a combination of button size and number that maximizes the predicted information rate and thereby achieves communication at the optimal channel capacity. Programming communication devices with each child's predicted optimal parameters improved the communication rate in five of the ten children, compared with programming by professionals. Therefore, measurement of information rate may provide an assessment of the effect of motor disorders on success in assisted communication. Optimization of the information rate may be useful for programming assisted communication devices.

On the sensitivity, selectivity, sensory information and optimal size of resistive chemical sensors

Information theoretical tools are applied for the study of the sensitivity and selectivity enhancements of resistive fluctuation-enhanced sensors. General considerations are given for the upper limit of selectivity enhancement. The signal-to- noise ratio and information channel capacity of fluctuation-enhanced chemical sensors is compared to that of classical sensors providing a single output. The considerations are done at the generic level with a few concrete examples and include the estimation of scaling relations between the sensor size, speed of measurements, sampling rate, measurement time, signal power and noise power.

Basic binary state-space model for time-varying communication channels: uncertainty and information capacity

This paper looks at capacity achieving detection strategies for information transfer over time-varying channels. The time-varying binary symmetric channel (TV-BSC) is identified as the basic binary state-space model. Separation of entropies principles and the TV-BSC model-based state-space approach are used to determine the performance bounds for coherent and non-coherent detection over time-varying communication channels. The mutual information rate over the TV-BSC, assuming channel estimation in the presence of channel noise, is shown to be below the channel information capacity because of lack of perfect channel knowledge. Furthermore, it is shown that TV-BSC model-based differential detection has a fundamental advantage over the channel estimation based detection since it theoretically preserves the TV-BSC information capacity when the observation interval approaches infinity. Simulation analysis corroborates the theoretical results, showing that multiple-symbol differential detection practically achieves the TV-BSC capacity in just a few symbol observation times.

Dynamic statistical information theory

In recent years we extended Shannon static statistical information theory to dynamic processes and established a Shannon dynamic statistical information theory, whose core is the evolution law of dynamic entropy and dynamic information. We also proposed a corresponding Boltzmman dynamic statistical information theory. Based on the fact that the state variable evolution equation of respective dynamic systems, i.e. Fokker-Planck equation and Liouville diffusion equation can be regarded as their information symbol evolution equation, we derived the nonlinear evolution equations of Shannon dynamic entropy density and dynamic information density and the nonlinear evolution equations of Boltzmann dynamic entropy density and dynamic information density, that describe respectively the evolution law of dynamic entropy and dynamic information. The evolution equations of these two kinds of dynamic entropies and dynamic informations show in unison that the time rate of change of dynamic entropy densities is caused by their drift, diffusion and production in state variable space inside the systems and coordinate space in the transmission processes; and that the time rate of change of dynamic information densities originates from their drift, diffusion and dissipation in state variable space inside the systems and coordinate space in the transmission processes. Entropy and information have been combined with the state and its law of motion of the systems. Furthermore we presented the formulas of two kinds of entropy production rates and information dissipation rates, the expressions of two kinds of drift information flows and diffusion information flows. We proved that two kinds of information dissipation rates (or the decrease rates of the total information) were equal to their corresponding entropy produc-tion rates (or the increase rates of the total entropy) in the same dynamic system. We obtained the formulas of two kinds of dynamic mutual informations and dynamic channel capacities reflecting the dynamic dissipation characteristics in the transmission processes, which change into their maximum—the present static mutual information and static channel capacity under the limit case where the proportion of channel length to informa-tion transmission rate approaches to zero. All these unified and rigorous theoretical for-mulas and results are derived from the evolution equations of dynamic information and dynamic entropy without adding any extra assumption. In this review, we give an overview on the above main ideas, methods and results, and discuss the similarity and difference between two kinds of dynamic statistical information theories.

Monotonicity Results for Coherent MIMO Rician Channels

The dependence of the Gaussian input information rate on the line-of-sight (LOS) matrix in multiple-input multiple-output coherent Rician fading channels is explored. It is proved that the outage probability and the mutual information induced by a multivariate circularly symmetric Gaussian input with any covariance matrix are monotonic in the LOS matrix D, or more precisely, monotonic in D'D in the sense of the Loewner partial order. Conversely, it is also demonstrated that this ordering on the LOS matrices is a necessary condition for the uniform monotonicity over all input covariance matrices. This result is subsequently applied to prove the monotonicity of the isotropic Gaussian input information rate and channel capacity in the singular values of the LOS matrix. Extensions to multiple-access channels are also discussed.

Capacity Analysis for Finite-State Markov Mapping of Flat-Fading Channels

In this paper, time-varying flat-fading channels are modeled as first-order finite-state Markov channels (FSMC). The effect of this modeling on the channel information capacity is addressed. The approximation accuracy of the first-order memory assumption in the Markov model is validated by comparing the FSMC capacity with the channel capacity assuming perfect state information at the receiver side. The results indicate that the first-order Markovian assumption is accurate for normalized Doppler frequencies f/sub d/T /spl lsim/ 0.01, in amplitude-only quantization of the channel gain for noncoherent binary signaling. In phase-only and joint phase and amplitude quantization of the channel gain for coherent binary signaling, the first-order Markovian assumption is accurate for f/sub d/T /spl lsim/ 0.001. Furthermore, the effect of channel quantization thresholds on the FSMC capacity is studied. In high signal-to-noise ratio (SNR) conditions, nonuniform two-level amplitude quantization scheme outperforms equiprobable quantization method by 0.8-1.5 dB.

ERROR RATE IN CURRENT-CONTROLLED LOGIC PROCESSORS WITH SHOT NOISE

The error rate in a current-controlled logic microprocessor dominated by shot noise has been investigated. It is shown that the error rate increases very rapidly with increasing cutoff frequency. The maximum clock frequency of the processor, which works without errors, is obtained as a function of the operational current. The information channel capacity of the system is also studied.

The adaptive classical capacity of a quantum channel, or Information capacities of three symmetric pure states in three dimensions

We investigate the capacity of three symmetric quantum states in three real dimensions to carry classical information. Several such capacities have already been defined, depending on what operations are allowed in the protocols that the sender uses to encode classical information into these quantum states, and that the receiver uses to decode it. These include the C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1,1</inf> capacity, which is the capacity achievable if separate measurements must be used for each of the received states, and the C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1,∞</inf> capacity, which is the capacity achievable if joint measurements are allowed on the tensor product of all of the received states. We discover a new classical information capacity of quantum channels, the adaptive capacity C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1,A</inf> , which lies strictly between the C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1,1</inf> and the C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1,∞</inf> capacities. The adaptive capacity allows the use of what is known as the LOCC (local operations and classical communication) model of quantum operations for decoding the channel outputs. This model requires each of the signals to be measured by a separate apparatus, but allows the quantum states of these signals to be measured in stages, with the first stage partially reducing their quantum states; measurements in subsequent stages may depend on the results of a classical computation taking as input the outcomes of the first round of measurements. We also show that even in three dimensions, with the information carried by an ensemble containing three pure states, achieving the C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1,1</inf> capacity may require a positive operator valued measure (POVM) with six outcomes.

Nonequilibrium statistical information theory

In t his paper, we propose a nonequilibrium statistical information theory, whose ke rnel is information(entropy) evolution equation describing information evolution law. A nonlinear evolution equation of Shannon information (entropy) is derived . The statistical physical information is introduced and its nonlinear evolution equation is derived. Both of these two information (entropy) evolution equatio ns show that the temporal change rate of statistical information (entropy) densi t y originates together from their drift, diffusion and dissipation (production) i n coodinate space (and state variable space). The expressions of drift informati on flow and diffusion information flow, the concise formulas of statistical entr opy production rate and statistical information dissipation are given. The stati stical information dissipation (or increase) rate being equal to its statistical entropy production (or decrease) rate of the dynamic system, the information di ffusion and information dissipation occuring at the same time are proved. The d ynamic mutual information and dynamic channel capacity reflecting the dynamic di ssipative character in transmission process is presented. The similarities and dissimilarities between Shannon information (entropy),its evolution equation an d physical information (entropy),its evolution equation are discussed.

Information capacity of optical fiber channels with zero average dispersion.

We study the statistics of optical data transmission in a noisy nonlinear fiber channel with a weak dispersion management and zero average dispersion. Applying analytical expressions for the output probability density functions both for a nonlinear channel and for a linear channel with additive and multiplicative noise we calculate in a closed form a lower bound estimate on the Shannon capacity for an arbitrary signal-to-noise ratio.

How much power does neural signal propagation need?

Two well known, biologically inspired non-dynamical models of stochastic resonance, the threshold-crossing model and the fluctuating rate model, are analyzed in terms of channel information capacity and dissipation of energy necessary for small-signal transduction. Using analogies to spike propagation in neurons we postulate the average output pulse rate as a measure of dissipation. The dissipation increases monotonically with the input noise. We find that for small dissipation both models give a close to linear dependence of the channel information capacity on dissipation. In both models the channel information capacity, as a function of dissipation, has a maximum at input noise amplitude that is different from that in the standard signal-to-noise ratio versus input noise plot. Though a direct comparison is not straightforward, for small signals the threshold model gives appreciably higher density of information per dissipation than the exponential fluctuating rate model. We show that a formal introduction of cooperativity in the rate fluctuating model permits us to imitate the response function of the threshold model and to enhance performance. This finding may have direct relevance to real neural spike generation where, due to a strong positive feedback, the ion channel currents add up in a synchronized way.

Additivity for unital qubit channels

Additivity of the Holevo capacity is proved for product channels, under the condition that one of the channels is a unital qubit channel, with the other completely arbitrary. As a byproduct this proves that the Holevo bound is the classical information capacity of such qubit channels, and provides an explicit formula for this classical capacity. Additivity of minimal entropy and multiplicativity of p-norms are also proved under the same assumptions. The proof relies on a new bound for the p-norm of an output state from the half-noisy phase-damping channel.