Shannon Channel Capacity Research Articles

Shannon Meets Fick on the Microfluidic Channel: Diffusion Limit to Sum Broadcast Capacity for Molecular Communication.

Molecular communication (MC) over a microfluidic channel with flow is investigated based on Shannon's channel capacity theorem and Fick's laws of diffusion. Specifically, the sum capacity for MC between a single transmitter and multiple receivers (broadcast MC) is studied. The transmitter communicates by using different types of signaling molecules with each receiver over the microfluidic channel. The transmitted molecules propagate through microfluidic channel until reaching the corresponding receiver. Although the use of different types of molecules provides orthogonal signaling, the sum broadcast capacity may not scale with the number of the receivers due to physics of the propagation (interplay between convection and diffusion based on distance). In this paper, the performance of broadcast MC on a microfluidic chip is characterized by studying the physical geometry of the microfluidic channel and leveraging the information theory. The convergence of the sum capacity for microfluidic broadcast channel is analytically investigated based on the physical system parameters with respect to the increasing number of molecular receivers. The analysis presented here can be useful to predict the achievable information rate in microfluidic interconnects for the biochemical computation and microfluidic multi-sample assays.

A Lower Bound for the Hardware Complexity of FIR Filters

Shannon's channel capacity theorem states that in a communication channel the available power budget (SNR) dictates the maximum realizable data rate. A relevant question for FIR filters then is this: Given a hardware budget, can we similarly define a general upper bound on the performance capabilities of a realizable FIR filter? And if yes, how close are the hardware complexities of existing FIR filter designs in relation to such performance limitations? Herein, we investigate such questions and we present a practical approach to derive a lower bound for the minimal hardware budget required for a practical FIR filter realization of a given set of target filter specifications. We believe this to be an important insight for fully understanding the design complexities of FIR filters and for properly managing their power consumption expectations.

On physical limit of wireless digital transmission from radio wave propagation perspective

AbstractUnder a time‐invariant condition with thermal noise, the physical limit of digital transmission ability is governed by Shannon's channel capacity. However, in this formula, it does not contain factors on radio wave propagation environments. In other words, for the ultimate information transmission, a sufficiently long time for the coding and signal processing is expected. However, since wave propagation prevents its premise, there is another physical limit for digital transmission in a different perspective with Shannon's channel capacity. Even if the S/N ratio is sufficiently high, there is the limit for information transmission. This paper deals with this matter concerning physical limit of wireless transmission from a radio wave propagation viewpoint.

Optimal Multiband Transmission Under Hostile Jamming

This paper considers optimal multiband transmission under hostile jamming, where both the authorized user and the jammer are power-limited and operate against each other. The strategic decision making of the authorized user and the jammer is modeled as a two-party zero-sum game, where the payoff function is the capacity that can be achieved by the authorized user in the presence of the jammer. First, we investigate the game under AWGN channels. It is found that: either for the authorized user to maximize its capacity, or for the jammer to minimize the capacity of the authorized user, the best strategy for both of them is to distribute the transmission power or jamming power uniformly over all the available spectrum. The minimax capacity can be calculated based on the channel bandwidth and the signal-to-jamming and noise ratio, and it matches with the Shannon channel capacity formula. Second, we consider frequency selective fading channels. We characterize the dynamic relationship between the optimal signal power allocation and the optimal jamming power allocation in the minimax game, and then propose an iterative water pouring algorithm to find the optimal power allocation schemes for both the authorized user and the jammer.

ISI-Aware Modeling and Achievable Rate Analysis of the Diffusion Channel

Analyzing the achievable rate of molecular communication via diffusion (MCvD) inherits intricacies due to its nature: MCvD channel has memory, and the heavy tail of the signal causes inter symbol interference (ISI). Therefore, using Shannon's channel capacity formulation for memoryless channel is not appropriate for the MCvD channel. Instead, a more general achievable rate formulation and system model must be considered to make this analysis accurately. In this letter, we propose an effective ISI-aware MCvD modeling technique in 3-D medium and properly analyze the achievable rate.

Normal and Impaired hearing recognition of speech segments in noise

The identification of very short phoneme segments of speech is the key to understanding speech in chopped noise. Over the last 12 years, UIUC has repeated Miller-Nicely's 1955 phone recognition experiment, with 60 subjects and six SNRs, from quiet to −22 dB SNR, providing an improve understanding of the fundamentals of phone perception in normal (NH) and impaired (HI) listeners. Our phone robustness metric (SNR50) is the SNR such that the phone error is 50% (Toscano and Allen (2014), JSHLR). The error rate at SNR50 + 5 [dB] is <0.33%. We interpret this to mean that above SNR50, phonemes are below the Shannon channel-capacity limit. This is a game-changer: We must reevaluate speech recognition methods. For example, in an experiment on HI ears, we found that HI ears make large errors (e.g., 100%) on a small subset of tokens (Trevino Allen, JASA 134, 607, 2012). Averaging across tokens or listeners for any given consonant conflates the scores. There is a good news: Since there are only a small number of subject-dependent difficult sounds, testing time is reduced and accuracy is increased for a fixed test duration.

A Tight Upper Bound on Channel Capacity for Visible Light Communications

Since the optical wireless channel in visible light communication (VLC) systems is subject to the non-negativity of the signal and the average optical power, the classic Shannon channel capacity formula is not applicable to VLC systems. To derive a simple closed-form upper bound on channel capacity, sphere packing argument method has been applied previously. However, there is an obvious gap between the existing sphere-packing upper bounds and the lower bounds at high optical signal-to-noise-ratios (OSNRs), which is mainly caused by the inaccurate mathematical approximation of the intrinsic volumes of the simplex. In this letter, a tight sphere-packing upper bound is derived with a new approximation method. Numerical results demonstrate that compared to the existing sphere-packing upper bounds, our proposed upper bound is tighter at high OSNRs.

Double-space-cooperation method for increasing channel capacity

Shannon channel capacity theorem poses highest bit-rate of error free transmission over additive white Gaussian noise channel. In addition, he proved that there exists channel code that can theoretically achieve the channel capacity. Indeed fortunately, the latter researchers found some practical channel codes approaching the channel capacity with insignificant losses of spectral efficiency under ignorable bit error rate (BER). The authors note, in general, that bits of the channel codes are not independent of each other in code space. Further, we note that the modulated symbols are not independent among them, as well, in Euclidean Space. By exploiting a usage of the dependencies jointly to signal design, we can transmit two independent signal streams through an additive white Gaussian channel and separate them in Euclidean space at the receiver. The capacity of this approach is found larger than that of Shannon capacity in the same channel assumptions. The numerical results confirm the theoretical procedures.

Energy and Delay Constrained Maximum Adaptive Schedule for Wireless Networked Control Systems

Communication system design for wireless networked control systems (WNCSs) is very challenging since the strict timing and reliability requirements of control systems should be met by the wireless communication systems that introduce non-zero packet error probability and non-zero delay at all times. Particularly, the scheduling algorithms for WNCSs should be designed to provide maximum level of adaptivity accommodating packet losses and changes in network topology while exploiting periodic nature of the sensor node transmissions. Creating such a schedule has been previously studied for an Ultra Wide Band (UWB) based WNCS. In this paper, we extend the joint optimization problem of power control, rate adaptation and scheduling with the objective of providing maximum adaptivity for general WNCSs employing continuous rate transmission model in which Shannon's channel capacity formulation is used for the achievable transmission rate. Upon proving the NP-hardness of the problem, we provide a framework for the design of a heuristic algorithm for scheduling and propose an optimal polynomial time algorithm for the power control and rate adaptation problem following the derivation of the optimality conditions. We demonstrate via extensive simulations that the proposed algorithms outperform the existing algorithms with performance close to optimal solution and average runtime admissible for practical WNCSs.

Detection Capacities of Distributed and Centralized Systems: A Comparative Study

Distributed systems using many inexpensive sensors widely distributed over a large area present an alternative way for target detection and potential paradigm change in environmental sensing. The diversity of opportunities for detection by widely distributed sensors seems attractive, but how one compares the detection performance based on the observations made from many distributed sensors, each with small gain, with that from a centralized system with a high array gain has not been studied theoretically or experimentally. Treating the target-radiated signal as a communication signal, transmitting continuous Gaussian-distributed alphabets, the Shannon channel capacity yields the maximum information that the receivers can learn about the (target) transmitted signal. For this idea case, the channel capacity can then be used as a metric to compare the performance of various sensor systems. Matched track processing is introduced to motivate a capacity-based detector and the corresponding detection capacity. Based on that, it is found that the distributed systems can achieve, in principle, an area of coverage two to three times larger than that of a centralized system under the right conditions, and the area of coverage by the entire system can be significantly larger than the sum of detection areas of individual nodes for distributed systems.

Shifting the Frontiers of Analog and Mixed-Signal Electronics

Nowadays, analog and mixed-signal (AMS) IC designs, mainly found in the frontends of large ICs, are highly dedicated, complex, and costly. They form a bottleneck in the communication with the outside world, determine an upper bound in quality, yield, and flexibility for the IC, and require a significant part of the power dissipation. Operating very close to physical limits, serious boundaries are faced. This paper relates, from a high-level point of view, these boundaries to the Shannon channel capacity and shows how the AMS circuitry forms a matching link in transforming the external analog signals, optimized for the communication medium, to the optimal on-chip signal representation, the digital one, for the IC medium. The signals in the AMS part itself are consequently not optimally matched to the IC medium. To further shift the frontiers of AMS design, a matching-driven design approach is crucial for AMS. Four levels will be addressed: technology-driven, states-driven, redundancy-driven, and nature-driven design. This is done based on an analysis of the various classes of AMS signals and their specific properties, seen from the angle of redundancy. This generic, but abstract way of looking at the design process will be substantiated with many specific examples.

A Unified Carnot Thermodynamic and Shannon Channel Capacity Information-Theoretic Energy Efficiency Analysis

In this paper we employ the Shannon channel capacity theorem and classical thermodynamic Carnot's Law to derive the kT ln 2 minimum energy dissipation per bit for a communications channel. We then extend the analysis, incorporating forward error correction (FEC) to show an asymptotic energy efficiency approach to the Carnot/Shannon limit. For the first time, we derive a generalized version of the Shannon channel capacity theorem which embraces non-Gaussian noise statistics. Finally, we apply the theory to different telecommunications technologies, thus offering commonality of absolute energy efficiency assessment.

Keyless Authentication in a Noisy Model

We study a keyless authentication problem in a new noisy model, where there is a discrete memoryless channel (DMC) $W_{1}$ from sender Alice to receiver Bob and a DMC $W_{2}$ from adversary Oscar to Bob. In addition, there is an insecure noiseless channel between Alice and Bob. Under this model, we characterize the condition under which an authentication from Alice to Bob is possible. We also construct a secure authentication protocol that has an authentication rate approaching infinity. Finally, we prove that the authentication capacity of a noninteractive authentication over binary symmetric channels is exactly 1. This is an interesting result as Shannon capacity of channel $W_{1}$ is strictly less than 1 while the noiseless channel is completely unreliable.

Performance comparison of distributed and centralized systems based on information

Distributed systems using many sensors widely distributed over a large area present an alternative way for target detection and environmental sensing. The diversity of opportunities for detection by widely distributed sensors seems attractive but the signals received on distributed sensors are usually not coherent due to the wide separations between the receivers. Hence, conventional approaches based on signal coherence/gain no longer apply. In this paper, information theory will be used to compare their performance. Treating the target radiated signal as a communication signal, transmitting continuous Gaussian-distributed alphabets, the Shannon channel capacity yields the maximum information that the receivers can learn about the (target) transmitted signal. The channel capacity can then be used as a metric to compare the performance of various sensor systems in the ideal case. Matched tracking processing is introduced to motivate a capacity-based detector and detection capacity. Based on that, it is found that the distributed systems can achieve in principle an area of coverage two to three times larger than that of a centralized system under the right conditions, and the area of coverage by the entire system can be significantly larger than the sum of detection areas of individual nodes.

A WLAN design problem with known mobile device locations: a global optimization approach

As wireless networks continue to see rapid growth, the next generation of networks needs to be designed to optimize fundamental capacity limits suggested by Shannon's Information Theory. The use of simple coverage models and linear objective functions based on number of users will not provide optimal performance. This manuscript describes an optimization model and software tools that may be used to assist a network engineer in the design of a wireless local area network. The model uses Shannon's channel capacity formula to calculate network capacity and is restricted to a single frequency with known locations for the mobile devices. Even under these assumptions, the model involves a complex nonlinear objective function with quadratic constraints. Multiple commercial software tools as well as a global optimizer developed by us are empirically evaluated on a set of test cases. One of the global solvers developed by LINDO Systems was found to work very well for all of the design cases tested.

Analysis of capacity and outage probability for intersatellite optical communications in the presence of random pointing jitter

This work is motivated by the need to achieve the Shannon channel capacity for intersatellite optical communication with intensity modulation and direct detection. In this paper, we investigate the tradeoff between the channel capacity and the outage probability in the presence of random pointing jitter for intersatellite optical links. First, we derive an analytical expression of the outage probability in the absence of bias error, and then the bias error is taken into account. By minimizing the outage probability, we obtain analytically the optimum design of the beam half-width divergence angle $ w_0^{\text{opt}} $ for a given channel code rate. Furthermore, we discuss the tradeoff between the outage probability and the channel capacity and illustrate the results by adopting $ w_0^{\text{opt}} $ in the presence of random pointing jitter. The results obtained are useful for the parametric-estimation optimization of intersatellite optical communication systems.

Spectral Efficient Multihop Relaying Based on Alternate Transmission

A spectral efficient multihop relaying scheme is proposed based on scheduling the source transmission in alternate time slots. Each relay processes its received signal and forwards it in the subsequent time slot. An interference cancellation mechanism is developed to eliminate the interference term at each relay caused by the source alternate-based transmission. The frame error probability and the achievable rate of the proposed scheme are evaluated. It is shown that the individual bit-by-bit detection at the destination performs almost as well as the optimal joint detection, and achieves a data rate that approaches the Shannon channel capacity limit. In addition, the proposed scheme significantly outperforms the conventional orthogonal multihop transmission in terms of the achievable data rate, especially for larger numbers of hops, with slightly inferior frame error rate performance.

Random Matrix Model for Nakagami–Hoyt Fading

Random matrix model for the Nakagami-q (Hoyt) fading in multiple-input multiple-output (MIMO) communication channels with arbitrary number of transmitting and receiving antennas is considered. The joint probability density for the eigenvalues of H{\dag}H (or HH{\dag}), where H is the channel matrix, is shown to correspond to the Laguerre crossover ensemble of random matrices and is given in terms of a Pfaffian. Exact expression for the marginal density of eigenvalues is obtained as a series consisting of associated Laguerre polynomials. This is used to study the effect of fading on the Shannon channel capacity. Exact expressions for higher order density correlation functions are also given which can be used to study the distribution of channel capacity.

Capacity of η – μ fading channels under different adaptive transmission techniques

An analysis for the Shannon channel capacity of digital receivers operating over η–μ fading channels is presented under four adaptive policies: constant power with optimal rate adaptation, optimal power and rate adaptation, channel inversion with fixed rate and truncated channel inversion with fixed rate. In this context, useful formulae for the average channel capacity for maximal-ratio combining receivers with not necessarily identically distributed branches are derived. The analysis also includes the performance of single-branch receivers. These expressions provide an effective tool to assess the spectral efficiency of the aforementioned adaptive transmission techniques over practical fading channels. Our newly derived expressions include several special cases that arise from the η–μ distribution, namely the Nakagami-m and the Hoyt distribution. Extensive numerical and computer simulation results are also presented that illustrate the proposed mathematical analysis.

Quantized Identification With Dependent Noise and Fisher Information Ratio of Communication Channels

System identification is studied in which the system output is quantized, transmitted through a digital communication channel, and observed afterwards. This paper explores strong convergence, efficiency, and complexity of identification algorithms under colored noise and dependent communication channels. It first presents algorithms for certain core identification problems using quantized observations on the basis of empirical measures and nonlinear mappings. Strong consistency (with-probability-one convergence) is established under ¿-mixing noises. Furthermore, with pre-quantization signal processing, it is shown that certain modified algorithms can achieve asymptotic efficiency under correlated noises. To improve convergence speeds, quantization threshold adaptation algorithms are introduced. These results are then used to study the impact of communication channels on system identification under dependent channels. The concept of fisher information ratio is introduced to characterize such impact. It is shown that the fisher information ratio can be calculated from certain channel characteristic matrices. The relationship between the fisher information ratio and Shannon's channel capacity is discussed from the angle of time and space information. The methods of identification input designs that link general system parameters to core identification problems are reviewed.