Information Channel Capacity Research Articles

INFORMATION TRANSFER RATE OF NEURONS: STOCHASTIC RESONANCE OF SHANNON'S INFORMATION CHANNEL CAPACITY

The information channel capacity of neurons is calculated in the stochastic resonance region using Shannon's formula. This quantity is an effective measure of the quality of signal transfer, unlike the information theoretic calculations previously used, which only characterize the entropy of the output and not the rate of information transfer. The Shannon channel capacity shows a well pronounced maximum versus input noise intensity. The location of the maximum is at a higher input noise level than has been observed for classical measures, such as signal-to-noise ratio.

Information Capacity of Channels with Partially Unknown Noise. II. Infinite-Dimensional Channels

Information capacity is considered for a communication channel in which the noise is the sum of a known Gaussian component and an independent component with unknown statistical distributions. A lower bound on capacity is sought; the unknown noise component is thus assumed to be under the control of an adversary---a jammer. The problem is modeled as a zero-sum two-person game with mutual information as the payoff function. Appropriate constraints are determined on the transmitted signal and the unknown noise component. Although the usual conditions sufficient for application of the general form of the von Neumann minimax theorem are shown not to hold, a solution is obtained for the game: a saddle value, saddle point, and minimax strategy for the jammer are obtained. The essential effect of jamming is to convert the infinite-dimensional channel into a finite-dimensional channel having the same constraints, with the dimensionality depending upon the problem parameters: the covariance of the known Gaussian noise component and the constraints on the transmitted signal and the unknown noise component.

Information Capacity of Channels with Partially Unknown Noise. I. Finite-Dimensional Channels

Information capacity is considered for a finite-dimensional communication channel in which the noise is the sum of a known Gaussian noise and an independent component with unknown statistical properties. The problem is modeled as a zero-sum two-person game between the coder and a jammer, with mutual information as the payoff function. The jammer does not control the ambient Gaussian noise. The constraints employed are appropriate for extension to infinite-dimensional channels and for consideration of coding capacity. The original problem is reformulated to prove existence of both a saddle value and a saddle point. The saddle value and a saddle point are determined, along with the jammer’s minimax strategy.

Criteria for positioning of sensors for a microphone array

Sensor positioning has an important influence on the performance of array processing. The main contributions of this paper are criteria to evaluate and to optimize the performance of an array of sensors through its geometric configuration-the number and the positions of sensors. The distance between the observed and the desired directivity patterns has been used as a criterion. We propose a generalization of this criterion. We obtain a second criterion by an orthogonality measurement between two subspaces, respectively, containing the noise and the desired signals. This measurement is obtained by an integration of the variance of the signal estimated by an unbiased linear estimator. The output power of a minimum variance distortionless response (MVDR) filter is considered as another criterion. Considering the channels between the desired sources and the array as a multidimensional communication channel, we obtain as a criterion the channel information capacity, which shows the maximum information that we may transfer by this channel. The simulation results confirm that the criteria are related and properly characterize the array configuration. For a linear equally spaced array and narrowband plane-wave signals, the classical results regarding the sensor spacing are confirmed.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Using a binocular field expander on a wide-field search task.

Five patients with constricted visual fields and good central acuity were fitted with an experimental binocular field expander. They undertook a laboratory visual search task in a stationary horizontal 90 degree field with and without the device. Four of five patients adapted relatively quickly to the inherent optical distortions of the field expander. When this binocular aid was worn search efficiency improved significantly with practice; search time was highly correlated (r = -0.92; p less than 0.01) with information channel capacity in the larger visual field.

The information index: An objective measure of speech transmission performance *

The maximal mutual information (channel capacity) between the input and output of a telephone connection was calculated, taking into account both the frequency and time domain aspects of this problem. This leads to define an « information index » which can be computed from objective measurements. Several examples are given, both for digital coders and for analogue transmission: they show that the information index has a very good correlation with the results of opinion tests. This index therefore constitutes the basis of a mathematical model for predicting the transmission performance of speech in a telephone network.

The Information Capacity of the Human Fingertip

An order-of-magnitude upper bound is placed on the information channel capacity for vibrotactile stimulation of human fingertips. Results are presented which suggest an order-of-magnitude limit of 100 bit/s. This limit is compared to previously published figures for similar limits in the human ear and eye, showing a progression of 102: 104: 106, respectively.

The information capacity of a multi-dimensional communications system

One-dimensional communications and two-dimensional optics may be considered special cases of a multi-dimensional communications system, the information capacity and channel capacity of which are derived. It is shown that the channel capacity of such a system may far exceed that of a conventional temporal communications system with the same bandwidth. In the case of imaging systems, an invariance theorem is proposed. This theorem is then invoked to explain the underlying principles behind optical super-resolution. The theorem is then applied to the analysis of analytic continuation and several previous results are predicted. The exploitation of multiple dimensions in communications is also discussed. The spatial dimensions of communications systems are expected to become increasingly important if future communication needs are to be satisfied.

On the capacity of channels with Gaussian and non-Gaussian noise

We evaluate the information capacity of channels for which the noise process is a Gaussian measure on a quasi-complete locally convex space. The coding capacity is calculated in this setting and for time-continuous Gaussian channels using the information capacity result. The coding capacity of channels with non-Gaussian noise having finite entropy with respect to Gaussian noise of the same covariance is shown not to exceed the coding capacity of the Gaussian channel. The sensitivity of the information capacity to deviations from normality in the noise process is also investigated.

The Effects of Vibration on Manual Control Performance

A simple systems approach is used to derive a basic framework for the effects of vibration on human operator performance. This framework is applied to an investigation of co-ordinated manual tracking during vibration. An experiment is described in which twelve subjects were required to perform zero order pursuit tracking of a quasi-random forcing function in a horizontal axis with freo-moving and spring centred control levers. The seated subjects were exposed to four conditions of vertical vibration by means cf a flat, hard seat. Each vibration condition consisted of a combination of three sinusoids (3, 5 and 8 Hz), each of equal acceleration amplitude (either 0, 0.25, 0.5 or 1 m/s2rms). Performance was measured in terms of information channel capacity and frequency dependent error. The frequency dependent measures wero used to distinguish error due to direct transmission of vibration to the control from that due to other sources. Vibration transmitted directly to the control caused only a small proportio...

Introduction to Diffraction, Information Processing and Holography

The field of coherent optics has stimulated much interest and indeed excitement over the past decade, and a number of engineering applications have been brought to light. The most significant of these are the subject of this senior- or graduate-level text, which was originally prepared--but is not limited to--electrical engineering students. It emphasizes the analogy between optical and electrical systems, both of which, for example, are capable of performing Fourier transform operations and signal filtering and processing.The book is designed for students without an intensive background in electromagnetic theory and classical optics. Its discussion of diffraction is based on scalar theory, and it approaches information processing and holography by means of the elementary point concept and linear system theory. This approach simplified the analysis so that solutions may be directly calculated, and it will appeal to engineering students because of their familiarity with the concepts of the impulse response of linear systems.After an opening presentation of the basic properties of linear systems and Fourier transformations, the book develops the theory of diffraction. The topics taken up include, among others, Fraunhofer and Fresnel diffraction, the reciprocity theorem, Huygens' principle, Kirchhoff's integral, the Fresnel zone plate, the Rayleigh criterion, and Abbe's sine condition. This part closes with a discussion of coherent theory and the mutual coherence function.The next part, on information processing, covers the Fourier transform properties of lenses and linear optical imaging systems, filtering, the basic properties of photographic film as a recording medium, film-grain noise and signal-to-noise ratio, the information channel capacity of photographic film, and optical resolving power and its relation to the uncertainty of information and physical realizability.The final part throws a clear light on the subject of holography. The presentation includes both linear and nonlinear holograms and takes up wavefront construction and reconstruction, magnifications, resolution limits and bandwidth requirements, finite-point analysis linear optimization techniques, syntheses of optimum nonlinear spatial filters, and applications. Among the latter are microscopic wavefront reconstruction, multiexposure holographic interferometry, time-average interferometry, and contour generation.

Measurement of Information Transfer in Retinal Ganglion Cells of Cat

The paper deals with questions concerning the meaning of single spikes and the information capacity of sensory channels. In order to obtain useful data, measurements were made, in cats, of the transinformation of retinal ganglion cell responses, using methods provided by information theory. It is concluded that there seems to be less information reduction (or loss) in the pathways from the retina to the cortex than has hitherto been assumed, especially by those authors who calculated information capacity by means of the Weber constant.

Information channel capacity of a photographic film (Corresp.)

A different approach toward the information channel capacity of a photographic film is presented. It is based on the total number of possible combinations of the smallest distinguishable regions in a piece of photographic film. The information channel capacity of a hologram is also discussed. It is found that the information capacity of a hologram is somewhat lower than that of a photographic film. However, the information content of a hologram is, in general, greater than that of a photograph. It is the recording process used in making a hologram that provides the advantage over an ordinary photograph. Nevertheless, if the object to be recorded were properly encoded, then the information content of a hologram could never reach that of a photographic film.

On Decision Theory Under Competition

Decision theory treats models in which a decision maker must choose an information function and a decision function, where the former transforms states of nature into messages, the latter transforms messages into actions, and the ultimate payoff depends upon the state of nature, the message, and the action. This paper treats similar models in which nature's role is played by a selfseeking rational opponent. These models fall into the purview of the theory of two-person games, and the main results concern the maximin and minimax theories of these games. The various games treated differ in the assumptions made about the decision maker's ability to probability mix his choices and about his opponent's information concerning these choices. Twenty different sets of assumptions are considered, but it is shown that the maximin and minimax theories of the resulting games are all reducible in substance to those of just three games: (i) in which the decision maker cannot probability mix his choices at all, and his opponent observes both choices; (ii) in which the decision maker can mix arbitrarily, and his opponent observes neither of his choices; or (iii) in which the decision maker can mix decision functions only, and his opponent observes only the information function. These games are represented in terms of matrix games, and the matrices are compressed by the deletion of dominated strategies for the decision maker. Further compressions are obtained for the case in which the cost of information is separable from the payoff resulting from the actions of the decision maker and his opponent. The game matrices are finally reduced to manageable proportions for the further specialized cases in which either the capacity of the decision maker's information channel is significantly limited and information cost is constant, or the capacity of the information channel is large and there is one low—cost “non-message” available. A numerical example is given, and the concluding comments point to directions for further research.

Information capacity of fading channels under conditions of intense interference

The newer, more efficient data transmission systems which are designed to combat channel variations or interference, or both, require information concerning the channel variations for proper operation. Although it is seldom recognized, such systems must receive and process two distinct types of information. The first, and quite obvious, type of information concerns the intelligence signal which is provided as an output from the receiving terminal of the system. The second, and much less obvious, type of information concerns medium variations. This information is normally used internally at the receiver terminal in order that proper processing of of the received signal may be obtained. As the channel SNR progressively worsens, the total information capacity of the channel must of necessity decrease. Eventually the total information capacity of the channel becomes so small that the receiver is denied the required information concerning medium variations and proper operation of systems of the above type is no longer possible. This paper is concerned with the limiting case of very small channel SNR's that are below the level required for analysis of the medium at the receiver. The channel capacity of an ideal or nonfading channel is derived for reference purposes and in order that an estimate of the information rate loss due to medium variations at low SNR's may be obtained. The problem of data transmission over a fading channel under conditions of intense interference is approached in an intuitive manner. A system for operation under such conditions is proposed which appears to offer the best performance possible. The performance of this proposed system is then analyzed to yield capacity formulas for the conditions specified. No proof is given that the system proposed is, in fact, ideal but the resulting capacity formulas are believed to be correct in form and sufficiently accurate in a quantitative sense for useful practical application. The mathematical approach used is sufficiently general to permit inclusion of random system variations (such as oscillator phase) as part of the medium variations and hence the results are believed to be applicable to a wide variety of communications situations including deep-space probe telemetry.

Information capacity of time-continuous channels

The maximum average mutual information in the observation of the output, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">y(t)</tex> , of a channel over the time interval <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[T_3,T_4]</tex> about the signal (input), <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s(t)</tex> , in the interval <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[T_1, T_2]</tex> is taken as the definition of channel capacity for the time-continuous case. In the case where the channel introduces additive Independent Gaussian noise of known correlation function, the capacity is evaluated subject to the constraint that the signal process have a given correlation function. For this evaluation a new joint expansion of the processes <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">y(t)</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s(t)</tex> is introduced which has the property that all coefficients in the expansion are uncorrelated. Thus, the expansion is a generalization of the Karkunen-Lo'{e}ve expansion to which it reduces when the noise is white and the time intervals coincide. The channel capacity is shown to be directly related to results in the theory of optimum filtering over a finite time interval. Closed form results for the capacity of several channels are given as well as some limiting expressions and bounds. For the case of white noise of spectral density <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N_o</tex> , the capacity is always bounded by <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\bar{E}/N_o</tex> where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\bar{E}</tex> is the average signal energy.

The theoretical channel capacity of a single neuron as determined by various coding systems

The information channel capacity of a model neuron with a fixed refractory period δ has been calculated for optimum continuous time interval coding. Two types of noise perturbations have been considered, a Gaussian probability distribution and a rectangular distribution of the time of occurrence of the response to a stimulus. Laboratory measurements indicate a Gaussian distribution with a standard deviation of latency σ, of about 5 μsec gives the best fit to an actual nerve fiber. This results in a maximum information transmission of slightly over 4000 bits per second. The results are compared with the discrete theory of MacKay and McCulloch.