Application Of Information Theory Research Articles

Applicability of Information Theory to the Quantification of Responses to Anthropogenic Noise by Southeast Alaskan Humpback Whales

We assess the effectiveness of applying information theory to the characterization and quantification of the affects of anthropogenic vessel noise on humpback whale (Megaptera novaeangliae) vocal behavior in and around Glacier Bay, Alaska. Vessel noise has the potential to interfere with the complex vocal behavior of these humpback whales which could have direct consequences on their feeding behavior and thus ultimately on their health and reproduction. Humpback whale feeding calls recorded during conditions of high vessel-generated noise and lower levels of background noise are compared for differences in acoustic structure, use, and organization using information theoretic measures. We apply information theory in a self-referential manner (i.e., orders of entropy) to quantify the changes in signaling behavior. We then compare this with the reduction in channel capacity due to noise in Glacier Bay itself treating it as a (Gaussian) noisy channel. We find that high vessel noise is associated with an increase in the rate and repetitiveness of sequential use of feeding call types in our averaged sample of humpback whale vocalizations, indicating that vessel noise may be modifying the patterns of use of feeding calls by the endangered humpback whales in Southeast Alaska. The information theoretic approach suggested herein can make a reliable quantitative measure of such relationships and may also be adapted for wider application to many species where environmental noise is thought to be a problem.

Osmotic modulation of stimulus‐evoked responses in the rat supraoptic nucleus

Neural information is conveyed by action potentials along axons to downstream synaptic targets. Synapses permit functionally relevant modulation of the information transmitted by converging inputs. Previous studies have measured the amount of information associated with a given stimulus based either on spike counts or on the relative frequencies of spike sequences represented as binary strings. Here we apply information theory to the phase-interval stimulus histogram (PhISH) to measure the extent of the stimulus-evoked response using the statistical relationship between each interspike interval and its phase within the stimulus cycle. We used the PhISH as a novel approach to investigate how different osmotic states affect the flow of information through the osmoreceptor complex of the hypothalamus. The amount of information conveyed from one (afferent) element of the complex, the anteroventral region of the third ventricle (AV3V), to another (an efferent element), the supraoptic nucleus, was increased by hypertonic stimulation (intravenous mannitol, z = 4.39, P < 0.001) and decreased by hypotonic stimulation (intragastric water, z = -3.37, P < 0.001). Supraoptic responses to AV3V stimulation differed from those that follow stimulation of a hypothalamic element outside the osmoreceptor complex, the suprachiasmatic nucleus (SCN), which also projects to the supraoptic nucleus. Thus osmosensitive gain control mechanisms differentially modulate osmotically dependent and osmotically independent inputs, and enhance the osmoresponsiveness of supraoptic cells within a physiological range. The value of the novel approach is that its use is not limited to the osmoreceptor ensemble but it can be used to investigate the flow of information throughout the central nervous system.

An evaluation tool for psychomotor performance during visual motor task: An application of information theory

On the basis of an approach inspired by Shannon's information theory, a Visual Motor Efficiency (VME) index is proposed. This index is an absolute measure of performance because it expresses a percentage of a perfect performance. By means of a Psycho Motor Performance Device (PMPD), this index is used to evaluate psychomotor performance during visual pointing task. Localising was carried out with the help of a unidirectional handle that only allows supination or pronation movements of the wrist to reach targets that appear on a computer screen in the form of a small square along the horizontal (left/right) and vertical (up/down) arms of a cross. The PMPD was designed so that the progress of the visual motor task would be immediately examined. In a pilot study dedicated to track the evolution of performance in six subjects for six consecutive days, we show that the degree of directional coherence between movement and effect produced is an important factor. Indeed the performance level is significantly lower when it is a question of pointing at targets located along the vertical arm compared to pointing at those along the horizontal. These results indicate that PMPD is a valid and sensitive measurement of human psychomotor performance in charting the progress of a subject during training.

Development of an e-learning system for manufacturing technology education

Recently, e–learning technology has been applied to every kind of specialised education field. However, these systems are developed based on a browser and HTML environment. The major capabilities of these systems are limited to presenting course contents to students. Because of the framework limitations for developing e–learning systems, students' practical abilities cannot be performed or disciplined through a developed e–learning system. Currently, practical training is still applied on the shop floor to discipline students' practice abilities. How to build up students' practice ability through an e–learning system is an important research issue. This study employs virtual reality techniques to develop an interactive virtual laboratory to discipline students' practice ability using the internet. Four kinds of manufacturing technology are established in this system: turning, milling, spraying and laser marking. This system allows students to select different process parameters to learn entire manufacturing processes in real time. Further, this study applies information theory to develop a semantic interface for students to present their queries in nature language. This study has opened up an opportunity of change to the traditional practical training method. The developed system provides students with a syntactic and semantic way to learn manufacturing technology.

Novel information theory techniques for phonon spectroscopy

The maximum entropy method (MEM) and spectral reverse Monte Carlo (SRMC) techniques are applied to the determination of the phonon density of states (PDOS) from heat-capacity data. The approach presented here takes advantage of the standard integral transform relating the PDOS with the specific heat at constant volume. MEM and SRMC are highly successful numerical approaches for inverting integral transforms. The formalism and algorithms necessary to carry out the inversion of specific heat curves are introduced, and where possible, I have concentrated on algorithms and experimental details for practical usage. Simulated data are used to demonstrate the accuracy of the approach. The main strength of the techniques presented here is that the resulting spectra are always physical: Computed PDOS is always positive and properly applied information theory techniques only show statistically significant detail. The treatment set out here provides a simple, cost-effective and reliable method to determine phonon properties of new materials. In particular, the new technique is expected to be very useful for establishing where interesting phonon modes and properties can be found, before spending time at large scale facilities.

An Information-Theoretical Approach to Contextual Processing in the Human Brain: Evidence from Prefrontal Lesions

Context shapes perception, thought, and action, but little is known about the neural mechanisms supporting these modulations. Here, we addressed the role of lateral prefrontal cortex (PFC) in context updating and maintenance from an information-theoretic perspective. Ten patients with PFC lesions and 10 age-matched controls responded to bilaterally displayed visual targets intermixed with repetitive and novel distracters in 2 different task contexts. In a predictable context, targets were always preceded by a novel event, whereas this temporal contingency was removed in an unpredictable context condition. We applied information theory to the analysis and interpretation of behavioral and electrophysiological data. The results revealed deficits in both the selection and the suppression of familiar versus novel information mainly observed at the visual hemifield contralateral to PFC damage due to disrupted frontocortical and frontosubcortical connectivity. The findings support a deficit in the representation of the temporal contingency between contextually related novel and familiar stimulation subsequent to lateral PFC damage.

Applications of information theory in ensemble data assimilation

AbstractWe apply information theory within an ensemble‐based data assimilation approach and define information matrix in ensemble subspace. The information matrix in ensemble subspace employs a flow‐dependent forecast error covariance and it is of relatively small dimensions (equal to the ensemble size). The information matrix in ensemble subspace can be directly linked to the information matrix typically used in non‐ensemble‐based data assimilation methods, such as the Kalman Filter (KF) and the three‐dimensional variational (3D‐Var) methods, which provides a framework for consistent comparisons of information measures between different data assimilation methods.We evaluate information measures, such as degrees of freedom for signal, within the Maximum Likelihood Ensemble Filter (MLEF) data assimilation approach and compare them with those obtained using the KF approach and the 3D‐Var approach. We assimilate model‐simulated observations and use the Goddard Earth Observing System Single Column Model (GEOS‐5 SCM) as a dynamical forecast model.The experimental results demonstrate that the proposed framework is useful for comparing information measures obtained in different data assimilation approaches. These comparisons indicate that using a flow‐dependent forecast error covariance matrix (e.g. as in the KF and the MLEF experiments) is fundamentally important for adequately describing prior knowledge about the true model state when calculating information measures of assimilated observations. We also demonstrate that data assimilation results obtained using the KF and the MLEF approach (when ensemble size is larger than 10 ensemble members) are superior to the results of the 3D‐Var approach. Copyright © 2007 Royal Meteorological Society

Prof. Dr‐Ing. Dr‐Ing. E.h. Joachim Hagenauer on occasion of his 65th birthday

AbstractThe following nine contributions are dedicated to Prof. Dr‐Ing. Dr‐Ing. E.h. Joachim Hagenauer, who was the Head of the Institute for Communications Engineering at the Munich University of Technology from April 1993 to September 2006. Among the authors are 22 Ph.D. students of Prof. Hagenauer. This editorial provides a brief overview on his enormous contributions to science around Applied Information Theory so far. Copyright © 2007 John Wiley &amp; Sons, Ltd.

INSIDER TRADING RULES AND PRICE FORMATION IN SECURITIES MARKETS: AN ENTROPY ANALYSIS OF STRATEGIC TRADING

This paper addresses the issue of how insider trading rules affect price formation in securities markets and suggests the application of information theory to market microstructure theory. We analyze a variant of the setting in [20] by simply introducing a more general criterion for informational efficiency borrowed from information theory — namely maximum information transmission. The analysis shows that both the insider's optimal trading strategy and the market price of the risky security depend on the insider trading restriction. Insider trading restrictions are reported to be detrimental to the liquidity of the securities market. We find that a unique insider trading rule exists which implements semi-strong form informational efficiency of the securities market. Alternative restrictions on insider trading give rise to either underreaction or overreaction in securities prices. Too strict insider trading rules are shown to account for excess volatility in securities prices. Contrary to common notion, the uninformed investors are shown to be hurt by too restrictive insider trading rules. We conclude that loose insider trading rules are preferred by the group of investors as a whole.

Information Theory in Living Systems, Methods, Applications, and Challenges

Living systems are distinguished in nature by their ability to maintain stable, ordered states far from equilibrium. This is despite constant buffeting by thermodynamic forces that, if unopposed, will inevitably increase disorder. Cells maintain a steep transmembrane entropy gradient by continuous application of information that permits cellular components to carry out highly specific tasks that import energy and export entropy. Thus, the study of information storage, flow and utilization is critical for understanding first principles that govern the dynamics of life. Initial biological applications of information theory (IT) used Shannon's methods to measure the information content in strings of monomers such as genes, RNA, and proteins. Recent work has used bioinformatic and dynamical systems to provide remarkable insights into the topology and dynamics of intracellular information networks. Novel applications of Fisher-, Shannon-, and Kullback-Leibler informations are promoting increased understanding of the mechanisms by which genetic information is converted to work and order. Insights into evolution may be gained by analysis of the the fitness contributions from specific segments of genetic information as well as the optimization process in which the fitness are constrained by the substrate cost for its storage and utilization. Recent IT applications have recognized the possible role of nontraditional information storage structures including lipids and ion gradients as well as information transmission by molecular flux across cell membranes. Many fascinating challenges remain, including defining the intercellular information dynamics of multicellular organisms and the role of disordered information storage and flow in disease.

New directions in information theory

Telecommunication systems technology is characterized by a continuous, steady progress led by the massive, ever increasing use of Internet in all aspects of everyday life. Within the broader scope of telecommunication, wireless systems are playing a peculiar role in their impetuous pace toward higher bandwidth efficiency, increased reliability, unlimited mobility. We are then witnessing a continuous evolution of telecommunication standards and paradigms, trying to immediately incorporate the last promising theoretical advances, such as powerful channel codes, multi-antenna technology, etc. In this exciting panorama, the present call for papers intends to attract researchers in Information Theory to submit new results and tutorial outlines aiming at setting frameworks, milestones, and new directions in present and future telecommunication systems. Multi-terminal Information Theory Capacity of Multi-antenna Systems in Multi-user Environments Joint Source and Channel Coding Network Coding and User Cooperation Layering and Separation Theorems in Networks Channels with Feedback Fundamental properties of message-passing algorithms. In addition to submitting new results in the areas above, researchers in related fields and researchers working on novel applications of information theory are encouraged to submit contributions. Submission Instructions: Papers should be no longer than 20 double-spaced pages excluding figures. It is essential that each manuscript be accompanied by a 75- to 200-word abstract clearly outline the scope and contributions of the paper. The papers must be submitted through on-line system Manuscript Central at http://mc.manuscriptcentral.com/ett, going to the Authoring Center, specifying “Special Issue on New directions in Information Theory”. All submitted papers must not have been previously published and must not be submitted for publication elsewhere while they are under consideration. The following deadlines will apply:

Information loss in visual assessments of medical images

We applied information theory to quantify information losses in assessing contrast-detail (C-D) analysis. Images of a C-D phantom were acquired with a flat panel detector (FPD) and a computed radiography (CR) by changing surface entrance doses. Six phantom radiographs (FPD: five images; CR: one image) were prepared for visual evaluations. Thirteen radiographers and two radiologists participated in the observation test. Detectability was defined as the shortest length of the cylinders of which border the observers could recognize from the background, and was recorded using row number. Information content was defined as the entropy ∑ p i log(1/ p i ) with detection probabilities p i , which were calculated from distribution of detection rate of the ith column. Information loss, in unit of bits, was calculated as the difference between information obtained and information content when all the columns were detected. The information losses decreased with the increase in cylinder diameters and with the increase in surface entrance dose. Because the information loss varies depending on distribution of detection rate, this method of using the information theory was expected to be more sensitive in evaluating the C-D image quality than using the averaged values of detectability.

Approaches to Information-Theoretic Analysis of Neural Activity.

Understanding how neurons represent, process, and manipulate information is one of the main goals of neuroscience. These issues are fundamentally abstract, and information theory plays a key role in formalizing and addressing them. However, application of information theory to experimental data is fraught with many challenges. Meeting these challenges has led to a variety of innovative analytical techniques, with complementary domains of applicability, assumptions, and goals.

Modelling fish growth: Model selection, multi-model inference and model selection uncertainty

Model selection based on information theory is a relatively new paradigm in biological sciences with several advantages over the classical approaches. The aim of the present study was to apply information theory in the area of modelling fish growth and to show how model selection uncertainty may be taken into account when estimating growth parameters. The methodology was applied for length–age data of four species of fish, taken from the literature. Five-candidate models were fitted to each dataset: von Bertalanffy growth model (VBGM), generalized VBGM, Gompertz growth model, Schnute–Richards growth model, and logistic. In each case, the ‘best’ model was selected by minimizing the small-sample, bias-corrected form of the Akaike information criterion (AIC). To quantify the plausibility of each model, given the data and the set of five models, the ‘Akaike weight’ w i of each model was calculated. The average model was estimated for each case based on w i . Following a multi-model inference (MMI) approach, the model-averaged asymptotic length L ¯ ∞ for each species was estimated, using all five models, by model-averaging estimations of L ∞ and weighting the prediction of each model by w i . In the examples of this study, model selection uncertainty caused a magnification of the standard error of the asymptotic length of the best model (up to 3.9 times) and thus in all four cases estimating L ∞ from just the best model would have caused overestimation of precision of the asymptotic length. The VBGM, when used for inference, without being the best model, could cause biased point estimation and false evaluation of precision. Model selection uncertainty should not be ignored even if VBGM is the best model. Multi-model inference by model-averaging, based on Akaike weights, is recommended for making robust parameter estimations and for dealing with uncertainty in model selection.

A prolegomenon to nonlinear empiricism in the human behavioral sciences

We propose a general framework for integrating theory and empiricism in human evolutionary ecology. We specifically emphasize the joint use of stochastic nonlinear dynamics and information theory. To illustrate critical ideas associated with historical contingency and complex dynamics, we review recent research on social preferences and social learning from behavioral economics. We additionally examine recent work on ecological approaches in history, the modeling of chaotic populations, and statistical application of information theory.

DIAGNOSTIC ACCURACY OF MODELS 1

Successful practical implementation of fault diagnosis methods depends on good diagnosability. For Model-Based Diagnosis, the diagnostic accuracy of models is an important metric to predict this diagnosability. In this paper a technique to predict diagnostic accuracy from a system model is presented, together with its efficient computation. The correlations of accuracy with observability and uncertainty are established. The developed methods are successfully demonstrated in industrial case studies. The correlation with uncertainty provides another useful application of information theory within Model-Based Diagnosis.

Objective Assessment of the Information Content of Visible and Infrared Radiance Measurements for Cloud Microphysical Property Retrievals over the Global Oceans. Part I: Liquid Clouds

Abstract The importance of accurately representing the role of clouds in climate change studies has become increasingly apparent in recent years, leading to a substantial increase in the number of satellite sensors and associated algorithms that are devoted to measuring the global distribution of cloud properties. The physics governing the radiative transfer through clouds is well understood, but the impact of uncertainties in algorithm assumptions and the true information content of the measurements in the inverse retrieval problem are generally not as clear, making it difficult to determine the best product to adopt for any particular application. This paper applies information theory to objectively analyze the problem of liquid cloud retrievals from an observing system modeled after the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument currently operating on the Aqua and Terra platforms. It is found that four diagnostics—the retrieval error covariance, the information content, the number of degrees of freedom for signal, and the effective rank of the problem—provide a rigorous test of an observing system. Based on these diagnostics, the combination of the 0.64- and 1.64-μm channels during the daytime and the 3.75- and 11.0-μm channels at night provides the most information for retrieving the properties of the wide variety of liquid clouds modeled. With an eye toward developing a coherent representation of the global distribution of cloud microphysical and radiative properties, these four channels may be integrated into a suitable multichannel inversion methodology such as the optimal estimation or Bayesian techniques to provide a common framework for cloud retrievals under varying conditions. The expected resolution of the observing system for such liquid cloud microphysical property retrievals over a wide variety of liquid cloud is also explored.

Reflections on the History of Computer-Assisted Music Analysis I: Predecessors and the Beginnings

This article is the first of a series that focuses on the history of computer-assisted music analysis. This first article discusses the philosophical basis of computer-assisted music analysis, i.e. the application of information theory to aesthetics, as well as representative applications of statistical and information-theoretical measurements to music analysis and other computational approaches to music analysis that did not include the use of electronic computers. In most cases, those approaches were direct models for computer-assisted applications. Finally, this article provides a short historical account of the development of early computers and summarizes the earliest computer-applications to music analysis, carried out during the 1950s.

Control of Peptide Product Sizes by the Energy-Dependent Protease ClpAP

Processive proteases can unfold proteins and cleave them into fragments of a characteristic size. The detailed mechanism by which product sizes are controlled is still in question. One possible mechanism for the control of product sizes would be translocation of unfolded polypeptides to the protease active sites in units of defined length. We have investigated the mechanism by which ClpAP, an energy-dependent protease from Escherichia coli, controls the sizes of its peptide products. We show that ClpAP generates peptide products with a distribution of sizes that has a pronounced peak at a peptide length of 6-8 amino acid residues. This product size distribution, which is similar to that observed previously for the proteasome, is robust to perturbations that interfere with translocation or proteolysis. To explain these results, we propose a mechanism in which translocation alternates with proteolysis, allowing peptides of more or less uniform length to be cleaved processively from a translocating substrate. To estimate the rate and energy efficiency of ClpAP-catalyzed measurements of product sizes, we apply information theory to quantify how precisely the product sizes are controlled. This analysis may also prove to be useful in characterizing the mechanisms of other proteases and nucleases, such as the proteasome and Dicer, which control the sizes of their products.

Spike coding from the perspective of a neurone

In this paper, we compare existing methods for quantifying the coding capacity of a spike train, and review recent developments in the application of information theory to neural coding. We present novel methods for characterising single-unit activity based on the perspective of a downstream neurone and propose a simple yet universally applicable framework to characterise the order of complexity of neural coding by single units. We establish four orders of complexity in the capacity for neural coding. First-order coding, quantified by firing rates, is conveyed by frequencies and is thus entirely described by first moment processes. Second-order coding, represented by the variability of interspike intervals, is quantified by the log interval entropy. Third-order coding is the result of spike motifs that associate adjacent inter-spike intervals beyond chance levels; it is described by the joint interval histogram, and is measured by the mutual information between adjacent log intervals. Finally, nonstationarities in activity represent coding of the fourth-order that arise from the effects of a known or unknown stimulus.