Information Theory Methods Research Articles

Probabilistic Analysis of Intermediate Floor Steel and Wooden Structures in the Old Urban Development Building

The article suggests an approach to determine structural elements technical condition, based on the mathematical probabilistic apparatus of technical diagnostics. Diagnostics are performed using probabilistic methods of complex technical systems conditions recognition. Probabilistic parameters are calculated according to Bayes’s rule. The paper shows a diagnostics example of intermediate floor elements and systems in the old urban development building. Both the suggested method and information theory methods are used during diagnostics.

Research on Information Management System of Equipment Based on Information Theory

Information theory and information methods are first introduced. And then information methods are applied into information management system of equipment, which can show relation and character of management processes, and provide scientific support for information management system. Finally, information entropy is used for measurement of information capacity, which can achieve rational and economic equipment management plan.

Differential neural responses to naturally occurring envelopes in the electrosensory system

Natural sensory stimuli frequently consist of a fine structure whose amplitude (i.e. envelope) varies more slowly. Previous studies have demonstrated that envelope signals are found across sensory systems and are necessary for perception. For example, envelope signals in the auditory system carry information pertaining to the perception of pitch fluctuations in communication vocalizations [1], while envelopes in the visual system are responsible for object grouping and the perception of illusory contours [2,3]. However, the neural mechanisms underlying the encoding of these envelope signals are poorly understood in general. Gymnotiform wave-type weakly electric fish constitute an attractive model system for studying envelope processing in the brain because of its well-characterized anatomy and physiology [4]. These fish generate around themselves a weak quasi-sinusoidal electric field through the electric organ discharge (EOD) [5]. Recent studies have shown that the beat amplitude (i.e. the envelope) can vary in time during different behavioral contexts and display differential frequency content: while envelopes caused by movement primarily contain low ( 1 Hz) temporal frequencies [7]. While it is known that electrosensory neurons can respond to envelopes [7], whether these neurons respond differentially to envelopes arising from movement and social interaction is unknown in part because their tuning to envelope frequencies has not been characterized. Here we used well established techniques [8] to record from electrosensory pyramidal neurons across different maps of the body surface within the electrosensory lateral line lobe (ELL) in response to envelope stimuli with different frequency content spanning the behaviorally relevant range (0.05-10 Hz). Using standard measures of linear systems identification techniques as well as more quantitative methods of information theory, we found that pyramidal neurons in different maps displayed differential responses to envelopes arising from different behavioral contexts. Our results suggest that information about envelopes arising from different behavioral contexts begins to be segregated at the level of the ELL and provide new insights as to how these behaviorally relevant stimuli are encoded in early sensory pathways.

Discovery of biological networks using an optimized partial correlation coefficient with information theory algorithm on Stampede's Xeon and Xeon Phi processors

SUMMARYThe partial correlation coefficient with information theory (PCIT) method is an important technique for detecting interactions between networks. The PCIT algorithm has been used in the biological context to infer complex regulatory mechanisms and interactions in genetic networks, in genome wide association studies, and in other similar problems. In this work, the PCIT algorithm is re‐implemented with exemplary parallel, vector, input/output (I/O), memory, and instruction optimizations for today's multi‐core and many‐core architectures. The evolution and performance of the new code targets the processor architectures of the Stampede supercomputer but will also benefit other architectures. The Stampede system consists of an Intel Xeon E5 processor base system with an innovative component consist of Intel Xeon Phi Coprocessors. Optimized results and an analysis are presented for both the Xeon and the Xeon Phi. Copyright © 2014 John Wiley & Sons, Ltd.

Soft-Decoding-Based Strategies for Relay and Interference Channels: Analysis and Achievable Rates Using LDPC Codes

We provide a rigorous mathematical analysis of two communication strategies: soft decode-and-forward (soft-DF) for relay channels, and soft partial interference-cancelation (soft-IC) for interference channels. Both strategies involve soft estimation, which assists the decoding process. We consider LDPC codes, not because of their practical benefits, but because of their analytic tractability, which enables an asymptotic analysis similar to random coding methods of information theory. Unlike some works on the closely-related demodulate-and-forward, we assume non-memoryless, code-structure-aware estimation. With soft-DF, we develop {\it simultaneous density evolution} to bound the decoding error probability at the destination. This result applies to erasure relay channels. In one variant of soft-DF, the relay applies Wyner-Ziv coding to enhance its communication with the destination, borrowing from compress-and-forward. To analyze soft-IC, we adapt existing techniques for iterative multiuser detection, and focus on binary-input additive white Gaussian noise (BIAWGN) interference channels. We prove that optimal point-to-point codes are unsuitable for soft-IC, as well as for all strategies that apply partial decoding to improve upon single-user detection (SUD) and multiuser detection (MUD), including Han-Kobayashi (HK).

Managing for resilience: early detection of regime shifts in complex systems

The broad implications of catastrophic regime shifts have prompted the need to find methods that are not only able to detect regime shifts but more importantly, identify them before they occur. Rising variance, skewness, kurtosis, and critical slowing down have all been proposed as indicators of impending regime shifts. However, these approaches typically do not signal a shift until it is well underway. Further, they have primarily been used to evaluate simple systems; hence, additional work is needed to adapt these methods, if possible, to real systems which typically are complex and multivariate. Fisher information is a key method in information theory and affords the ability to characterize the dynamic behavior of systems. In this work, Fisher information is compared to traditional indicators through the assessment of model and real systems and identified as a leading indicator of impending regime shifts. Evidenced by the great deal of activity in this research area, it is understood that such work could lead to better methods for detecting and managing systems that are of significant importance to humans. Thus, we believe the results of this work offer great promise for resilience science and sustainability.

Parametric sensitivity analysis for biochemical reaction networks based on pathwise information theory.

BackgroundStochastic modeling and simulation provide powerful predictive methods for the intrinsic understanding of fundamental mechanisms in complex biochemical networks. Typically, such mathematical models involve networks of coupled jump stochastic processes with a large number of parameters that need to be suitably calibrated against experimental data. In this direction, the parameter sensitivity analysis of reaction networks is an essential mathematical and computational tool, yielding information regarding the robustness and the identifiability of model parameters. However, existing sensitivity analysis approaches such as variants of the finite difference method can have an overwhelming computational cost in models with a high-dimensional parameter space.ResultsWe develop a sensitivity analysis methodology suitable for complex stochastic reaction networks with a large number of parameters. The proposed approach is based on Information Theory methods and relies on the quantification of information loss due to parameter perturbations between time-series distributions. For this reason, we need to work on path-space, i.e., the set consisting of all stochastic trajectories, hence the proposed approach is referred to as “pathwise”. The pathwise sensitivity analysis method is realized by employing the rigorously-derived Relative Entropy Rate, which is directly computable from the propensity functions. A key aspect of the method is that an associated pathwise Fisher Information Matrix (FIM) is defined, which in turn constitutes a gradient-free approach to quantifying parameter sensitivities. The structure of the FIM turns out to be block-diagonal, revealing hidden parameter dependencies and sensitivities in reaction networks.ConclusionsAs a gradient-free method, the proposed sensitivity analysis provides a significant advantage when dealing with complex stochastic systems with a large number of parameters. In addition, the knowledge of the structure of the FIM can allow to efficiently address questions on parameter identifiability, estimation and robustness. The proposed method is tested and validated on three biochemical systems, namely: (a) a protein production/degradation model where explicit solutions are available, permitting a careful assessment of the method, (b) the p53 reaction network where quasi-steady stochastic oscillations of the concentrations are observed, and for which continuum approximations (e.g. mean field, stochastic Langevin, etc.) break down due to persistent oscillations between high and low populations, and (c) an Epidermal Growth Factor Receptor model which is an example of a high-dimensional stochastic reaction network with more than 200 reactions and a corresponding number of parameters.

Entropy-Based Nuclear Fuel Cycle Sensitivities and Covariances

Technology development and deployment decisions are justified by weighing their costs against the expected benefits. Multiple nuclear fuel cycle (NFC) simulation models have been devised, some with the aim of quantifying cyclewide sensitivities to variations from base-case scenarios. Base-case sensitivity studies often perturb only one parameter at a time and only in the region around the initial value. This paper details a sensitivity study methodology that applies entropy-based statistical methods of information theory to describe outcomes produced by an NFC model. This supersedes past efforts at sensitivity and uncertainty analysis by allowing a much larger space to be explored. Here, 30 independent fuel cycle parameters for a fast reactor-light water reactor hybrid scenario are varied simultaneously and stochastically. This fuel cycle schema was chosen as a well-known, sufficiently complex model; the underlying statistical methods could be applied to any cycle. This study uses the uncertainty coefficient computed from contingency tables (CTs) to represent the sensitivity of a technology-defining input to the response. The response of interest here was taken to be the deep geologic repository capacity for a given realization of fuel cycle inputs. After computing the uncertainty coefficients, the inputs themselves are sorted based on decreasing sensitivities. Fast reactor used fuel plutonium separations were found to be most important to the cycle. Furthermore, to represent input covariances (the effect of one input on the sensitivity of a second input to the response), a new measure is defined on three-dimensional CTs. This metric is the coefficient of the variation of uncertainty coefficient of two-dimensional slices of the original table. Sorting by this sensitivity of sensitivity metric, the input pair of fast reactor americium separations together with high-level-waste storage time was found to have the largest joint effect on the repository capacity.

Encore: Genetic Association Interaction Network Centrality Pipeline and Application to SLE Exome Data

Open source tools are needed to facilitate the construction, analysis, and visualization of gene-gene interaction networks for sequencing data. To address this need, we present Encore, an open source network analysis pipeline for genome-wide association studies and rare variant data. Encore constructs Genetic Association Interaction Networks or epistasis networks using two optional approaches: our previous information-theory method or a generalized linear model approach. Additionally, Encore includes multiple data filtering options, including Random Forest/Random Jungle for main effect enrichment and Evaporative Cooling and Relief-F filters for enrichment of interaction effects. Encore implements SNPrank network centrality for identifying susceptibility hubs (nodes containing a large amount of disease susceptibility information through the combination of multivariate main effects and multiple gene-gene interactions in the network), and it provides appropriate files for interactive visualization of a network using tools from our online Galaxy instance. We implemented these algorithms in C++ using OpenMP for shared-memory parallel analysis on a server or desktop. To demonstrate Encore's utility in analysis of genetic sequencing data, we present an analysis of exome resequencing data from healthy individuals and those with Systemic Lupus Erythematous (SLE). Our results verify the importance of the previously associated SLE genes HLA-DRB and NCF2, and these two genes had the highest gene-gene interaction degrees among the susceptibility hubs. An additional 14 genes previously associated with SLE emerged in our epistasis network model of the exome data, and three novel candidate genes, ST8SIA4, CMTM4, and C2CD4B, were implicated in the model. In summary, we present a comprehensive tool for epistasis network analysis and the first such analysis of exome data from a genetic study of SLE.

A Methodology for Improving Business Process Performance through Positive Deviance

The importance of process improvement and the role that best practice reference models play in the achievement of process improvement are both well recognized. Best practice reference models are generally created by experts who are external to the organisation. However, best practices can be implicitly derived from the work practices of actual workers within the organisation, especially when there is opportunity for variance within the work, i.e. there may be different approaches to achieve the same process goal. In this paper, the authors propose to support improvement of process performance intrinsically by utilizing the experiences and knowledge of business process users to inform and improve the current practices. The proposed methodology is inspired by the theory of positive deviance. By utilizing a multiple criteria decision making approach and Shannon’s entropy method of information theory in determining useful information from uncertain data within execution log of business process, the authors are able to define the “best” and most suitable previous practices as a recommendation that fits with the current competence/experience levels of individuals. The authors demonstrate that the proposed method is capable to generate meaningful recommendations from large data sets and effectively facilitating learning within organisation leading to process performance improvement.

Kolmogorov complexity of epithelial pattern formation: The role of regulatory network configuration

The tissues of multicellular organisms are made of differentiated cells arranged in organized patterns. This organization emerges during development from the coupling of dynamic intra- and intercellular regulatory networks. This work applies the methods of information theory to understand how regulatory network structure both within and between cells relates to the complexity of spatial patterns that emerge as a consequence of network operation. A computational study was performed in which undifferentiated cells were arranged in a two dimensional lattice, with gene expression in each cell regulated by identical intracellular randomly generated Boolean networks. Cell–cell contact signalling between embryonic cells is modeled as coupling among intracellular networks so that gene expression in one cell can influence the expression of genes in adjacent cells. In this system, the initially identical cells differentiate and form patterns of different cell types. The complexity of network structure, temporal dynamics and spatial organization is quantified through the Kolmogorov-based measures of normalized compression distance and set complexity. Results over sets of random networks that operate in the ordered, critical and chaotic domains demonstrate that: (1) ordered and critical networks tend to create the most information-rich patterns; (2) signalling configurations in which cell-to-cell communication is non-directional mostly produce simple patterns irrespective of the internal network domain; and (3) directional signalling configurations, similar to those that function in planar cell polarity, produce the most complex patterns, but only when the intracellular networks function in non-chaotic domains.

Metabolic cost of neuronal information in an empirical stimulus-response model

The limits on maximum information that can be transferred by single neurons may help us to understand how sensory and other information is being processed in the brain. According to the efficient-coding hypothesis (Barlow, Sensory Comunication, MIT press, Cambridge, 1961), neurons are adapted to the statistical properties of the signals to which they are exposed. In this paper we employ methods of information theory to calculate, both exactly (numerically) and approximately, the ultimate limits on reliable information transmission for an empirical neuronal model. We couple information transfer with the metabolic cost of neuronal activity and determine the optimal information-to-metabolic cost ratios. We find that the optimal input distribution is discrete with only six points of support, both with and without a metabolic constraint. However, we also find that many different input distributions achieve mutual information close to capacity, which implies that the precise structure of the capacity-achieving input is of lesser importance than the value of capacity.

Dynamics of the Information Exchange and the Causal- and-Effect Relationships in Plants under Controlled Conditions

Information theory methods were used to process extensive test data on the content of chemical elements in wheat and tomato plant organs obtained during their year-round and intensive cultivation on initially abiogenic mineral crushed stone (granite and zeolite). The directions of information flows related to redistribution chemical elements (Ca, K, P, S, Na, Si, Al, Fe, Zn, Mn, Mg, and Cl) between plant organs were revealed. Cause-and-effect relations in the dynamics of heterogeneity of chemical elements in plant organs under condition of primary soil formation were established. It is demonstrated that the total state of the chemical elements that determined by the integral indicator – information function – is differentiated for various plant organs. A cause-and-effect relation between the composition of organic matter of the root-inhabited environment and the elemental chemical composition of plant organs (root, stem, leaf and fruit, grain) is established by the method of information theory.

Hospital Status Admission Determination

To evaluate methods of logic, set theory, and information theory in developing a conceptual framework that would be useful in an educational process as well as in developing a consistent and rational method for hospital status determination. To implement these methods on a daily basis in interaction with nurse case managers, physicians, and in documentation of the process. A tertiary private, not-for-profit institution within the department of case management and utilization review. These methods were well accepted by those involved in the decision process and allowed a Case Management Assignment Protocol to function well in the hospital environment with a low level of disagreement and conflict. Medical information can be processed effectively with conceptual models of logic and information theory. The used commercial screening systems are described well by set theory and are intersecting sets of patient variables and characteristics. These methods can be used in educational processes in practice settings apart from those using the Case Management Assignment Protocol. It provides a basis for evaluation of patients' presentations that use important factors such as clinical uncertainty, patient specific data, and reference to preexisting admission criteria.

Building gene co-expression networks using transcriptomics data for systems biology investigations: Comparison of methods using microarray data.

Gene co-expression networks (GCN), built using high-throughput gene expression data are fundamental aspects of systems biology. The main aims of this study were to compare two popular approaches to building and analysing GCN. We use real ovine microarray transcriptomics datasets representing four different treatments with Metyrapone, an inhibitor of cortisol biosynthesis. We conducted several microarray quality control checks before applying GCN methods to filtered datasets. Then we compared the outputs of two methods using connectivity as a criterion, as it measures how well a node (gene) is connected within a network. The two GCN construction methods used were, Weighted Gene Co-expression Network Analysis (WGCNA) and Partial Correlation and Information Theory (PCIT) methods. Nodes were ranked based on their connectivity measures in each of the four different networks created by WGCNA and PCIT and node ranks in two methods were compared to identify those nodes which are highly differentially ranked (HDR). A total of 1,017 HDR nodes were identified across one or more of four networks. We investigated HDR nodes by gene enrichment analyses in relation to their biological relevance to phenotypes. We observed that, in contrast to WGCNA method, PCIT algorithm removes many of the edges of the most highly interconnected nodes. Removal of edges of most highly connected nodes or hub genes will have consequences for downstream analyses and biological interpretations. In general, for large GCN construction (with > 20000 genes) access to large computer clusters, particularly those with larger amounts of shared memory is recommended.

Evaluation and Comparison of Diagnostic Test Performance Based on Information Theory

A fundamental concept of information theory, relative entropy and mutual information, is directly applicable to evaluation of diagnostic test performance. The aim of this study is to demonstrate how basic concepts in information theory apply to the problem of quantifying major depressive disorder diagnostic test performance. In this study, the per- formances of the Dexamethasone Suppression Test-DST and the Thyroid-Stimulating Hormone Test-TSH, two of the di- agnosis tests of Major Depressive Disorder, are evaluated with the method of Information Theory. The amount of informa- tion gained by performing a diagnostic test can be quantified by calculating the relative entropy between the posttest and pretest probability distributions. And also demonstrates that diagnostic test performance can be quantified as the average amount of information the test result provides about the disease state. It is aimed that this study will hopefully give various points of view to the researchers who want to make research on this subject by explaining how the tests used for the diag- nosis of various diseases are evaluated with this way.

A computation neuroscience stresstest for coding strategies in cochlear implants

Event Abstract Back to Event A computation neuroscience stresstest for coding strategies in cochlear implants Michele Nicoletti1* and Werner Hemmert1* 1 Technische Universität München, IMETUM Institute of Medical Engineering, Germany Although modern cochlear implants (CI) are able to restore speech perception to a high degree, there is still a large potential for improvements e.g. in music perception and speech discrimination in noise. To evaluate and optimize novel coding strategies, we have developed a toolbox which codes sound signals into spike-trains of the auditory nerve. We have previously developed a model of the intact inner ear, which we have complemented with detailed models of a CI speech processor, channel crosstalk and spiral ganglion neuron models. With our toolbox we present qualitative comparisons of neurograms elicited by different coding strategies with the situation in the healthy inner ear. Moreover, we conducted quantitative evaluations using two methods: i) With the framework of automatic speech recognition we evaluated speech discrimination using a noisy database. ii) With the methods of information theory we quantified the transmitted information coded in neuronal spike trains, which allows us to evaluate especially well how well temporal information is coded. The major advantage of our approach is that we are able to evaluate both spectral and temporal aspects of novel coding strategies before we conduct extensive clinical studies. Acknowledgements This work was funded by a grant from MED-EL Innsbruck and within the Munich Bernstein Center for Computational Neuroscience by the German Federal Ministry of Education and Research (reference number 01GQ0441 and 01GQ1004B). References • Holmberg, M., Gelbart, D. and Hemmert, W. (2007), Speech Communication, 49(12), pp. 917-932. • Negm, M. and Bruce, I. (2008). EMBS 2008, pp 5539-5542. • Shepherd, R. K. & Javel, E. (1997). Hear Res. , 108, pp. 112-144 • Wilson, B. S.; Finley, C. C.; Lawson, D. T.; Wolford, R. D.; Eddington, D. K. & Rabinowitz, W. M. (1991). Nature 352, pp. 236-238 • Kral, A.; Hartmann, R.; Mortazavi, D. & Klinke, R. (1998). Hearing Research, 121, pp.11-28 Keywords: automatic speech recognition, Cochlear Implants, coding strategies, Information Theory, Speech Perception Conference: Bernstein Conference 2012, Munich, Germany, 12 Sep - 14 Sep, 2012. Presentation Type: Poster Topic: Neural encoding and decoding Citation: Nicoletti M and Hemmert W (2012). A computation neuroscience stresstest for coding strategies in cochlear implants . Front. Comput. Neurosci. Conference Abstract: Bernstein Conference 2012. doi: 10.3389/conf.fncom.2012.55.00007 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 04 May 2012; Published Online: 12 Sep 2012. * Correspondence: Mr. Michele Nicoletti, Technische Universität München, IMETUM Institute of Medical Engineering, Garching, 85748, Germany, michele.nicoletti@tum.de Prof. Werner Hemmert, Technische Universität München, IMETUM Institute of Medical Engineering, Garching, 85748, Germany, werner.hemmert@tum.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Michele Nicoletti Werner Hemmert Google Michele Nicoletti Werner Hemmert Google Scholar Michele Nicoletti Werner Hemmert PubMed Michele Nicoletti Werner Hemmert Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

An Information Theory Analysis of Gene-Environmental Interactions in Count/Rate Data

Objective: To develop and critically evaluate an information theory method for identifying gene-gene and gene-environment interactions in count and rate data. Methods: The entropy-based metric k-way interaction information (KWII) was critically assessed for utility in detecting interactions with count data and over-dispersed count data in three simulation studies of increasing complexity and in datasets from animal models of depression and colitis. The results were compared to Poisson regression. The power and effect size dependence of the KWII for detecting interactions was also assessed. Results: The KWII was capable of effectively identifying the genetic and environmental predictors and their interactions in all three simulated datasets. The results indicate that the KWII approach may produce more parsimonious results than regression. In a rat model of depression, we successfully identified a prominent gender effect as well as other published associations. Analysis of severity scores from an animal model of colitis identified markers from chromosome 3, as well as unique first- and second-order associations for the individual sections of the colon and cecum. Conclusions: The results demonstrate the utility and versatility of our entropy-based method for gene-environment interaction analysis of count and rate data with Poisson and over-dispersed distributions.

Evaluating the sustainability of a regional system using Fisher information in the San Luis Basin, Colorado

This paper describes the theory, data, and methodology necessary for using Fisher information to assess the sustainability of the San Luis Basin (SLB) regional system over time. Fisher information was originally developed as a measure of the information content in data and is an important method in information theory. Our adaptation of Fisher information provides a means of monitoring the variables of a system to characterize dynamic order, and, therefore, its regimes and regime shifts. This work is part of the SLB Sustainability Metrics Project, which aimed to evaluate movement over time towards or away from regional sustainability. One of the key goals of this project was to use readily available data to assess the sustainability of the system including its environmental, social and economic aspects. For this study, Fisher information was calculated for fifty-three variables which characterize the consumption of food and energy, agricultural production, environmental characteristics, demographic properties and changes in land use for the SLB system from 1980 to 2005. Our analysis revealed that while the system displayed small changes in dynamic order over time with a slight decreasing trend near the end of the period, there is no indication of a regime shift. Therefore, the SLB system is stable with very slight movement away from sustainability in more recent years.

Information flows between organic matter of the root environment and elemental chemical composition of plants under primary pedogenic conditions

The results of a comprehensive multiple growth-period experiment on intensive year-round cultivation of wheat and tomato plants on an originally abiogenic mineral substrate under controlled conditions are given. An interrelation is established between the ash content of plant roots and reproductive organs and the dynamics of the forming multicomponent organic matter of the root environment. It is shown that the effect of crop rotations leads to an intermittent change in the dynamics of the ash composition of plant tissues. A cause-and-effect relation between the composition of organic matter of the root environment and the elemental chemical composition of plant tissues is established by information theory methods.