Information Theory Approach Research Articles

Evaluating methods for modeling epistasis networks with application to head and neck cancer.

Epistasis helps to explain how multiple single-nucleotide polymorphisms (SNPs) interact to cause disease. A variety of tools have been developed to detect epistasis. In this article, we explore the strengths and weaknesses of an information theory approach for detecting epistasis and compare it to the logistic regression approach through simulations. We consider several scenarios to simulate the involvement of SNPs in an epistasis network with respect to linkage disequilibrium patterns among them and the presence or absence of main and interaction effects. We conclude that the information theory approach more efficiently detects interaction effects when main effects are absent, whereas, in general, the logistic regression approach is appropriate in all scenarios but results in higher false positives. We compute epistasis networks for SNPs in the FSD1L gene using a two-phase head and neck cancer genome-wide association study involving 2,185 cases and 4,507 controls to demonstrate the practical application of the methods.

Optimal Placement of Water Quality Monitoring Stations in Sewer Systems: An Information Theory Approach

A core problem associated with the water quality monitoring in the sewer system is the optimal placement of a limited number of monitoring sites. A methodology is provided for optimally design water quality monitoring stations in sewer networks. The methodology is based on information theory, formulated as a multi-objective optimization problem and solved using NSGA-II. Computer code is written to estimate two entropy quantities, namely Joint Entropy, a measure of information content, and Total Correlation, a measure of redundancy, which are maximized and minimized, respectively. The test on a real sewer network suggests the effectiveness of the proposed methodology.

An Integrated Risk Measure and Information Theory Approach for Modeling Financial Data and Solving Decision Making Problems

In this paper we build some integrated techniques for modeling financial data and solving decision making problems, based on risk theory and information theory. Several risk measures and entropy measures are investigated and compared with respect to their analytical properties and effectiveness in solving real problems. Some criteria for portfolio selection are derived combining the classical risk measure approach with the information theory approach. We analyze the performance of the methods proposed in case of some financial applications.Computational results are provided.

Understanding complexity: Are we making progress?

In recent years a new conceptual tool called Complexity Theory has come to the attention of scientists and philosophers. This approach is concerned with the emergent properties of interacting systems. It has found wide applicability from cosmology to Social Structure Analysis. However, practitioners are still struggling to find the best way to define complexity and then to measure it. A new book Complexity and the arrow of time by Lineweaver et al. (2013) contains contributions from scholars who provide critical reviews of Complexity Theory and its wider applications. This is a huge task and this essay examines how well the authors have succeeded in satisfying the claim made by the book’s three editors to have clarified the leading questions. I also explore the application of Complexity Theory to Biology as a means to explain the popular view that biological complexity (as viewed in many guises and at many levels) has increased over time. In this regard, I conclude by recommending an Information Theory approach which urges that physical complexity arises from accumulation of genetic coding sequences.

AN ALGORITHMIC INFORMATION THEORY CHALLENGE TO INTELLIGENT DESIGN

AbstractWilliam Dembski claims to have established a decision process to determine when highly unlikely events observed in the natural world are due to Intelligent Design. This article argues that, as no implementable randomness test is superior to a universal Martin‐Löf test, this test should be used to replace Dembski's decision process. Furthermore, Dembski's decision process is flawed, as natural explanations are eliminated before chance. Dembski also introduces a fourth law of thermodynamics, his “law of conservation of information,” to argue that information cannot increase by natural processes. However, this article, using algorithmic information theory, shows that this law is no more than the second law of thermodynamics. The article concludes that any discussions on the possibilities of design interventions in nature should be articulated in terms of the algorithmic information theory approach to randomness and its robust decision process.

2D-3D Face Recognition Method Basedon a Modified CCA-PCA Algorithm

This paper presents a proposed methodology for face recognition based on an information theory approach to coding and decoding face images. In this paper, we propose a 2D-3D face-matching method based on a principal component analysis (PCA) algorithm using canonical correlation analysis (CCA) to learn the mapping between a 2D face image and 3D face data. This method makes it possible to match a 2D face image with enrolled 3D face data. Our proposed fusion algorithm is based on the PCA method, which is applied to extract base features. PCA feature-level fusion requires the extraction of different features from the source data before features are merged together. Experimental results on the TEXAS face image database have shown that the classification and recognition results based on the modified CCA-PCA method are superior to those based on the CCA method. Testing the 2D-3D face match results gave a recognition rate for the CCA method of a quite poor 55% while the modified CCA method based on PCA-level fusion achieved a very good recognition score of 85%.

Describing the Complexity of Systems: Multivariable “Set Complexity” and the Information Basis of Systems Biology

Context dependence is central to the description of complexity. Keying on the pairwise definition of "set complexity," we use an information theory approach to formulate general measures of systems complexity. We examine the properties of multivariable dependency starting with the concept of interaction information. We then present a new measure for unbiased detection of multivariable dependency, "differential interaction information." This quantity for two variables reduces to the pairwise "set complexity" previously proposed as a context-dependent measure of information in biological systems. We generalize it here to an arbitrary number of variables. Critical limiting properties of the "differential interaction information" are key to the generalization. This measure extends previous ideas about biological information and provides a more sophisticated basis for the study of complexity. The properties of "differential interaction information" also suggest new approaches to data analysis. Given a data set of system measurements, differential interaction information can provide a measure of collective dependence, which can be represented in hypergraphs describing complex system interaction patterns. We investigate this kind of analysis using simulated data sets. The conjoining of a generalized set complexity measure, multivariable dependency analysis, and hypergraphs is our central result. While our focus is on complex biological systems, our results are applicable to any complex system.

Size at the onset of maturity (SOM) revealed in length–weight relationships of brackish amphipods and isopods: An information theory approach

In amphipods and other small-sized crustaceans, allometric relationships are conventionally analysed by fitting the standard model Y = a·Xb (X and Y are, e.g., body length and weight, respectively) whose scaling exponent b is assumed to be constant. However, breakpoints in allometric relationships have long been documented in large-sized crustaceans, ultimately determined by ontogenetic, abrupt variations in the value of b. Here, the existence of breakpoints in length–weight relationships was investigated in four amphipod (i.e., Gammarus aequicauda, Gammarus insensibilis, Microdeutopus gryllotalpa, and Dexamine spinosa) and three isopod species (i.e., Lekanesphaera hookeri, Sphaeroma serratum, and Cymodoce truncata) from three Mediterranean lagoons. The power of two candidate linear models fitted to log10-transformed data – a simple model assuming a constant exponent b and a segmented model assuming b to vary after a breakpoint - was compared using a parsimonious selection strategy based on the Akaike information criterion. The segmented model with a breakpoint provided the most accurate fitting of length–weight data in the majority of the species analysed; non-conclusive results were obtained only for D. spinosa and C. truncata, of which a limited number of specimens was examined. Model parameters were consistent for amphipod and isopod species collected across the three different habitats; the generality of the results was further supported by a literature search confirming that the identified breakpoints corresponded with ontogenetic discontinuities related with sexual maturation in all the species investigated. In this study, segmented regression models were revealed to provide a statistically accurate and biologically meaningful description of length–weight relationships of common amphipod and isopod species. The methodological limitations of the approach are considered, while the practical implications for secondary production estimates are discussed.

Age Dependent Face Recognition using Eigenface

Face recognition is the most successful form of human surveillance. Face recognition technology, is being used to improve human efficiency when recognition faces, is one of the fastest growing fields in the biometric industry. In the first stage, the age is classified into eleven categories which distinguish the person oldness in terms of age. In the second stage of the process is face recognition based on the predicted age. Age prediction has considerable potential applications in human computer interaction and multimedia communication. In this paper proposes an Eigen based age estimation algorithm for estimate an image from the database. Eigenface has proven to be a useful and robust cue for age prediction, age simulation, face recognition, localization and tracking. The scheme is based on an information theory approach that decomposes face images into a small set of characteristic feature images called eigenfaces, which may be thought of as the principal components of the initial training set of face images. The eigenface approach used in this scheme has advantages over other face recognition methods in its speed, simplicity, learning capability and robustness to small changes in the face image.

Changing the Direction of Sleep Medicine: Business can Boom, but it is Not as Usual

Changing the Direction of Sleep Medicine: Business can Boom, but it is Not as Usual

Multi-Level Determinants of Parasitic Fly Infection in Forest Passerines

The study of myiasis is important because they may cause problems to the livestock industry, public health, or wildlife conservation. The ecology of parasitic dipterans that cause myiasis is singular, as they actively seek their hosts over relatively long distances. However, studies that address the determinants of myiasis dynamics are very scarce. The genus Philornis include species that may be excellent models to study myiasis ecology, as they exclusively parasitize bird nestlings, which stay in their nests until they are fully fledged, and larvae remain at the point of entry until the parasitic stage is over, thus allowing the collection of sequential individual-level infection data from virtually all the hosts present at a particular area. Here we offer a stratified multi-level analysis of longitudinal data of Philornis torquans parasitism in replicated forest bird communities of central Argentina. Using Generalized Linear Models and Generalized Linear Mixed Models and an information theory approach for model selection, we conducted four groups of analyses, each with a different study unit, the individual, the brood, the community at a given week, and the community at a given year. The response variable was larval abundance per nestling or mean abundance per nestling. At each level, models included the variables of interest of that particular level, and also potential confounders and effect modifiers of higher levels. We found associations of large magnitude at all levels, but only few variables truly governed the dynamics of this parasite. At the individual level, the infection was determined by the species and the age of the host. The main driver of parasite abundance at the microhabitat level was the average height of the forest, and at the community level, the density of hosts and prior rainfall. This multi-level approach contributed to a better understanding of the ecology of myiasis.

An Information-Theoretic Machine Learning Approach to Expression QTL Analysis

Expression Quantitative Trait Locus (eQTL) analysis is a powerful tool to study the biological mechanisms linking the genotype with gene expression. Such analyses can identify genomic locations where genotypic variants influence the expression of genes, both in close proximity to the variant (cis-eQTL), and on other chromosomes (trans-eQTL). Many traditional eQTL methods are based on a linear regression model. In this study, we propose a novel method by which to identify eQTL associations with information theory and machine learning approaches. Mutual Information (MI) is used to describe the association between genetic marker and gene expression. MI can detect both linear and non-linear associations. What’s more, it can capture the heterogeneity of the population. Advanced feature selection methods, Maximum Relevance Minimum Redundancy (mRMR) and Incremental Feature Selection (IFS), were applied to optimize the selection of the affected genes by the genetic marker. When we applied our method to a study of apoE-deficient mice, it was found that the cis-acting eQTLs are stronger than trans-acting eQTLs but there are more trans-acting eQTLs than cis-acting eQTLs. We compared our results (mRMR.eQTL) with R/qtl, and MatrixEQTL (modelLINEAR and modelANOVA). In female mice, 67.9% of mRMR.eQTL results can be confirmed by at least two other methods while only 14.4% of R/qtl result can be confirmed by at least two other methods. In male mice, 74.1% of mRMR.eQTL results can be confirmed by at least two other methods while only 18.2% of R/qtl result can be confirmed by at least two other methods. Our methods provide a new way to identify the association between genetic markers and gene expression. Our software is available from supporting information.

Generalized Chemistry–Internal Energy Coupling Model Using Prior Recombination Distribution

In this work, the interaction between the internal energy states and the dissociation/recombination process for a diatomic molecule is studied in detail. In particular, the state-specific recombination and dissociation rate constants are derived as a function of internal energy distribution and other molecular parameters. As a special case, applying this result with specific vibrational energy distributions, we obtain the classic Marrone and Treanor models. An information theory approach to obtain the initial recombination distribution for any diatomic molecule is also presented. The general idea is to incorporate the vibrational and the rotational effects in the dissociation and recombination process without a full vibrational-rotational state-specific model. The effect of this rovibrational coupling is to produce an initial recombination distribution that is biased to the high internal energy states. These initial recombination distributions are then thermally averaged to obtain temperature-dependent expressions for different diatomic molecules to characterize the internal energy distribution of recombining molecules.

Affective State Level Recognition in Naturalistic Facial and Vocal Expressions

Naturalistic affective expressions change at a rate much slower than the typical rate at which video or audio is recorded. This increases the probability that consecutive recorded instants of expressions represent the same affective content. In this paper, we exploit such a relationship to improve the recognition performance of continuous naturalistic affective expressions. Using datasets of naturalistic affective expressions (AVEC 2011 audio and video dataset, PAINFUL video dataset) continuously labeled over time and over different dimensions, we analyze the transitions between levels of those dimensions (e.g., transitions in pain intensity level). We use an information theory approach to show that the transitions occur very slowly and hence suggest modeling them as first-order Markov models. The dimension levels are considered to be the hidden states in the Hidden Markov Model (HMM) framework. Their discrete transition and emission matrices are trained by using the labels provided with the training set. The recognition problem is converted into a best path-finding problem to obtain the best hidden states sequence in HMMs. This is a key difference from previous use of HMMs as classifiers. Modeling of the transitions between dimension levels is integrated in a multistage approach, where the first level performs a mapping between the affective expression features and a soft decision value (e.g., an affective dimension level), and further classification stages are modeled as HMMs that refine that mapping by taking into account the temporal relationships between the output decision labels. The experimental results for each of the unimodal datasets show overall performance to be significantly above that of a standard classification system that does not take into account temporal relationships. In particular, the results on the AVEC 2011 audio dataset outperform all other systems presented at the international competition.

Comparing Surface and Mid-Tropospheric CO2 Concentrations from Central U.S. Grasslands

Comparisons of eddy covariance (EC) tower measurements of CO2 concentration with mid-tropospheric observations from the Atmospheric Infrared Sounder (AIRS) allow for evaluation of the rising global signal of this greenhouse gas in relation to surface carbon dynamics. Using an information theory approach combining relative entropy and wavelet multi-resolution analysis, this study has explored correlations and divergences between mid-tropospheric and surface CO2 concentrations in grasslands of northeastern Kansas. Results show that surface CO2 measurements at the Kansas Field Station (KFS) and the Konza Prairie Biological Stations 1B (KZU) and 4B (K4B) with different land-cover types correlate well with mid-tropospheric CO2 in this region at the 512-day timescale between 2007 and 2010. Relative entropy further reveals that AIRS observations are indicative of surface CO2 concentrations for all land-cover types on monthly (32-day) and longer timescales. AIRS observations are also similar to CO2 concentrations at shorter timescales at sites KFS and K4B experiencing woody encroachment, though these results require further investigation. Differences in species composition and microclimate add to the variability of surface concentrations compared with mid-tropospheric observations.

Assessment of measurement efficiency in laser- and phase-Doppler techniques: an information theory approach

Laser- and phase-Doppler diagnostic techniques provide information on particle characteristics in the form of discrete probability distribution functions. Most methods assess the amount of information required for an accurate measurement through the first- and second-order moments of these distributions. However, considering that a measurement is the distribution and not its moments, a different approach is developed based on information theory (IT) concepts in order to assess if the information provided to the experimentalist is enough to ensure an accurate statistical analysis. The methodology and stopping criteria are presented and used in previously reported measurements obtained with laser- and phase-Doppler techniques. Results show that using an IT approach to assess the reliability of data provided by a measurement means evaluating the degree of stabilization of a discrete probability distribution, where more information acquired does not necessarily imply a more accurate measurement. The statistical analysis performed using the number of samples indicated by the IT method, compared to the total sample size previously measured, shows similar results. Moreover, measurement time is substantially reduced if the IT method is used, thus improving measurement efficiency.

Factors influencing the prevalence of stereotypical behaviour in captive cheetahs (Acinonyx jubatus)

The purpose of the current study was to investigate the factors which influence the prevalence of stereotypical behaviour in captive cheetahs. An information theory approach highlighted that the most optimal model was the controllable husbandry factors model with the size of enclosures, group membership, feeding regime and the ability to view other cheetahs in adjacent enclosures being the significant factors within the model which determined the occurrence of stereotypical behaviour. Increasing size of enclosure decreased this behaviour, whilst being solitary, being fed on a predictable feeding regime and having the ability to view other cheetahs in adjacent enclosures increased levels of stereotypical behaviour. These findings allow zoological institutions to focus on these factors in order to reduce the occurrence of stereotypical behaviour in captive cheetahs. Although cheetahs were used as the subject animal in this study, this type of research can be utilised for any captive species in order to understand unwanted behaviours or behaviours that zoological institutions wish to promote, so a behaviourally healthy captive population can be exhibited, reproduced and subsequently conserved.

Entropy assessment of supply chain disruption

PurposeManufacturing organizations and networks are heavily dependent on the flow of information within and across organization boundaries. A disruption in information flow might interrupt the operations of the organization and make management even more difficult. The purpose of this paper is to incorporate information theory approach to investigate the perturbation introduced into a manufacturing organization as a result of disruption in the flow of critical information needed in manufacturing operations.Design/methodology/approachThis study proposes the use of entropy theory to assess the level of risk introduced by different sources of perturbation into the material flow stream and the use of discrete event simulation to investigate the impact of the resulting disruption on collaborating members.FindingsThe result of the analysis carried out on the effect of system failure on supply chain performance revealed that the retailer experiences the most uncertainty in the supply chain while the holding cost constitutes the most unpredictable cost measure when a system failure breach occurs. For the manufacturer and wholesaler, the holding cost is responsible for most of the uncertainty in predicting the impact of the threat on inventory management cost, while the backlog cost holds the highest complexity level for the retailer.Practical implicationsOnce this methodology is well developed for use in industrial networks, it can serve as a risk assessment, risk monitoring and risk prediction tool. The paper also calls for a proactive approach to disruption risk management.Originality/valueThis paper proposes a novel approach to assess the impact of information disruption, using entropy theory coupled with simulation methodology.

Distinguishing chaotic and stochastic dynamics from time series by using a multiscale symbolic approach

In this paper we introduce a multiscale symbolic information-theory approach for discriminating nonlinear deterministic and stochastic dynamics from time series associated with complex systems. More precisely, we show that the multiscale complexity-entropy causality plane is a useful representation space to identify the range of scales at which deterministic or noisy behaviors dominate the system's dynamics. Numerical simulations obtained from the well-known and widely used Mackey-Glass oscillator operating in a high-dimensional chaotic regime were used as test beds. The effect of an increased amount of observational white noise was carefully examined. The results obtained were contrasted with those derived from correlated stochastic processes and continuous stochastic limit cycles. Finally, several experimental and natural time series were analyzed in order to show the applicability of this scale-dependent symbolic approach in practical situations.

Measuring international trade-related technology spillover: a composite approach of network analysis and information theory

The study of international technology spillover focuses on bilateral relationship between host country and home country, and the study of transnational technology network focuses on the position and power of individual country in a network. Introducing the network model and information theory into the research framework, we propose a new measure of international technology spillover in this present article, in which the bilateral relationship was transformed to multilateral relationship between countries, and network features at the whole network level were investigated instead of at the individual level. By using data from CHELEM-International Trade Database, we measure three transnational spillover networks in the high-tech field from 1979 to 2009, and analyze the results of various technology networks at different time points.