Information Gain Measure Research Articles

Characterizing gene-gene interactions in a statistical epistasis network of twelve candidate genes for obesity.

BackgroundRecent findings have reemphasized the importance of epistasis, or gene-gene interactions, as a contributing factor to the unexplained heritability of obesity. Network-based methods such as statistical epistasis networks (SEN), present an intuitive framework to address the computational challenge of studying pairwise interactions between thousands of genetic variants. In this study, we aimed to analyze pairwise interactions that are associated with Body Mass Index (BMI) between SNPs from twelve genes robustly associated with obesity (BDNF, ETV5, FAIM2, FTO, GNPDA2, KCTD15, MC4R, MTCH2, NEGR1, SEC16B, SH2B1, and TMEM18).MethodsWe used information gain measures to identify all SNP-SNP interactions among and between these genes that were related to obesity (BMI > 30 kg/m2) within the Framingham Heart Study Cohort; interactions exceeding a certain threshold were used to build an SEN. We also quantified whether interactions tend to occur more between SNPs from the same gene (dyadicity) or between SNPs from different genes (heterophilicity).ResultsWe identified a highly connected SEN of 709 SNPs and 1241 SNP-SNP interactions. Combining the SEN framework with dyadicity and heterophilicity analyses, we found 1 dyadic gene (TMEM18, P-value = 0.047) and 3 heterophilic genes (KCTD15, P-value = 0.045; SH2B1, P-value = 0.003; and TMEM18, P-value = 0.001). We also identified a lncRNA SNP (rs4358154) as a key node within the SEN using multiple network measures.ConclusionThis study presents an analytical framework to characterize the global landscape of genetic interactions from genome-wide arrays and also to discover nodes of potential biological significance within the identified network.Electronic supplementary materialThe online version of this article (doi:10.1186/s13040-015-0077-x) contains supplementary material, which is available to authorized users.

Effects of omics data combinations on in silico tumor-normal tissue classification

A disease can be characterized by various attributes such as genomic, epigenetic, and transcriptomic features beyond physiological symptoms. The accumulation of vast datasets allows us to investigate the relative effectiveness of each omics data and their combinations for in silico analysis of diseases. Here, we employed a classification method with the well-established measure of information gain for the computational analysis of the effect of the aggregation of omics data, especially for the task of in silico classification of tumor-normal samples for bladder urothelial carcinoma and kidney renal papillary cell carcinoma. We observed that the combination of multi-omics data such as copy number variation, DNA methylation, RNA-Seq, and somatic mutations have beneficial effects. The quantitative analysis using information gain and various measures for classification-performance showed that the combination of multiple omics data improved the performance in general. The qualitative analysis referring previous researches also confirmed the relevance of genes with higher information gain to target diseases. Our results report that the combination of multiple omics data is beneficial and the information gain which focuses on the distribution of attributes across target domains could be useful as an indicator of the effect of each omics data on tumor-normal sample classification.

On a measure of information gain for regression models in survival analysis

Papers dealing with measures of predictive power in survival analysis have seen their independence of censoring, or their estimates being unbiased under censoring, as the most important property. We argue that this property has been wrongly understood. Discussing the so-called measure of information gain, we point out that we cannot have unbiased estimates if all values, greater than a given time τ, are censored. This is due to the fact that censoring before τ has a different effect than censoring after τ. Such τ is often introduced by design of a study. Independence can only be achieved under the assumption of the model being valid after τ, which is impossible to verify. But if one is willing to make such an assumption, we suggest using multiple imputation to obtain a consistent estimate. We further show that censoring has different effects on the estimation of the measure for the Cox model than for parametric models, and we discuss them separately. We also give some warnings about the usage of the measure, especially when it comes to comparing essentially different models.

Sample Skewness as a Statistical Measurement of Neuronal Tuning Sharpness

We propose using the statistical measurement of the sample skewness of the distribution of mean firing rates of a tuning curve to quantify sharpness of tuning. For some features, like binocular disparity, tuning curves are best described by relatively complex and sometimes diverse functions, making it difficult to quantify sharpness with a single function and parameter. Skewness provides a robust nonparametric measure of tuning curve sharpness that is invariant with respect to the mean and variance of the tuning curve and is straightforward to apply to a wide range of tuning, including simple orientation tuning curves and complex object tuning curves that often cannot even be described parametrically. Because skewness does not depend on a specific model or function of tuning, it is especially appealing to cases of sharpening where recurrent interactions among neurons produce sharper tuning curves that deviate in a complex manner from the feedforward function of tuning. Since tuning curves for all neurons are not typically well described by a single parametric function, this model independence additionally allows skewness to be applied to all recorded neurons, maximizing the statistical power of a set of data. We also compare skewness with other nonparametric measures of tuning curve sharpness and selectivity. Compared to these other nonparametric measures tested, skewness is best used for capturing the sharpness of multimodal tuning curves defined by narrow peaks (maximum) and broad valleys (minima). Finally, we provide a more formal definition of sharpness using a shape-based information gain measure and derive and show that skewness is correlated with this definition.

A Modified Entropy-Based Approach for Identifying Gene-Gene Interactions in Case-Control Study

Gene-gene interactions may play an important role in the genetics of a complex disease. Detection and characterization of gene-gene interactions is a challenging issue that has stimulated the development of various statistical methods to address it. In this study, we introduce a method to measure gene interactions using entropy-based statistics from a contingency table of trait and genotype combinations. We also developed an exploration procedure by using graphs. We propose a standardized relative information gain (RIG) measure to evaluate the interactions between single nucleotide polymorphism (SNP) combinations. To identify the k th order interactions, contingency tables of trait and genotype combinations of k SNPs are constructed, with which RIGs are calculated. The RIGs are standardized using the mean and standard deviation from the permuted datasets. SNP combinations yielding high standardized RIG are chosen for gene-gene interactions. Detection of high-order interactions and comparison of interaction strengths between different orders are made possible by using standardized RIG. We have applied the proposed standardized entropy-based method to two types of data sets from a simulation study and a real genetic association study. We have compared our method and the multifactor dimensionality reduction (MDR) method through power analysis of eight different genetic models with varying penetrance rates, number of SNPs, and sample sizes. Our method shows successful identification of genetic associations and gene-gene interactions both in simulation and real genetic data. Simulation results suggest that the proposed entropy-based method is better able to detect high-order interactions and is superior to the MDR method in most cases. The proposed method is well suited for detecting interactions without main effects as well as for models including main effects.

Information gain on variable neuronal firing rate

The question of how much information can be theoretically gained from variable neuronal firing rate with respect to constant mean firing rate is investigated. For this purpose, we employ the Kullback-Leibler divergence as a measure of information gain. We first give a statistical interpretation of this information in terms of detectability of rate variation: the lower bound of detectable rate variation, below which the temporal variation of firing rate is undetectable with a Bayesian decoder, is entirely determined by this information [1]. We derive a formula for the lower bound, which tells how much information is necessary for the rate variation to be detected from spike trains. For instance, if a spike, on average, is expected to be observed in the characteristic timescale of the rate variation, it is necessary for the spike train to carry more than 0.36 bits per spike information so that the underlying rate variation is detectable. We show that the information depends not only of the variation of firing rates (i.e., signals), but also significantly on the dispersion properties of neuronal firing described by the shape of interspike interval (ISI) distribution (i.e., noise properties). It is shown that under certain condition, the gamma distribution attains the theoretical lower bound of the information among all ISI distributions when the coefficient of variation of ISIs is given [2]. We also give a useful formula for the (approximate) information, which provides a theoretical prediction for the range of the information gain. With the basis of the theoretical investigations, we propose a practical procedure for estimating the information from spike trains, and apply this method to biological spike data recorded from a cortical area. The estimated information ranges up to 0.8 bits/spike, which roughly matches the theoretical prediction. Acknowledgements The author was supported by JSPS KAKENHI Grant Number 24700287.

Regional Earthquake Likelihood Models I: First-Order Results

Abstract The Regional Earthquake Likelihood Models (RELM) working group designed a 5-year experiment to forecast the number, spatial distribution, and magnitude distribution of subsequent target earthquakes, defined to be those with magnitude ≥4.95 (M 4.95+) in a well-defined California testing region. Included in the experiment specification were the description of the data source, the methods for data processing, and the proposed evaluation metrics. The RELM experiment began on 1 January 2006 and involved 17 time-invariant forecasts constructed by seismicity modelers; by the end of the experiment on 1 January 2011, 31 target earthquakes had occurred. We analyze the experiment outcome by applying the proposed consistency tests based on likelihood measures and additional comparison tests based on a measure of information gain. We find that the smoothed seismicity forecast by Helmstetter et al., 2007 based on M 2+ earthquakes since 1981, is the best forecast, regardless of whether aftershocks are included in the analysis. The RELM experiment has helped to clarify ideas about testing that can be applied to more wide-ranging earthquake forecasting experiments conducted by the Collaboratory for the Study of Earthquake Predictability (CSEP). Online Material: Figures and tables showing the RELM testing region and collection region definitions, numerical results associated with the RELM experiment, and the uncorrected forecast by Ebel et al. (2007).

Analysis and evolution of air quality monitoring networks using combined statistical information indexes

In this work, we present combined statistical indexes for evaluating air quality monitoring networks based on concepts derived from the information theory and Kullback–Liebler divergence. More precisely, we introduce: (1) the standard measure of complementary mutual information or ‘specificity’ index; (2) a new measure of information gain or ‘representativity’ index; (3) the information gaps associated with the evolution of a network and (4) the normalised information distance used in clustering analysis. All these information concepts are illustrated by applying them to 14 yr of data collected by the air quality monitoring network in Santiago de Chile (33.5 S, 70.5 W, 500 m a.s.l.). We find that downtown stations, located in a relatively flat area of the Santiago basin, generally show high ‘representativity’ and low ‘specificity’, whereas the contrary is found for a station located in a canyon to the east of the basin, consistently with known emission and circulation patterns of Santiago. We also show interesting applications of information gain to the analysis of the evolution of a network, where the choice of background information is also discussed, and of mutual information distance to the classifications of stations. Our analyses show that information as those presented here should of course be used in a complementary way when addressing the analysis of an air quality network for planning and evaluation purposes.

Quantitative microscopy approach for shape‐based erythrocytes characterization in anaemia

Anaemia is one of the most common diseases in the world population. Primarily anaemia is identified based on haemoglobin level; and then microscopically examination of peripheral blood smear is required for characterizing and confirmation of anaemic stages. In conventional approach, experts visually characterize abnormality present in the erythrocytes under light microscope, and this evaluation process is subjective in nature and error prone. In this study, we have proposed a methodology using machine learning techniques for characterizing erythrocytes in anaemia associated with anaemia using microscopic images of peripheral blood smears. First, peripheral blood smear images are preprocessed based on grey world assumption technique and geometric mean filter for reducing unevenness of background illumination and noise reduction. Then erythrocyte cells are segmented using marker-controlled watershed segmentation technique. The erythrocytes in anaemia, such as, tear drop, echinocyte, acanthocyte, elliptocyte, sickle cells and normal erythrocytes cells have been characterized and classified based on their morphological changes. Optimal subset of features, ranked by information gain measure provides highest classification performance using logistic regression classifier in comparison with other standard classifiers.

An information theoretical algorithm for analyzing supersaturated designs for a binary response

A supersaturated design is a factorial design in which the number of effects to be estimated is greater than the number of runs. It is used in many experiments, for screening purpose, i.e., for studying a large number of factors and identifying the active ones. In this paper, we propose a method for screening out the important factors from a large set of potentially active variables through the symmetrical uncertainty measure combined with the information gain measure. We develop an information theoretical analysis method by using Shannon and some other entropy measures such as Renyi entropy, Havrda–Charvat entropy, and Tsallis entropy, on data and assuming generalized linear models for a Bernoulli response. This method is quite advantageous as it enables us to use supersaturated designs for analyzing data on generalized linear models. Empirical study demonstrates that this method performs well giving low Type I and Type II error rates for any entropy measure we use. Moreover, the proposed method is more efficient when compared to the existing ROC methodology of identifying the significant factors for a dichotomous response in terms of error rates.

A Novel Mathematical Approach to Diagnose Premenstrual Syndrome

Diagnosis of Premenstrual syndrome (PMS) is a research challenge due to its subjective presentation. An undiagnosed PMS case is often termed as 'borderline' ('B') that further add to the diagnostic fuzziness. This study proposes a methodology to diagnose PMS cases using a combined knowledge engineering and soft computing techniques. According to the guidelines of American College of Gynecology (ACOG), ten symptoms have been selected and technically processed for 50 cases each having class labels-'B' or 'NB' (not borderline) using domain expertise. Any Attribute that fails normality test has been excluded from the study. Decision tree (DT) has then been induced in obtaining the initial class boundaries and mining the important Attributes to classify PMS cases. Prior doing so, the best split criterion has been set using the maximum information gain measure. Initial information about classification boundaries are finally used to measure fuzzy membership values and the corresponding firing strengths have been measured for final classification of PMS 'B' cases.

Inversion of Bayes formula and measures of Bayesian information gain and pairwise dependence

By inverting the Bayes formula in a point-wise manner, we develop measures quantifying the information gained by the Bayesian process, in reference to the Fisher information. Simple examples are used for focused illustrations of the ideas. Numerical computation for the measures is discussed with formulae. By extending the information gain concept to the broader context of distribution theory, we arrive at a pairwise dependence measure, which can handle the case of functional dependence and becomes Pearson's phi(2) when the joint probability density function is defined.

Learning random forests for ranking

The random forests (RF) algorithm, which combines the predictions from an ensemble of random trees, has achieved significant improvements in terms of classification accuracy. In many real-world applications, however, ranking is often required in order to make optimal decisions. Thus, we focus our attention on the ranking performance of RF in this paper. Our experimental results based on the entire 36 UC Irvine Machine Learning Repository (UCI) data sets published on the main website of Weka platform show that RF doesn't perform well in ranking, and is even about the same as a single C4.4 tree. This fact raises the question of whether several improvements to RF can scale up its ranking performance. To answer this question, we single out an improved random forests (IRF) algorithm. Instead of the information gain measure and the maximum-likelihood estimate, the average gain measure and the similarity-weighted estimate are used in IRF. Our experiments show that IRF significantly outperforms all the other algorithms used to compare in terms of ranking while maintains the high classification accuracy characterizing RF.

Analyzing supersaturated designs with entropic measures

A supersaturated design is a design for which there are fewer runs than effects to be estimated. In this paper, we propose a method for screening out the important factors from a large set of potentially active variables, based on an information theoretical approach. Three entropy measures: Rényi entropy, Tsallis entropy and Havrda–Charvát entropy, have been associated with the measure of information gain, in order to identify the significant factors using data and assuming generalized linear models. The investigation of the proposed method performance and the comparison of each entropic measure application have been accomplished through simulation experiments. A noteworthy advantage of this paper is the use of generalized linear models for analyzing data from supersaturated designs, a fact that, to the best of our knowledge, has not yet been studied.

Effective Linkage Learning Using Low-Order Statistics and Clustering

The adoption of probabilistic models for selected individuals is a powerful approach for evolutionary computation. Probabilistic models based on high-order statistics have been used by estimation of distribution algorithms (EDAs), resulting better effectiveness when searching for global optima for hard optimization problems. This paper proposes a new framework for evolutionary algorithms, which combines a simple EDA based on order 1 statistics and a clustering technique in order to avoid the high computational cost required by higher order EDAs. The algorithm uses clustering to group genotypically similar solutions, relying that different clusters focus on different substructures and the combination of information from different clusters effectively combines substructures. The combination mechanism uses an information gain measure when deciding which cluster is more informative for any given gene position, during a pairwise cluster combination. Empirical evaluations effectively cover a comprehensive range of benchmark optimization problems.

Recognizing speculative language in biomedical research articles: a linguistically motivated perspective.

BackgroundDue to the nature of scientific methodology, research articles are rich in speculative and tentative statements, also known as hedges. We explore a linguistically motivated approach to the problem of recognizing such language in biomedical research articles. Our approach draws on prior linguistic work as well as existing lexical resources to create a dictionary of hedging cues and extends it by introducing syntactic patterns.Furthermore, recognizing that hedging cues differ in speculative strength, we assign them weights in two ways: automatically using the information gain (IG) measure and semi-automatically based on their types and centrality to hedging. Weights of hedging cues are used to determine the speculative strength of sentences.ResultsWe test our system on two publicly available hedging datasets. On the fruit-fly dataset, we achieve a precision-recall breakeven point (BEP) of 0.85 using the semi-automatic weighting scheme and a lower BEP of 0.80 with the information gain weighting scheme. These results are competitive with the previously reported best results (BEP of 0.85). On the BMC dataset, using semi-automatic weighting yields a BEP of 0.82, a statistically significant improvement (p <0.01) over the previously reported best result (BEP of 0.76), while information gain weighting yields a BEP of 0.70.ConclusionOur results demonstrate that speculative language can be recognized successfully with a linguistically motivated approach and confirms that selection of hedging devices affects the speculative strength of the sentence, which can be captured reasonably by weighting the hedging cues. The improvement obtained on the BMC dataset with a semi-automatic weighting scheme indicates that our linguistically oriented approach is more portable than the machine-learning based approaches. Lower performance obtained with the information gain weighting scheme suggests that this method may benefit from a larger, manually annotated corpus for automatically inducing the weights.

Fuzzy classification systems based on fuzzy information gain measures

In this paper, we present a new method for handling classification problems using a new fuzzy information gain measure. Based on the proposed fuzzy information gain measure, we propose an algorithm for constructing membership functions, calculating the class degree of each subset of training instances with respect to each class and calculating the fuzzy entropy of each subset of training instances. Based on the constructed membership function of each fuzzy set of each feature, the obtained class degree of each subset of training instances with respect to each class and the obtained fuzzy entropy of each subset of training instances, we propose an evaluating function for classifying testing instances. The proposed method gets higher average classification accuracy rates than the methods presented in [John, G. H., & Langley, P. (1995). Estimating continuous distributions in Bayesian classifiers. In Proceedings of the 11th conference on uncertainty in artificial intelligence, Montreal, Canada (pp. 338–345); Platt, J. C. (1999). Using analytic QP and sparseness to speed training of support vector machines. In Proceedings of the 13th annual conference on neural information processing systems, Denver, Colorado (pp. 557–563); Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco: Morgan Kaufmann].

Using information gain to improve multi-modal information retrieval systems

Nowadays, access to information requires managing multimedia databases effectively, and so, multi-modal retrieval techniques (particularly images retrieval) have become an active research direction. In the past few years, a lot of content-based image retrieval (CBIR) systems have been developed. However, despite the progress achieved in the CBIR, the retrieval accuracy of current systems is still limited and often worse than only textual information retrieval systems. In this paper, we propose to combine content-based and text-based approaches to multi-modal retrieval in order to achieve better results and overcome the lacks of these techniques when they are taken separately. For this purpose, we use a medical collection that includes both images and non-structured text. We retrieve images from a CBIR system and textual information through a traditional information retrieval system. Then, we combine the results obtained from both systems in order to improve the final performance. Furthermore, we use the information gain (IG) measure to reduce and improve the textual information included in multi-modal information retrieval systems. We have carried out several experiments that combine this reduction technique with a visual and textual information merger. The results obtained are highly promising and show the profit obtained when textual information is managed to improve conventional multi-modal systems.

Cluster Generation and Labeling for Web Snippets: A Fast, Accurate Hierarchical Solution

This paper describes Armil, a meta-search engine that groups the web snippets returned by auxiliary search engines into disjoint labeled clusters. The cluster labels generated by Armil provide the user with a compact guide to assessing the relevance of each cluster to his/her information need. Striking the right balance between running time and cluster well-formedness was a key point in the design of our system. Both the clustering and the labeling tasks are performed on the fly by processing only the snippets provided by the auxiliary search engines, and they use no external sources of knowledge. Clustering is performed by means of a fast version of the furthest-pointfirst algorithm for metric k-center clustering. Cluster labeling is achieved by combining intra-cluster and inter-cluster term extraction based on a variant of the information gain measure. We have tested the clustering effectiveness of Armil against Vivisimo, the de facto industrial standard in web snippet clustering, using as benchmark a c...

A new approach to classification based on association rule mining

Classification is one of the key issues in the fields of decision sciences and knowledge discovery. This paper presents a new approach for constructing a classifier, based on an extended association rule mining technique in the context of classification. The characteristic of this approach is threefold: first, applying the information gain measure to the generation of candidate itemsets; second, integrating the process of frequent itemsets generation with the process of rule generation; third, incorporating strategies for avoiding rule redundancy and conflicts into the mining process. The corresponding mining algorithm proposed, namely GARC (Gain based Association Rule Classification), produces a classifier with satisfactory classification accuracy, compared with other classifiers (e.g., C4.5, CBA, SVM, NN). Moreover, in terms of association rule based classification, GARC could filter out many candidate itemsets in the generation process, resulting in a much smaller set of rules than that of CBA.