Mutual Information Function Research Articles

Entropy rate estimates from mutual information

We show how to estimate the Kolmogorov-Sinai entropy rate for chaotic systems using the mutual information function, easily obtainable from experimental time series. We state the conditions under which the relationship is exact, and explore the usefulness of the approach for both maps and flows. We also explore refinements of the method, and study its convergence properties as a function of time series length.

Mutual information analysis reveals bigeminy patterns in Andersen–Tawil syndrome and in subjects with a history of sudden cardiac death

Herein we introduce the Mutual Information Function (MIF) as a mathematical method to analyze ventricular bigeminy in certain pathological conditions of the heart known to be associated with frequent ventricular arrhythmias. In particular, we show that the MIF is sensitive enough to detect the bigeminy pattern in symbolic series from patients with Andersen–Tawil syndrome as well as in a group of patients from the Sudden Cardiac Death Holter Databases. The results confirm that MIF is an adequate method to detect the autocorrelation between the appearance of sinus and ventricular premature beats resulting in a bigeminy pattern. It is also shown that MIF reflects the bigeminy patterns as a function of the percentage of ventricular premature beats present in the symbolic series and also as a function of the percentage of bigeminy. The MIF was also useful to establish a consistent difference in the bigeminy pattern related to the diurnal and nocturnal periods presumably associated to the circadian rhythm of the heart. Understanding of the ventricular bigeminy patterns throughout 24-hours could provide some insights into the pathogenesis of ventricular tachyarrhythmias in these pathological conditions.

Chaotic and statistical tests on fiber optic dynamic data taken from the riser section of a circulating fluidized bed

Dynamical tests have been applied to fiber optic data taken from a cold-flow circulating fluidized bed to characterize flow conditions, identify three characteristic scales (macro, meso, and micro), and understand the contribution these scales have on the raw data. The characteristic variable analyzed is the raw voltage signal obtained from a fiber-optic probe taken at various axial and radial positions under different loading conditions. These experiments were carried out with the bed material of cork particles with a medium particle size of 812μm. The characterization was accomplished through analysis of the distribution of the signal through the third and fourth moments of skewness and excess kurtosis. A generalization of the autocorrelation function known as the average mutual information function was analyzed by examining the function's first minimum, identifying the point at which successive elements are no longer correlated. Further characterization was accomplished through the correlation dimension, a measure of the complexity of the chaotic attractor in deterministic chaos theory. Lastly, the amount of disorder of the system is described by a Kolmogorov-type entropy estimate. All six aforementioned tests were also implemented on ten levels of detail coefficients resulting from a discrete wavelet transformation of the same signal as used above. Through this analysis it is possible to identify and describe micro (particle level), meso (clustering or turbulence level), and macro (physical or dimensional level) length scales even though some literature considers these scales inseparable as is discussed. This investigation also used detail wavelet coefficients in conjunction with ANOVA to show which scales have the most impact on the raw signal resulting from local hydrodynamic conditions.

Chaotic and statistical tests on fiber optic dynamic data taken from the riser section of a circulating fluidized bed

Dynamical tests have been applied to fiber optic data taken from a cold-flow circulating fluidized bed to characterize flow conditions, identify three characteristic scales (macro, meso, and micro), and understand the contribution these scales have on the raw data. The characteristic variable analyzed is the raw voltage signal obtained from a fiber-optic probe taken at various axial and radial positions under different loading conditions. These experiments were carried out with the bed material of cork particles with a medium particle size of 812 μm. The characterization was accomplished through analysis of the distribution of the signal through the third and fourth moments of skewness and excess kurtosis. A generalization of the autocorrelation function known as the average mutual information function was analyzed by examining the function's first minimum, identifying the point at which successive elements are no longer correlated. Further characterization was accomplished through the correlation dimension, a measure of the complexity of the chaotic attractor in deterministic chaos theory. Lastly, the amount of disorder of the system is described by a Kolmogorov-type entropy estimate. All six aforementioned tests were also implemented on ten levels of detail coefficients resulting from a discrete wavelet transformation of the same signal as used above. Through this analysis it is possible to identify and describe micro (particle level), meso (clustering or turbulence level), and macro (physical or dimensional level) length scales even though some literature considers these scales inseparable as is discussed. This investigation also used detail wavelet coefficients in conjunction with ANOVA to show which scales have the most impact on the raw signal resulting from local hydrodynamic conditions.

General properties of transcriptional time series in Escherichia coli

Gene activity is described by the time-series of discrete, stochastic mRNA production events. This transcriptional time-series exhibits intermittent, bursty behavior. One consequence of this temporal intricacy is that gene expression can be tuned by varying different features of the time-series. What schemes for varying the transcriptional time-series are observed in the cell? Are the observed properties of these time-series optimized for cellular function? To address these questions, we characterize mRNA copy-number statistics at single-molecule resolution from multiple Escherichia coli promoters. We find that the degree of burstiness depends only on the gene expression level, while being independent of the details of gene regulation. The observed behavior is explained by the underlying variation in the duration of bursting events. Using information theory, we find that the properties of the transcriptional time series allow the cell to efficiently map the extracellular concentration of inducer molecules to intracellular levels of mRNA and proteins.

RING: networking interacting residues, evolutionary information and energetics in protein structures

Residue interaction networks (RINs) have been used in the literature to describe the protein 3D structure as a graph where nodes represent residues and edges physico-chemical interactions, e.g. hydrogen bonds or van-der-Waals contacts. Topological network parameters can be calculated over RINs and have been correlated with various aspects of protein structure and function. Here we present a novel web server, RING, to construct physico-chemically valid RINs interactively from PDB files for subsequent visualization in the Cytoscape platform. The additional structure-based parameters secondary structure, solvent accessibility and experimental uncertainty can be combined with information regarding residue conservation, mutual information and residue-based energy scoring functions. Different visualization styles are provided to facilitate visualization and standard plugins can be used to calculate topological parameters in Cytoscape. A sample use case analyzing the active site of glutathione peroxidase is presented. The RING server, supplementary methods, examples and tutorials are available for non-commercial use at URL: http://protein.bio.unipd.it/ring/.

Robustness Analysis of QIM Watermarking against Additional Noise

A mutual information function was defined as a criterion measuring the robustness of watermarking algorithm. Considering QIM scheme, error probability of watermarking can be calculated to validate the measurement of mutual information function. By mean of numerical computation, mutual information under Gaussian noise and uniform noise is calculated with change of noise standard deviation. In the experiment, an audio section is selected as the host and their third lever wavelet detail coefficients are quantified according to watermark bit series. Experiment results show that statistic Bit Error Rate (BER) is matched with evaluation conclusion of mutual information method when step is on the small side. Mutual information function can be selected as a cost function to evaluate the robustness of watermarking algorithm, and predict the BER.

Automatic registration and fast volume reconstruction from serial histology sections

The scope of this research is to propose a novel method for automatic registration and fast volume reconstruction of serial tissue sections based on common-configured computer, on the premise that the algorithm can achieve accurate result with fast speed and little interaction as much as possible. The whole flowchart comprises the four main parts, of which image registration and image reconstruction are of great significance, materialized the following innovative ideas. Firstly, mutual information measure is combined with morphological gradient information to contain fewer erroneous maxima and lead to the global maximum when registration. Secondly, the hybrid optimizer combined Particle Swarm Optimization (PSO) with Powell algorithm is proposed to restrain local maxima of mutual information function and to improve the registration accuracy. Thirdly, the multiresolution data structure based on Mallat decomposition not only improves the behavior of registration function, but also accelerates the algorithm speed. Finally, an improved Shear–Warp algorithm is proposed based on the sorted volumetric dataset for registered image stack, which can skip all of the transparent voxels and achieve faster speed. Experimental results demonstrate that the novel registration and reconstruction algorithms are effective and efficient to achieve the virtual three-dimensional gekko’s cervical spinal cord successfully and the reconstruction result can be rotated, scaled, incised and measured arbitrarily, which is valuable to be applied to the other kinds of serial histology sections.

Reducing interpolation artifacts for mutual information based image registration

Medical image registration methods which use mutual information as similarity measure have been improved in recent decades. Mutual Information is a basic concept of Information theory which indicates the dependency of two random variables (or two images). In order to evaluate the mutual information of two images their joint probability distribution is required. Several interpolation methods, such as Partial Volume (PV) and bilinear, are used to estimate joint probability distribution. Both of these two methods yield some artifacts on mutual information function. Partial Volume-Hanning window (PVH) and Generalized Partial Volume (GPV) methods are introduced to remove such artifacts. In this paper we show that the acceptable performance of these methods is not due to their kernel function. It's because of the number of pixels which incorporate in interpolation. Since using more pixels requires more complex and time consuming interpolation process, we propose a new interpolation method which uses only four pixels (the same as PV and bilinear interpolations) and removes most of the artifacts. Experimental results of the registration of Computed Tomography (CT) images show superiority of the proposed scheme.

MEG Connectivity Analysis in Patients with Alzheimer’s Disease Using Cross Mutual Information and Spectral Coherence

Alzheimer's disease (AD) is an irreversible brain disorder which represents the most common form of dementia in western countries. An early and accurate diagnosis of AD would enable to develop new strategies for managing the disease; however, nowadays there is no single test that can accurately predict the development of AD. In this sense, only a few studies have focused on the magnetoencephalographic (MEG) AD connectivity patterns. This study compares brain connectivity in terms of linear and nonlinear couplings by means of spectral coherence and cross mutual information function (CMIF), respectively. The variables defined from these functions provide statistically significant differences (p<0.05) between AD patients and control subjects, especially the variables obtained from CMIF. The results suggest that AD is characterized by both decreases and increases of functional couplings in different frequency bands as well as by an increase in regularity, that is, more evident statistical deterministic relationships in AD patients' MEG connectivity. The significant differences obtained indicate that AD could disturb brain interactions causing abnormal brain connectivity and operation. Furthermore, the combination of coherence and CMIF features to perform a diagnostic test based on logistic regression improved the tests based on individual variables for its robustness.

SU-GG-I-105: Ultrafast Deformable Image Registration for Potential Adaptive Total Body Irradiation Therapy Using Helical TomoTherapy

Purpose: An ultrafast deformable image registration model was developed to register planning computed tomography(CT) with the on‐treatment megavoltage CT (MVCT) of the total body irradiation (TBI) patients using Helical TomoTherapy. The proposed technique constitutes a valuable platform for MVCT‐based dose verification and MVCT auto‐contouring for adaptive radiotherapy.Methods and materials: MVCTs acquired prior to each treatment using Helical TomTherapy unit (HT, TomoTherapy Inc, Madison, WI) can not only be used for patient's setup, but it also contains the patient's updated anatomy, forming the basis for adaptive radiotherapy. A free form B‐Spline deformable registration was carried out to derive the patient's interfraction anatomical variations by co‐registering the planning CT with the individual treatment fraction MVCT. A Mattes mutual information function was used as the metric function and the limited memory Broyden‐Fletcher‐Goldfarb‐Shanno algorithm (L‐BFGS) was employed to optimize the multi‐dimensional metric function. Upon successful registration, the resulting deformation field was extracted and utilized to transform the manual contours of regions of interest (ROIs) on planning CT to MVCTs. The registration accuracy was evaluated by comparing the mapped contours and manually segmented contours on MVCT. Four patients with acute myelogenous leukemia treated with TBI using HT were used to test the proposed algorithm. Results: A free form deformable image registration model has been developed. Application of the algorithm to four leukemia patients indicates that clinically satisfactory results are achievable with a spatial accuracy better than 2mm for ROI contours mappings. As compared to the cross correlation metric function, the proposed registration model was found to be two orders of magnitude more efficient whilst a same accuracy level could be achieved. Conclusions: A regional deformable registration model was implemented and it provides a powerful tool for auto‐contouring MVCTs to maximize the potential of the adaptive TBI therapy using HT.

Geometry of repeated measurements in chaotic systems

We use joint probability matrices for measurements at different times to describe chaotic systems. By coarse graining the range of the measured variable into uniformly sized bins we can generate matrices that contain both topological and metric information about the systems being studied. Armed with this tool we examine two extreme families of chaotic systems. In the case of one-dimensional piecewise linear maps, we can construct transfer matrices that depend on the map and partition used, and which allow us to generate the respective joint probability matrices for all times as well as the exact time evolution of the mutual information function. We find that the mutual information decays linearly or exponentially depending on whether the second-largest eigenvalue of the transfer matrix is zero or not. In the case of three-dimensional, continuous-time chaotic systems we generate the joint probability matrices directly from numerical data. We show that these matrices directly provide attractor reconstructions with information about the attractor's probability measure.

Evaluation model for robustness of digital watermarking

Robustness is one of the most important requirements when digital watermarking is applied. Different from the StirMark test and various simulation tests, a mutual information function is defined as a criterion to measure the robustness of watermarking algorithm. Taking the additive spread spectrum watermarking scheme and quantization index modulation (QIM) watermarking scheme as two examples, the calculation formulas of mutual information function are derived to evaluate the robustness of the algorithms. Numerical computation of mutual information is performed with change of watermark noise rate (WNR). In the experiment, spread spectrum watermarking is implemented in discrete cosine transform (DCT) and QIM watermarking is implemented in discrete wavelet transform (DWT). The statistic bit error rate (BER) is derived against Gaussian distribution noise and JPEG compression. Experiment results show that the evaluation conclusion of mutual information method is in accordance with the empirical BER. Mutual information can be selected as a cost function to predict the BER.

Study on the parameters determination for reconstructing phase-space in chaos time series

The choice of delay time and embedding dimension have no correlation in the process of phase-space reconstruction. Based on this principle, a new idea for determining the embedding dimension is proposed. It uses CAO theory after determining the delay time by the mutual information function. Numerical simulations show that the method is applicable for determining the appropriate delay time and the best embedding dimension. The method can recover the original phase space from the time series effectively, and can be used as an effective method to identify the chaotic signal.

Drug effect on EEG connectivity assessed by linear and nonlinear couplings.

Quantitative analysis of human electroencephalogram (EEG) is a valuable method for evaluating psychopharmacological agents. Although the effects of different drug classes on EEG spectra are already known, interactions between brain locations remain unclear. In this work, cross mutual information function and appropriate surrogate data were applied to assess linear and nonlinear couplings between EEG signals. The main goal was to evaluate the pharmacological effects of alprazolam on brain connectivity during wakefulness in healthy volunteers using a cross-over, placebo-controlled design. Eighty-five pairs of EEG leads were selected for the analysis, and connectivity was evaluated inside anterior, central, and posterior zones of the scalp. Connectivity between these zones and interhemispheric connectivity were also measured. Results showed that alprazolam induced significant changes in EEG connectivity in terms of information transfer in comparison with placebo. Trends were opposite depending on the statistical characteristics: decreases in linear connectivity and increases in nonlinear couplings. These effects were generally spread over the entire scalp. Linear changes were negatively correlated, and nonlinear changes were positively correlated with drug plasma concentrations; the latter showed higher correlation coefficients. The use of both linear and nonlinear approaches revealed the importance of assessing changes in EEG connectivity as this can provide interesting information about psychopharmacological effects.

Possible dependence between the total solar irradiance and dimethylsulphide

La variabilidad solar es una de las principales influencias naturales del clima terrestre, y que afecta profundamente a los procesos biológicos. Se ha propuesto que algunos de estos procesos biológicos alteran el albedo y por tanto tienen un impacto en el clima. En este trabajo investigamos la relación entre la irradiancia total solar (ITS) y la concentración de Dimetilsulfuro (DMS), que es producido por algas marinas. El DMS se ha mencionado frecuentemente como un forzamiento climático a través de su efecto en las nubes y por tanto en el albedo. En el presente trabajo encontramos que las series temporales de la IST y el DMS muestran una correlación. Para ello utilizamos un esquema probabilístico, la Función de Información Mutua (FIM), que es una medida de la dependencia entre los parámetros de interés. La FIM presenta una dependencia con el ciclo solar: valores mayores aparecen durante épocas de baja actividad solar (épocas de menor IST), mientras que los valores menores se presentan durante tiempos de alta actividad solar (mayor IST). En otras palabras, la dependencia entre IST y DMS es más fuerte durante épocas de actividad solar menor que durante actividad solar mayor.

Forecasting stream water temperature using regression analysis, artificial neural network, and chaotic non-linear dynamic models

Stream water temperature is considered both a dominant factor in determining the longitudinal distribution pattern of aquatic biota and as a general metabolic indicator for the water body, since so many biological processes are temperature dependent. Moreover, the plunging depth of stream water, its associated pollutant load, and its potential impact on lake/reservoir ecology is dependent on water temperature. Lack of detailed datasets and knowledge on physical processes of the stream system limits the use of a phenomenological model to estimate stream temperature. Rather, empirical models have been used as viable alternatives. In this study, an empirical model (artificial neural networks (ANN)), a statistical model (multiple regression analysis (MRA)), and the chaotic non-linear dynamic algorithms (CNDA) were examined to predict the stream water temperature from the available solar radiation and air temperature. Observed time series data were non-linear and non-Gaussian, thus the method of time delay was applied to form the new dataset that closely represent the inherent system dynamics. Phase-space reconstruction plots show that time lag equal to 0 and greater than 10 result in highly dependent (a well-defined attractor) and highly independent (no attractor at all) reconstructions, respectively, and, therefore, may not be appropriate to use. Delayed vector was found to be strongly correlated with the original vector when time lag is small (i.e. less than 3-day) and vice versa. Power spectrum analysis and autocorrelation function suggested that the time series data was chaotic and mutual information function indicates that optimum time lag was approximately 3-day. The chaotic non-linear dynamic algorithm and four-layer back propagation neural network (4BPNN) optimized by micro-genetic algorithms (μGA) showed that the prediction performance was optimum when data are presented to the model with 1-day and 3-day time lag, respectively. The prediction performance efficiency of MRA is higher for time lag greater than 3-day, however, the incremental performance efficiency rate significantly decreased after 3-day time lag. The prediction performance efficiency of μGA-4BPNN was found to be the highest among all algorithms considered in this study. Air temperature was found to be the most important variable in stream temperature forecasting; however, the prediction performance efficiency was somewhat higher if short wave radiation was included.

Characterization of Healthy and Pathological Voice Through Measures Based on Nonlinear Dynamics

In this paper, we propose to quantify the quality of the recorded voice through objective nonlinear measures. Quantification of speech signal quality has been traditionally carried out with linear techniques since the classical model of voice production is a linear approximation. Nevertheless, nonlinear behaviors in the voice production process have been shown. This paper studies the usefulness of six nonlinear chaotic measures based on nonlinear dynamics theory in the discrimination between two levels of voice quality: healthy and pathological. The studied measures are first- and second-order Renyi entropies, the correlation entropy and the correlation dimension. These measures were obtained from the speech signal in the phase-space domain. The values of the first minimum of mutual information function and Shannon entropy were also studied. Two databases were used to assess the usefulness of the measures: a multiquality database composed of four levels of voice quality (healthy voice and three levels of pathological voice); and a commercial database (MEEI Voice Disorders) composed of two levels of voice quality (healthy and pathological voices). A classifier based on standard neural networks was implemented in order to evaluate the measures proposed. Global success rates of 82.47% (multiquality database) and 99.69% (commercial database) were obtained.

Short-Term Solar Flare Prediction Using a Sequential Supervised Learning Method

Solar flares are powered by the energy stored in magnetic fields, so evolutionary information of the magnetic field is important for short-term prediction of solar flares. However, the existing solar flare prediction models only use the current information of the active region. A sequential supervised learning method is introduced to add the evolutionary information of the active region into a prediction model. The maximum horizontal gradient, the length of the neutral line, and the number of singular points extracted from SOHO/MDI longitudinal magnetograms are used in the model to describe the nonpotentiality and complexity of the photospheric magnetic field. The evolutionary characteristics of the predictors are analyzed by using autocorrelation functions and mutual information functions. The analysis results indicate that a flare is influenced by the 3-day photospheric magnetic field information before flare eruption. A sliding-window method is used to add evolutionary information of the predictors into machine learning algorithms, then C4.5 decision tree and learning vector quantization are employed to predict the flare level within 48 hours. Experimental results indicate that the performance of the short-term solar flare prediction model within the sequential supervised learning framework is significantly improved.

Detecting and tracking moving targets on omnidirectional vision

A method was presented to implement the detecting and tracking of moving targets through omnidirectional vision. The method combined optical flow with particle filter arithmetic, in which optical flow field was used to detect and locate moving targets and particle filter was used to track the detected moving objects. According to the circular image character of omnidirectional vision, the calculation equation of optical flow field and the tracking arithmetic of particle filter were improved based on the polar coordinates at the omnidirectional center. The edge of a randomly moving object could be detected by optical flow field and was surrounded by a reference region in the particle filter. A dynamic motion model was established to predict particle state. Histograms were used as the features in the reference region and candidate regions. The mutual information (MI) and Gaussian function were combined to calculate particle weights. Finally, the state of tracked object was computed by the total particle states with weights. Experiment results show that the proposed method could detect and track moving objects with better real-time performance and accuracy.