Distribution Of Mutual Information Research Articles

Voltage-fault diagnosis for battery pack in electric vehicles using mutual information

The safety of battery system is compromised by the abusive operation and aging, potentially resulting in the abnormal voltage levels. Rapid detection and accurate diagnosis of voltage fault are crucial for ensuring the safety of battery packs. A battery voltage fault diagnosis method is proposed by using the mutual information in this work, which can identify faulty cells timely. Specifically, the voltage of battery pack in an electric vehicle is collected, and the mutual information of voltages between each paired-cells is calculated. The presence of faulty cells disturbs the original distribution of mutual information. Therefore, the Ryan Joiner (RJ) test is used to judge whether the normal distribution of calculated mutual information or not. The interquartile test is used to detect the outliers. The faulty cell is diagnosed by matching the cell corresponding to the outliers. The defaults resulting from an internal short circuit and mechanical vibration are analyzed and diagnosed. The method can avoid misdiagnosis caused by the mutual influence between different faults. Furthermore, the fault diagnosis capability of presented method is discussed under various states of aged cells. The experimental results verify the effectiveness, robustness, model-free, and easiest-to-implement of the presented method.

Image preprocessing to improve the accuracy and robustness of mutual-information-based automatic image registration in proton therapy

PurposeWe propose a method that potentially improves the outcome of mutual-information-based automatic image registration by using the contrast enhancement filter (CEF). MethodsSeventy-six pairs of two-dimensional X-ray images and digitally reconstructed radiographs for 20 head and neck and nine lung cancer patients were analyzed retrospectively. Automatic image registration was performed using the mutual-information-based algorithm in VeriSuite®. Images were preprocessed using the CEF in VeriSuite®. The correction vector for translation and rotation error was calculated and manual image registration was compared with automatic image registration, with and without CEF. In addition, the normalized mutual information (NMI) distribution between two-dimensional images was compared, with and without CEF. ResultsIn the correction vector comparison between manual and automatic image registration, the average differences in translation error were < 1 mm in most cases in the head and neck region. The average differences in rotation error were 0.71 and 0.16 degrees without and with CEF, respectively, in the head and neck region; they were 2.67 and 1.64 degrees, respectively, in the chest region. When used with oblique projection, the average rotation error was 0.39 degrees with CEF. CEF improved the NMI by 17.9 % in head and neck images and 18.2 % in chest images. ConclusionsCEF preprocessing improved the NMI and registration accuracy of mutual-information-based automatic image registration on the medical images. The proposed method achieved accuracy equivalent to that achieved by experienced therapists and it will significantly contribute to the standardization of image registration quality.

GTT: Leveraging data characteristics for guiding the tensor train decomposition

The demand for searching, querying multimedia data such as image, video and audio is omnipresent, how to effectively access data for various applications is a critical task. Nevertheless, these data usually are encoded as multi-dimensional arrays, or tensor, and traditional data mining techniques might be limited due to the curse of dimensionality. Tensor decomposition is proposed to alleviate this issue. Commonly used tensor decomposition algorithms include CP-decomposition (which seeks a diagonal core) and Tucker-decomposition (which seeks a dense core). Naturally, Tucker maintains more information, but due to the denseness of the core, it also is subject to exponential memory growth with the number of tensor modes. Tensor train (TT) decomposition addresses this problem by seeking a sequence of three-mode cores: but unfortunately, currently, there are no guidelines to select the decomposition sequence. In this paper, we propose a GTT method for guiding the tensor train in selecting the decomposition sequence. GTT leverages the data characteristics (including number of modes, length of the individual modes, density, distribution of mutual information, and distribution of entropy) as well as the target decomposition rank to pick a decomposition order that will preserve information. Experiments with various data sets demonstrate that GTT effectively guides the TT-decomposition process towards decomposition sequences that better preserve accuracy.

Density (de)localization and statistical correlation in the Van der Waals interactions and the chemical bond between two hydrogens

Localization and statistical correlation of the molecule H2 and the Van der Waals system H(1s)−H(1s) are studied as a function of its internuclear distance by the use of Shannon entropies, mutual information, and density distribution variance and covariance. Comparative analysis of the behavior of the different localization measures are made, as well as the comparison between the statistical correlation ones. A comparative analysis between the two systems is performed, where the Kullback–Leibler statistical distance between the two probability densities is computed and discussed. We show that there is a region where the valence bond and the VdW densities are closer. Hence, statistical analysis is able to detect regions with notable similarities between the Heitler and London density, based on valence bond wave function and the VdW density, based on the correspondent wave function. This leads to the concern if VdW interactions could be detected using valence bond wave functions by analyzing its correlation and localization properties.

Modeling NDVI Using Joint Entropy Method Considering Hydro-Meteorological Driving Factors in the Middle Reaches of Hei River Basin

Terrestrial vegetation dynamics are closely influenced by both hydrological process and climate change. This study investigated the relationships between vegetation pattern and hydro-meteorological elements. The joint entropy method was employed to evaluate the dependence between the normalized difference vegetation index (NDVI) and coupled variables in the middle reaches of the Hei River basin. Based on the spatial distribution of mutual information, the whole study area was divided into five sub-regions. In each sub-region, nested statistical models were applied to model the NDVI on the grid and regional scales, respectively. Results showed that the annual average NDVI increased at a rate of 0.005/a over the past 11 years. In the desert regions, the NDVI increased significantly with an increase in precipitation and temperature, and a high accuracy of retrieving NDVI model was obtained by coupling precipitation and temperature, especially in sub-region I. In the oasis regions, groundwater was also an important factor driving vegetation growth, and the rise of the groundwater level contributed to the growth of vegetation. However, the relationship was weaker in artificial oasis regions (sub-region III and sub-region V) due to the influence of human activities such as irrigation. The overall correlation coefficient between the observed NDVI and modeled NDVI was observed to be 0.97. The outcomes of this study are suitable for ecosystem monitoring, especially in the realm of climate change. Further studies are necessary and should consider more factors, such as runoff and irrigation.

Statistical Approach for Performance Analysis of Multipath Scattering Environment

In realistic scenarios, multipath scattering environments usually require an approach which can easily model signal behavior. We have considered a Rayleigh multipath scattering channel. The multipath scattering environment captures degradation of signals in fading channels. In this paper, performance of the channel is investigated through the distribution of mutual information in the multipath scattering channel using matrix method. The probability density function for multipath scattering channels is still a developing area of research. The need of the hour is performance evaluation of these communication links in an efficient manner. By using techniques of probability theory, the variance of mutual information is derived for multiple cases based on fading environment. In this paper, we have considered a matrix based multipath scattering model for communication link. We have derived closed-form expression for mutual information in a multipath scattering channel. The matrix method is used to substitute this effect in terms of the Gaussian distribution. This substitution will prove to be very accurate for different cases as analyzed in this paper.

On the Exact Distribution of Mutual Information of Two-User MIMO MAC Based on Quotient Distribution of Wishart Matrices

We propose an exact calculation of the probability density function (PDF) and cumulative distribution function (CDF) of mutual information (MI) for a two-user multiple-input multiple-output (MIMO) multiple access channel (MAC) network over block Rayleigh fading channels. This scenario can be found in the uplink channel of MIMO non-orthogonal multiple access (NOMA) system, a promising multiple access technique for 5G networks. So far, the PDF and CDF have been numerically evaluated since MI depends on the quotient of two Wishart matrices, and no closed form for this quotient was available. We derive exact results for the PDF and CDF of extreme (the smallest/the largest) eigenvalues. Based on the results of quotient ensemble, the exact calculation for PDF and CDF of mutual information is presented via Laplace transform approach and by direct integration of joint PDF of quotient ensemble’s eigenvalues. Furthermore, our derivations also provide the parameters to apply the Gaussian approximation method, which is comparatively easier to implement. We show that approximation matches the exact results remarkably well for outage probability, i.e., CDF, above 10%. However, the approximation could also be used for 1% outage probability with a relatively small error. We apply the derived expressions to investigate the effects of adding antennas in the receiver and its ability to decode the weak user signal. By supposing no channel knowledge at transmitters and successive decoding at receiver, the capacity of the weak user increases and its outage probability decreases with the increment of extra antennas at the receiver end.

Asymptotic Analysis of Rayleigh Product Channels: A Free Probability Approach

The Rayleigh product channel model is useful in capturing the performance degradation due to rank deficiency of MIMO channels. In this paper, such a performance degradation is investigated via the distribution of mutual information assuming the block fading channels and the uniform power transmission scheme. Using techniques of free probability theory, the asymptotic variance of mutual information is derived when the dimensions of the channel matrices approach infinity. In this asymptotic regime, the mutual information is rigorously proven to be Gaussian distributed. Using the obtained results, a fundamental tradeoff between multiplexing gain and diversity gain of Rayleigh product channels under the uniform power transmission can be characterized by the closed-form expression at any finite signal-to-noise ratio. Numerical results are provided to compare the outage performance between the Rayleigh product channels and the conventional Rayleigh MIMO channels.

Reorganized DCT-based image representation for reduced reference stereoscopic image quality assessment

In this paper, a novel reduced reference (RR) stereoscopic image quality assessment (SIQA) is proposed by characterizing the statistical properties of the stereoscopic image in the reorganized discrete cosine transform (RDCT) domain. Firstly, the difference image between the left and right view images is computed. Afterwards, the left and right view images, as well as the difference image, are decomposed by block-based discrete cosine transform (DCT). The DCT coefficients are further reorganized into a three-level coefficient tree, resulting in ten RDCT subbands. For each RDCT subband, the statistical property of the coefficient distribution is modeled by the generalized Gaussian density (GGD) function. And the mutual information (MI) and energy distribution ratio (EDR) are employed to depict the statistical properties across different RDCT subbands. Moreover, EDR can further model the mutual masking property of the human visual system (HVS). By considering the GGD modeling behavior within each RDCT subband and MI together EDR characterizing behavior across RDCT subbands, the statistical properties of the stereoscopic image are fully exploited, including the left view, right view, and the difference image. Experimental results demonstrated that the statistical properties of the difference image can well represent the perceptual quality of the stereoscopic image, which outperforms the representative RR quality metrics for stereoscopic image and even some full reference (FR) quality metrics. By considering the left view, right view, and difference image together, the performances of the proposed RR SIQA can be further improved, which presenting a more closely relationship between the quality metric output and human visual perception.

Research on the evolution of stock correlation based on maximal spanning trees

In this study, we choose the daily closing price of 268 constituent stocks of the S&P 500 index, 221 stocks of London Stock Exchange, 148 constituent stocks of the Shanghai Composite index and 152 constituent stocks of the Hang Seng index as the research objects and select the sample of all the stock markets from 2 January, 2003, to 16 September, 2013. For each stock market, first, using a moving window to scan through every stock return series and mutual information to measure the statistical interdependence between stock returns, we construct a corresponding weighted network in every given window. Then we study the evolution of stock correlation by analyzing the average mutual information, mutual information distribution and topology structure’s variation of the maximal spanning tree extracting from every weighted network. All the obtained results indicate that for all the stock markets, both the average mutual information and the standard deviation of mutual information distribution first gradually increase and they reach a peak during the full-outbreak periods, and finally, they decrease again. In addition, the topology structure of the maximal spanning tree also changes from compact star-like to loose chain-like first and then turns to compact star-like once more. All the facts tell us that the crisis does change the stock correlation and the stock correlation is from weak to strong first, and then becomes weak again.

Distribution of Mutual Information in Multipartite States

Using the relative entropy of total correlation, we derive an expression relating the mutual information of $n$-partite pure states to the sum of the mutual informations and entropies of its marginals and analyze some of its implications. Besides, by utilizing the extended strong subadditivity of von Neumann entropy, we obtain generalized monogamy relations for the total correlation in three-partite mixed states. These inequalities lead to a tight lower bound for this correlation in terms of the sum of the bipartite mutual informations. We use this bound to propose a measure for residual three-partite total correlation and discuss the non-applicability of this kind of quantifier to measure genuine multiparty correlations.

Further Results on the Asymptotic Mutual Information of Rician Fading MIMO Channels

Rician fading multiple-input multiple-output (MIMO) channels with Kronecker-decomposable covariance structure are considered in this study to derive information measures characterizing their achievable rate. Analytic results are obtained in the asymptotic setting corresponding to infinitely many transmit and receive antennas, whose numbers approach a finite limiting ratio. More specifically, the asymptotic mean and variance of the mutual information are derived in this asymptotic regime, and it is shown that the corresponding probability distribution converges to a Gaussian distribution. All these results are based on the replica method, whose mathematical principles are discussed in this study. The ergodic capacity is also addressed by deriving an iterative water-filling algorithm yielding the capacity-achieving input covariance matrix (under an average power constraint). Resorting to the asymptotic Gaussianity of the mutual information distribution, an analytic expression of the outage capacity is derived based on the mean and variance of the mutual information. Finally, numerical results are reported to assess the level of approximation of the analytic results in comparison with those obtained by Monte Carlo simulation.

On the Distribution of MIMO Mutual Information: An In-Depth Painlevé-Based Characterization

This paper builds upon our recent work which computed the moment generating function of the MIMO mutual information exactly in terms of a Painlev\'{e} V differential equation. By exploiting this key analytical tool, we provide an in-depth characterization of the mutual information distribution for sufficiently large (but finite) antenna numbers. In particular, we derive systematic closed-form expansions for the high order cumulants. These results yield considerable new insight, such as providing a technical explanation as to why the well known Gaussian approximation is quite robust to large SNR for the case of unequal antenna arrays, whilst it deviates strongly for equal antenna arrays. In addition, by drawing upon our high order cumulant expansions, we employ the Edgeworth expansion technique to propose a refined Gaussian approximation which is shown to give a very accurate closed-form characterization of the mutual information distribution, both around the mean and for moderate deviations into the tails (where the Gaussian approximation fails remarkably). For stronger deviations where the Edgeworth expansion becomes unwieldy, we employ the saddle point method and asymptotic integration tools to establish new analytical characterizations which are shown to be very simple and accurate. Based on these results we also recover key well established properties of the tail distribution, including the diversity-multiplexing-tradeoff.

Coulumb Fluid, Painlevé Transcendents, and the Information Theory of MIMO Systems

In this paper, we compute two important information-theoretic quantities which arise in the application of multiple-input multiple-output (MIMO) antenna wireless communication systems: the distribution of the mutual information of multiantenna Gaussian channels, and the Gallager random coding upper bound on the error probability achievable by finite-length channel codes. We show that the mathematical problem underpinning both quantities is the computation of certain Hankel determinants generated by deformed versions of classical weight functions. For single-user MIMO systems, it is a deformed Laguerre weight; for multiuser MIMO systems, it is a deformed Jacobi weight. We apply two different methods to characterize each of these Hankel determinants. First, we employ the ladder operators of the corresponding monic rthogonal polynomials to give an exact characterization of the Hankel determinants in terms of Painleve differential equations. This turns out to be a Painleve V for the single-user MIMO scenario and a Painleve VI for the multiuser scenario. We then introduce Coulomb fluid linear statistics methods to derive closed-form approximations for the MIMO mutual information distribution and the error probability which, although formally valid for large matrix dimensions, are shown to give accurate results even when the matrix dimensions are small. Focusing on the single-user mutual information distribution, we then employ the exact Painleve V representation with the help of the Coulomb fluid linear statistics approximation to yield deeper insights into the scaling behavior in terms of the number of antennas and signal-to-noise ratio (SNR). Among other things, these results allow us to study the asymptotic Gaussianity of the distribution as the number of antennas increase, and to investigate when and why such approximations break down as the SNR increases. Based on the Painleve, we also derive recursive formulas for explicitly computing in closed form any desired number of correction terms to the asymptotic mean and variance, as well as closed-form asymptotic expressions for any desired number of higher order cumulants. Using these cumulants, we propose new closed-form approximations to the mutual information distribution which are shown to be very accurate, not only in the bulk but also in the tail region of interest for the outage probability.

Joint Design of Transceivers for Multiple-Access Channels Using MMSE Decision Feedback Detection

We consider the joint design of transceivers for a multiple-access multi-input-multi-output (MIMO) system communicating over intersymbol interference (ISI) channels. The system we consider is equipped with the minimum mean-square error (MMSE) decision-feedback (DF) detector. Here, we explore a novel perspective of designing a transceiver that minimizes the arithmetic mean-square error (MSE) of MMSE-DF detection and satisfies a fixed-sum Gaussian mutual information constraint. For this optimization problem, a direct explicit solution is obtained. We show that the optimal solution is achieved if and only if uniform user mutual information and uniform symbol mutual information are both achieved. Uniform user mutual information is attained by successively solving a series of problems of minimizing individual user transmission power under fixed mutual information, whereas uniform symbol mutual information is maintained by applying the equal-diagonal QRS decomposition. We also show that, in addition to minimizing the arithmetic MSE of MMSE-DF detection, such uniform mutual information distribution has also other desirable properties: 1) The optimal detection order in terms of signal-to-interference-noise ratios (SINRs) for both the users and the symbols is the natural transmission order of the system; and 2) when the sum Gaussian mutual information is large, the performance of the optimum transceiver using the MMSE-DF receiver asymptotically approaches that of the maximum-likelihood (ML) receiver.

Time Course of Information Representation of Macaque AIP Neurons in Hand Manipulation Task Revealed by Information Analysis

We used mutual information analysis of neuronal activity in the macaque anterior intraparietal area (AIP) to examine information processing during a hand manipulation task. The task was to reach-to-grasp a three-dimensional (3D) object after presentation of a go signal. Mutual information was calculated between the spike counts of individual neurons in 50-ms-wide time bins and six unique shape classifications or 15 one-versus-one classifications of these shapes. The spatiotemporal distribution of mutual information was visualized as a two-dimensional image ("information map") to better observe global profiles of information representation. In addition, a nonnegative matrix factorization technique was applied for extracting its structure. Our major finding was that the time course of mutual information differed significantly according to different classes of task-related neurons. This strongly suggests that different classes of neurons were engaged in different information processing stages in executing the hand manipulation task. On the other hand, our analysis revealed the heterogeneous nature of information representation of AIP neurons. For example, "information latency" (or information onset) varied among individual neurons even in the same neuron class and the same shape classification. Further, some neurons changed "information preference" (i.e., shape classification with the largest amount of information) across different task periods. These suggest that neurons encode different information in the different task periods. Taking the present result together with previous findings, we used a Gantt chart to propose a hypothetical scheme of the dynamic interactions between different types of AIP neurons.

Asymptotic Statistics of the Mutual Information for Spatially Correlated Rician Fading MIMO Channels With Interference

The statistics of the mutual information of a separately correlated Rician fading multiple-input multiple-output (MIMO) channel in the presence of multiple-access interference are addressed in this paper. The approach followed is asymptotic in the number of transmit, receive, and interfering antennas, which are all assumed to grow asymptotically large while approaching finite ratios. In this asymptotic regime: i) the mean and the variance of the mutual information are calculated; and ii) the mutual information distribution is shown to converge to the Gaussian distribution, specified completely by the mean and variance, under some mild technical conditions. The asymptotic method adopted relies on two powerful tools developed in the context of theoretical physics: the replica method and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">superanalysis</i> . The former has been already successfully applied in several research studies on MIMO systems. The application of these asymptotic results takes advantage of the fact that, in spite of being developed under the assumption of an asymptotically large number of antennas, they still represent a very accurate approximation even when the number of antennas is limited to a few units, as supported by the ample set of numerical results obtained by Monte Carlo simulations and reported in this paper.

Leveraging sentiment analysis for topic detection

The emergence of new social media such as blogs, message boards, news, and web content in general has dramatically changed the ecosystems of corporations. Consumers, non-profit organizations, and other forms of communities are extremely vocal about their opinions and perceptions on companies and their brands on the web. The ability to leverage such “voice of the web” to gain consumer, brand, and market insights can be truly differentiating and valuable to today's corporations. In particular, one important form of insights can be derived from sentiment analysis on web content. Sentiment analysis traditionally emphasizes on classification of web comments into positive, neutral, and negative categories. This paper goes beyond sentiment classification by focusing on techniques that could detect the topics that are highly correlated with the positive and negative opinions. Such techniques, when coupled with sentiment classification, can help the business analysts to understand both the overall sentiment scope as well as the drivers behind the sentiment. In this paper, we describe our overall sentiment analysis system that consists of such sentiment analysis techniques, including the bootstrapping method for word polarities weighting, automatic filtering and expansion for domain word, and a sentiment classification method. We then detail a novel topic detection method using point-wise mutual information and term frequency distribution. We demonstrate the effectiveness of our overall approaches via several case studies on different social media data sets.

On the Mutual Information Distribution of MIMO Rician Fading Channels

In a fading environment, the statistical distribution of the mutual information of a multiple-input multiple-output (MIMO) system depends on the joint distribution of the eigenvalues of a Wishart matrix, and is quite complex in general. We obtain here simple expressions for the distributions of the determinant and the trace of a Wishart matrix. Based on the obtained distributions, we derive some simple and tight bounds on the complementary cumulative distribution function (CCDF) of the mutual information of a MIMO system in Rician fading environments. The bounds obtained on the CCDF of mutual information provide further insights into the channel mutual information, and show the effects of the system parameters on the mutual information distribution explicitly. In addition, results for the Rayleigh channels are obtained as a special case.

A smoothed bootstrap test for independence based on mutual information

A test for independence of multivariate time series based on the mutual information measure is proposed. First of all, a test for independence between two variables based on i.i.d. (time-independent) data is constructed and is then extended to incorporate higher dimensions and strictly stationary time series data. The smoothed bootstrap method is used to estimate the null distribution of mutual information. The experimental results reveal that the proposed smoothed bootstrap test performs better than the existing tests and can achieve high powers even for moderate dependence structures. Finally, the proposed test is applied to assess the actual independence of components obtained from independent component analysis (ICA).