Mutual Information Measure Research Articles

Generalized Coupled Matrix Tensor Factorization Method Based on Normalized Mutual Information for Simultaneous EEG-fMRI Data Analysis.

The complementary properties of both modalities can be exploited through the fusion of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data. Thus, a joint analysis of both modalities can be used in brain studies to estimate brain activity's shared and unshared components. This study introduces a comprehensive approach for jointly analyzing EEG and fMRI data using the advanced coupled matrix tensor factorization (ACMTF) method. The similarity of the components based on normalized mutual information (NMI) was defined to overcome the restrictive equality assumption of shared components in the common dimension of the ACMTF method. Because the mutual information (MI) measure can identify both linear and nonlinear relationships between the components, the proposed method can be viewed as a generalization of the ACMTF method; thus, it is called the generalized coupled matrix tensor factorization (GCMTF). The proposed GCMTF method was applied to simulated data, in which the components exhibited a nonlinear relationship. The results demonstrate that the average match score increased by 23.46% compared with the ACMTF model, even with different noise levels. Furthermore, applying this method to real data from an auditory oddball paradigm demonstrated that three shared components with frequency responses in the alpha and theta bands were identified. The proposed MI-based method cannot only extract shared components with any nonlinear or linear relationship but can also identify more active brain areas corresponding to an auditory oddball paradigm compared to ACMTF and other similar methods.

The End of Mean-Variance? Tsallis Entropy Revolutionises Portfolio Optimisation in Cryptocurrencies

Has the mean-variance framework become obsolete? In this paper, we replace traditional variance–covariance methods of portfolio optimisation with relative Tsallis entropy and mutual information measures. Its goal is to enhance risk management and diversification in complicated finance ecosystems. We utilize the S&P 500 and Bitwise 10 cryptocurrency indices’ daily returns (2019–2024 data) and conduct our analysis to the year 2020 under extreme shocks. Many models were trained with different configurations, like mean-variance (MV), mean-entropy (ME), and mean-mutual information (MI) traders and their corresponding variants, using Sharpe’s ratio, Jensen’s alpha, and entropy value of risk (EVAR). The findings indicate that entropic models outperform conventional models in terms of diversification and, especially, extreme risk management. Because the appropriate normalization conditions often fail to be satisfied, we can informally see that after a recalibration of the effective frontier, we obtain from EVAR an accumulated resilience aspect to these rare events while also observing the great potential of entropy-based models to replicate non-linear dependencies between assets. The results show that models combining entropy and mutual information optimise the gain–loss ratio (GLR), providing stable diversification and improved risk management, while maximising returns in complex and volatile market environments.

Cardiac sound classification using a hybrid approach: MFCC-based feature fusion and CNN deep features

The detection of cardiovascular diseases through the analysis of phonocardiograms (PCGs), which are digital recordings of heartbeat sounds, is crucial for early diagnosis. Conventional feature extraction methods often face challenges in distinguishing non-stationary signals like healthy and pathological PCG signals. Our research addresses these challenges by adopting a hybrid feature extraction scheme that leverages deep learning and handcrafted techniques. This approach allows for a more effective analysis and classification of PCG signals. This paper presents a novel approach to PCG signal classification, leveraging a fusion of deep learning features and handcrafted features based on mutual information measurements. High-level features are obtained through a pretrained deep network applied to time-frequency representations of PCG signals. Additionally, Mel-Frequency Cepstral Coefficients of empirical wavelet subbands serve as handcrafted features. Canonical correlation analysis is employed for feature fusion, effectively combining crucial information from both feature types. Classification is performed using support vector machines, k-nearest neighbor, and multilayer perceptron (MLP) classifiers with a fivefold cross-validation approach. Evaluation using the Physionet Challenge 2016 database demonstrates the superior performance of our proposed approach compared to existing state-of-the-art studies, showcasing its efficacy in PCG signal classification.

Non‐Markovian Dynamics in Fiber Delay‐Line Buffers

AbstractThe non‐Markovian effect is studied on a two‐photon polarization entangled state, in which one photon from the pair is stored in a fiber delay‐line buffer. A model of a photonic qubit coupled to fiber birefringence and a fiber reservoir representing the environment is proposed. Analytically, a non‐Markovian probability function is derived for the buffered photon and its paired photon. To verify the probability function, full quantum state tomography of the photon pairs is performed. The probability function fits well with the experimental data and physical values. These results indicate that the quantum system operates slightly above the threshold for a non‐Markovian transition. We observe a unique polarization dynamic of the buffered photon. Measures of quantum mutual information are further exploited to study the quantumness of the photon pairs. Werner's well‐known separability criterion occurs at a buffer time of about 0.9 ms. These results imply that quantum discord can surpass Werner's criterion, and hence, quantum bi‐partite correlation can exist for buffer times greater than 0.9 ms.

Mutual Information Measure for Glass Ceiling Effect in Preferential Attachment Models

Mutual Information Measure for Glass Ceiling Effect in Preferential Attachment Models

Tuning Pair and Higher‐Order Mutual Information Measures in Oscillator Systems with N$N$‐Dependent Frequencies

Tuning Pair and Higher‐Order Mutual Information Measures in Oscillator Systems with N$N$‐Dependent Frequencies

Network mutual information measures for graph similarity

A wide range of tasks in network analysis, such as clustering network populations or identifying anomalies in temporal graph streams, require a measure of the similarity between two graphs. To provide a meaningful data summary for downstream scientific analyses, the graph similarity measures used for these tasks must be principled, interpretable, and capable of distinguishing meaningful overlapping network structure from statistical noise at different scales of interest. Here we derive a family of graph mutual information measures that satisfy these criteria and are constructed using only fundamental information theoretic principles. Our measures capture the information shared among networks according to different encodings of their structural information, with our mesoscale mutual information measure allowing for network comparison under any specified network coarse-graining. We test our measures in a range of applications on real and synthetic network data, finding that they effectively highlight intuitive aspects of network similarity across scales in a variety of systems.

Cross-Modal Clustering With Deep Correlated Information Bottleneck Method.

Cross-modal clustering (CMC) intends to improve the clustering accuracy (ACC) by exploiting the correlations across modalities. Although recent research has made impressive advances, it remains a challenge to sufficiently capture the correlations across modalities due to the high-dimensional nonlinear characteristics of individual modalities and the conflicts in heterogeneous modalities. In addition, the meaningless modality-private information in each modality might become dominant in the process of correlation mining, which also interferes with the clustering performance. To tackle these challenges, we devise a novel deep correlated information bottleneck (DCIB) method, which aims at exploring the correlation information between multiple modalities while eliminating the modality-private information in each modality in an end-to-end manner. Specifically, DCIB treats the CMC task as a two-stage data compression procedure, in which the modality-private information in each modality is eliminated under the guidance of the shared representation of multiple modalities. Meanwhile, the correlations between multiple modalities are preserved from the aspects of feature distributions and clustering assignments simultaneously. Finally, the objective of DCIB is formulated as an objective function based on a mutual information measurement, in which a variational optimization approach is proposed to ensure its convergence. Experimental results on four cross-modal datasets validate the superiority of the DCIB. Code is released at https://github.com/Xiaoqiang-Yan/DCIB.

Mutual information sums and relations with interaction energies in N-particle quantum systems.

The sums of position- and momentum-space mutual information measures are used to examine the pairwise and higher-order statistical correlation in the ground states of N-particle coupled oscillators. Analytical expressions for these measures are shown to be related to the logarithmic interaction energies of these states, plus those of mirror states where the intensities of the one- and two-body potentials are interchanged, and the nature of the attractive or repulsive interaction is opposite to that in the original state. The measures separate the contributions from the interactionsand those from the effective interactions due to marginalization into different terms. The pairwise mutual information sum is linearly related to the Shannon entropy sum in two particle systems, while the total correlation sum exhibits a similar relationship in three particle ones. In the latter instance, the interaction information sum can be related to entropy differences. This illustrates how entropy sums are connected to correlation measure sums in these systems. All measures approach zero with large N, when the magnitudes of the one- and two-body potentials are fixed. The pair mutual information and total correlation sums decay monotonically with N in the presence of an attractive potential and monotonically increase with a repulsive potential. On the other hand, the interaction information sum exhibits a minimum at small N, with an attractive potential. This is a consequence of the higher-order correlations governing behavior at smaller N while the pairwise ones dominate at larger N. Results are presented when the magnitude of the one-body potential is set to N.

Characterizing the nonmonotonic behavior of mutual information along biochemical reaction cascades.

Cells sense environmental signals and transmit information intracellularly through changes in the abundance of molecular components. Such molecular abundances can be measured in single cells and exhibit significant heterogeneity in clonal populations even in identical environments. Experimentally observed joint probability distributions can then be used to quantify the covariability and mutual information between molecular abundances along signaling cascades. However, because stationary state abundances along stochastic biochemical reaction cascades are not conditionally independent, their mutual information is not constrained by the data-processing inequality. Here, we report the conditions under which the mutual information between stationary state abundances increases along a cascade of biochemical reactions. This nonmonotonic behavior can be intuitively understood in terms of noise propagation and time-averaging stochastic fluctuations that are short-lived compared to an extrinsic signal. Our results reemphasize that mutual information measurements of stationary state distributions of cellular components may be of limited utility for characterizing cellular signaling processes because they do not measure information transfer.

Copula-Based Mutual Information Measures and Mutual Entropy: A Brief Survey

Copula-Based Mutual Information Measures and Mutual Entropy: A Brief Survey

Consistency of the central bank of the Republic of Türkiye’s policy stance with its discourses: a fuzzy logic analysis

ABSTRACT The purpose of the study is to use fuzzy logic methods to assess the stance of the Central Bank of the Republic of Türkiye (CBRT) towards monetary policy. These methods enable us to examine the coherence and consistency between the CBRT’s discourses and policy stance with its declarations covering the period 2011:09 - 2019:12. Key variables are included that effectively explain the proxy for the central bank’s policy stance by mutual information measure for variable selection. The policy stance of the CBRT is classified by fuzzy C-means clustering within the scope of the rule-based fuzzy logic model. This analysis allows us to group the policy stance as loose, moderate, and tight. Within the analysis timeframe, the CBRT maintained a rhetoric of consistent tightness in statements regarding its monetary policy stance. However, the only overlap occurred between June 2018 and July 2019, when the policy stance declared by the CBRT and the model results aligned. In the remaining periods, the policy stance declared by the CBRT does not correspond with the model outputs.

Hybrid Filtering and Probabilistic Techniques for Privacy-Preserving Community Detection in OSNs

Online Social Networks (OSNs) face the major challenge of protecting participant's privacy, due to the high dimensionality and volume of the data. In real-time social networks, where hundreds of personal details of people are shared every day, there remains a significant threat to privacy. Privacy preservation is challenging for a community detection problem due to the high computational complexity and memory requirements, especially in larger real-world OSN graphs. Although weighted nodes provide better results, as they allow capturing the frequencies of the values, the privacy preservation of sensitive attributes such as specific profiles becomes harder compared to these models. This problem can result in queries and subsequent learnings from social network profiles of specific individuals, which may be of personal, political or otherwise concern. Online social networks (OSNs) grapple with significant privacy challenges due to the extensive dimensions and vast quantities of data involved. This research fills the void in current privacy-preserving community detection methodologies, which face problems in computational complexity and memory usage in large-scale OSNs. The proposed framework seeks to bolster privacy preservation through a comprehensive multi-step process. Data filtering uses a blended data filter system to remove outliers and irrelevant data, thus enhancing the quality of the input data. Density-Oriented Clustering phase employs a density-oriented clustering model to identify communities, with each cluster representing a distinct community. Privacy Preservation component introduces a new privacy preservation technique for sensitive OSN attributes, surpassing existing k-anonymization methods. The developed density-based social network community detection model and its novel privacy-preserving scheme are evaluated using the datasets Yelp, Football, Zachary and Dolphin from the SNAP dataset. Experimental results on these datasets embed a comprehensive evaluation based on the order of each node and the graph-based networks, where each node is laden with weights as proximity values, indicating the semantic proximity between communities and individuals. The proposed framework employs the normalized mutual information (NMI), modularity (Q), Rand index and runtime measurements to demonstrate widespread advantages in the multi-dimensional functional space, including greater accuracy, cluster compatibility, and computational tractability over the existing prominent traditional models for OSNs.

Deep learning insights into non-universality in the halo mass function

ABSTRACT The abundance of dark matter haloes is a key cosmological probe in forthcoming galaxy surveys. The theoretical understanding of the halo mass function (HMF) is limited by our incomplete knowledge of the origin of non-universality and its cosmological parameter dependence. We present a deep-learning model which compresses the linear matter power spectrum into three independent factors which are necessary and sufficient to describe the $z=0$ HMF from the state-of-the-art Aemulus emulator to sub-per cent accuracy in a wCDM$+N_\mathrm{eff}$ parameter space. Additional information about growth history does not improve the accuracy of HMF predictions if the matter power spectrum is already provided as input, because required aspects of the former can be inferred from the latter. The three factors carry information about the universal and non-universal aspects of the HMF, which we interrogate via the information-theoretic measure of mutual information. We find that non-universality is captured by recent growth history after matter-dark-energy equality and $N_{\rm eff}$ for $M\sim 10^{13} \, \mathrm{M_\odot }\, h^{-1}$ haloes, and by $\Omega _{\rm m}$ for $M\sim 10^{15} \, \mathrm{M_\odot }\, h^{-1}$. The compact representation learnt by our model can inform the design of emulator training sets to achieve high emulator accuracy with fewer simulations.

Analyzing the Quality of Distorted Images by the Normalized Mutual Information Measure

This research explores how different types of distorting algorithms impact the Full-Reference image quality assessment, particularly when subjective quality evaluations are incorporated. We draw upon the TID2013 database, which contains 3000 images distorted by 24 distinct algorithms, in conjunction with Mean Opinion Scores (MOS) for quality ratings. We compare the results of Normalized Mutual Information (NMI) for image quality score with W2, based on Weibull distribution, the common PSNR similarity measure and MOS. We advocate for integrating of NMI into the repertoire of image quality assessment metrics.

Measurement of information content of Perovskite solar cell’s synthesis descriptors related to performance parameters

AbstractPerovskite solar cells (PSC) are formed by different layers composed of thin films of various materials, in which the properties of every thin layer affect the performance of the cell. The identification of those most relevant properties (or descriptors) has a significant impact on the optimization and cost reduction of the Perovskite solar cell. This relevance is typically evaluated by adjusting a model using subsets of features, but in the present work, we propose to use the mutual information measure to quantify the statistical association between input descriptors and Perovskite solar cell performance parameters (Voc, Jsc, FF, PCE). As a result, it is found that ion X is the factor that most impacts the performance of the solar cell. On the other hand, variables such as band gap, Perovskite layer thickness, and A and B ions are also important. In this work, we identify some of the most important factors affecting Perovskite solar cells’ performance, and it could help to improve the efficiency of Perovskite solar cells. In addition, this proposed method could also be applied to other types of functional coatings, thin films, and surfaces.

Strong Asymptotic Composition Theorems for Mutual Information Measures

We characterize the growth of the Sibson and Arimoto mutual informations and α-maximal leakage, of any order that is at least unity, between a random variable and a growing set of noisy, conditionally independent and identically-distributed observations of the random variable. Each of these measures increases exponentially fast to a limit that is order- and measure-dependent, with an exponent that is order- and measure-independent.

A computational linguistic methodology for assessing semiotic structure in prehistoric art and the meaning of southern Scandinavian Mesolithic ornamentation

Non-figurative prehistoric art is comparatively common yet challenging to decode and hence to understand. At the heart of the difficulty of assessing the presence of semiotic structure in prehistoric art is a lack of appropriate, replicable, and case-transferable methodologies. We here propose a novel approach derived from computational linguistics, in which k-skip-n-gram (skipgram) models and associated pointwise mutual information (PMI) measures are customised to the analysis of prehistoric art. In applying this methodology to a large corpus of portable art from the South Scandinavian Mesolithic, we demonstrate how the mutual relationship between individual motifs can be established. In the case of Mesolithic portable art, our analysis strongly suggests that there is no evident semiotic structure – this was likely not a form of proto-writing – but salient changes of motif occurrence over time are detectable. These changes are sensitive to changes in population density, structure, and connectedness, and may relate to increased territoriality in the Late Mesolithic. The method presented here is readily case-transferable and renders possible further linguistic and semiotic analyses of prehistoric art.

Private outsourced model predictive control via secure multi-party computation

Protecting data privacy is essential in developing practical cloud-based control systems to benefit from remote and distributed computation while avoiding the risks of disclosing sensitive information to potentially untrusted parties. This paper proposes a privacy-preserving framework to outsource the computation of model predictive control law to several untrusted cloud-based servers. A secret sharing scheme is employed to maintain the privacy of the system’s data in all the stages of the control loop in a secure multi-party computation environment. In this regard, a privacy-preserving algorithm is derived to securely implement a projected gradient method to solve the underlying optimization problem, without revealing private data to external eavesdroppers and curious cloud-based servers. The proposed method is solely based on secret sharing, and all computations can be performed securely by cloud servers without the need to designate target nodes or engage the system’s actuator in computing the intermediate steps. Therefore, the proposed method is less computationally demanding than the existing results based on homomorphic encryption and is applicable to systems with limited computing resources. Information-theoretic privacy assessments based on mutual information measures are provided. The efficacy of the proposed method is investigated on the cruise control problem of a freight train system.

The causality measure of partial mutual information from mixed embedding (PMIME) revisited.

The measure of partial mutual information from mixed embedding (PMIME) is an information theory-based measure to accurately identify the direct and directional coupling, termed Granger causality or simply causality, between the observed variables or subsystems of a high-dimensional dynamical and complex system, without any a priori assumptions about the nature of the coupling relationship. In its core, it is a forward selection procedure that aims to iteratively identify the lag-dependence structure of a given observed variable (response) to all the other observed variables (candidate drivers). This model-free approach is capable of detecting nonlinear interactions, abundantly present in real-world complex systems, and it was shown to perform well on multivariate time series of moderately high dimension. However, the PMIME presents some inefficiencies in its performance mainly when applied on strongly stochastic (linear or nonlinear) systems as it may falsely detect non-existent relationships. Moreover, and by construction, the measure cannot extract purely synergetic relationships present in a system. In the current work, the issue of false detections is addressed by introducing an improved resampling significance test and a procedure of rechecking the identified drivers (backward revision). Regarding the inability to detect synergetic relationships, the PMIME is further enhanced by checking pairs as candidate drivers for the response variable after having considered all drivers individually. The effects of these modifications are investigated in a systematic simulation study on properly designed systems involving strong stochasticity, regressor terms with synergetic effects, and a system dimension ranging from 3 to 30. The overall results of the simulations indicate that these modifications indeed improve the performance of PMIME and alleviate to a significant degree the issues of the original algorithm. Guidelines for balancing between accuracy and computational efficiency are also given, particularly relevant for real-world applications. Finally, the measure performance is investigated in the study of futures of various government bonds and stock market indices in the period around COVID-19 pandemic.