Abstract
In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target/source classifications and that are sufficient to describe the relationships among all PID lattices. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system’s entropy, hence it characterizes completely all the interdependencies in the system.
Highlights
Shannon’s mutual information [1] provides a well established, widely applicable tool to characterize the statistical relationship between two stochastic variables
non-source redundancy (NSR)( X : {Y; Z }) of the inputs about the output are plotted as a function of λ. (a) Since the output variable X copies the inputs (Y, Z ), all of SI ( X : {Y; Z }) can only come from the correlations between the inputs, which is reflected in NSR( X : {Y; Z }) being identically 0 for all values of λ. (b) For all values of λ > 0 we find NSR( X : {Y; Z }) > 0, i.e., there is a part of the redundancy SI ( X : {Y; Z })
This is compatible with our description of non-source redundancy as redundancy that is not related to the source correlations; NSR > 0 implies that the sources carry synergistic information about the target
Summary
Shannon’s mutual information [1] provides a well established, widely applicable tool to characterize the statistical relationship between two stochastic variables. By tracking how the PID information modes change across different lattices, we move beyond the partial perspective intrinsic to a single PID lattice and unveil the finer structure common to all PID lattices We find that this structure can be fully described in terms of a unique minimal set of seven information-theoretic quantities, which is invariant to different classifications of the variables. The second result is that the minimal set induces a unique nonnegative decomposition of the full joint entropy H ( X, Y, Z ) of the system This allows us to dissect completely the distribution of information of any trivariate system in a general way that is invariant with respect to the source/target classification of the variables. We briefly discuss how our methods might be extended to the analysis of systems with more than three variables
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
