Abstract This study investigates the optimization of broadband communication channel capacity through an integrative information-theoretic framework. Leveraging Shannon’s theory, it examines fundamental constraints such as bandwidth limitations, channel noise, modulation techniques, error correction mechanisms, and adaptive systems. A comprehensive literature review of 118 articles identified 18 critical enablers, which were evaluated by domain experts. The Fuzzy DEMATEL method was employed to prioritize enablers based on interdependencies and influence. Results indicate that Security Considerations, Channel Access Protocols, and Propagation Characteristics exert the most significant impact on capacity optimization. The findings offer a structured decision-making model for stakeholders, enabling efficient allocation of technological, infrastructural, and human resources. By bridging theoretical principles with practical implementation, this research provides actionable insights for academic researchers and industry practitioners in designing robust, high-capacity broadband systems. The integrative modeling approach advances the application of information theory in modern communication networks, supporting informed technology adoption and system integration.

Brain functional networks have studied amnestic mild cognitive impairment diagnosis. However, the abnormal patterns of amnestic mild cognitive impairment brain networks based on electroencephalography have not yet been resolved. In this study, a new method, the genuine low-order functional connectivity and high-order functional connection, integrates multilayer network information and dynamic network theory to analyze the characteristics of amnestic mild cognitive impairment function coupling. Meanwhile, the assessment framework combined dynamic brain functional networks and multidimensional metrics to diagnose amnestic mild cognitive impairment. Using the new method, this paper analyzed 28 amnestic mild cognitive impairment cases and 21 normal controls from clinical electroencephalography data. The results indicate that, except for the delta band, the connection strength of different amnestic mild cognitive impairment networks is lower than that of normal controls. The alpha and beta bands normal control's network metric greater than amnestic mild cognitive impairment showed significant differences (P < 0.05). Meanwhile, the significant difference in state entropy between the amnestic mild cognitive impairment and normal controls disappeared in the delta band of the 10 s window and the beta band of the 2s window in the dynamic high-order functional connection. The amnestic mild cognitive impairment brain functional network exhibits abnormal features. Meanwhile, the alpha and beta bands could be sensitive to diagnose amnestic mild cognitive impairment.

In this article, a modified mutual information maximization (InfoMax) framework, named channel capacity maximization (CapMax), is proposed and applied to learn informative representations for dynamic networks with time-varying topology and/or time-evolving node attributes. The CapMax is based on the network information theory for multiuser communication, where the representation model is treated as a multiaccess communication channel with memory and feedback. Without requirements of the backbone structure, the learning objective of our CapMax is maximizing the channel capacity, which is measured by directed information (DI) rather than mutual information. For efficient implementation, we design an estimator of the channel capacity through the combination of graph neural networks (GNNs) and recurrent neural networks (RNNs). Under some mild conditions, we theoretically prove that DI is a better measure than mutual information in capturing useful information. The experiments are conducted on multiple real-world dynamic network datasets, and the outperformance of our CapMax on different backbone models on link detection and prediction validates the effectiveness of modeling the representation model as a communication channel.

Music has a complex structure that expresses emotion and conveys information. Humans process that information through imperfect cognitive instruments that produce a gestalt, smeared version of reality. How can we quantify the information contained in a piece of music? Further, what is the information inferred by a human, and how does that relate to (and differ from) the true structure of a piece? To tackle these questions quantitatively, we present a framework to study the information conveyed in a musical piece by constructing and analyzing networks formed by notes (nodes) and their transitions (edges). Using this framework, we analyze music composed by J. S. Bach through the lens of network science, information theory, and statistical physics. Regarded as one of the greatest composers in the Western music tradition, Bach's work is highly mathematically structured and spans a wide range of compositional forms, such as fugues and choral pieces. Conceptualizing each composition as a network of note transitions, we quantify the information contained in each piece and find that different kinds of compositions can be grouped together according to their information content and network structure. Moreover, using a model for how humans infer networks of information, we find that the music networks communicate large amounts of information while maintaining small deviations of the inferred network from the true network, suggesting that they are structured for efficient communication of information. We probe the network structures that enable this rapid and efficient communication of information—namely, high heterogeneity and strong clustering. Taken together, our findings shed light on the information and network properties of Bach's compositions. More generally, our simple framework serves as a stepping stone for exploring further musical complexities, creativity, and questions therein. Published by the American Physical Society 2024

The optimal transport problem studies how to transport one measure to another in the most cost-effective way and has wide range of applications from economics to machine learning. In this paper, we introduce and study an information constrained variation of this problem. Our study yields a strengthening and generalization of Talagrand’s celebrated transportation cost inequality. Following Marton’s approach, we show that the new transportation cost inequality can be used to recover old and new concentration of measure results. Finally, we provide an application of this new inequality to network information theory. We show that it can be used to recover almost immediately a recent solution to a long-standing open problem posed by Cover regarding the capacity of the relay channel.

Gaussian networks are fundamental objects in network information theory. Here many senders and receivers are connected by physically motivated Gaussian channels, while auxiliary Gaussian components, such as Gaussian relays, are entailed. Whilst the theoretical backbone of classical Gaussian networks is well established, the quantum analog is yet immature. Here, we theoretically tackle composable security of arbitrary Gaussian quantum networks, with generally untrusted nodes, in the finite-size regime. We put forward a general methodology for parameter estimation, which is only based on the data shared by the remote end-users. Taking a chain of identical quantum links as an example, we further demonstrate our study. Additionally, we find that the key rate of a quantum amplifier-assisted chain can ideally beat the fundamental repeaterless limit with practical block sizes. However, this objective is practically questioned leading the way to future network/chain designs.

Many biological phenomena or social events critically depend on how information evolves in complex networks. However, a general theory to characterize information evolution is yet absent. Consequently, numerous unknowns remain about the mechanisms underlying information evolution. Among these unknowns, a fundamental problem, being a seeming paradox, lies in the coexistence of local randomness, manifested as the stochastic distortion of information content during individual-individual diffusion, and global regularity, illustrated by specific non-random patterns of information content on the network scale. Here, we attempt to formalize information evolution and explain the coexistence of randomness and regularity in complex networks. Applying network dynamics and information theory, we discover that a certain amount of information, determined by the selectivity of networks to the input information, frequently survives from random distortion. Other information will inevitably experience distortion or dissipation, whose speeds are shaped by the diversity of information selectivity in networks. The discovered laws exist irrespective of noise, but noise accounts for disturbing them. We further demonstrate the ubiquity of our discovered laws by analyzing the emergence of neural tuning properties in the primary visual and medial temporal cortices of animal brains and the emergence of extreme opinions in social networks.

Activity-dependent self-organization plays an important role in the formation of specific and stereotyped connectivity patterns in neural circuits. By combining neuronal cultures, and tools with approaches from network neuroscience and information theory, we can study how complex network topology emerges from local neuronal interactions. We constructed effective connectivity networks using a transfer entropy analysis of spike trains recorded from rat embryo dissociated hippocampal neuron cultures between 6 and 35 days in vitro to investigate how the topology evolves during maturation. The methodology for constructing the networks considered the synapse delay and addressed the influence of firing rate and population bursts as well as spurious effects on the inference of connections. We found that the number of links in the networks grew over the course of development, shifting from a segregated to a more integrated architecture. As part of this progression, three significant aspects of complex network topology emerged. In agreement with previous in silico and in vitro studies, a small-world architecture was detected, largely due to strong clustering among neurons. Additionally, the networks developed in a modular topology, with most modules comprising nearby neurons. Finally, highly active neurons acquired topological characteristics that made them important nodes to the network and integrators of modules. These findings leverage new insights into how neuronal effective network topology relates to neuronal assembly self-organization mechanisms.

Activity-dependent self-organization plays an important role in the formation of specific and stereotyped connectivity patterns in neural circuits. By combining neuronal cultures, and tools with approaches from network neuroscience and information theory, we can study how complex network topology emerges from local neuronal interactions. We constructed effective connectivity networks using a transfer entropy analysis of spike trains recorded from rat embryo dissociated hippocampal neuron cultures between 6 and 35 days in vitro to investigate how the topology evolves during maturation. The methodology for constructing the networks considered the synapse delay and addressed the influence of firing rate and population bursts as well as spurious effects on the inference of connections. We found that the number of links in the networks grew over the course of development, shifting from a segregated to a more integrated architecture. As part of this progression, three significant aspects of complex network topology emerged. In agreement with previous in silico and in vitro studies, a small-world architecture was detected, largely due to strong clustering among neurons. Additionally, the networks developed in a modular topology, with most modules comprising nearby neurons. Finally, highly active neurons acquired topological characteristics that made them important nodes to the network and integrators of modules. These findings leverage new insights into how neuronal effective network topology relates to neuronal assembly self-organization mechanisms.

In the last two decades, there has been an explosion of interest in modeling the brain as a network, where nodes correspond variously to brain regions or neurons, and edges correspond to structural or statistical dependencies between them. This kind of network construction, which preserves spatial, or structural, information while collapsing across time, has become broadly known as “network neuroscience.” In this work, we provide an alternative application of network science to neural data: network-based analysis of non-linear time series and review applications of these methods to neural data. Instead of preserving spatial information and collapsing across time, network analysis of time series does the reverse: it collapses spatial information, instead preserving temporally extended dynamics, typically corresponding to evolution through some kind of phase/state-space. This allows researchers to infer a, possibly low-dimensional, “intrinsic manifold” from empirical brain data. We will discuss three methods of constructing networks from nonlinear time series, and how to interpret them in the context of neural data: recurrence networks, visibility networks, and ordinal partition networks. By capturing typically continuous, non-linear dynamics in the form of discrete networks, we show how techniques from network science, non-linear dynamics, and information theory can extract meaningful information distinct from what is normally accessible in standard network neuroscience approaches.

Modeling communication dynamics in the brain is a key challenge in network neuroscience. We present here a framework that combines two measurements for any system where different communication processes are taking place on top of a fixed structural topology: path processing score (PPS) estimates how much the brain signal has changed or has been transformed between any two brain regions (source and target); path broadcasting strength (PBS) estimates the propagation of the signal through edges adjacent to the path being assessed. We use PPS and PBS to explore communication dynamics in large-scale brain networks. We show that brain communication dynamics can be divided into three main “communication regimes” of information transfer: absent communication (no communication happening); relay communication (information is being transferred almost intact); and transducted communication (the information is being transformed). We use PBS to categorize brain regions based on the way they broadcast information. Subcortical regions are mainly direct broadcasters to multiple receivers; Temporal and frontal nodes mainly operate as broadcast relay brain stations; visual and somatomotor cortices act as multichannel transducted broadcasters. This work paves the way toward the field of brain network information theory by providing a principled methodology to explore communication dynamics in large-scale brain networks.

ABSTRACTA primary aspect of applied marine ecology assesses how food webs change in response to ecosystem disturbances. In 2010, the drilling rig Deepwater Horizon (DWH) discharged ~3.19 million barrels of crude oil into the northern Gulf of Mexico. The spill, followed by widespread dispersant application to enhance oil degradation, represented a significant anthropogenic disturbance in the region. We created network models of four multi‐year periods, to represent the acute and chronic food web responses to the DWH spill. Using ecological network analysis (ENA) and information theory, we compared multiple food web metrics among these periods in the context of food web resilience theory. This analysis was conducted at three levels of hierarchical organization: whole ecosystem, nekton community, and individual nekton taxa. We analyzed how individual taxa contribute to resilience of the food web with a novel informational index: Redundancy/Ascendency. Apparent responses to the disturbance differed across hierarchical levels. Some metrics dependent on biomass change and flow distribution temporarily increased during the years immediately following the discharge and subsequently returned to pre‐DWH levels. Metrics of energy flow linked to primary production rose in the last two periods, perhaps reflecting eutrophication. Other metrics changed little or had no obviously explainable patterns. Overall, our results indicate the nektonic food web in this region is flexible to disturbance and likely has redundant energy pathways explaining the reported ecosystem resilience to the DWH spill. We show that an ENA, when applied to multiple levels of ecosystem hierarchy, may aid understanding of marine food web resilience.

In this article, we aim to develop a unified view of causal and non-causal feature selection methods. The unified view will fill in the gap in the research of the relation between the two types of methods. Based on the Bayesian network framework and information theory, we first show that causal and non-causal feature selection methods share the same objective. That is to find the Markov blanket of a class attribute, the theoretically optimal feature set for classification. We then examine the assumptions made by causal and non-causal feature selection methods when searching for the optimal feature set, and unify the assumptions by mapping them to the restrictions on the structure of the Bayesian network model of the studied problem. We further analyze in detail how the structural assumptions lead to the different levels of approximations employed by the methods in their search, which then result in the approximations in the feature sets found by the methods with respect to the optimal feature set. With the unified view, we can interpret the output of non-causal methods from a causal perspective and derive the error bounds of both types of methods. Finally, we present practical understanding of the relation between causal and non-causal methods using extensive experiments with synthetic data and various types of real-world data.

A fundamental tool to prove inner bounds in classical network information theory is the so-called ‘conditional joint typicality lemma’. In addition to the lemma, one often uses unions and intersections of typical sets in the inner bound arguments without so much as giving them a second thought. These arguments do not work in the quantum setting. This bottleneck shows up in the fact that so-called ‘simultaneous decoders’, as opposed to ‘successive cancellation decoders’, are known for very few channels in quantum network information theory. Another manifestation of this bottleneck is the lack of so-called ‘simultaneous smoothing’ theorems for quantum states. In this paper, we overcome the bottleneck by proving for the first time a one-shot quantum joint typicality lemma with robust union and intersection properties. To do so we develop two novel tools in quantum information theory, which may be of independent interest. The first tool is a simple geometric idea called tilting, which increases the angles between a family of subspaces in orthogonal directions. The second tool, called smoothing and augmentation, is a way of perturbing a multipartite quantum state such that the partial trace over any subset of registers does not increase the operator norm much. Our joint typicality lemma allows us to construct simultaneous quantum decoders for many multiterminal quantum channels. It provides a powerful tool to extend many results in classical network information theory to the one-shot quantum setting.

We introduce a fundamental lemma called the Poisson matching lemma, and apply it to prove one-shot achievability results for various settings, namely channels with state information at the encoder, lossy source coding with side information at the decoder, joint source-channel coding, broadcast channels, distributed lossy source coding, multiple access channels, channel resolvability and wiretap channels. Our one-shot bounds improve upon the best known one-shot bounds in most of the aforementioned settings (except multiple access channels, channel resolvability and wiretap channels, where we recover bounds comparable to the best known bounds), with shorter proofs in some settings even when compared to the conventional asymptotic approach using typicality. The Poisson matching lemma replaces both the packing and covering lemmas, greatly simplifying the error analysis. This paper extends the work of Li and El Gamal on Poisson functional representation, which mainly considered variable-length source coding settings, whereas this paper studies fixed-length settings, and is not limited to source coding, showing that the Poisson functional representation is a viable alternative to typicality for most problems in network information theory.

International audience

We prove new inner bounds for several multiterminal channels with classical inputs and quantum outputs. Our inner bounds are all proved in the one-shot setting and are natural analogues of the best classical inner bounds for the respective channels. For some of these channels, similar quantum inner bounds were unknown even in the asymptotic independent and identically distributed setting. We prove our inner bounds by appealing to a new classical–quantum joint typicality lemma established in a companion paper. This lemma allows us to lift to the quantum setting many inner bound proofs for classical multiterminal channels that use intersections and unions of typical sets.

Network neuroscience employs graph theory to investigate the human brain as a complex network, and derive generalizable insights about the brain’s network properties. However, graph-theoretical results obtained from network construction pipelines that produce idiosyncratic networks may not generalize when alternative pipelines are employed. This issue is especially pressing because a wide variety of network construction pipelines have been employed in the human network neuroscience literature, making comparisons between studies problematic. Here, we investigate how to produce networks that are maximally representative of the broader set of brain networks obtained from the same neuroimaging data. We do so by minimizing an information-theoretic measure of divergence between network topologies, known as the portrait divergence. Based on functional and diffusion MRI data from the Human Connectome Project, we consider anatomical, functional, and multimodal parcellations at three different scales, and 48 distinct ways of defining network edges. We show that the highest representativeness can be obtained by using parcellations in the order of 200 regions and filtering functional networks based on efficiency-cost optimization—though suitable alternatives are also highlighted. Overall, we identify specific node definition and thresholding procedures that neuroscientists can follow in order to derive representative networks from their human neuroimaging data.

Information theory, synchronization and topological order in complete dynamical networks of discontinuous maps

Aggressive frequency reuse alleviates the frequency-spectrum shortage in multibeam satellite systems targeting multi-Terabits-per-second throughput but creates harsh co-channel interference (CCI) environment. For forward-link, state-of-the-art receivers at user terminals assume memoryless CCI to avoid exponential increase in complexity. However, memory effects of CCI are inevitable, arising when co-channel signals combine asynchronously. First, we provide analytical characterization of CCI accounting for memory, a formulation that captures synchronization impairments. Then, low-complexity receivers, capable of realizing full frequency reuse, are developed to compensate for spatial and temporal CCI profiles, with computational complexity increasing linearly in the memory span. This is achieved by our novel soft-in soft-out, iterative divide-and-conquer (IDAC) paradigm, decomposing interference into smaller sets depending on intensity. A set is created from weak interferers which are not decoded but considered as thermal noise. The interferers which are decoded are further split into a set that subtracts strong ones and another set addressing intense interferers in optimal-Bayesian fashion. Extensive numerical studies reveal that IDAC architecture offers lossless compensation of dominant CCI with memory for beam-edge terminals, when rate coordination is guided by network information theory. Further, its performance is superior compared with approaches using iterative subtractive cancellation, commonly employed for return-link, random-access satellite applications.