Perspective Of Information Theory Research Articles

Spatial-Temporal Dynamic Hypergraph Information Bottleneck for Brain Network Classification.

Recently, Graph Neural Networks (GNNs) have gained widespread application in automatic brain network classification tasks, owing to their ability to directly capture crucial information in non-Euclidean structures. However, two primary challenges persist in this domain. First, within the realm of clinical neuro-medicine, signals from cerebral regions are inevitably contaminated with noise stemming from physiological or external factors. The construction of brain networks heavily relies on set thresholds and feature information within brain regions, making it susceptible to the incorporation of such noises into the brain topology. Additionally, the static nature of the artificially constructed brain network's adjacent structure restricts real-time changes in brain topology. Second, mainstream GNN-based approaches tend to focus solely on capturing information interactions of nearest neighbor nodes, overlooking high-order topology features. In response to these challenges, we propose an adaptive unsupervised Spatial-Temporal Dynamic Hypergraph Information Bottleneck (ST-DHIB) framework for dynamically optimizing brain networks. Specifically, adopting an information theory perspective, Graph Information Bottleneck (GIB) is employed for purifying graph structure, and dynamically updating the processed input brain signals. From a graph theory standpoint, we utilize the designed Hypergraph Neural Network (HGNN) and Bi-LSTM to capture higher-order spatial-temporal context associations among brain channels. Comprehensive patient-specific and cross-patient experiments have been conducted on two available datasets. The results demonstrate the advancement and generalization of the proposed framework.

STAR-RIS-assisted key generation method in quasi-static environment

In this paper, we introduce a new method to solve the low-entropy problem in a quasi-static environment. We utilize a simultaneous transmission and reflection reconfigurable intelligent surface (STAR-RIS) that transmits and reflects incoming signals to users located on different sides of the surface. Specifically, we design the flow of multi-user channel probing when STAR-RIS is assisted, analyze and derive the analytical expression of the secret key capacity (SKC) of STAR-RIS-assisted system from the perspective of information theory, and finally analyze how to maximize the SKC and sum of all users under three different working protocols of STAR-RIS. The simulation results show that: (1) STAR-RIS can achieve better SKC performance than the conventional reflection/transmission RIS; and (2) the SKC is determined by the number of STAR-RIS elements, correlation coefficient, pilot length and RIS-reflected channel quality, and the time switching protocol can obtain higher SKC than the energy splitting protocol and mode switching protocol.

Visual information and appearance: The impact of visual attributes of user-generated photos on review helpfulness

Images have become integral to consumers’ sharing of consumption experiences due to their abilities of carrying rich and vivid information. Drawing from the perspective of information theory and the elaboration likelihood model (ELM), this study investigates impacts of visual information and visual appearance of user-generated photos (UGPs) on customers’ perceived review helpfulness. We utilize deep learning techniques to calculate the breadth and depth of photos (visual information), and evaluate the aesthetic value of the photos (visual appearance). By collecting a substantial amount of review, reviewer and restaurant information from the Yelp platform, our results demonstrate that the visual breadth and visual depth of review photos have a significant positive impact on review helpfulness. Notably, reviewer status and review length moderate these effects. These insights offer valuable strategies for both restaurant managers and online restaurant platforms regarding UGP management.

Security enhancement by intelligent reflecting surfaces for visible light communications

As a revolutionizing technology, intelligent reflecting surfaces (IRS) are expected to enhance the physical layer security (PLS) by constructing a controllable wireless environment. In this paper, we consider an IRS-aided visible light communication (VLC) system with a legitimate user and a malicious eavesdropper, in which the IRS is implemented by an intelligent rotatable mirror array. Our objective is to provide a desired optical reflecting environment with enriched spatial degrees of freedom by jointly optimizing the rotation angles of arrayed mirrors. First, from the perspective of information theory, we derive the achievable secrecy rate for the IRS-aided VLC system. Then, we formulate an achievable secrecy rate maximization problem with the orientations of mirror elements as optimization variables. Since the effect of mirror elements’ orientations on the IRS channel gain makes the formulated problem hard to solve, we propose a reflected spot search (RSS) method to transform the original problem into one with reduced dimension of optimization variables. Besides, bearing in mind that the VLC system has the dual task of communication and illumination, we define and analyse the illumination uniformity of the IRS-aided secure VLC system. Our simulation results show that by utilizing the proposed method, the VLC system can achieve significant secrecy enhancement, especially when the eavesdropper is close to the legitimate user.

Land surface temperature and transboundary air pollution: a case of Bangkok Metropolitan Region

In a rapidly urbanizing world, heavy air pollution and increasing surface temperature pose significant threats to human health and lives, especially in densely populated cities. In this study, we took an information theory perspective to investigate the causal relationship between diel land surface temperature (LST) and transboundary air pollution (TAP) from 2003 to 2020 in the Bangkok Metropolitan Region (BMR), which includes Bangkok Metropolis and its five adjacent provinces. We found an overall increasing trend of LST over the study region, with the mean daytime LST rising faster than nighttime LST. Evident seasonal variations showed high aerosol optical depth (AOD) loadings during the dry period and low loadings at the beginning of the rainy season. Our study revealed that TAP affected diel surface temperature in Bangkok Metropolis significantly. Causality tests show that air pollutants of two adjacent provinces west of Bangkok, i.e., Nakhon Pathom and Samut Sakhon, have a greater influence on the LST of Bangkok than other provinces. Also, the bidirectional relationship indicates that air pollution has a greater impact on daytime LST than nighttime LST. While LST has an insignificant influence on AOD during the daytime, it influences AOD significantly at night. Our study offers a new approach to understanding the causal impact of TAP and can help policymakers to identify the most relevant locations that cause pollution, leading to appropriate planning and management.

How does node centrality in a financial network affect asset price prediction?

In complex financial networks, systemically important nodes usually play crucial roles. Asset price forecasting is important for describing the evolution of a financial network. Naturally, the question arises as to whether node centrality affects the effectiveness of price forecasting. To explore this, we examine networks composed of major global assets and investigate how node centrality affects price forecasting using a hybrid random forest algorithm. Our findings reveal two counterintuitive phenomena: (i) factors with low centrality usually have better prediction ability, and (ii) nodes with low centrality can be predicted more accurately in direction. These unexpected observations can be explained from the perspective of information theory. Moreover, our research suggests a criterion for factor selection: when predicting an asset price in a complex system, factors with low centrality should be selected rather than only factors with high centrality.

BVT-IMA: Binary Vision Transformer with Information-Modified Attention

As a compression method that can significantly reduce the cost of calculations and memories, model binarization has been extensively studied in convolutional neural networks. However, the recently popular vision transformer models pose new challenges to such a technique, in which the binarized models suffer from serious performance drops. In this paper, an attention shifting is observed in the binary multi-head self-attention module, which can influence the information fusion between tokens and thus hurts the model performance. From the perspective of information theory, we find a correlation between attention scores and the information quantity, further indicating that a reason for such a phenomenon may be the loss of the information quantity induced by constant moduli of binarized tokens. Finally, we reveal the information quantity hidden in the attention maps of binary vision transformers and propose a simple approach to modify the attention values with look-up information tables so that improve the model performance. Extensive experiments on CIFAR-100/TinyImageNet/ImageNet-1k demonstrate the effectiveness of the proposed information-modified attention on binary vision transformers.

Expand-and-Quantize: Unsupervised Semantic Segmentation Using High-Dimensional Space and Product Quantization

Unsupervised semantic segmentation (USS) aims to discover and recognize meaningful categories without any labels. For a successful USS, two key abilities are required: 1) information compression and 2) clustering capability. Previous methods have relied on feature dimension reduction for information compression, however, this approach may hinder the process of clustering. In this paper, we propose a novel USS framework called Expand-and-Quantize Unsupervised Semantic Segmentation (EQUSS), which combines the benefits of high-dimensional spaces for better clustering and product quantization for effective information compression. Our extensive experiments demonstrate that EQUSS achieves state-of-the-art results on three standard benchmarks. In addition, we analyze the entropy of USS features, which is the first step towards understanding USS from the perspective of information theory.

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Self-Supervised Learning (SSL) methods harness the concept of semantic invariance by utilizing data augmentation strategies to produce similar representations for different deformations of the same input. Essentially, the model captures the shared information among multiple augmented views of samples, while disregarding the non-shared information that may be beneficial for downstream tasks. To address this issue, we introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks, to make the learned representations more comprehensive. Specifically, we update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features. Additionally, guided by the constrained extraction of features using maximum entropy coding, the self-supervised learning model learns more comprehensive features on top of learning consistent features. In addition, we provide theoretical support for our proposed method from information theory and causal counterfactual perspective. Experimental results show that our method achieves significant improvement in classification, object detection and semantic segmentation tasks on multiple benchmark datasets.

Multi-view graph pooling with coarsened graph disentanglement

Multi-view graph pooling utilizes information from multiple perspectives to generate a coarsened graph, exhibiting superior performance in graph-level tasks. However, existing methods mainly focus on the types of multi-view information to improve graph pooling operations, lacking explicit control over the pooling process and theoretical analysis of the relationships between views. In this paper, we rethink the current paradigm of multi-view graph pooling from an information theory perspective, subsequently introducing GDMGP, an innovative method for multi-view graph pooling derived from the principles of graph disentanglement. This approach effectively simplifies the original graph into a more structured, disentangled coarsened graph, enhancing the clarity and utility of the graph representation. Our approach begins with the design of a novel view mapper that dynamically integrates the node and topology information of the original graph. This integration enhances its information sufficiency. Next, we introduce a view fusion mechanism based on conditional entropy to accurately regulate the task-relevant information in the views, aiming to minimize information loss in the pooling process. Finally, to further enhance the expressiveness of the coarsened graph, we disentangle the fused view into task-relevant and task-irrelevant subgraphs through mutual information minimization, retaining the task-relevant subgraph for downstream tasks. We theoretically demonstrate that the performance of the coarsened graph generated by our GDMGP is superior to that of any single input view. The effectiveness of GDMGP is further validated by experimental results on seven public datasets.

Indistinguishability of Identical Bosons from a Quantum Information Theory Perspective.

Using tools from quantum information theory, we present a general theory of indistinguishability of identical bosons in experiments consisting of passive linear optics followed by particle number detection. Our results do neither rely on additional assumptions on the input state of the interferometer, such as, for instance, a fixed mode occupation, nor on any assumption on the degrees of freedom that potentially make the particles distinguishable. We identify the expectation value of the projector onto the N-particle symmetric subspace as an operationally meaningful measure of indistinguishability, and derive tight lower bounds on it that can be efficiently measured in experiments. Moreover, we present a consistent definition of perfect distinguishability and characterize the corresponding set of states. In particular, we show that these states are diagonal in the computational basis up to a permutationally invariant unitary. Moreover, we find that convex combinations of states that describe partially distinguishable and perfectly indistinguishable particles can lead to perfect distinguishability, which itself is not preserved under convex combinations.

Frequency properties of channeled spectropolarimetry: an information theory perspective.

Channeled spectropolarimetry (CSP) has emerged as a notable technique due to its unique capacity to instantaneously measure either the polarization state of light or the Mueller matrix of a sample over a broad spectral range. Leveraging the quasi-linear relation between phase retardances of thick birefringent retarders and wavenumber, the target signal undergoes wavelength encoding. For the first time, we present a theoretical framework for the general CSP from a perspective of information theory. This framework comprehensively addresses the frequency properties of CSP, encompassing signal bandwidth, modulation frequency, sampling relationships, and filter window width during the demodulation process. Drawing from the frequency properties of CSP, we establish a theoretical foundation that informs the design of versatile CSPs and evaluates their measurement capabilities. Simulations for both Stokes CSP and Mueller CSP validate the efficacy of the proposed approach.

A Survey of Physical Layer Secret Key Generation Enhanced by Intelligent Reflecting Surface

As wireless communication scenarios grow more complicated, security issues are becoming increasingly prominent and severe. In the Internet of Things and vehicle-to-everything scenarios, conventional cryptographic technology faces numerous challenges. These include difficulties in secret key distribution and management, low update rates of secret keys, and vulnerability to quantum attacks. Physical layer secret key generation is considered a promising solution to security issues. The perfect secrecy proposed by Shannon can be achieved by combining secret key generation and the one-time pad when the length of secret keys is equal to that of plaintext. Hence, it is important to increase secret key generation rates. Intelligent reflecting surfaces demonstrate great advantages in improving the secret key generation performance. This paper provides a comprehensive review of current research efforts related to secret key generation assisted by intelligent reflecting surfaces, which is divided into three main categories: introducing the randomness of intelligent reflecting surfaces, optimizing the reflecting coefficients, and designing probing protocols. Comparative results of existing optimization approaches are provided and discussed. Furthermore, we emphasize the significance of selecting a random source of secret key generation from the perspective of information theory. Finally, two significant application scenarios, the Industrial Internet of Things and vehicle-to-everything, are discussed, and some challenges and opportunities are presented.

Strategies of Dealing with Redundancy in Chinese-to-English Simultaneous Interpreting from the Perspective of Information Theory

Strategies of Dealing with Redundancy in Chinese-to-English Simultaneous Interpreting from the Perspective of Information Theory

Understanding the Quality of User Experience in Telepresence Systems from an Information Theory Perspective

Understanding the Quality of User Experience in Telepresence Systems from an Information Theory Perspective

Brain network classification based on dynamic graph attention information bottleneck

Background and ObjectivesGraph neural networks (GNN) have demonstrated remarkable encoding capabilities in the context of brain network classification tasks. They excel at uncovering hidden static connections between brain states. However, brain network signals can be influenced by physiological traits and external variables during clinical detection, resulting in noisy brain graphs. Additionally, many existing algorithms for brain networks primarily focus on static topologies determined by threshold-based criteria, thereby overlooking the real-time variability in brain channel connectivity. These sources of noise and the persistence of static structures inevitably hinder the effective exchange of information during brain network computations. MethodsTo address these challenges, we propose a novel framework called the dynamic graph attention information bottleneck (DGAIB). This framework is designed to dynamically enhance the input raw brain graph structure from the perspective of information theory and graph theory. First, we employ the Spearman function to construct a raw graph. Then, we use a graph information bottleneck (GIB) to optimize the internal graph connections by selectively masking redundant feature embeddings. Finally, we enhance the feature aggregation of each brain state by utilizing a graph attention network (GAT), which promotes improved information exchange among distinct brain regions within the model. These processed representations serve as input for subsequent classification tasks. Experiment and ResultsWe systematically evaluated the robustness and generalizability of our proposed framework through a series of experiments. This evaluation included patient-specific experiments using the electroencephalography (EEG)-based CHB-MIT dataset and cross-patient experiments leveraging the functional magnetic resonance imaging (fMRI)-based ABIDE-I dataset from multiple perspectives.

RIS-Aided Multiuser MIMO-OFDM With Linear Precoding and Iterative Detection: Analysis and Optimization

In this paper, we consider a reconfigurable intelligent surface (RIS) aided uplink multiuser multi-input multi-output (MIMO) orthogonal frequency division multiplexing (OFDM) system, where the receiver is assumed to conduct low-complexity iterative detection. We aim to minimize the total transmit power by jointly designing the precoder of the transmitter and the passive beamforming of the RIS. This problem can be tackled from the perspective of information theory. But this information-theoretic approach may involve prohibitively high complexity since the number of rate constraints that specify the capacity region of the uplink multiuser channel is exponential in the number of users. To avoid this difficulty, we formulate the design problem of the iterative receiver under the constraints of a maximal iteration number and target bit error rates of users. To tackle this challenging problem, we propose a groupwise successive interference cancellation (SIC) optimization approach, where the signals of users are decoded and canceled in a group-by-group manner. We present a heuristic user grouping strategy, and resort to the alternating optimization technique to iteratively solve the precoding and passive beamforming sub-problems. Specifically, for the precoding sub-problem, we employ fractional programming to convert it to a convex problem; for the passive beamforming sub-problem, we adopt successive convex approximation to deal with the unit-modulus constraints of the RIS. We show that the proposed groupwise SIC approach has significant advantages in both performance and computational complexity, as compared with the counterpart approaches.

Self-Information Domain-Based Neural CSI Compression With Feature Coupling

Deep learning (DL)-based channel state information (CSI) feedback methods compressed the CSI matrix by exploiting its delay and angle features straightforwardly, while the measure in terms of information contained in the CSI matrix has rarely been considered. Based on this observation, we introduce self-information as an informative CSI representation from the perspective of information theory, which reflects the amount of information of the original CSI matrix in an explicit way. Then, a novel DL-based network is proposed for temporal CSI compression in the self-information domain, namely SD-CsiNet. The proposed SD-CsiNet projects the raw CSI onto a self-information matrix in the newly-defined self-information domain, extracts both temporal and spatial features of the self-information matrix, and then couples these two features for effective compression. Experimental results verify the effectiveness of the proposed SD-CsiNet by exploiting the self-information of CSI. Particularly for compression ratios <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1/8$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$1/16$</tex-math></inline-formula> , the SD-CsiNet respectively achieves 7.17 dB and 3.68 dB performance gains compared to state-of-the-art methods.

Структурно-семантические модели простых повторов в английской разговорной речи

The aim of the study is to identify the models of sentences with duplication of sentence composition elements presented in the works of American playwrights. The paper analyzes the main typical models of simple repetitions without explication of additional components in the sentence structure based on the notion of informative-syntagmatic duplication proposed by Professor M. Ya. Blokh within the theory of paradigmatic syntax. The study makes it possible to take a fresh look at the problem of the classical rhetorical figure of repetition, considering it from the perspective of grammar and information theory. The study is novel in that it is the first to highlight the models of simple repetitions at the sentence level and assess their functionality in question-and-answer interaction. As a result of the analysis, it was found that despite the emerging structural and semantic redundancy of the repetition models without modification of duplicated elements, it gives the utterance an emotionally expressive content, serves as a means of logical emphasis for important information and allows solving various stylistic tasks.

Information theory-guided heuristic progressive multi-view coding

Multi-view representation learning aims to capture comprehensive information from multiple views of a shared context. Recent works intuitively apply contrastive learning to different views in a pairwise manner, which is still scalable: view-specific noise is not filtered in learning view-shared representations; the fake negative pairs, where the negative terms are actually within the same class as the positive, and the real negative pairs are coequally treated; evenly measuring the similarities between terms might interfere with optimization. Importantly, few works study the theoretical framework of generalized self-supervised multi-view learning, especially for more than two views. To this end, we rethink the existing multi-view learning paradigm from the perspective of information theory and then propose a novel information theoretical framework for generalized multi-view learning. Guided by it, we build a multi-view coding method with a three-tier progressive architecture, namely Information theory-guided heuristic Progressive Multi-view Coding (IPMC). In the distribution-tier, IPMC aligns the distribution between views to reduce view-specific noise. In the set-tier, IPMC constructs self-adjusted contrasting pools, which are adaptively modified by a view filter. Lastly, in the instance-tier, we adopt a designed unified loss to learn representations and reduce the gradient interference. Theoretically and empirically, we demonstrate the superiority of IPMC over state-of-the-art methods.