Information-theoretic Analysis Research Articles

A Simple Cooperative Diversity Method Based on Deep-Learning-Aided Relay Selection

Opportunistic relay selection (ORS) has been recognized as a simple but efficient method for mobile nodes to achieve cooperative diversity in slow fading channels. However, the wrong selection of the best relay arising from outdated channel state information (CSI) in fast time-varying channels substantially degrades its performance. With the proliferation of high-mobility applications and the adoption of higher frequency bands in 5G and beyond systems, the problem of outdated CSI will become more serious. Therefore, the design of a novel cooperative method that is applicable to not only slow fading but also fast fading is increasingly of importance. To this end, we develop and analyze a deep-learning-aided cooperative method coined predictive relay selection (PRS) in this article. It can remarkably improve the quality of CSI through fading channel prediction while retaining the simplicity of ORS by selecting a single opportunistic relay so as to avoid the complexity of multi-relay coordination and synchronization. Information-theoretic analysis and numerical results in terms of outage probability and channel capacity reveal that PRS achieves full diversity gain in slow fading wireless environments and substantially outperforms the existing schemes in fast fading channels.

A new look at universals and specificities in the songs of humpback whales.

In this issue's featured article, Mercado and Perazio (2021) describe songs of humpback whales using acoustic qualities, in part to determine the degree of similarity in songs across time and space that have not been revealed by information-theoretic analyses. They are particularly interested in evaluating alternative explanations of song variations in humpback whales. They argue that if humpback whales' songs "are . . . transmitted through acoustic contact followed by imitation" (p. 29) then (a) songs of populations not in acoustic contact should diverge, (b) songs of the same population should diverge increasingly over time, and (c) song forms separated by multiple decades either within or across populations should be dissimilar. Alternatively, acoustic similarities in song structure across populations and/or across decades in the same population would challenge the hypothesis that socially mediated learning is the primary driver of variation in the structure of humpback whales' songs over time. If that is the case, then identifying universal properties of song composition is important to move the field forward. Broadening the analytical tool kit can help in this effort. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Bladder pressure encoding by sacral dorsal root ganglion fibres: implications for decoding

Objective. We aim at characterising the encoding of bladder pressure (intravesical pressure) by a population of sensory fibres. This research is motivated by the possibility to restore bladder function in elderly patients or after spinal cord injury using implanted devices, so called bioelectronic medicines. For these devices, nerve-based estimation of intravesical pressure can enable a personalized and on-demand stimulation paradigm, which has promise of being more effective and efficient. In this context, a better understanding of the encoding strategies employed by the body might in the future be exploited by informed decoding algorithms that enable a precise and robust bladder-pressure estimation. Approach. To this end, we apply information theory to microelectrode-array recordings from the cat sacral dorsal root ganglion while filling the bladder, conduct surrogate data studies to augment the data we have, and finally decode pressure in a simple informed approach. Main results. We find an encoding scheme by different main bladder neuron types that we divide into three response types (slow tonic, phasic, and derivative fibres). We show that an encoding by different bladder neuron types, each represented by multiple cells, offers reliability through within-type redundancy and high information rates through semi-independence of different types. Our subsequent decoding study shows a more robust decoding from mean responses of homogeneous cell pools. Significance. We have here, for the first time, established a link between an information theoretic analysis of the encoding of intravesical pressure by a population of sensory neurons to an informed decoding paradigm. We show that even a simple adapted decoder can exploit the redundancy in the population to be more robust against cell loss. This work thus paves the way towards principled encoding studies in the periphery and towards a new generation of informed peripheral nerve decoders for bioelectronic medicines.

Similarities in composition and transformations of songs by humpback whales (Megaptera novaeangliae) over time and space.

The complex songs produced by humpback whales have been cited as evidence of prodigious memory, innovativeness, sophisticated auditory scene analysis, vocal imitation, and even culture. Researchers believe humpbacks learn their songs culturally because songs appear to change rapidly, consistently, and irreversibly across whales within a population. Here, we present evidence of similarities in song structure both across populations and decades that strongly challenge claims that social learning is the main driver of variations in humpback whale songs over time. Groups of humpback whales that were not in acoustic contact (recorded in Puerto Rico in 1970, Hawaii in 2012, and Colombia in 2013-2019) produced songs in acoustically comparable cycles, suggesting that progression through sound patterns within and across songs is not simply determined by vocal imitation of innovative patterns, but may instead be controlled by production templates that prescribe how singers construct and transform songs over time. Identifying universal constraints on song production is critical to evaluating the role of vocal imitation and cultural transmission in the progressive changes that humpback whales make to their songs and for evaluating the functional relevance of such changes. The current findings illustrate how information theoretic analyses of vocal sequences can potentially obscure key acoustic qualities of signals that may be critical to understanding how vocalizers produce, perceive, and use those sequences. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Information Theoretic Analysis and Performance Gains of 3-Color Shift Keying

In this letter, we investigate the information theoretic performance of 3-Color Shift Keying (3-CSK), a promising modulation scheme for visible light communications that is based on the use of sequences of symbols. More specifically, considering its advantages, which include improved error performance and lower power consumption compared to 4-CSK and On-Off Keying, we evaluate its performance in terms of the maximum achievable rate, which is expressed as a function of the number of symbols per sequence, while a closed-form lower-bound and a tractable approximation are also obtained. Monte-Carlo simulations of the maximum achievable rate verify the analysis and illustrate that 3-CSK outperforms 4-CSK, when channel state information is not available at the receiver, with 3-CSK -in contrast to 4-CSK- reaching its maximum capacity in this scenario. Finally, to get further insights on the inferred 3-CSK gains, the bit-error-rate is also analyzed under different channel realizations, assuming no channel state information is available.

The main drivers of methane emissions differ in the growing and flooded fallow seasons in Mediterranean rice fields

PurposeTo assess 1) the cumulative greenhouse gas emissions –GHG- and global warming potential (methane – CH4- and nitrous oxide) from rice fields in the growing and fallow seasons, and 2) the environmental and agronomic drivers of CH4 emissions, and their relative capacity to explain CH4 variation.MethodsA two-year multisite field experiment covering the agronomic and environmental variability of a rice growing area in NE Iberian Peninsula was conducted with monthly samplings of GHG and monitoring of both environmental and agronomic factors. Information-theoretic framework analysis was used to assess the relative contribution of the environmental and agronomic variables on methane emissions.ResultsTwo thirds of the CH4 is emitted in the fallow season. Edaphic factors exert more influence during the growing season whereas agronomic factors have a higher impact in the fallow. The implications of these findings on the design of improved mitigation options rice are discussed.ConclusionsSoils with higher soil sulphate concentration, bulk density and clay content emit less CH4 in growing season. In the fallow season, the rates of both straw input and nitrogen fertilization stimulate CH4 emissions.Graphical abstract

Coherence and Adaptivity in Frameless Rendering-A Practical and Information Theoretic Analysis

Virtual and Augmented Reality applications demand low-latency rendering. Adaptive Frameless Rendering (AFR) techniques offer a potential alternative to currently dominating non-adaptive double-buffered real-time 3D rendering techniques. However, due to their adaptive non-contiguous importance sampling, AFR does not benefit fully from the performance improvements offered by caching the local scene information, resulting in a lower sampling rate. This article presents an experimental, followed by an information-theoretic analysis, of this coherence vs. adaptivity trade-off in AFR. It introduces a novel way to utilize entropy to evaluate the efficiency of a rendering technique. It also presents various spatial coherence exploiting techniques in 3D path tracing and their application to AFR, paving the way to future research in the field.

Rainbow smelt spawning population and habitat characteristics in a Missouri River reservoir

Rainbow Smelt Osmerus mordax are the primary coldwater forage species in Lake Oahe, South Dakota, yet most aspects of their life history remain unknown. As Missouri River reservoirs present a novel habitat compared to elsewhere in the Rainbow Smelt’s range, we investigated their spawning site use relative to available physical and thermal habitat from spring 2016 − 2018 using a mobile horizontal-beaming hydroacoustic survey. We mapped spawning habitat using side-scanning sonar to establish relationships with site bathymetry and thermal spawning conditions. From our site mapping, we determined the depth, slope, bottom hardness, and ruggedness of individual sites. We used a PCA to condense our physical habitat metrics, with PC1 representing 77% of cumulative variance. Peak abundance was linked to a physical habitat gradient, representing [steep slope, deep, soft bottom] to [low slope, shallow, hard bottom]. Peak Rainbow Smelt spawning aggregations were observed at 6.8 ± 1.7 °C (Mean ± StDev). An information-theoretic regression analysis revealed an interactive effect of temperature and habitat, whereby shallower, warmer sites typically had a lower peak abundance. These results represent the first step toward understanding habitat use characteristics of spawning Rainbow Smelt in reservoir systems.

Alternative Codes and Phase Rotation Extensions for Alternating Space-Time Coding-Based Physical Layer Security

In this article, several extensions to an alternating space-time (ST) code (STC) physical layer security (PLS) scheme from the literature are proposed. These contributions include alternative orthogonal STCs as well as non-orthogonal and spatially-multiplexed (SM) STCs for improved bandwidth efficiency. Phase rotation (PR) algorithms are provided to build very large sets of unique alternative STCs for use with this scheme. Decoding methods are discussed for alternating non-orthogonal and SM STCs. Monte Carlo simulations are provided to compare bit error rate (BER) performance between different decoding methods. Secrecy system nomenclature is adapted to the alternative STCs and proposed algorithms. Information-theoretic security analysis including message and key equivocation is provided along with expected eavesdropper BER based on an assumed attack methodology. A comparison of security offered by the alternating STC PLS scheme with and without incorporation of proposed PR algorithms is performed. Substantially greater exhaustive key search attack complexity is achieved by using the PR algorithms proposed in this article.

An Information Theoretical Analysis of Gap Junction Channels

The propagation of molecules, ions and electrochemical signals in various cell types occurs through hundreds of gap junction channels. A gap junction channel is composed of hundreds of units which are further decomposed to hemichannels. From the computational modeling perspective, each gap junction channel acts as a state machine, switching between different permeability states at known rates. Both the permeabilities and the rates jointly depend on transmembrane and transjunctional voltages. To the best of our knowledge, the effects of the permeability states on the communication performance between cells remain unknown to-date. Here, we use the example of gap junctions between cardiac myocytes (cardiomyocytes) to model a gap junction channel as a finite-state channel. We then apply the Shannon’s information theory to compute the achievable information rates. Our results demonstrate the information theoretical limits of communication over cardiac gap junction channels. The proposed analysis is anticipated to address open-research problems in cardiac modeling, as well as in nano-networking where man-made miniature devices inter-connect with cardiomyocytes to enable novel groundbreaking applications in healthcare.

The Evolution of Faster-Than-Nyquist Signaling

The fifty-year progress of faster-than-Nyquist (FTN) signaling is surveyed. FTN signaling exploits non-orthogonal dense symbol packing in the time domain for the sake of increasing the data rate attained. After reviewing the system models of both the conventional Nyquist-based and FTN signaling transceivers, we survey the evolution of FTN techniques, including their low-complexity detection and channel estimation. Furthermore, in addition to the classic FTN signaling philosophy, we introduce the recent frequency-domain filtering and precoding aided schemes. When relying on precoding, the information rate of FTN signaling becomes related to the eigenvalues of an FTN-specific intersymbol interference matrix, which provides a unified framework for the associated information-theoretic analysis and simplifies the associated power allocation specifically designed for increasing the information rate attained. We show that the FTN signaling scheme combined with bespoke power allocation employing a realistic raised-cosine shaping filter achieves the Shannon capacity associated with ideal rectangular shaping filters.

Information Thermodynamics and Reducibility of Large Gene Networks.

Gene regulatory networks (GRNs) control biological processes like pluripotency, differentiation, and apoptosis. Omics methods can identify a large number of putative network components (on the order of hundreds or thousands) but it is possible that in many cases a small subset of genes control the state of GRNs. Here, we explore how the topology of the interactions between network components may indicate whether the effective state of a GRN can be represented by a small subset of genes. We use methods from information theory to model the regulatory interactions in GRNs as cascading and superposing information channels. We propose an information loss function that enables identification of the conditions by which a small set of genes can represent the state of all the other genes in the network. This information-theoretic analysis extends to a measure of free energy change due to communication within the network, which provides a new perspective on the reducibility of GRNs. Both the information loss and relative free energy depend on the density of interactions and edge communication error in a network. Therefore, this work indicates that a loss in mutual information between genes in a GRN is directly coupled to a thermodynamic cost, i.e., a reduction of relative free energy, of the system.

Information theoretic perspective on genome clustering.

Shannon’s information theoretic perspective of communication helps one to understand the storage and processing of information in one-dimensional sequences. An information theoretic analysis of 937 available completely sequenced prokaryotic genomes and 238 eukaryotic chromosomes is presented. Information content (Id) values were used to cluster these chromosomes. Chargaff’s second parity rule i.e compositional self-complementarity, an empirical fact is observed in all the genomes, except for the proteobacteria Candidatus Hodgkinia cicadicola. High information content, arising out of biased base composition in all the 14 chromosomes of Plasmodium falciparum is found among two other genomes of prokaryotes viz. Buchnera aphidicola str. Cc (Cinara cedri) and Candidatus Carsonella ruddii PV. Despite size and compositional variations, both prokaryotic and eukaryotic genomes do not deviate significantly from an equiprobable and random situation. Eukaryotic chromosomes of an organism tend to have similar informational restraints as seen when a simple distance based method is used to cluster them. In eukaryotes, in certain cases, Id values are also similar for the two arms (p and q arm) of the chromosomes. The results of this current study confirm that the information content can provide insights into the clustering of genomes and the evolution of messaging strategies of the genomes. An efficient and robust Perl CGI standalone tool is created based on this information theory algorithm for the analysis of the whole genomes and is made available at https://github.com/AlagurajVeluchamy/InformationTheory.

Euclid: A Fully In-Network, P4-Based Approach for Real-Time DDoS Attack Detection and Mitigation

Distributed Denial-of-Service (DDoS) attacks have been steadily escalating in frequency, scale, and disruptiveness - with outbreaks reaching multiple terabits per second and compromising the availability of highly-resilient networked systems. Existing defenses require frequent interaction between forwarding and control planes, making it difficult to reach a satisfactory trade-off between accuracy (higher is better), resource usage, and defense response delay (lower is better). Recently, high-performance programmable data planes have made it possible to develop a new generation of mechanisms to analyze and manage traffic at line rate. In this article, we explore P4 language constructs and primitives to design Euclid, a fully in-network fine-grained, low-footprint, and low-delay traffic analysis mechanism for DDoS attack detection and mitigation. Euclid utilizes information-theoretic and statistical analysis to detect attacks and classify packets as either legitimate or malicious, thus enabling the enforcement of policies (e.g., discarding, inspection, or throttling) to prevent attack traffic from disrupting the operation of its victims. We experimentally evaluate our proposed mechanism using packet traces from CAIDA. The results indicate that Euclid can detect attacks with high accuracy (98.2%) and low delay (≈250 ms), and correctly identify most of the attack packets (>96%) without affecting more than 1% of the legitimate traffic. Furthermore, our approach operates under a small resource usage footprint (tens of kilobytes of static random-access memory per 1 Gbps link and a few hundred ternary content-addressable memory entries), thus enabling its deployability on high-throughput, high-volume scenarios.

Using enriched semantic event chains to model human action prediction based on (minimal) spatial information.

Predicting other people's upcoming action is key to successful social interactions. Previous studies have started to disentangle the various sources of information that action observers exploit, including objects, movements, contextual cues and features regarding the acting person's identity. We here focus on the role of static and dynamic inter-object spatial relations that change during an action. We designed a virtual reality setup and tested recognition speed for ten different manipulation actions. Importantly, all objects had been abstracted by emulating them with cubes such that participants could not infer an action using object information. Instead, participants had to rely only on the limited information that comes from the changes in the spatial relations between the cubes. In spite of these constraints, participants were able to predict actions in, on average, less than 64% of the action's duration. Furthermore, we employed a computational model, the so-called enriched Semantic Event Chain (eSEC), which incorporates the information of different types of spatial relations: (a) objects' touching/untouching, (b) static spatial relations between objects and (c) dynamic spatial relations between objects during an action. Assuming the eSEC as an underlying model, we show, using information theoretical analysis, that humans mostly rely on a mixed-cue strategy when predicting actions. Machine-based action prediction is able to produce faster decisions based on individual cues. We argue that human strategy, though slower, may be particularly beneficial for prediction of natural and more complex actions with more variable or partial sources of information. Our findings contribute to the understanding of how individuals afford inferring observed actions' goals even before full goal accomplishment, and may open new avenues for building robots for conflict-free human-robot cooperation.

An Optimality Summary: Secret Key Agreement with Physical Unclonable Functions.

We address security and privacy problems for digital devices and biometrics from an information-theoretic optimality perspective to conduct authentication, message encryption/decryption, identification or secure and private computations by using a secret key. A physical unclonable function (PUF) provides local security to digital devices and this review gives the most relevant summary for information theorists, coding theorists, and signal processing community members who are interested in optimal PUF constructions. Low-complexity signal processing methods are applied to simplify information-theoretic analyses. The best trade-offs between the privacy-leakage, secret-key, and storage rates are discussed. Proposed optimal constructions that jointly design the vector quantizer and error-correction code parameters are listed. These constructions include modern and algebraic codes such as polar codes and convolutional codes, both of which can achieve small block-error probabilities at short block lengths, corresponding to a small number of PUF circuits. Open problems in the PUF literature from signal processing, information theory, coding theory, and hardware complexity perspectives and their combinations are listed to stimulate further advancements in the research on local privacy and security.

An information-theoretical analysis of gene nucleotide sequence structuredness for a selection of aging and cancer-related genes.

We provide an algorithm for the construction and analysis of autocorrelation (information) functions of gene nucleotide sequences. As a measure of correlation between discrete random variables, we use normalized mutual information. The information functions are indicative of the degree of structuredness of gene sequences. We construct the information functions for selected gene sequences. We find a significant difference between information functions of genes of different types. We hypothesize that the features of information functions of gene nucleotide sequences are related to phenotypes of these genes.

Do galactic bars depend on environment?: an information theoretic analysis of Galaxy Zoo 2

ABSTRACT We use an information theoretic framework to analyse data from the Galaxy Zoo 2 project and study if there are any statistically significant correlations between the presence of bars in spiral galaxies and their environment. We measure the mutual information between the barredness of galaxies and their environments in a volume limited sample (Mr ≤ −21) and compare it with the same in data sets where (i) the bar/unbar classifications are randomized and (ii) the spatial distribution of galaxies are shuffled on different length scales. We assess the statistical significance of the differences in the mutual information using a t-test and find that both randomization of morphological classifications and shuffling of spatial distribution do not alter the mutual information in a statistically significant way. The non-zero mutual information between the barredness and environment arises due to the finite and discrete nature of the data set that can be entirely explained by mock Poisson distributions. We also separately compare the cumulative distribution functions of the barred and unbarred galaxies as a function of their local density. Using a Kolmogorov–Smirnov test, we find that the null hypothesis cannot be rejected even at $75{{\ \rm per\ cent}}$ confidence level. Our analysis indicates that environments do not play a significant role in the formation of a bar, which is largely determined by the internal processes of the host galaxy.

Characterization of Pathogen Airborne Inoculum Density by Information Theoretic Analysis of Spore Trap Time Series Data.

In a previous study, air sampling using vortex air samplers combined with species-specific amplification of pathogen DNA was carried out over two years in four or five locations in the Salinas Valley of California. The resulting time series data for the abundance of pathogen DNA trapped per day displayed complex dynamics with features of both deterministic (chaotic) and stochastic uncertainty. Methods of nonlinear time series analysis developed for the reconstruction of low dimensional attractors provided new insights into the complexity of pathogen abundance data. In particular, the analyses suggested that the length of time series data that it is practical or cost-effective to collect may limit the ability to definitively classify the uncertainty in the data. Over the two years of the study, five location/year combinations were classified as having stochastic linear dynamics and four were not. Calculation of entropy values for either the number of pathogen DNA copies or for a binary string indicating whether the pathogen abundance data were increasing revealed (1) some robust differences in the dynamics between seasons that were not obvious in the time series data themselves and (2) that the series were almost all at their theoretical maximum entropy value when considered from the simple perspective of whether instantaneous change along the sequence was positive.

Explaining the Behavior of Neuron Activations in Deep Neural Networks

Deep Neural Networks has shown superior performance in various applications. But it is often seen as black box in real world applications, which is challenging to explain from the viewpoint of humans. It is important to understand the behavior of deep neural networks so as to trust the decision made and improve the classification accuracy of deep neural networks. In this study, the information theoretical analysis is used to investigate the behavior of layer-wise neurons in deep neural networks. The activation patterns of individual neurons in fully connected layers can provide insights for the performance of the neural network model. The behavior of neuron activation is investigated based on state-of-art classification network model. We study and compare the layer-wise pattern of neurons activation in fully connected layers given the same image input. Experiments are conducted on various data sets. We find that in a well trained classification model, the randomness level of the neurons activation pattern is reduced with the depth of the fully connected layers. This means that the neuron activation patterns of deep layers is more stable than that of shallow layers. The results in this study can also answer the question of how many layers are needed to avoid overfitting in deep neural networks. Corresponding experiments are conducted to validate the assumptions.