Information-theoretic Approach Research Articles

Derivation of Standard Quantum Theory via State Discrimination

Abstract It is a key issue to characterize the model of standard quantum theory out of general models by an operational condition. The framework of General Probabilistic Theories (GPTs) is a new information theoretical approach to single out standard quantum theory. It is known that traditional properties, for example, Bell-CHSH inequality, are not sufficient to single out standard quantum theory among possible models in GPTs. As a more precise property, we focus on the bound of the performance for an information task called state discrimination in general models. We give an equivalent condition for outperforming the minimum discrimination error probability under the standard quantum theory given by the trace norm. Besides, by applying the equivalent condition, we characterize standard quantum theory out of general models in GPTs by the bound of the performance for state discrimination.

Hybrid Machine Learning Approach for Early Diagnosis of Polycystic Ovary Syndrome with Stable Features

Polycystic Ovary Syndrome (PCOS) is a hormonal condition that typically affects female during the time of their reproduction. It is identified by the disruptions in hormonal balance, particularly an increase in levels of androgen (male hormone) in the female body. PCOS can lead to various symptoms and health complications including irregular menstrual cycles, ovarian cysts, fertility issues, insulin resistance, weight gain, acne, and excess hair growth. The real-world PCOS detection is a challenging task whilst PCOS specific cause is unknown and its symptoms are unclear. Thus, accurate and timely diagnosis of PCOS is crucial for effective management and prevention of long-term complications. In such cases, Machine learning based PCOS prediction model support diagnostic process, address potential errors and time constraints. Machine learning algorithms can analyze large set of patient data, including medical history, hormonal profiles, and imaging results, to assist in the diagnosis of PCOS. In particular, the performance of data analysis chore and prediction model is improved by ensemble feature selection strategies. These methods concentrate on selecting a subset of pertinent features from a broader range of features. The unstable nature of the outcome of feature selection algorithm is a frequent issue in practical applications, when it is applied multiple times on similar dataset or with slight modifications in the data. Thus, evaluating the robustness of feature selection algorithm is most important. To address these issues and quantify the robustness, this study uses Jenson-Shannon divergence, an information theoretic approach with ensemble feature selection method to handle the various findings, such as complete ranking, half ranking and top-k lists (without ranking). Furthermore, this article proposes a hybrid machine learning classifier with SMOTE – SVM for the prompt detection of PCOS and the performance of the model is compared with a number of other individual classifiers including KNN (K-Nearest Neighbour), Support Vector Machine (SVM), AdaBoost, LR –Logistic Regression, NB –Nave Bayes, RF –Random Forest, Decision Tree. The proposed SWISS-AdaBoost classifier surpassed other models with 97.81% of accuracy and AUC of 99.08%.

An Information Theoretic Approach to Analyze Irregularity of Graph Signals and Network Topological Changes Based on Bubble Entropy

An Information Theoretic Approach to Analyze Irregularity of Graph Signals and Network Topological Changes Based on Bubble Entropy

Nor climate nor human impact factors: Chytrid infection shapes the skin bacterial communities of an endemic amphibian in a biodiversity hotspot.

The bacterial communities of the amphibian skin (i.e., the bacteriome) are critical to the host's innate immune system. However, it is unclear how different drivers can alter this function by modulating the bacteriome's structure. Our aim was to assess the extent to which different host attributes and extrinsic factors influence the structure of the bacterial communities of the skin. Skin bacterial diversity was examined in 148 individuals of the four-eyed frog (Pleurodema thaul) from 16 localities spanning almost 1800 km in latitude. The richness and beta diversity of bacterial families and the richness and abundance of Bd-inhibitory bacterial genera were used to describe their structure. Predictors associated with the host (developmental stage, genetic lineage, individual Batrachochytrium dendrobatidis [Bd] infection status) and the landscape (current climate, degree of anthropogenic disturbance) were used in the statistical modeling in an information theoretical approach. Bd infection and host developmental stage were the only predictors affecting bacteriome richness, with Bd+ individuals and postmetamorphic stages (adults and juveniles) having higher richness than Bd- ones and tadpoles. High diversity in Bd+ individuals is not driven by bacterial genera with known anti-Bd properties. Beta diversity was not affected by Bd infection and was mainly a consequence of bacterial family turnover rather than nestedness. Finally, for those bacterial genera known to have inhibitory effects on chytrid, Bd+ individuals had a slightly higher diversity than Bd- ones. Our study confirms an association between Bd infection and the host developmental stage with the bacterial communities of the skin of P. thaul. Unexpectedly, macroclimate and human impact factors do not seem to play a role in shaping the amphibian skin microbiome. Our study exemplifies that focusing on a single host-parasite system over a large geographic scale can provide essential insights into the factors driving host-parasite-bacteriome interactions.

Out-of-Sample Tuning for Causal Discovery.

Causal discovery is continually being enriched with new algorithms for learning causal graphical probabilistic models. Each one of them requires a set of hyperparameters, creating a great number of combinations. Given that the true graph is unknown and the learning task is unsupervised, the challenge to a practitioner is how to tune these choices. We propose out-of-sample causal tuning (OCT) that aims to select an optimal combination. The method treats a causal model as a set of predictive models and uses out-of-sample protocols for supervised methods. This approach can handle general settings like latent confounders and nonlinear relationships. The method uses an information-theoretic approach to be able to generalize to mixed data types and a penalty for dense graphs to penalize for complexity. To evaluate OCT, we introduce a causal-based simulation method to create datasets that mimic the properties of real-world problems. We evaluate OCT against two other tuning approaches, based on stability and in-sample fitting. We show that OCT performs well in many experimental settings and it is an effective tuning method for causal discovery.

Quantifying the efficacy of voltage protocols in characterising ion channel kinetics: A novel information-theoretic approach.

Voltage-clamp experiments are commonly utilised to characterise cellular ion channel kinetics. In these experiments, cells are stimulated using a known time-varying voltage, referred to as the voltage protocol, and the resulting cellular response, typically in the form of current, is measured. Parameters of models that describe ion channel kinetics are then estimated by solving an inverse problem which aims to minimise the discrepancy between the predicted response of the model and the actual measured cell response. In this paper, a novel framework to evaluate the information content of voltage-clamp protocols in relation to ion channel model parameters is presented. Additional quantitative information metrics that allow for comparisons among various voltage protocols are proposed. These metrics offer a foundation for future optimal design frameworks to devise novel, information-rich protocols. The efficacy of the proposed framework is evidenced through the analysis of seven voltage protocols from the literature. By comparing known numerical results for inverse problems using these protocols with the information-theoretic metrics, the proposed approach is validated. The essential steps of the framework are: (i) generate random samples of the parameters from chosen prior distributions; (ii) run the model to generate model output (current) for all samples; (iii) construct reduced-dimensional representations of the time-varying current output using proper orthogonal decomposition (POD); (iv) estimate information-theoretic metrics such as mutual information, entropy equivalent variance, and conditional mutual information using non-parametric methods; (v) interpret the metrics; for example, a higher mutual information between a parameter and the current output suggests the protocol yields greater information about that parameter, resulting in improved identifiability; and (vi) integrate the information-theoretic metrics into a single quantitative criterion, encapsulating the protocol's efficacy in estimating model parameters.

Irruption and Absorption: A 'Black-Box' Framework for How Mind and Matter Make a Difference to Each Other.

Cognitive science is confronted by several fundamental anomalies deriving from the mind-body problem. Most prominent is the problem of mental causation and the hard problem of consciousness, which can be generalized into the hard problem of agential efficacy and the hard problem of mental content. Here, it is proposed to accept these explanatory gaps at face value and to take them as positive indications of a complex relation: mind and matter are one, but they are not the same. They are related in an efficacious yet non-reducible, non-observable, and even non-intelligible manner. Natural science is well equipped to handle the effects of non-observables, and so the mind is treated as equivalent to a hidden 'black box' coupled to the body. Two concepts are introduced given that there are two directions of coupling influence: (1) irruption denotes the unobservable mind hiddenly making a difference to observable matter, and (2) absorption denotes observable matter hiddenly making a difference to the unobservable mind. The concepts of irruption and absorption are methodologically compatible with existing information-theoretic approaches to neuroscience, such as measuring cognitive activity and subjective qualia in terms of entropy and compression, respectively. By offering novel responses to otherwise intractable theoretical problems from first principles, and by doing so in a way that is closely connected with empirical advances, irruption theory is poised to set the agenda for the future of the mind sciences.

Fragment-Based Deep Learning for Simultaneous Prediction of Polarizabilities and NMR Shieldings of Macromolecules and Their Aggregates.

Simultaneous prediction of the molecular response properties, such as polarizability and the NMR shielding constant, at a low computational cost is an unresolved issue. We propose to combine a linear-scaling generalized energy-based fragmentation (GEBF) method and deep learning (DL) with both molecular and atomic information-theoretic approach (ITA) quantities as effective descriptors. In GEBF, the total molecular polarizability can be assembled as a linear combination of the corresponding quantities calculated from a set of small embedded subsystems in GEBF. In the new GEBF-DL(ITA) protocol, one can predict subsystem polarizabilities based on the corresponding molecular wave function (thus electron density and ITA quantities) and DL model rather than calculate them from the computationally intensive coupled-perturbed Hartree-Fock or Kohn-Sham equations and finally obtain the total molecular polarizability via a linear combination equation. As a proof-of-concept application, we predict the molecular polarizabilities of large proteins and protein aggregates. GEBF-DL(ITA) is shown to be as accurate enough as GEBF, with mean absolute percentage error <1%. For the largest protein aggregate (>4000 atoms), GEBF-DL(ITA) gains a speedup ratio of 3 compared with GEBF. It is anticipated that when more advanced electronic structure methods are used, this advantage will be more appealing. Moreover, one can also predict the NMR chemical shieldings of proteins with reasonably good accuracy. Overall, the cost-efficient GEBF-DL(ITA) protocol should be a robust theoretical tool for simultaneously predicting polarizabilities and NMR shieldings of large systems.

Information-Theoretic Exploration for Adaptive Robotic Grasping in Clutter Based on Real-Time Pixel-Level Grasp Detection

Robust robotic grasping in clutter still remains a challenging problem despite its great practical value. This paper presents an information-theoretic exploration approach that aims to improve the grasp estimation quality in complex environment. First, a light-weight grasp detector that composed by inverted residual blocks and a pyramid pooling module is proposed to make more accurate pixel-wise predictions in real-time, which are then projected to the workspace. To measure the uncertainty of estimations in the workspace, a two-dimensional grasp entropy (2D-GE) is defined and Gaussian process is applied to regress the variation of the information gain that approximated by 2D-GEs. Finally, guided by the information gain, depth cost, and distance cost, our approach is able to actively collect and fuse estimations from multiple informative viewpoints through the exploration that finally converges to a refined best grasp, resulting an improved grasping performance with a higher success rate and environmental adaptability in clutter. Simulation and robotic experiment are both performed to demonstrate the effectiveness of our method and compare it to baselines.

Performance Comparison of Different HTM-Spatial Pooler Algorithms Based on Information-Theoretic Measures

Hierarchical temporal memory (HTM) is a promising unsupervised machine-learning algorithm that models key principles of neocortical computation. One of the main components of HTM is the spatial pooler (SP), which encodes binary input streams into sparse distributed representations (SDRs). In this paper, we propose an information-theoretic framework for the performance comparison of HTM-spatial pooler (SP) algorithms, specifically, for quantifying the similarities and differences between sparse distributed representations in SP algorithms. We evaluate SP's standalone performance, as well as HTM's overall performance. Our comparison of various SP algorithms using Renyi mutual information, Renyi divergence, and Henze–Penrose divergence measures reveals that the SP algorithm with learning and a logarithmic boosting function yields the most effective and useful data representation. Moreover, the most effective SP algorithm leads to superior HTM results. In addition, we utilize our proposed framework to compare HTM with other state-of-the-art sequential learning algorithms. We illustrate that HTM exhibits superior adaptability to pattern changes over time than long short term memory (LSTM), gated recurrent unit (GRU) and online sequential extreme learning machine (OS-ELM) algorithms. This superiority is evident from the lower Renyi divergence of HTM (0.23) compared to LSTM6000 (0.33), LSTM3000 (0.38), GRU (0.41), and OS-ELM (0.49). HTM also achieved the highest Renyi mutual information value of 0.79, outperforming LSTM6000 (0.73), LSTM3000 (0.71), GRU (0.68), and OS-ELM (0.62). These findings not only confirm the numerous advantages of HTM over other sequential learning algorithm, but also demonstrate the effectiveness of our proposed information-theoretic approach as a powerful framework for comparing and evaluating various learning algorithms.

Predictors of teachers’ behavioral intentions in inclusive education and their changes over time: A competitive test of hypotheses

Teachers' attitudes, self-efficacy, and subjective norm influence the realization of inclusive education. Recent educational changes in Germany may have affected these variables and their influence on behavioral intentions. Applying the Theory of Planned Behavior, this study examines how teachers' inclusive intentions and their predictors have evolved over eight years. We analyzed two samples of German teachers from 2014 (N = 211) and 2022 (N = 311) and performed a competitive test of hypotheses with different change patterns according to the information-theoretic approach. Results show that teachers' subjective norm increased and that teachers’ self-efficacy predicts their intentions more strongly in 2022.

The confined helium atom: An information–theoretic approach

AbstractIn this article, we study the helium atom confined in a spherical impenetrable cavity by using informational measures. We use the Ritz variational method to obtain the energies and wave functions of the confined helium atom as a function of the cavity radius . As trial wave functions we use one uncorrelated function and five explicitly correlated basis sets in Hylleraas coordinates with different degrees of electronic correlation. We computed the Shannon entropy, Fisher information, Kullback–Leibler entropy, Tsallis entropy, disequilibrium and Fisher–Shannon complexity, as a function of . We found that these entropic measures are sensitive to electronic correlation and can be used to measure it. As expected these entropic measures are less sensitive to electron correlation in the strong confinement regime ( a.u.).

Network toxicity analysis: an information-theoretic approach to studying the social dynamics of online toxicity

AbstractThe rise of social media has corresponded with an increase in the prevalence and severity of online toxicity. While much work has gone into understanding its nature, we still lack knowledge of its emergent structural dynamics. This work presents a novel method—network toxicity analysis—for the inductive analysis of the dynamics of discursive toxicity within social media. Using an information-theoretic approach, this method estimates toxicity transfer relationships between communicating agents, yielding an effective network describing how those entities influence one another, over time, in terms of their produced discursive toxicity. This method is applied to Telegram messaging data to demonstrate its capacity to induce meaningful, interpretable toxicity networks that provide valuable insight into the social dynamics of toxicity within social media.

Sexual dichromatism in the fur of a bat: An exploration of color differences and potential signaling functions.

Sex differences in body color (i.e., sexual dichromatism) are rare in bats and, more broadly, in mammals. The eastern red bat (Lasiurus borealis) is a common tree-roosting bat that occupies much of North America and has long been described as sexually dichromatic. However, previous research on this species found that absolute body size and collection year were better predictors of fur color in preserved specimens than sex. We revisited this issue and photographed 82 live eastern red bats under standardized conditions, then used image analysis to quantify pelage hue, saturation, and value. We used an information theoretic approach to evaluate four competing hypotheses about the principal drivers of color differences in the fur of eastern red bats. Our analyses demonstrated that sex was a better predictor of pelage color than body size; males had redder, more saturated, and lighter pelages than females. Additionally, the fur color of juvenile versus adult bats differed somewhat, as juveniles were darker than adults. In general, absolute body size (i.e., forearm length in bats) was a poor predictor of color in live eastern red bats. In an exploratory post-hoc analysis, we confirm that fur color is related to body mass (i.e., a proxy for body condition in bats), suggesting color might serve as a sexually selected signal of mate quality in this partially diurnal species. Future work should investigate the functional role of sexual dichromatism in this species, which could be related to signaling or possibly thermoregulation.

An Information-Theoretic Approach to Joint Sensing and Communication

A communication setup is considered where a single transmitter wishes to convey messages to one or two receivers and simultaneously estimate the states of the receivers through the backscattered signals of the emitted waveform. The scenario at hand is motivated by joint radar and communication, which aims to co-design radar sensing and communication over shared spectrum and hardware. In this paper, we model the communication channel as a simple memoryless channel with independent and identically distributed (i.i.d.) time-varying state sequences and we model the backscattered signals by (strictly causal) generalized feedback. For single-receiver systems of this form, we fully characterize the capacity-distortion tradeoff, defined as the largest rate at which a message can reliably be conveyed to the receiver while simultaneously allowing the transmitter to sense the state sequence with a given allowed distortion. Our results show a tradeoff between the achievable rates and distortions, and that this tradeoff only stems from a common choice of the input distribution (the waveform) but not from other properties of the utilized codes. To better illustrate the capacity-distortion tradeoff, we propose a numerical method to compute the optimal inputs (waveforms) that achieve the desired tradeoff. For two-receiver systems with two states, we characterize the capacity-distortion tradeoff region of physically degraded broadcast channels (BC) as a rather straightforward extension of the single receiver case. Here, a tradeoff not only arises between sensing and communication performances but also between the various rates and the distortions of the different states. Similarly to the single-receiver case, the optimal co-design scheme exploits the generalized feedback signals only for sensing but not for improving communication performance. This is different for general two-receiver BCs, where optimal co-design schemes exploit generalized feedback also to improve capacity. However, as we show, also for BCs the optimal sensing performance only depends on the chosen input distribution (waveform) but not on the code construction used to accomplish the communication task. For general BCs, we provide inner and outer bounds on the capacity-distortion region, as well as a sufficient condition when this capacity-distortion region is equal to the product of the capacity region and the set of achievable distortions, in which case no tradeoff between sensing and communication occurs. A number of illustrative examples demonstrate that the optimal co-design schemes outperform conventional schemes that split the resources between sensing and communication, both for single-receiver and BC systems.

Planktic diatom responses to spatiotemporal environmental variation in high‐mountain tropical lakes

Abstract Tropical lakes harbour high levels of biodiversity, but the temporal and spatial variability of biological communities are still inadequately characterised, making it difficult to predict the impact of accelerated rates of environmental change in these regions. Our goal was to identify the spatiotemporal dynamics of the planktic diatom community in the Cajas Massif in the tropical Andes. We analysed seasonal diatom and environmental data over a period of 1 year from 10 lakes located in geologically distinct basins and modelled community–environment relationships using multivariate ordination and variation partitioning techniques. Generalised additive models with a full‐subset information theoretic approach also were used to determine which environmental variables explain single‐species abundance. Although the lakes are monomictic and thus have variable thermal structure across the year, seasonal variability of water chemistry conditions was negligible, and seasonal differences in diatom community composition were small. Across space, diatom community composition was correlated primarily with ionic content (divalent cations and alkalinity), related to bedrock composition, and secondly with lake thermal structure and productivity. The ionic gradient overrode the effect of the thermal structure–productivity gradient at the diatom community level, whereas individual diatom species responded more sensitively to variables related to in‐lake and catchment productivity, including chlorophyll‐a and iron, and the proportion of wetlands in the catchment. Our results indicate that the spatiotemporal variability of Cajas lakes and their diatom communities is the result of multiple intertwined environmental factors. The emergence of the ionic and thermal structure–productivity gradients in a rather small tropical lake district suggests segregation of ecological niches for diatoms that also may be important in other high‐elevation lake regions. Future studies that track tropical Andean lakes under natural and anthropogenically mediated change, both in contemporary times and in palaeoenvironmental reconstructions, would benefit from the modelling approach (community and species levels) developed here.

Entropy Cost of ‘Erasure’ in Physically Irreversible Processes

A restricted form of Landauer’s principle, independent of computational considerations, is shown to hold for thermal systems by reference to the joint entropy associated with conjugate observables. It is shown that the source of the compensating entropy for irreversible physical processes is due to the ontological uncertainty attending values of such mutually incompatible observables, rather than due to epistemic uncertainty, as traditionally assumed in the information-theoretic approach. In particular, it is explicitly shown that erasure of logical (epistemic) information via reset operations is not equivalent to erasure of thermodynamic entropy, so that the traditional, information-theoretic form of Landauer’s principle is not supported by the physics. A further implication of the analysis is that, in principle, there can be no Maxwell’s Demon in the real world.

Behavioural Economics: Appropriate Framework for Studying the Life Insurance Market

This article rests on establishing a unique standpoint that life insurance products (unlike other financial assets) are purchased for protection, not as an investment for return. This aims at identifying the most suitable theoretical framework to study the life insurance market and to suggest appropriate tools of analysis. Economic and financial theories starting from neo-classical economics, going up to behavioural economics (BE) have been studied vis-à-vis life insurance markets. A distinction has been drawn between BE and behavioural finance (BF). Accordingly, neo-classical economics, standard theory of finance and BF have been found to be inappropriate to understand life insurance as a market. The main theoretical edifice of BE, with a blend of transaction-cost approach and information theoretic approach, is argued to be the right theoretical framework to study the life insurance market, given the idiosyncrasies involved. A novel attempt has been made to create a ‘construct’ to depict how the BE framework is most relevant. It is hypothesised that bounded rationality is adversely influenced by emotions, search costs, intermediaries and socio-psychological influences, to embed behavioural biases and heuristics into life insurance purchase decisions. Combinedly, this leads to demand and supply distortions that create an incomplete market. To test this framework, it has been suggested that Heuristic z-test; Murthy’s Index of Rank Dominance (MIrd &amp; MRird); and logistic regression, be harnessed. These tools would help in measuring market inefficiency and underperformance. Policymakers, the regulator, insurers and buyers alike would benefit from a better understanding of the market.

Bounds and Protocols for Graph-Based Distributed Secret Sharing

Distributed Secret Sharing is a (multi) secret sharing model in which the shares are distributed over storage nodes of a network and each participant is able to reconstruct a specific secret by accessing a subset of the storage nodes. In this work, we provide new Distributed (multi) Secret Sharing Protocols for a specific class of access structures, namely those that can be described with a graph. The protocols improve on previous results allowing a faster encoding and decoding phase while maintaining optimal storage requirements. Moreover, our protocols can manage any kind of graph, while previous protocols have been designed only for complete graphs, and we provide a complete characterization of graph-based protocols. We also prove some tight bounds on the size of the information held in the storage nodes and communication complexity by using an information-theoretic approach. Finally, we also introduce a computationally secure technique for the general case that allows improvements in the size of the needed disk space if secrecy is computational, that is, if the scheme is robust against resource-bounded adversaries.

Securing Large-Scale D2D Networks Using Covert Communication and Friendly Jamming

We exploit both covert communication and friendly jamming to propose a friendly jamming-assisted covert communication and use it to doubly secure a large-scale device-to-device (D2D) network against eavesdroppers (i.e., wardens). The D2D transmitters defend against the wardens by: 1) hiding their transmissions with enhanced covert communication, and 2) leveraging friendly jamming to ensure information secrecy even if the D2D transmissions are detected. We model the combat between the wardens and the D2D network (the transmitters and the friendly jammers) as a two-stage Stackelberg game. Therein, the wardens are the followers at the lower stage aiming to minimize their detection errors, and the D2D network is the leader at the upper stage aiming to maximize its utility (in terms of link reliability and communication security) subject to the constraint on communication covertness. We apply stochastic geometry to model the network spatial configuration so as to conduct a system-level study. We develop a bi-level optimization algorithm to search for the equilibrium of the proposed Stackelberg game based on the successive convex approximation (SCA) method and Rosenbrock method. Numerical results reveal interesting insights. We observe that without the assistance from the jammers, it is difficult to achieve covert communication on D2D transmission. Moreover, we illustrate the advantages of the proposed friendly jamming-assisted covert communication by comparing it with the information-theoretical secrecy approach in terms of the secure communication probability and network utility.