Information-theoretic Analysis Research Articles

An analytical assessment of credit card fraud detection techniques: Supervised, Unsupervised, and Reinforcement Learning

Objective. Bank card fraud is an increasingly serious problem for individuals, businesses and financial institutions. There is a need for effective fraud detection measures to protect consumers and businesses from financial losses. Method. information-theoretical analysis of methods for detecting fraud with bank cards, machine learning algorithms in improving the accuracy of fraud detection. Result. An analytical evaluation of fraud detection methods is provided, covering different learning approaches: supervised, unsupervised and reinforcement learning. Conclusion. The choice of a fraud detection method should be based on an understanding of the available data, the specific requirements of the application domain and the trade-offs between methods in terms of performance, adaptability and computational complexity.

Adaptation optimizes sensory encoding for future stimuli.

Sensory neurons continually adapt their response characteristics according to recent stimulus history. However, it is unclear how such a reactive process can benefit the organism. Here, we test the hypothesis that adaptation actually acts proactively in the sense that it optimally adjusts sensory encoding for future stimuli. We first quantified human subjects' ability to discriminate visual orientation under different adaptation conditions. Using an information theoretic analysis, we found that adaptation leads to a reallocation of coding resources such that encoding accuracy peaks at the mean orientation of the adaptor while total coding capacity remains constant. We then asked whether this characteristic change in encoding accuracy is predicted by the temporal statistics of natural visual input. Analyzing the retinal input of freely behaving human subjects showed that the distribution of local visual orientations in the retinal input stream indeed peaks at the mean orientation of the preceding input history (i.e., the adaptor). We further tested our hypothesis by analyzing the internal sensory representations of a recurrent neural network trained to predict the next frame of natural scene videos (PredNet). Simulating our human adaptation experiment with PredNet, we found that the network exhibited the same change in encoding accuracy as observed in human subjects. Taken together, our results suggest that adaptation-induced changes in encoding accuracy prepare the visual system for future stimuli.

Information Theoretical Analysis of Quantum Mixedness in a Finite Model of Interacting Fermions.

In this study, we utilize information theory tools to investigate notable features of the quantum degree of mixedness (Cf) in a finite model of N interacting fermions. This model serves as a simplified proxy for an atomic nucleus, capturing its essential features in a more manageable form compared to a realistic nuclear model, which would require the diagonalization of matrices with millions of elements, making the extraction of qualitative features a significant challenge. Specifically, we aim to correlate Cf with particle number fluctuations and temperature, using the paradigmatic Lipkin model. Our analysis reveals intriguing dependencies of Cf on the total fermion number, showcasing distinct behaviors at different temperatures. Notably, we find that the degree of quantum mixedness exhibits a strong dependence on the total fermion number, with varying trends across different temperature regimes. Remarkably, this dependence remains unaffected by the strength of the fermion-fermion interaction (as long as it is non-zero), underscoring the robustness of the observed phenomena. Through comprehensive numerical simulations, we provide illustrative graphs depicting these dependencies, offering valuable insights into the fundamental characteristics of quantum many-body fermion systems. Our findings illuminate the intricate dynamics of the degree of mixedness, a crucial quantum property, with potential implications for diverse fields ranging from condensed matter physics to quantum information science.

Probabilistic Explanations for Entropic Knowledge Extraction for Automated Satellite Component Detection

The escalating risk of collisions and the accumulation of space debris in Low Earth Orbit (LEO) has reached critical concern due to the ever-increasing number of spacecraft. Addressing this crisis, especially in dealing with noncooperative and unidentified space debris, is of paramount importance. This paper contributes to efforts in enabling autonomous swarms of small chaser satellites for target geometry determination and safe flight-trajectory planning for proximity operations in LEO. Our research explores on-orbit use of the You Only Look Once v5 object detection model trained to detect satellite components. Although this model has shown promise, its inherent lack of interpretability hinders human understanding, a critical aspect of validating algorithms for use in safety-critical missions. To analyze the decision processes, we introduce Probabilistic Explanations for Entropic Knowledge extraction (PEEK), a method that utilizes information theoretic analysis of the latent representations within the hidden layers of the model. Through both synthetic and hardware-in-the-loop experiments, PEEK illuminates the decision-making processes of the model, helping to identify its strengths, limitations, and biases.

Cortisol awakening response prompts dynamic reconfiguration of brain networks in emotional and executive functioning

Emotion and cognition involve an intricate crosstalk of neural and endocrine systems that support dynamic reallocation of neural resources and optimal adaptation for upcoming challenges, an active process analogous to allostasis. As a hallmark of human endocrine activity, the cortisol awakening response (CAR) is recognized to play a critical role in proactively modulating emotional and executive functions. Yet, the underlying mechanisms of such proactive effects remain elusive. By leveraging pharmacological neuroimaging and hidden Markov modeling of brain state dynamics, we show that the CAR proactively modulates rapid spatiotemporal reconfigurations (state) of large-scale brain networks involved in emotional and executive functions. Behaviorally, suppression of CAR proactively impaired performance of emotional discrimination but not working memory (WM), while individuals with higher CAR exhibited better performance for both emotional and WM tasks. Neuronally, suppression of CAR led to a decrease in fractional occupancy and mean lifetime of task-related brain states dominant to emotional and WM processing. Further information-theoretic analyses on sequence complexity of state transitions revealed that a suppressed or lower CAR led to higher transition complexity among states primarily anchored in visual-sensory and salience networks during emotional task. Conversely, an opposite pattern of transition complexity was observed among states anchored in executive control and visuospatial networks during WM, indicating that CAR distinctly modulates neural resources allocated to emotional and WM processing. Our findings establish a causal link of CAR with brain network dynamics across emotional and executive functions, suggesting a neuroendocrine account for CAR proactive effects on human emotion and cognition.

An information-theoretic analysis of security behavior intentions amongst United States poll workers.

In light of recent events related to national elections in the United States, safeguarding the security and integrity of forthcoming elections stands as a critical national priority. Elections equipment in the United States constitutes critical national infrastructure, and its operation relies on poll workers, who are trusted insiders. However, those insiders may pose risks if they make mistakes with detrimental consequences or act with malice. This research analyzes a large dataset of 2213 responses obtained from a survey of poll workers and potential poll workers in 13 states. The survey includes the Security Behavior Intentions Scale, which has been previously established and validated in the security literature. We use the responses to assess poll workers' intentions of complying with established security-related practices. We develop a novel model using information theory to examine potential weaknesses in security behaviors and identify poll worker security practices to improve to ensure the integrity of our elections. We also recommend action items and countermeasures for states and localities based upon this empirical analysis.

When to Simply Use Passive RIS as Beamformer: An Information-Theoretic Analysis and a Novel Single-RF MIMO Transceiver Architecture

When to Simply Use Passive RIS as Beamformer: An Information-Theoretic Analysis and a Novel Single-RF MIMO Transceiver Architecture

DNA methylation stochasticity is linked to transcriptional variability and identifies convergent epigenetic disruption across genetically-defined subtypes of AML.

Disruption of the epigenetic landscape is of particular interest in acute myeloid leukemia (AML) due to its relatively low mutational burden and frequent occurrence of mutations in epigenetic regulators. Here, we applied an information-theoretic analysis of methylation potential energy landscapes, capturing changes in mean methylation level and methylation entropy, to comprehensively analyze DNA methylation stochasticity in subtypes of AML defined by mutually exclusive genetic mutations. We identified AML subtypes with CEBPA double mutation and those with IDH mutations as distinctly high-entropy subtypes, marked by methylation disruption over a convergent set of genes. We found a core program of epigenetic landscape disruption across all AML subtypes, with discordant methylation stochasticity and transcriptional dysregulation converging on functionally important leukemic signatures, suggesting a genotype-independent role of stochastic disruption of the epigenetic landscape in mediating leukemogenesis. We further established a relationship between methylation entropy and gene expression variability, connecting the disruption of the epigenetic landscape to transcription in AML. This approach identified a convergent program of epigenetic dysregulation in leukemia, clarifying the contribution of specific genetic mutations to stochastic disruption of the epigenetic and transcriptional landscapes of AML.

Information–Theoretic Analysis of Visibility Graph Properties of Extremes in Time Series Generated by a Nonlinear Langevin Equation

In this study, we examined how the nonlinearity α of the Langevin equation influences the behavior of extremes in a generated time series. The extremes, defined according to run theory, result in two types of series, run lengths and surplus magnitudes, whose complex structure was investigated using the visibility graph (VG) method. For both types of series, the information measures of the Shannon entropy measure and Fisher Information Measure were utilized for illustrating the influence of the nonlinearity α on the distribution of the degree, which is the main topological parameter describing the graph constructed by the VG method. The main finding of our study was that the Shannon entropy of the degree of the run lengths and the surplus magnitudes of the extremes is mostly influenced by the nonlinearity, which decreases with with an increase in α. This result suggests that the run lengths and surplus magnitudes of extremes are characterized by a sort of order that increases with increases in nonlinearity.

Signatures of hierarchical temporal processing in the mouse visual system.

A core challenge for the brain is to process information across various timescales. This could be achieved by a hierarchical organization of temporal processing through intrinsic mechanisms (e.g., recurrent coupling or adaptation), but recent evidence from spike recordings of the rodent visual system seems to conflict with this hypothesis. Here, we used an optimized information-theoretic and classical autocorrelation analysis to show that information- and correlation timescales of spiking activity increase along the anatomical hierarchy of the mouse visual system under visual stimulation, while information-theoretic predictability decreases. Moreover, intrinsic timescales for spontaneous activity displayed a similar hierarchy, whereas the hierarchy of predictability was stimulus-dependent. We could reproduce these observations in a basic recurrent network model with correlated sensory input. Our findings suggest that the rodent visual system employs intrinsic mechanisms to achieve longer integration for higher cortical areas, while simultaneously reducing predictability for an efficient neural code.

Kinetic Trajectories of Glucose Uptake in Single Cancer Cells Reveal a Drug-Induced Cell-State Change Within Hours of Drug Treatment.

The development of drug resistance is a nearly universal phenomenon in patients with glioblastoma multiforme (GBM) brain tumors. Upon treatment, GBM cancer cells may initially undergo a drug-induced cell-state change to a drug-tolerant, slow-cycling state. The kinetics of that process are not well understood, in part due to the heterogeneity of GBM tumors and tumor models, which can confound the interpretation of kinetic data. Here, we resolve drug-adaptation kinetics in a patient-derived in vitro GBM tumor model characterized by the epithelial growth factor receptor (EGFR) variant(v)III oncogene treated with an EGFR inhibitor. We use radiolabeled 18F-fluorodeoxyglucose (FDG) to monitor the glucose uptake trajectories of single GBM cancer cells over a 12 h period of drug treatment. Autocorrelation analysis of the single-cell glucose uptake trajectories reveals evidence of a drug-induced cell-state change from a high- to low-glycolytic phenotype after 5-7 h of drug treatment. Information theoretic analysis of a bulk transcriptome kinetic series of the GBM tumor model delineated the underlying molecular mechanisms driving the cellular state change, including a shift from a stem-like mesenchymal state to a more differentiated, slow-cycling astrocyte-like state. Our results demonstrate that complex drug-induced cancer cell-state changes of cancer cells can be captured via measurements of single cell metabolic trajectories and reveal the extremely facile nature of drug adaptation.

A quantum information theoretic analysis of reinforcement learning-assisted quantum architecture search

In the field of quantum computing, variational quantum algorithms (VQAs) represent a pivotal category of quantum solutions across a broad spectrum of applications. These algorithms demonstrate significant potential for realising quantum computational advantage. A fundamental aspect of VQAs involves formulating expressive and efficient quantum circuits (namely ansatz), and automating the search of such ansatz is known as quantum architecture search (QAS). Recently reinforcement learning (RL) techniques is utilized to automate the search for ansatzes, know as RL-QAS. This study investigates RL-QAS for crafting ansatz tailored to the variational quantum state diagonalisation problem. Our investigation includes a comprehensive analysis of various dimensions, such as the entanglement thresholds of the resultant states, the impact of initial conditions on the performance of RL-agent, the phase transition behaviour of correlation in concurrence bounds, and the discrete contributions of qubits in deducing eigenvalues through conditional entropy metrics. We leverage these insights to devise an entanglement-guided admissible ansatz in QAS to diagonalise random quantum states using optimal resources. Furthermore, the methodologies presented herein offer a generalised framework for constructing reward functions within RL-QAS applicable to variational quantum algorithms.

Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis.

We study stochastic linear contextual bandits (CB) where the agent observes a noisy version of the true context through a noise channel with unknown channel parameters. Our objective is to design an action policy that can "approximate" that of a Bayesian oracle that has access to the reward model and the noise channel parameter. We introduce a modified Thompson sampling algorithm and analyze its Bayesian cumulative regret with respect to the oracle action policy via information-theoretic tools. For Gaussian bandits with Gaussian context noise, our information-theoretic analysis shows that under certain conditions on the prior variance, the Bayesian cumulative regret scales as O˜(mT), where m is the dimension of the feature vector and T is the time horizon. We also consider the problem setting where the agent observes the true context with some delay after receiving the reward, and show that delayed true contexts lead to lower regret. Finally, we empirically demonstrate the performance of the proposed algorithms against baselines.

Flying Base Station Channel Capacity Limits: Dependent on Stationary Base Station and Independent of Positioning

Flying base stations, also known as aerial base stations, provide wireless connectivity to the user and utilize their aerial mobility to improve communication performance. Flying base stations depend on traditional stationary terrestrial base stations for connectivity, as stationary base stations act as the gateway to the backhaul/cloud via a wired connection. We introduce the flying base station channel capacity to build on the Shannon channel capacity, which quantifies the upper-bound limit of the rate at which information can be reliably transmitted using the communication channel regardless of the modulation and coding techniques used. The flying base station’s channel capacity assumes aerial mobility and ideal positioning for maximum channel capacity. Therefore, the channel capacity limit holds for any digital and signal processing technique used and for any location or positioning of the flying base station. Because of its inherent reliance on the stationary terrestrial base station, the flying base station channel capacity depends on the stationary base station’s parameters, such as its location and SNR performance to the user, in contrast to previous research, which focused on the link between the user and the flying base station without the stationary base station. For example, the beneficial region (where there is a positive flying base station capacity gain) depends on the stationary base station’s power and channel SNR in addition to the flying base station’s own transmission power and whether it has full duplex vs. half-duplex capability. We jointly study the mobility and the wireless communications of the flying base station to analyze its position, channel capacity, and beneficialness over the stationary terrestrial base station (capacity gain). As communication protocols and implementations for flying base stations undergo development for next-generation wireless networking, we focus on information-theoretical analyses and channel capacity to inform future research and development in flying base station networking.

Quantum information-theoretical analysis on the two-photon transitions in hydrogen isoelectronic ions under plasma confinement

Quantum information-theoretical analysis on the two-photon transitions in hydrogen isoelectronic ions under plasma confinement

A Twist for Graph Classification: Optimizing Causal Information Flow in Graph Neural Networks

Graph neural networks (GNNs) have achieved state-of-the-art results on many graph representation learning tasks by exploiting statistical correlations. However, numerous observations have shown that such correlations may not reflect the true causal mechanisms underlying the data and thus may hamper the ability of the model to generalize beyond the observed distribution. To address this problem, we propose an Information-based Causal Learning (ICL) framework that combines information theory and causality to analyze and improve graph representation learning to transform information relevance to causal dependence. Specifically, we first introduce a multi-objective mutual information optimization objective derived from information-theoretic analysis and causal learning principles to simultaneously extract invariant and interpretable causal information and reduce reliance on non-causal information in correlations. To optimize this multi-objective objective, we enable a causal disentanglement layer that effectively decouples the causal and non-causal information in the graph representations. Moreover, due to the intractability of mutual information estimation, we derive variational bounds that enable us to transform the above objective into a tractable loss function. To balance the multiple information objectives and avoid optimization conflicts, we leverage multi-objective gradient descent to achieve a stable and efficient transformation from informational correlation to causal dependency. Our approach provides important insights into modulating the information flow in GNNs to enhance their reliability and generalization. Extensive experiments demonstrate that our approach significantly improves the robustness and interpretability of GNNs across different distribution shifts. Visual analysis demonstrates how our method converts informative dependencies in representations into causal dependencies.

Information-Theoretical Analysis of the Cycle of Creation of Knowledge and Meaning in Brains under Multiple Cognitive Modalities.

It is of great interest to develop advanced sensory technologies allowing non-invasive monitoring of neural correlates of cognitive processing in people performing everyday tasks. A lot of progress has been reported in recent years in this research area using scalp EEG arrays, but the high level of noise in the electrode signals poses a lot of challenges. This study presents results of detailed statistical analysis of experimental data on the cycle of creation of knowledge and meaning in human brains under multiple cognitive modalities. We measure brain dynamics using a HydroCel Geodesic Sensor Net, 128-electrode dense-array electroencephalography (EEG). We compute a pragmatic information (PI) index derived from analytic amplitude and phase, by Hilbert transforming the EEG signals of 20 participants in six modalities, which combine various audiovisual stimuli, leading to different mental states, including relaxed and cognitively engaged conditions. We derive several relevant measures to classify different brain states based on the PI indices. We demonstrate significant differences between engaged brain states that require sensory information processing to create meaning and knowledge for intentional action, and relaxed-meditative brain states with less demand on psychophysiological resources. We also point out that different kinds of meanings may lead to different brain dynamics and behavioral responses.

Information-theoretical analysis of the neural code for decoupled face representation.

Processing faces accurately and efficiently is a key capability of humans and other animals that engage in sophisticated social tasks. Recent studies reported a decoupled coding for faces in the primate inferotemporal cortex, with two separate neural populations coding for the geometric position of (texture-free) facial landmarks and for the image texture at fixed landmark positions, respectively. Here, we formally assess the efficiency of this decoupled coding by appealing to the information-theoretic notion of description length, which quantifies the amount of information that is saved when encoding novel facial images, with a given precision. We show that despite decoupled coding describes the facial images in terms of two sets of principal components (of landmark shape and image texture), it is more efficient (i.e., yields more information compression) than the encoding in terms of the image principal components only, which corresponds to the widely used eigenface method. The advantage of decoupled coding over eigenface coding increases with image resolution and is especially prominent when coding variants of training set images that only differ in facial expressions. Moreover, we demonstrate that decoupled coding entails better performance in three different tasks: the representation of facial images, the (daydream) sampling of novel facial images, and the recognition of facial identities and gender. In summary, our study provides a first principle perspective on the efficiency and accuracy of the decoupled coding of facial stimuli reported in the primate inferotemporal cortex.

Information representation in an oscillating neural field model modulated by working memory signals.

We study how stimulus information can be represented in the dynamical signatures of an oscillatory model of neural activity-a model whose activity can be modulated by input akin to signals involved in working memory (WM). We developed a neural field model, tuned near an oscillatory instability, in which the WM-like input can modulate the frequency and amplitude of the oscillation. Our neural field model has a spatial-like domain in which an input that preferentially targets a point-a stimulus feature-on the domain will induce feature-specific activity changes. These feature-specific activity changes affect both the mean rate of spikes and the relative timing of spiking activity to the global field oscillation-the phase of the spiking activity. From these two dynamical signatures, we define both a spike rate code and an oscillatory phase code. We assess the performance of these two codes to discriminate stimulus features using an information-theoretic analysis. We show that global WM input modulations can enhance phase code discrimination while simultaneously reducing rate code discrimination. Moreover, we find that the phase code performance is roughly two orders of magnitude larger than that of the rate code defined for the same model solutions. The results of our model have applications to sensory areas of the brain, to which prefrontal areas send inputs reflecting the content of WM. These WM inputs to sensory areas have been established to induce oscillatory changes similar to our model. Our model results suggest a mechanism by which WM signals may enhance sensory information represented in oscillatory activity beyond the comparatively weak representations based on the mean rate activity.

Propofol Reversibly Attenuates Short-Range Microstate Ordering and 20Hz Microstate Oscillations.

Microstate sequences summarize the changing voltage patterns measured by electroencephalography, using a clustering approach to reduce the high dimensionality of the underlying data. A common approach is to restrict the pattern matching step to local maxima of the global field power (GFP) and to interpolate the microstate fit in between. In this study, we investigate how the anesthetic propofol affects microstate sequence periodicity and predictability, and how these metrics are changed by interpolation. We performed two frequency analyses on microstate sequences, one based on time-lagged mutual information, the other based on Fourier transform methodology, and quantified the effects of interpolation. Resting-state microstate sequences had a 20Hz frequency peak related to dominant 10Hz (alpha) rhythms, and the Fourier approach demonstrated that all five microstate classes followed this frequency. The 20Hz periodicity was reversibly attenuated under moderate propofol sedation, as shown by mutual information and Fourier analysis. Characteristic microstate frequencies could only be observed in non-interpolated microstate sequences and were masked by smoothing effects of interpolation. Information-theoretic analysis revealed faster microstate dynamics and larger entropy rates under propofol, whereas Shannon entropy did not change significantly. In moderate sedation, active information storage decreased for non-interpolated sequences. Signatures of non-equilibrium dynamics were observed in non-interpolated sequences, but no changes were observed between sedation levels. All changes occurred while subjects were able to perform an auditory perception task. In summary, we show that low dose propofol reversibly increases the randomness of microstate sequences and attenuates microstate oscillations without correlation to cognitive task performance. Microstate dynamics between GFP peaks reflect physiological processes that are not accessible in interpolated sequences.