Information Theory Techniques Research Articles

Onshore versus offshore capacity factor and reliability for wind energy production in Germany: 2010–2022

AbstractSimilarities and differences between the features related to the productivity of onshore and offshore wind energy are developed with the aid of information theory techniques complemented by normal statistics. The data comes from the 13‐year period between 2010 and 2022 for the registered turbines in Germany (practically all). The information content of the generated power is dynamically measured by the mutability of the files storing the information. Monthly statistics show that in spite of the Summer months being relatively unproductive, the corresponding mutability shows the possibility of making use of short periods of intermediate productivity in the case of offshore plants. Favorable conditions for wind energy generation in Wintertime are reached for both onshore and offshore production, although the latter is favored. More homogeneity and stability in the data are still necessary to generalize algorithms, protocols, and criteria. This general study shows the success of the information theory techniques in describing wind ramps. Applications to specific zones could improve the efficiency and capacity factor of particular wind turbines depending on their exposure to wind streams or the blocking of nearby mountains and forestry. The information theory techniques presented here allow for a different and novel viewpoint to detect favorable and unfavorable wind energy periods.

Large-scale analysis of the N-terminal regulatory elements of the kinase domain in plant Receptor-like kinase family

BackgroundThe N-terminal regulatory element (NRE) of Receptor-like kinases (RLKs), consisting of the juxtamembrane segment in receptor kinases (RKs) and the N-terminal extension segment in RLCKs, is a crucial component that regulates the activities of these proteins. However, the features and functions of the NRE have remained largely unexplored. Herein, we comprehensively analyze 510,233 NRE sequences in RLKs from 528 plant species, using information theory and data mining techniques to unravel their common characteristics and diversity. We also use recombinant RKs to investigate the function of the NRE in vitro.ResultsOur findings indicate that the majority of NRE segments are around 40–80 amino acids in length and feature a serine-rich region and a 14-amino-acid consensus sequence, ‘FSYEELEKAT[D/N]NF[S/D]’, which contains a characteristic α-helix and ST motif that connects to the core kinase domain. This conserved signature sequence is capable of suppressing FERONIA’s kinase activity. A motif discovery algorithm identifies 29 motifs with highly conserved phosphorylation sites in RK and RLCK classes, especially the motif ‘VGPWKpTGLpSGQLQKAFVTGVP’ in LRR-VI-2 class. Phosphorylation of an NRE motif in an LRR-VI-2 member, MDIS1, modulates the auto-phosphorylation of its co-receptor, MIK1, indicating the potential role of NRE as a ‘kinase switch’ in RLK activation. Furthermore, the characterization of phosphorylatable NRE motifs improves the accuracy of predicting phosphorylatable sites.ConclusionsOur study provides a comprehensive dataset to investigate NRE segments from individual RLKs and enhances our understanding of the underlying mechanisms of RLK signal transduction and kinase activation processes in plant adaptation.

SPILLOVER EFFECTS OF COVID-19 ON USA EDUCATION GROUP STOCKS

In this paper, we explore the price dynamics of 16 representative records of USA Education Group stocks, encompassing two non-overlapping periods (before, during, and after COVID-19). Based on information theory and cluster analysis techniques, our study provides insights into disorder, predictability, efficiency, similarity, and resilience/weakness, considering the most diverse financial stakeholders. Our findings contribute significantly to understanding price dynamics in the education sector, encompassing non-crisis and crisis periods, and provide valuable information for financial risk management, highlighting the importance of considering financial assets’ efficiency and resilience/weakness during economic turbulence, such as the COVID-19 crisis.

The effective enhancement of information in 3D small-world networks of biological neuronal cells

The cardiovascular system, the kidney, or the brain, are examples of complex systems - where the properties of the systems arise because of the layout of cells in those systems. One way to characterize systems is using networks, where elements and interactions of the systems are represented as nodes and links of a graph. Network’s topology can be, in turn, measured by the small-world coefficient. Small world networks feature increased clustering and shorter paths compared to random or periodic networks of the same size. This suggests that systems with small world attributes can also efficiently transport signals, nutrients, or information within their body. While several reports in literature have illustrated that real biological systems are small-world, yet little is known about how information varies as a function of the small-world-ness () of three dimensional graphs. Here, we used a model of the brain to estimate quantitatively the information processed in 3D networks. In the model, nodes of the graph are neuronal units capable to receive, integrate and transmit signals to other neurons of the system in parallel. The information encoded in the signals was then extracted using the techniques of information theory. In simulations where the topology of networks of nodes was varied over large intervals, we found that in the range information scales linearly with the small world coefficient, with a five-fold increase. Results of the paper and review of the existing literature on model organisms suggest that a small-world architecture may offer an evolutionary advantage to biological systems.

Examining the origins of observed terahertz modes from an optically pumped atomistic model protein in aqueous solution.

The microscopic origins of terahertz (THz) vibrational modes in biological systems are an active and open area of current research. Recent experiments [Phys Rev X. 8, 031061 (2018)] have revealed the presence of a pronounced mode at ∼0.3 THz in fluorophore-decorated bovine serum albumin (BSA) protein in aqueous solution under nonequilibrium conditions induced by optical pumping. This result was heuristically interpreted as a collective elastic fluctuation originating from the activation of a low-frequency phonon mode. In this work, we show that the sub-THz spectroscopic response emerges in a statistically significant manner () from such collective behavior, illustrating how photoexcitation can alter specific THz vibrational modes. We revisit the theoretical analysis with proof-of-concept molecular dynamics that introduce optical excitations into the simulations. Using information theory techniques, we show that these excitations can give rise to a multiscale response involving two optically excited chromophores (tryptophans), other amino acids in the protein, ions, and water. Our results motivate new experiments and fully nonequilibrium simulations to probe these phenomena, as well as the refinement of atomistic models of Fröhlich condensates that are fundamentally determined by nonlinear interactions in biology.

Complexity analysis of VMAT prostate plans: insights from dimensionality reduction and information theory techniques

Abstract Introduction: Volumetric Modulated Arc Therapy (VMAT) is a state-of-the-art prostate cancer treatment, defined by high dose gradients around targets. Its unique dose shaping incurs hidden complexity, impacting treatment deliverability, carcinogenesis, and machine strain. This study compares various aperture-based VMAT complexity indices in prostate cases using principal component and mutual information analyses. It suggests essential properties for an ideal complexity index from an information-theoretic viewpoint. Material and methods: The following ten complexity indices were calculated in 217 VMAT prostate plans: circumference over area (CoA), edge metric (EM), equivalent square field (ESF), leaf travel (LT), leaf travel modulation complexity score for VMAT (LTMCSV), mean-field area (MFA), modulation complexity score (standard MCS and VMAT variant MCSV), plan irregularity (PI), and small aperture score (SAS5mm). Principal component analysis (PCA) was applied to explore the correlations between the metrics. The differential entropy of all metrics was also calculated, along with the mutual information for all 45 metric pairs. Results: Whole-pelvis plans had greater complexity across all indices. The first three principal components explained 96.2% of the total variance. The complexity metrics formed three groups with similar conceptual characteristics, particularly ESF, LT, MFA, PI, and EM, SAS5mm. The differential entropy varied across the complexity metrics (PI having the smallest vs. EM the largest). Mutual information analysis (MIA) confirmed some metrics’ interdependence, although other pairs, such as LTMCSV/SAS5mm, LT/MCSV, and EM/SAS5mm, were found to share minimal MI. Conclusions: There are many complexity indices for VMAT described in the literature. PCA and MIA analyses can uncover significant overlap among them. However, this is not entirely reducible through dimensionality reduction techniques, suggesting that there also exists some reciprocity. When designing predictive models of quality assurance metrics, PCA and MIA may prove useful for feature engineering.

Evolution of Sequence and Structure of SARS-CoV-2 Spike Protein: A Dynamic Perspective.

Novel coronavirus (SARS-CoV-2) enters its host cell through a surface spike protein. The viral spike protein has undergone several modifications/mutations at the genomic level, through which it modulated its structure-function and passed through several variants of concern. Recent advances in high-resolution structure determination and multiscale imaging techniques, cost-effective next-generation sequencing, and development of new computational methods (including information theory, statistical methods, machine learning, and many other artificial intelligence-based techniques) have hugely contributed to the characterization of sequence, structure, function of spike proteins, and its different variants to understand viral pathogenesis, evolutions, and transmission. Laying on the foundation of the sequence-structure-function paradigm, this review summarizes not only the important findings on structure/function but also the structural dynamics of different spike components, highlighting the effects of mutations on them. As dynamic fluctuations of three-dimensional spike structure often provide important clues for functional modulation, quantifying time-dependent fluctuations of mutational events over spike structure and its genetic/amino acidic sequence helps identify alarming functional transitions having implications for enhanced fusogenicity and pathogenicity of the virus. Although these dynamic events are more difficult to capture than quantifying a static, average property, this review encompasses those challenging aspects of characterizing the evolutionary dynamics of spike sequence and structure and their implications for functions.

Evaluation Method of Equalization of Basic Medical Services from the Spatial Perspective: The Case of Xinjiang, China

Protecting residents’ health and improving equality are important goals of the United Nations Sustainable Development Goals. The recent outbreak of COVID-19 has placed a heavy burden on the medical systems of many countries and been disastrous for the low-income population of the world, which has further increased economic, health, and lifelong inequality in society. One way to improve the population’s health is to equalize basic medical services. A scientific evaluation of the status quo or the equalization of basic medical services (EBMS) is the basic prerequisite and an important basis for realizing the equitable allocation of medical resources. Traditional evaluation methods ignore the spatial characteristics of medical services, mostly using the indicator of equal weight evaluation, which restricts the objectivity of the evaluation results. Given this, this research proposes a set of EBMS evaluation methods from a spatial perspective and takes the Xinjiang Uygur Autonomous Region of China (Xinjiang) as an example for studying the status quo of EBMS. This study puts forward a set of EBMS evaluation methods from a geospatial perspective and makes full use of spatial analysis and information theory techniques to construct a two-level evaluation indicator that takes into account the spatial characteristics of EBMS. The entropy weight method and the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method have been used to reveal the current status quo of EBMS in Xinjiang to objectively reflect the differences in EBMS. When using the entropy and TOPSIS methods, the evaluation is always based on the data so that the results can more objectively reveal the medical resources available to the residents. Therefore, the government can realize a reasonable allocation of medical resources.

IBD: An Interpretable Backdoor-Detection Method via Multivariate Interactions.

Recent work has shown that deep neural networks are vulnerable to backdoor attacks. In comparison with the success of backdoor-attack methods, existing backdoor-defense methods face a lack of theoretical foundations and interpretable solutions. Most defense methods are based on experience with the characteristics of previous attacks, but fail to defend against new attacks. In this paper, we propose IBD, an interpretable backdoor-detection method via multivariate interactions. Using information theory techniques, IBD reveals how the backdoor works from the perspective of multivariate interactions of features. Based on the interpretable theorem, IBD enables defenders to detect backdoor models and poisoned examples without introducing additional information about the specific attack method. Experiments on widely used datasets and models show that IBD achieves a 78% increase in average in detection accuracy and an order-of-magnitude reduction in time cost compared with existing backdoor-detection methods.

Role of Age and Education as the Determinant of Income Inequality in Poland: Decomposition of the Mean Logarithmic Deviation.

Measures of inequality can be used to illustrate inequality between and within groups, but the choice of the appropriate measure can have different implications. This study focused on the Mean Logarithmic Deviation, the measure proposed by Theil and based on the techniques of statistical information theory. The MLD was selected because of its attractive properties: fulfillment of the principle of monotonicity and the possibility of additive decomposition. The following study objectives were formulated: (1) to assess the degree of inequality in the population and in the distinguished subgroups, (2) to determine the extent to which education and age influence the level of inequality, and (3) to ascertain what factors contribute to changes in the level of inequality in Poland. The study confirmed an association between the level of education and the average income of the groups distinguished on this basis. The education level of the household head remains an important determinant of household income inequality in Poland, despite the decline in the “educational bonus”. The study also found that differences in the age of the household head had a smaller effect on income inequality than the level of education. However, it can be concluded that the higher share of older people may contribute to an increase in income inequality between groups, as the income from pension in Poland is more homogeneous than the income from work in younger groups. Moreover, the current paper seeks to situate Theil’s approach in the context of scholarly writings since 1967.

Multi-label feature selection based on label correlations and feature redundancy

The task of multi-label feature selection (MLFS) is to reduce redundant information and generate the optimal feature subset from the original multi-label data. A variety of MLFS methods utilize pseudo-label matrix to explore label correlations for identifying the most informative features. Moreover, some methods consider feature redundancy by virtue of information theory technique, but no prior literature unites them in a framework to perform feature selection. To remedy the deficiency, we propose a novel MLFS method based on label correlations and feature redundancy, namely LFFS. To be specific, we first utilize the ridge regression to create a feature selection matrix and a low dimensional embedding, and impose ℓ2,1-norm on the feature selection matrix. Then, the low-dimensional embedding is devoted to mine label correlations, which can keep the global and local structure of original label space. Finally, cosine similarity is employed to analyze feature redundancy, so as to generate a low redundancy feature subset. By virtue of the above process, we design an objective function followed with an optimization solution. Comprehensive experiments results demonstrate the effectiveness and superiority of the proposed method LFFS among ten competition methods.

An independent search for annual modulation and its significance in ANAIS-112 data

Abstract We perform an independent search for sinusoidal-based modulation in the recently released ANAIS-112 data, which could be induced by dark matter scatterings. We then evaluate this hypothesis against the null hypothesis that the data contain only background, using four different model comparison techniques. These include frequentist, Bayesian, and two information theory-based criteria (Akaike and Bayesian information criteria). This analysis was done on both the residual data (by subtracting the exponential fit obtained from the ANAIS-112 Collaboration) as well as the total (non-background subtracted) data. We find that according to the Bayesian model comparison test, the null hypothesis of no modulation is decisively favored over a cosine-based annual modulation for the non-background subtracted dataset in the 2–6 keV energy range. None of the other model comparison tests decisively favor any one hypothesis over another. This is the first application of Bayesian and information theory techniques to test the annual modulation hypothesis in ANAIS-112 data, extending our previous work on the DAMA/LIBRA and COSINE-100 data. Our analysis codes have also been made publicly available.

Information Acquisition Driven by Reinforcement in Non-Deterministic Environments

What is the fastest way for an agent living in a non-deterministic Markov environment (NME) to learn about its statistical properties? The answer is to create "optimal" experiment sequences by carrying out action sequences that maximize expected knowledge gain. This idea is put into practice by integrating information theory and reinforcement learning techniques. Experiments demonstrate that the resulting method, reinforcement-driven information acquisition (RDIA), is substantially faster than standard random exploration for exploring particular NMEs. Exploration was studied apart from exploitation and we evaluated the performance of different reinforcement-driven information acquisition variations to that of traditional random exploration.

Energy use prediction with information theory and machine learning technique

Appliances energy consumption plays an increasingly important role in the overall building electric energy consumption and its temporal trending. However, predicting appliances energy consumption is complicated by lack of causal understanding of the appliances energy use as well as too many potential predictors that might be relevant to the appliances energy use. In this study, we apply information theory and advance machine learning neural network technique to first rank the importance of potential drivers that dominate appliances energy consumption and secondly model the temporal evolution of appliances energy consumption with a restricted set of environmental predictors. Our results showed that temperature and humidity were the two most important environmental drivers in the house appliances energy consumption modeling. Furthermore, using those environmental drivers, the machine learning model was able to accurately capture the temporal dynamics of appliances energy consumption.

Brays and bits: information theory applied to acoustic communication sequences of bottlenose dolphins

Acoustic call sequences are important components of vocal repertoires for many animal species. Bottlenose dolphins (Tursiops truncatus) produce a wide variety of vocalizations, in different behavioural contexts, including some conspicuous vocal sequences – the ‘bray series’. The occurrence of brays is still insufficiently documented, contextually and geographically, and the specific functions of these multi-unit emissions are yet to be understood. Here, acoustic emissions produced by bottlenose dolphins in the Sado estuary, Portugal, were used to provide a structural characterization of the discrete elements that compose the bray series. Information theory techniques were applied to analyse bray sequences and explore the complexity of these calls. Log-frequency analysis, based on bout criterion interval, confirmed the bout structure of the bray series. A first-order Markov model revealed a distinct pattern of emission for the bray series’ elements, with uneven transitions between elements. The order in these sequential emissions was not random and consecutive decreases in higher order entropy values support the notion of a well-defined structure in the bray series. The key features of animal signal sequences here portrayed suggest the presence of relevant information content and highlight the complexity of the bottlenose dolphin’s acoustic repertoire.

Stochastic and Information Theory Techniques to Reduce Large Datasets and Detect Cyberattacks in Ambient Intelligence Environments

Ambient intelligence refers a new technological paradigm, where everyday environments behave in a smart way and are sensitive to their inhabitants. In order to reach this objective, complex pervasive sensing platforms are deployed, together with artificial intelligence solutions. In these new, complex, and highly interdependent systems, traditional security policies and defense strategies are not effective, as thousands of heterogeneous cyber and physical elements are mixed and connected. New security solutions try to learn about the expected behavior from the system and its components, so if a strange event occurs; adequate preventive, corrective, and/or reactive security actions to detect and stop the potential cyber-physical attack being performed are triggered in an intelligent way. In order to learn about the system and select and apply the adequate security actions, very large datasets containing records of previous behaviors should be analyzed, sometimes in a very fast way. This fact enormously complicates the implementation of these new security solutions, as it is necessary a huge storage capacity, which many domestic systems do not have, and it is needed to work with huge data sets whose processing time prevents making decisions with the required speed. Therefore, in this paper, we investigate and propose a procedure to reduce large datasets, with the objective of enabling new security techniques to detect cyberattacks in a fast and efficient way. The proposed procedure is based on the calculation of small sets of samples, whose statistic configuration is as similar as desired to the original large dataset. Stochastic models and information theory techniques and theorems are composed and combined in order to define a mathematical framework which allows the obtention of these equivalent reduced datasets. We also describe and evaluate a first implementation of the proposed solution, using both, a simulation scenario and a real deployment.

Phase transitions in a system of long rods on two-dimensional lattices by means of information theory.

The orientational phase transitions that occur in the deposition of longitudinal polymers of length k (in terms of lattice units) are characterized by information theory techniques. We calculate the absolute value of an order parameter δ, which weights the relative orientations of the deposited rods, which varies between 0.0 (random orientation) and 1.0 (fully oriented in either of the two equivalent directions in an L×L square lattice). A Monte Carlo (MC) algorithm is implemented to induce a dynamics allowing for accommodation of the rods for any given density or coverage θ (ratio of the occupied sites over all the sites in the lattice). The files storing δ(t) (with time t measured in MC steps) are then treated by data recognizer wlzip based on data compressor techniques yielding the information content measured by a parameter η(θ). This allows us to recognize two maxima separated by a well-defined minimum for η(θ) provided k≥7. The first maximum is associated with an isotropic-nematic (I-N) phase transition occurring at intermediate density, while the second maximum is associated with some kind of nematic-isotropic transition at high coverage. In the cases of k<7, the curves for η(θ) are almost constant, presenting a very broad maximum which can hardly be associated with a phase transition. The study varies L and k, allowing for a basic scaling of the found critical densities towards the thermodynamic limit. These calculations confirm the tendency obtained by different methods in the case of the intermediate-density I-N phase transition, while this tendency is established here in the case of the high-density phase transition.

Inferring neuronal network functional connectivity with directed information

A major challenge in neuroscience is to develop effective tools that infer the circuit connectivity from large-scale recordings of neuronal activity patterns. In this study, context tree maximizing (CTM) was used to estimate directed information (DI), which measures causal influences among neural spike trains in order to infer putative synaptic connections. In contrast to existing methods, the method presented here is data driven and can readily identify both linear and nonlinear relations between neurons. This CTM-DI method reliably identified circuit structures underlying simulations of realistic conductance-based networks. It also inferred circuit properties from voltage-sensitive dye recordings of the buccal ganglion of Aplysia. This method can be applied to other large-scale recordings as well. It offers a systematic tool to map network connectivity and to track changes in network structure such as synaptic strengths as well as the degrees of connectivity of individual neurons, which in turn could provide insights into how modifications produced by learning are distributed in a neural network.NEW & NOTEWORTHY This study brings together the techniques of voltage-sensitive dye recording and information theory to infer the functional connectome of the feeding central pattern generating network of Aplysia. In contrast to current statistical approaches, the inference method developed in this study is data driven and validated by conductance-based model circuits, can distinguish excitatory and inhibitory connections, is robust against synaptic plasticity, and is capable of detecting network structures that mediate motor patterns.

Quantitative assessment of drivers of recent global temperature variability: an information theoretic approach

Identification and quantification of possible drivers of recent climate variability remain a challenging task. This important issue is addressed adopting a non-parametric information theory technique, the Transfer Entropy and its normalized variant. It distinctly quantifies actual information exchanged along with the directional flow of information between any two variables with no bearing on their common history or inputs, unlike correlation, mutual information etc. Measurements of greenhouse gases, CO2, CH4, and N2O; volcanic aerosols; solar activity: UV radiation, total solar irradiance (TSI ) and cosmic ray flux (CR); El Nino Southern Oscillation (ENSO) and Global Mean Temperature Anomaly (GMTA) made during 1984-2005 are utilized to distinguish driving and responding climate signals. Estimates of their relative contributions reveal that CO 2 (~24%), CH 4 (~19%) and volcanic aerosols (~23%) are the primary contributors to the observed variations in GMTA. While, UV (~9%) and ENSO (~12%) act as secondary drivers of variations in the GMTA, the remaining play a marginal role in the observed recent climate variability. Interestingly, ENSO and GMTA mutually drive each other at varied time lags. This study assists future modelling efforts in climate science.

Imaging via complete cantilever dynamic detection: general dynamic mode imaging and spectroscopy in scanning probe microscopy

We develop and implement a multifrequency spectroscopy and spectroscopic imaging mode, referred to as general dynamic mode (GDM), that captures the complete spatially- and stimulus dependent information on nonlinear cantilever dynamics in scanning probe microscopy (SPM). GDM acquires the cantilever response including harmonics and mode mixing products across the entire broadband cantilever spectrum as a function of excitation frequency. GDM spectra substitute the classical measurements in SPM, e.g. amplitude and phase in lock-in detection. Here, GDM is used to investigate the response of a purely capacitively driven cantilever. We use information theory techniques to mine the data and verify the findings with governing equations and classical lock-in based approaches. We explore the dependence of the cantilever dynamics on the tip–sample distance, AC and DC driving bias. This approach can be applied to investigate the dynamic behavior of other systems within and beyond dynamic SPM. GDM is expected to be useful for separating the contribution of different physical phenomena in the cantilever response and understanding the role of cantilever dynamics in dynamic AFM techniques.