Information Theoretic Indices Research Articles

Physical but not virtual presence of others potentiates implicit and explicit learning.

E-learning activities are becoming more and more common. Whilst it is well known that the physical presence of others motivates individuals to engage in perceptual and learning tasks, systematic investigations comparing the effects of physical and virtual co-presence of others on knowledge acquisition are still scarce. Here we investigate the effects of physical and virtual co-presence of others on explicit and implicit learning. In Experiment 1 (discovery sample), retrieval accuracy in a spatial memory task and EEG indexes (mismatch negativity-MMN) of implicit perceptual learning were recorded when participants were alone or in presence of another individual. In Experiment 2 (replicating sample), we added a "virtual" condition, where the same tasks were performed during a video-conference call. In both experiments, MMN was demonstrated to encode for perceptual learning as revealed by the significant correlation with Bayesian Surprise (a consolidated information-theoretic index of Bayesian learning). Furthermore, In Experiments 1 and 2 physical co-presence systematically ameliorated memorization performances and increased MMN indexes related to implicit learning. These positive effects were absent in the virtual condition, thus suggesting that only physical, but not virtual co-presence is effective in potentiating learning dynamics.

Nice and easy: Mismatch negativity responses reveal a significant correlation between aesthetic appreciation and perceptual learning.

Neurocomputational models of cognition have framed aesthetic appreciation within the domain of knowledge acquisition and learning, suggesting that aesthetic appreciation might be considered as a hedonic feedback on successful perceptual learning dynamics. Such hypothesis, however, has never been empirically demonstrated yet. In order to investigate the relationship between aesthetic appreciation and learning, we measured the EEG mismatch negativity (MMN) response to more or less appreciated musical intervals, which is considered as a reliable index of perceptual learning. To this end, we measured the MMN to frequency (Hz) standard and frequency deviant musical intervals (Experiment 1) while participants were asked to judge their beauty. For each single stimulus, we also computed an information-theoretic index of perceptual learning (Bayesian surprise). We found that more appreciated musical intervals were associated with a larger MMN responses, which, in turn, correlated with trial-by-trial fluctuations in Bayesian surprise (Experiment 1). Coherently with previous results, Bayesian surprise was also found to correlate with slower RTs in a detection task of the same stimuli, evidencing that motor behavior is inhibited in presence of surprising sensory states triggering perceptual learning (Experiment 2). Our results provide empirical evidence of the existence of a positive correlation between aesthetic appreciation and EEG indexes of perceptual learning. We argue that the sense of beauty might have evolved to signal the nervous system new sensory knowledge acquisition and motivate the individual to search for informationally profitable stimuli. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The interdependence of frequency, predictability, and informativity in the segmental domain

Abstract It has long been noted that language production seems to reflect a correlation between message redundancy and signal reduction. More frequent words and contextually predictable instances of words, for example, tend to be produced with shorter and less clear signals. The same tendency is observed in the language code (e.g. the phonological lexicon), where more frequent words and words that are typically contextually predictable tend to have fewer segments or syllables. Average predictability in context (informativity) also seems to be an important factor in understanding phonological alternations. What has received little attention so far is the relation between various information-theoretic indices – such as frequency, contextual predictability, and informativity. Although each of these indices has been associated with different theories about the source of the redundancy-reduction link, different indices tend to be highly correlated in natural language, making it difficult to tease apart their effects. We present a computational approach to this problem. We assess the correlations between frequency, predictability, and informativity, and assess when these correlations are likely to create spurious (null or non-null) effects depending on, for example, the amount of data available to the researcher.

Compressing strongly connected subgroups in social networks: An entropy-based approach

ABSTRACTTo detect and study cohesive subgroups of actors is a main objective in social network analysis. What are the respective relations inside such groups and what separates them from the outside. Entropy-based analysis of network structures is an up-and-coming approach. It turns out to be a powerful instrument to detect certain forms of cohesive subgroups and to compress them to superactors without loss of information about their embeddedness in the net: Compressing strongly connected subgroups leaves the whole net’s and the (super-)actors’ information theoretical indices unchanged; i.e., such compression is information-invariant. The actual article relates on the reduction of networks with hundreds of actors. All entropy-based calculations are realized in an expert system shell.

Stochastic Fuzzy Modeling for Ear Imaging Based Child Identification

The unique identification of children is crucial for information technology supported vaccine delivery to the unprivileged population of third world countries. New robust image matching algorithms are required to match two ear photographs taken under nonstandard real-world conditions such as the presence of unwanted background objects in the photographs. This paper applies stochastic fuzzy models to the robust matching of ear images. The local features of the image regions are extracted using a “force-field-like” transformation. The extracted features of an image region are modeled by a stochastic fuzzy system. A region of an image is matched to a region of another image by matching the features of an image’s region with the model of another image’s region. As the model is fuzzy as well as stochastic, a robust matching of features’ data to a model is facilitated by handling any uncertainties arising from fuzziness and randomness of the image features. The study introduces an information-theoretic index for measuring the degree of matching between image features and a model of the features. Several experiments are performed on a database of 750 ear-photographs of children (0–6 years) to justify the novel stochastic fuzzy image matching method.

Critical Changes in Cortical Neuronal Interactions in Anesthetized and Awake Rats.

Neuronal interactions are fundamental for information processing, cognition, and consciousness. Anesthetics reduce spontaneous cortical activity; however, neuronal reactivity to sensory stimuli is often preserved or augmented. How sensory stimulus-related neuronal interactions change under anesthesia has not been elucidated. In this study, the authors investigated the visual stimulus-related cortical neuronal interactions during stepwise emergence from desflurane anesthesia. Parallel spike trains were recorded with 64-contact extracellular microelectrode arrays from the primary visual cortex of chronically instrumented, unrestrained rats (N = 6) at 8, 6, 4, and 2% desflurane anesthesia and wakefulness. Light flashes were delivered to the retina by transcranial illumination at 5- to 15-s randomized intervals. Information theoretical indices, integration and interaction complexity, were calculated from the probability distribution of coincident spike patterns and used to quantify neuronal interactions before and after flash stimulation. Integration and complexity showed significant negative associations with desflurane concentration (N = 60). Flash stimulation increased integration and complexity at all anesthetic levels (N = 60); the effect on complexity was reduced in wakefulness. During stepwise withdrawal of desflurane, the largest increase in integration (74%) and poststimulus complexity (35%) occurred before reaching 4% desflurane concentration-a level associated with the recovery of consciousness according to the rats' righting reflex. Neuronal interactions in the cerebral cortex are augmented during emergence from anesthesia. Visual flash stimuli enhance neuronal interactions in both wakefulness and anesthesia; the increase in interaction complexity is attenuated as poststimulus complexity reaches plateau. The critical changes in cortical neuronal interactions occur during transition to consciousness.

Information content of molecular graph and prediction of gas phase thermal entropy of organic compounds

Entropy is a fundamental thermodynamic property that has attracted a wide attention across domains, including chemistry. Inference of entropy of chemical compounds using various approaches has been a widely studied topic. However, many aspects of entropy in chemical compounds remain unexplained. In the present work, we propose two new information-theoretical molecular descriptors for the prediction of gas phase thermal entropy of organic compounds. The descriptors reflect the bulk and size of the compounds as well as the gross topological symmetry in their structures, all of which are believed to determine entropy. A high correlation () between the entropy values and our information-theoretical indices have been found and the predicted entropy values, obtained from the corresponding statistically significant regression model, have been found to be within acceptable approximation. We provide additional mathematical result in the form of a theorem and proof that might further help in assessing changes in gas phase thermal entropy values with the changes in molecular structures. The proposed information-theoretical molecular descriptors, regression model and the mathematical result are expected to augment predictions of gas phase thermal entropy for a large number of chemical compounds.

Information-theoretic multiplicities of chemical bond in Shull’s model of H2

Alternative information-theoretic (IT) measures of the chemical bond multiplicity and its covalent/ionic composition in the orbital communication theory (OCT) are examined using Shull’s natural orbital (NO) model of the homopolar bond in H2. In OCT a molecule is treated as an information (probability-scattering) system, generated by the network of conditional probabilities (from the quantum mechanical superposition principle) linking elementary events of the adopted perspective. For the first time this atomic orbital (AO) invariant, two-NO description of Shull allows one to examine in several alternative representations the behavior of the previously adopted IT indices, of the channel average communication noise (OCT-covalency) and information flow (OCT-ionicity), with changing internuclear distance R, from the united atom (R = 0 ) to the separated atoms limit (SAL) (R → ∞). The adopted references include the two-electron atomic and ionic functions of the model, as well as the alternative one-electron functions, of the AO and NO sets, respectively. The numerical results for the Wang function description of H2 are reported and a general agreement with the accepted chemical intuition is tested. Joint probabilities of Shull’s reference states are linked to the energy partitioning. The incorrect SAL behavior of the OCT-ionicity index, giving rise to the constant (interaction independent) overall multiplicity measure, emphasizes a need for a revision of these IT bond descriptors. The modified set of indices is proposed, reflecting the complementary localization (determinicity) and delocalization (indeterminicity) aspects of the communication system in question. The novel IT-ionicity now reflects the diagonal (intra-orbital, additive) information propagation in the molecular channel, while the modified IT-covalency accordingly measures the effect of its off-diagonal (inter-orbital, nonadditive) probability scatterings. These components are shown to give rise to the interaction strength dependent overall IT bond-order, which adequately reflects the chemical intuition.

Context enhancement to reveal a camouflaged target and to assist target localization by fusion of multispectral surveillance videos

Camouflage is an attempt to conceal a target by making it look similar to the background to make the detection and recognition of the target difficult for the observer (a man or a machine or both). Detecting a camouflaged target in a video, captured in visible band is a big challenge. One can use the infrared (IR) video where the visibility of the same target is much better, but the problem with IR band is that most of the contrast, color, and edge information about the background are lost and thus localizing the target becomes difficult. The objective of this work is to fuse registered videos, captured in visible and IR band such that the target is no longer camouflaged and hence clearly visible to the human monitor. More importantly, this should be done without losing the background details. We have proposed four different video fusion methods. All the proposed methods are intensity invariant so that camouflaged targets are detected independent of the illumination conditions. The performance of these methods has been compared using Wang–Bovik and Petrovic–Xydeas fusion metric along with other information theoretic indices. Experimental video results show that the proposed methods improve the perception and thus facilitates detection and localization of a camouflaged target. Moreover, the fused video has minimum artefacts as indicated by the highest peak signal-to-noise ratio, which is one of the most desired quality of a good fusion method.

Tuning and regulating the repertoire of glycan determinants

AbstractWe apply Tlusty's information-theoretic index theorem analysis of the genetic code to the glycome, using a cognitive paradigm by which external information sources constrain and tune the glycan code error network, in the context of available metabolic energy. The model suggests spontaneous symmetry breaking of the glycan code as a function of metabolic energy intensity, an effect that may be currently present, or embedded in evolutionary trajectory, recording large-scale ecosystem resilience shifts in energy availability such as the aerobic transition. Once focused on a subset of the glycan error code network however, the glycan production machinery must then be regulated by an elaborate cognitive process to ensure that what is produced matches what has been chosen for production. Index theorem methods emerge in a highly natural manner for this composite system whose failures will be represented by a spectrum of glycan disorders.

Use of the bond-projected superposition principle in determining the conditional probabilities of orbital events in molecular fragments

The problem of defining and determining the multi-conditional probabilities of many-orbital events in the chemical bond system of a molecule is addressed anew within theoretical framework of the one-determinantal orbital representation of molecular electronic structure. Its solution is vital for determining the information-theoretic indices of bond couplings between molecular fragments or the reactant/product subsystems in chemical reactions. The superposition principle of quantum mechanics, appropriately projected into the occupied subspace of molecular orbitals, is used to condition the atomic orbitals or general basis functions of the self-consistent-field calculations. The conditional probabilities between the subspaces of basis functions (atomic orbitals) are derived from an appropriate generalization of the bond-projected superposition principle. They are then used to define the triply-conditional probabilities, relating one conditional event to another. The resulting expression is shown to satisfy the relevant non-negativity and symmetry requirements. It is applied to probe the π-bond coupling in butadiene and benzene.

Molecular modeling of wine polyphenols

Due to wide range of health effects of wine polyphenols, it is important to investigate the relationship between their structure and physical properties (quantitative structure–property relationship, QSPR). We have investigated linear, nonlinear (polynomial), and multiple linear relationships between given topological indices and molecular properties of main pharmacological active components of wine, such as molecular weight (MW), van der Waals volume (Vw), molar refractivity (MR), polar molecular surface area (PSA) and lipophilicity (log P). Partition coefficient (log P) was calculated using three different computer program (CLOGP, ALOGPS and MLOGP). The best models were achieved using the MLOGP program. Topological indices used for correlation analysis include: the Wiener index, W(G); connectivity indices, χ(G); the Balaban index, J(G); information-theoretic index, I(G); and the Schultz index, MTI(G). QSPR was performed on the set of 19 polyphenols and, particularly, on the group of phenolic acids, and on the group of flavonoids with resveratrol. The connectivity index has been successfully used for describing almost all parameters. Significant correlations were achieved between the Wiener index and van der Waals volume, as well as molecular weight.

A Z-weighted information content index

In the study of the effects of chemical structure variations on pharmacological activity, the information content of a molecule may be expressed by derived numeric indices. Any one of these may be used as a parametric descriptor in correlation trials. In a series of azabutadiene sensory irritants, two new information theoretic indices, the Modified Information Content (MIC) Index and the Z-Modified Information Content (ZMIC) Index were used to derive correlation equations relating these indices to respiratory inhibition in mice. The MIC Index uses the atomic weight of constituent atoms of a compound to weigh the individual terms of the Information Content (IC) Index prior to summation; the ZMIC Index uses the atomic number (Z) to weigh the terms. Linear regression trials indicated that the unmodified IC Index and the related Structural Information Content (SIC) Index did not explain the variance in the pharmacological data. However, increasingly adequate correlations were obtained with the MIC Index, followed by molecular weight and then the ZMIC Index. The latter appears to give some topographical character to what would otherwise be a topological index and has the potential for enhancing the usefulness of these methods.

Chemical information theory: Structural aspects

Abstract Information is a concept of no less importance than matter or energy. Information has been de- scribed as a measure of the variety in a given system. We review several of the indices of molecular information, including those indices related to molecular composition, molecular symmetry, graph automorphism, graph coloring, and connectivity as measured by adjacency, path lengths, and cy- clicity. Electronic information indices include those related to the division of electronic structures into u and ?r bond spaces, core and valence spaces, and lone pair, bond pair, or molecular fragment spaces. General molecular information indices have been formed by combining two or more specific indices. Information theoretic indices haye been supplied to problems of isomer discrimination, classification of molecular complexity, and structure-property and structure-activity correlations. The potential for further applications is great. Introduction The concept of information appears to be one of the most fundamental con- ceptions in the science of the 20th century, a concept of no less importance than that of matter or energy. This assertion follows from the very definitions of in- formation. According to Wiener

Information-theoretic indices and an approximate significance test for testing the molecular clock hypothesis with genetic distances

Distance-based phylogenetic methods are widely used in biomedical research. However, distance-based dating of speciation events and the test of the molecular clock hypothesis are relatively underdeveloped. Here I develop an approximate test of the molecular clock hypothesis for distance-based trees, as well as information-theoretic indices that have been used frequently in model selection, for use with distance matrices. The results are in good agreement with the conventional sequence-based likelihood ratio test. Among the information-theoretic indices, AICu is the most consistent with the sequence-based likelihood ratio test. The confidence in model selection by the indices can be evaluated by bootstrapping. I illustrate the usage of the indices and the approximate significance test with both empirical and simulated sequences. The tests show that distance matrices from protein gel electrophoresis and from genome rearrangement events do not violate the molecular clock hypothesis, and that the evolution of the third codon position conforms to the molecular clock hypothesis better than the second codon position in vertebrate mitochondrial genes. I outlined evolutionary distances that are appropriate for phylogenetic reconstruction and dating.

Indices for Testing Neural Codes

One of the most critical challenges in systems neuroscience is determining the neural code. A principled framework for addressing this can be found in information theory. With this approach, one can determine whether a proposed code can account for the stimulus-response relationship. Specifically, one can compare the transmitted information between the stimulus and the hypothesized neural code with the transmitted information between the stimulus and the behavioral response. If the former is smaller than the latter (i.e., if the code cannot account for the behavior), the code can be ruled out. The information-theoretic index most widely used in this context is Shannon's mutual information. The Shannon test, however, is not ideal for this purpose: while the codes it will rule out are truly nonviable, there will be some nonviable codes that it will fail to rule out. Here we describe a wide range of alternative indices that can be used for ruling codes out. The range includes a continuum from Shannon information to measures of the performance of a Bayesian decoder. We analyze the relationship of these indices to each other and their complementary strengths and weaknesses for addressing this problem.

Manifestations of Pauli exclusion principle in communication-theory of the chemical bond

The implications of the Pauli exclusion principle for the entropy/informa- tion indices of the chemical bond formulated within the communication theory approach are explored. The spatial information channels in the local, two-electron resolution are derived for the singlet and triplet states of two electrons, modeling the chemical bonding and non-bonding states in a molecule, respectively. Their average conditional-entropy (covalency) and mutual-information (ionicity) descriptors are compared against those characterizing the separate atoms and an upper-bound to the information-theoretic bond-order for the molecular orbital “events” is determined. An illustrative application to AO channels in H2 generates numerical values of the information-theoretic indices for this prototype covalent bond. The molecular information systems are interpreted as the ensemble averages of the elementary deterministic (zero-covalency) information networks. Examples of such a channel synthesis include model binary channels and that representing the elementary valence-bond (VB) covalent structure in H2. The ensemble representation of the spin channel for the triplet state of two electrons, averaged over the three spin-projection components, offers an entropic perspective on the spin-pairing in the bond-formation process. The spin-paired (singlet) communication system is obtained by maximizing in the ensemble-average communication system of the triplet state the information-flow (ionicity) to its capacity level.

Role of physico-chemical parameters in Quantitative Structure-Activity Relationship based modeling of CYP26A1 inhibitory activity

Quantitative Structure-Activity Relationship (QSAR) studies were performed on CYP26A1 inhibitory activity using physico-chemical parameters like Balaban centric index (BAC), mean Weiner index (WA), Balaban distance connectivity index (J), information theoretic index (Id), zero order ( 0 Χ), first order ( 1 Χ) and second order ( 2 Χ) of molecular connectivity, polarizibility (Pz) and partition coefficient (log P) along with appropriate dummy parameter. The QSAR models were tested for their statistical significance and reliability by using leave one out cross validation method. Some significant models have been reported.

Encoding uncertainty in the hippocampus

The medial temporal lobe may play a critical role in binding successive events into memory while encoding contextual information in implicit and explicit memory tasks. Information theory provides a quantitative basis to model contextual information engendered by conditional dependence between, or conditional uncertainty about, consecutive events in a sequence. We show that information theoretic indices characterizing contextual dependence within a sequential reaction time task (SRTT) predict regional responses, measured by fMRI, in areas associated with sequence learning and navigation. Specifically, activity of a distributed paralimbic system, centered on the left hippocampus, correlated selectively with predictability as measured with mutual information. This is clear evidence that the brain is sensitive to the probabilistic context in which events are encountered. This is potentially important for theories about how the brain represents uncertainty and makes perceptual inferences, particularly those based on predictive coding and hierarchical Bayes.

Partial communication channels of molecular fragments and their entropy/information indices

Within the ‘communication’ theory of the chemical bond the additive decomposition of the global entropy/information indices into corresponding quantities characterizing the partial communication channels of the embedded molecular fragments is examined. The so called row and column channels of molecular subsystems are introduced. They represent alternative communication networks of the constituent fragments in the molecule, which fully take into account the internal (intra-subsystem) and external (between the molecular fragment and its molecular environment) communications (bonds). These concepts are illustrated using the two-orbital model of a chemical bond. The three-orbital model of the triatomic, symmetric transition-state complex is then examined to explore the influence of the system spin polarization on the entropy/information bond descriptors of the internal and external chemical interactions of the model diatomic fragments. Applications to the π-bond systems (butadiene and benzene) in the Hückel theory approximation are also reported and the results are compared with predictions from molecular orbital (MO) theory. These results indicate that the information-theoretic indices of the partial channels of molecular fragments emphasize the information equilibrium of the molecular subsystems, which is manifested by a remarkable equalization of various atomic and diatomic indices, irrespective of the fragment identity. This is in contrast to the complementary MO bond indices, which strongly differentiate between different subsystems, in accordance with the molecule structural formula.