Information-based Measures Research Articles

A novel clustering method for complex signals and feature extraction based on advanced information-based dissimilarity measure

In this paper, a new dissimilarity measure for more accurate feature extraction and clustering is put forward. The method is proposed from the perspective of the weighted-probability distribution of dispersion patterns and their rank order statistics, where an effective quantization procedure is provided and the loss of information can be reduced. The proposed dissimilarity is applied in the multidimensional scaling (MDS) method to investigate simulated and reality-based signals. The comparative experiment shows that the clustering results of the proposed technique are clearer and more appropriate. State-of-the-art techniques and conventional methods are both included in the comparative experiments. In particular, for the heartbeat signals, it is discovered that the distribution of the weighted-probabilities of the dispersion patterns can discriminate subjects with different physiological conditions and exhibit visible changes of the subject’s dynamical features when aging and disease attacks are taken place, which can be regarded as a microscopic insight of the dynamical mechanisms.

Short-Form Video Exposure and Its Two-Sided Effect on the Physical Activity of Older Community Women in China: Secondary Data Analysis.

There is a tendency for older adults to become more physically inactive, especially older women. Physical inactivity has been exacerbated since the COVID-19 pandemic. Lockdowns and information-based preventive measures for COVID-19 increased the number of short-form video app users and short-form video exposure, including content exposure and the duration of exposure, which has demonstrated important effects on youths' health and health-related behaviors. Despite more older adults viewing short-form videos, less is known about the status of their short-form video exposure or the impacts of the exposure on their physical activity. This study aims to describe physical activity-related content exposure among older adults and to quantify its impacts along with the duration of short-form video exposure on step counts, low-intensity physical activity (LPA), and moderate-to-vigorous physical activity (MVPA). We analyzed a subsample (N=476) of older women who used smartphones and installed short-form video apps, using the baseline data collected from an ongoing cohort study named the Physical Activity and Health in Older Women Study (PAHIOWS) launched from March to June 2021 in Yantai, Shandong Province, China. The information on short-form video exposure was collected by unstructured questions; physical activity-related content exposure was finalized by professionals using the Q-methodology, and the duration of exposure was transformed into hours per day. Step counts, LPA, and MVPA were assessed with ActiGraph wGT3X-BT accelerometers. Multiple subjective and objective covariates were assessed. Linear regression models were used to test the effects of short-form video exposure on step counts, LPA, and MVPA. MVPA was dichotomized into less than 150 minutes per week and 150 minutes or more per week, and the binary logistic regression model was run to test the effects of short-form video exposure on the achievement of spending 150 minutes or more on MVPA. Of 476 older women (mean age 64.63, SD 2.90 years), 23.7% (113/476) were exposed to physical activity-related short-form videos, and their daily exposure to short-form videos was 1.5 hours. Physical activity-related content exposure increased the minutes spent on MVPA by older women (B=4.14, 95% CI 0.13-8.15); the longer duration of short-form video exposure was associated with a reduced step count (B=-322.58, 95% CI -500.24 to -144.92) and minutes engaged in LPA (B=-6.95, 95% CI -12.19 to -1.71) and MVPA (B=-1.56, 95% CI -2.82 to -0.29). Neither content exposure nor the duration of exposure significantly increased or decreased the odds of older women engaging in MVPA for 150 minutes or more per week. Short-form video exposure has both positive and negative impacts on the physical activity of older adults. Efforts are needed to develop strategies to leverage the benefits while avoiding the harms of short-form videos.

Dynamical independence: Discovering emergent macroscopic processes in complex dynamical systems.

We introduce a notion of emergence for macroscopic variables associated with highly multivariate microscopic dynamical processes. Dynamical independence instantiates the intuition of an emergent macroscopic process as one possessing the characteristics of a dynamical system "in its own right," with its own dynamical laws distinct from those of the underlying microscopic dynamics. We quantify (departure from) dynamical independence by a transformation-invariant Shannon information-based measure of dynamical dependence. We emphasize the data-driven discovery of dynamically independent macroscopic variables, and introduce the idea of a multiscale "emergence portrait" for complex systems. We show how dynamical dependence may be computed explicitly for linear systems in both time and frequency domains, facilitating discovery of emergent phenomena across spatiotemporal scales, and outline application of the linear operationalization to inference of emergence portraits for neural systems from neurophysiological time-series data. We discuss dynamical independence for discrete- and continuous-time deterministic dynamics, with potential application to Hamiltonian mechanics and classical complex systems such as flocking and cellular automata.

Nonclassicality versus quantum non-Gaussianity of photon-subtracted displaced Fock state

In this paper, a quantitative investigation of the non-classical and quantum non-Gaussian characters of the photon-subtracted displaced Fock state [Formula: see text], where k is the number of photons subtracted and n is Fock parameter, is performed by using a collection of measures like Wigner logarithmic negativity, linear entropy potential, skew information-based measure, and relative entropy of quantum non-Gaussianity. It is noticed that the number of photons subtracted ( k) changes the nonclassicality and quantum non-Gaussianity in a significant amount in the regime of small values of the displacement parameter, whereas the Fock parameter ( n) presents a notable change in the large regime of the displacement parameter. In this respect, the role of the Fock parameter is found to be stronger as compared to the photon subtraction number. Finally, the Wigner function dynamics considering the effects of photon loss channel is used to show that the Wigner negativity can only be detected by highly efficient detectors.

An entropy-based measure of correlation for time series

In this paper, an information-based measure of association between time series, called information-based correlation coefficient (ICC), is introduced to potentially overcome some of the problems related to Pearson's correlation coefficient and other commonly used coefficients. The ICC, besides its simple definition, seems to have other interesting advantages over traditional correlation coefficients, such as a more natural interpretation in terms of the reciprocal informativeness between two series, higher reliability for independent white noise, Student's t, and AR(1) processes, higher reliability in the presence of spurious correlation and outliers, and a well-developed mathematical theory based on entropy. The ICC can also detect a number of nonlinear relationships, although it may not be equitable and general. Moreover, the ICC can be computed also for ordinal data, and it offers higher reliability for independent ordinal-valued time series. Monte Carlo simulations, applications to real data, properties of the estimator and asymptotics under independence are also discussed in this paper. In particular, the paper includes applications to signal processing, chaotic time series, macroeconomic and weather data.

Information-based estimation of causality networks from high-dimensional multivariate time series

Abstract One of the most challenging aspects in the study of the complex dynamical systems is the estimation of their underlying, interdependence structure. Being in the era of Big Data, this problem gets even more complicated since more observed variables are available. To estimate direct causality effects in this setting, dimension reduction has to be employed in the Granger causality measure. The measure should also be capable to detect non-linear effects, persistently present in real-world complex systems. The model-free information-based measure of partial mutual information from mixed embedding (PMIME) has been developed to address these issues and it was found to perform well on multivariate time series of moderately high dimension. Here, the problem of forming complex networks from direct, possibly non-linear, high-dimensional time series at the order of hundreds is investigated. The performance of the measure PMIME is tested on two coupled dynamical systems in discrete time (coupled Hénon maps) and continuous time (coupled Mackey–Glass delay differential equations). It is concluded that the correct detection of the underlying causality network depends mainly on the network density rather than on its size (number of nodes). Finally, the effect of network size is investigated in the study of the British stock market in the period around Brexit.

Should we adjust health expenditure for age structure on health systems efficiency? A worldwide analysis

IntroductionHealthcare expenditure, a common input used in health systems efficiency analyses is affected by population age structure. However, while age structure is usually considered to adjust health system outputs, health expenditure and other inputs are seldom adjusted. We propose methods for adjusting Health Expenditure per Capita (HEpC) for population age structure on health system efficiency analyses and assess the goodness-of-fit, correlation, reliability and disagreement of different approaches.MethodsWe performed a worldwide (188 countries) cross-sectional study of efficiency in 2015, using a stochastic frontier analysis. As single outputs, healthy life expectancy (HALE) at birth and at 65 years-old were considered in different models. We developed five models using as inputs: (1) HEpC (unadjusted); (2) age-adjusted HEpC; (3) HEpC and the proportion of 0–14, 15–64 and 65 + years-old; (4) HEpC and 5-year age-groups; and (5) HEpC ageing index. Akaike and Bayesian information criteria, Spearman’s rank correlation, intraclass correlation coefficient and information-based measure of disagreement were computed.ResultsModels 1 and 2 showed the highest correlation (0.981 and 0.986 for HALE at birth and HALE at 65 years-old, respectively) and reliability (0.986 and 0.988) and the lowest disagreement (0.011 and 0.014). Model 2, with age-adjusted HEpC, presented the lowest information criteria values.ConclusionsDespite different models showing good correlation and reliability and low disagreement, there was important variability when age structure is considered that cannot be disregarded. The age-adjusted HE model provided the best goodness-of-fit and was the closest option to the current standard.

Efficient quantum information probes of nonequilibrium quantum criticality

Quantum information-based approaches, in particular the fidelity, have been flexible probes for phase boundaries of quantum matter. A major hurdle to a more widespread application of fidelity and other quantum information measures to strongly correlated quantum materials is the inaccessibility of the fidelity susceptibility to most state-of-the-art numerical methods. This is particularly apparent away from equilibrium where, at present, no general critical theory is available and many standard techniques fail. Motivated by the usefulness of quantum information-based measures we show that a widely accessible quantity, the single-particle affinity, is able to serve as a versatile instrument to identify phase transitions beyond Landau’s paradigm. We demonstrate that it not only is able to signal previously identified nonequilibrium phase transitions but also has the potential to detect hitherto unknown phases in models of quantum matter far from equilibrium.

Association Measure and Compact Prediction for Chemical Process Data from an Information-Theoretic Perspective

Mutual information (MI) has been widely used for association mining in complex chemical processes, but how to precisely estimate MI between variables of different numerical types, discriminate their association relationships with targets and finally achieve compact and interpretable prediction has not been discussed in detail, which may limit MI in more complicated industrial applications. Therefore, this paper first reviews the existing information-based association measures and proposes a general framework, GIEF, to consistently detect associations and independence between different types of variables. Then, the study defines four mutually exclusive association relations of variables from an information-theoretic perspective to guide feature selection and compact prediction in high-dimensional processes. Based on GIEF and conditional mutual information maximization (CMIM), a new algorithm, CMIM-GIEF, is proposed and tested on a fluidized catalytic cracking (FCC) process with 217 variables, one which achieves significantly improved accuracies with fewer variables in predicting the yields of four crucial products. The compact variables identified are also consistent with the results of Shapley Additive exPlanations (SHAP) and industrial experience, proving good adaptivity of the method for chemical process data.

A Unified Understanding of Deep NLP Models for Text Classification.

The rapid development of deep natural language processing (NLP) models for text classification has led to an urgent need for a unified understanding of these models proposed individually. Existing methods cannot meet the need for understanding different models in one framework due to the lack of a unified measure for explaining both low-level (e.g., words) and high-level (e.g., phrases) features. We have developed a visual analysis tool, DeepNLPVis, to enable a unified understanding of NLP models for text classification. The key idea is a mutual information-based measure, which provides quantitative explanations on how each layer of a model maintains the information of input words in a sample. We model the intra- and inter-word information at each layer measuring the importance of a word to the final prediction as well as the relationships between words, such as the formation of phrases. A multi-level visualization, which consists of a corpus-level, a sample-level, and a word-level visualization, supports the analysis from the overall training set to individual samples. Two case studies on classification tasks and comparison between models demonstrate that DeepNLPVis can help users effectively identify potential problems caused by samples and model architectures and then make informed improvements.

Measuring health expectancy in the European Union

Abstract Background Healthy life expectancy (HLE) is a population health measure that combines mortality and morbidity, which can be calculated using different methods. In this study, we aimed to assess the correlation, reliability and (dis)agreement between two estimates monitored in the European Union (EU), that is, the European Commission's HLE based on self-perceived health (SPH-HLE) and the Institute for Health Metrics and Evaluation's HLE based on disability weight (DW-HLE), by sex, and comparing these results with LE and proportion of life spent in good health (%GH). Methods We performed a retrospective study in the EU28 countries, between 2010 and 2017. The HLE methods differ in definition, measurement and valuation of health states. While SPH-HLE relies directly on one question, DW-HLE relies on epidemiological data adjusted for DW. Spearman's r, intraclass correlation coefficient, information-based measure of disagreement and Bland-Altman plots were used to assess reliability, correlation and disagreement in HLE resulting from both methods and in LE or %GH measured by both institutions. Results Correlation and reliability between SPH-HLE and DW-HLE were good (better for males), with low disagreement, and were even better for LE between both institutions. The HLE Bland-Altman plots suggest a variability range of approximately 6 years for both sexes, higher for females. There was also an increasing HLE difference between methods with higher average HLE for both sexes. Conclusions We showed wide variations between both methods with a clear and different high impact on female and male HLE, showing a tendency for countries with higher health expectancies to yield larger gaps between SPH-HLE and DW-HLE. Acknowledgements: This presentation was supported by National Funds through FCT - Fundação para a Ciência e a Tecnologia,I.P., within CINTESIS, R&D Unit (reference UIDP/4255/2020) Key messages • Different methods for evaluating health expectancy lead to significantly different results. • There is a systematic tendency with countries with higher health expectancies to yield larger gaps between SPH-HLE and DW-HLE.

Skew information-based coherence generating power of quantum channels

We study the ability of a quantum channel to generate quantum coherence when it applies to incoherent states. We define the measure of coherence generating power (CGP) for a generic quantum channel to be the average coherence generated by the quantum channel acting on a uniform ensemble of incoherent states based on the skew information-based coherence measure. We present explicitly the analytical formulae of the CGP for any arbitrary finite dimensional unitary channels. We derive the mean value of the CGP over the unitary groups and investigate the typicality of the normalized CGP. Furthermore, we give an upper bound of the CGP for the convex combinations of unitary channels. Detailed examples are provided to calculate exactly the values of the CGP for the unitary channels related to specific quantum gates and for some qubit channels.

Unsupervised Adversarially Robust Representation Learning on Graphs

Unsupervised/self-supervised pre-training methods for graph representation learning have recently attracted increasing research interests, and they are shown to be able to generalize to various downstream applications. Yet, the adversarial robustness of such pre-trained graph learning models remains largely unexplored. More importantly, most existing defense techniques designed for end-to-end graph representation learning methods require pre-specified label definitions, and thus cannot be directly applied to the pre-training methods. In this paper, we propose an unsupervised defense technique to robustify pre-trained deep graph models, so that the perturbations on the input graph can be successfully identified and blocked before the model is applied to different downstream tasks. Specifically, we introduce a mutual information-based measure, graph representation vulnerability (GRV), to quantify the robustness of graph encoders on the representation space. We then formulate an optimization problem to learn the graph representation by carefully balancing the trade-off between the expressive power and the robustness (i.e., GRV) of the graph encoder. The discrete nature of graph topology and the joint space of graph data make the optimization problem intractable to solve. To handle the above difficulty and to reduce computational expense, we further relax the problem and thus provide an approximate solution. Additionally, we explore a provable connection between the robustness of the unsupervised graph encoder and that of models on downstream tasks. Extensive experiments demonstrate that even without access to labels and tasks, our model is still able to enhance robustness against adversarial attacks on three downstream tasks (node classification, link prediction, and community detection) by an average of +16.5% compared with existing methods.

Do information-based measures affect active travel, and if so, for whom, when and under what circumstances? Evidence from a longitudinal case-control study

Soft, information-based measures to encourage walking and cycling for travel are increasingly being recommended alongside infrastructure investments. Using principles of realist evaluation, we evaluate measures implemented as part of the European Physical Activity Through Sustainable Approaches (PASTA) study in Vienna (Austria), Örebro (Sweden), Rome (Italy), and Antwerp (Belgium) over a 3-year cohort study, and a further follow-up 2.5 years later in Vienna and Örebro. Increases in active modes of travel due to the interventions were most significant for walking, one year after the intervention, and for people in full-time employment. Increases in e-bike use were associated with changes in perceptions of cycling, while increases in walking were not associated with any changes in perceptions of walking. We find evidence supporting previous findings that information provision is unlikely to work as a standalone intervention in the longer run, but may be effective when combined with other policies. • Robust longer term evaluation of measures encouraging active travel in cities. • Soft measures helped increase walking and e-bicycling in all 4 cities 2014–2016. • Soft measures had the most long-term effect on e-bicycling, per the 2019 survey. • Public transit improvements can increase walking and may reduce driving.

Group-level inference of information-based measures for the analyses of cognitive brain networks from neurophysiological data

The reproducibility crisis in neuroimaging and in particular in the case of underpowered studies has introduced doubts on our ability to reproduce, replicate and generalize findings. As a response, we have seen the emergence of suggested guidelines and principles for neuroscientists known as Good Scientific Practice for conducting more reliable research. Still, every study remains almost unique in its combination of analytical and statistical approaches. While it is understandable considering the diversity of designs and brain data recording, it also represents a striking point against reproducibility. Here, we propose a non-parametric permutation-based statistical framework, primarily designed for neurophysiological data, in order to perform group-level inferences on non-negative measures of information encompassing metrics from information-theory, machine-learning or measures of distances. The framework supports both fixed- and random-effect models to adapt to inter-individuals and inter-sessions variability. Using numerical simulations, we compared the accuracy in ground-truth retrieving of both group models, such as test- and cluster-wise corrections for multiple comparisons. We then reproduced and extended existing results using both spatially uniform MEG and non-uniform intracranial neurophysiological data. We showed how the framework can be used to extract stereotypical task- and behavior-related effects across the population covering scales from the local level of brain regions, inter-areal functional connectivity to measures summarizing network properties. We also present an open-source Python toolbox called Frites1 that includes the proposed statistical pipeline using information-theoretic metrics such as single-trial functional connectivity estimations for the extraction of cognitive brain networks. Taken together, we believe that this framework deserves careful attention as its robustness and flexibility could be the starting point toward the uniformization of statistical approaches.

Effectiveness of an Energy-Counseling Intervention in Reducing Energy Poverty: Evidence from a Quasi-Experimental Study in a Southern European City.

Energy poverty is a serious social problem with well-known adverse health consequences. This problem has been addressed mainly through improvements in the energy efficiency of housing. Still, little is known about the effects of information-based measures on energy poverty and their impacts on health. A quasi-experimental study was implemented to assess the effectiveness of an energy-counseling home visit intervention targeting the vulnerable population in a southern European city, Barcelona, in alleviating energy poverty and improving health. The intervention had beneficial impacts on keeping homes at an adequate indoor temperature and reducing primary care visits. No effects were found on self-perceived health or self-reported anxiety and depression. After the intervention, participants reported a decrease in arrears on utility bills, but less pronounced than in the comparison group. In conclusion,the study showed that information-based measures lead to psychosocial gains and reduced healthcare use. Nevertheless, the impact of these measures could be enhanced by combining them with policies and programmes that address the structural determinants of energy poverty.

Information-based summary statistics for spatial genetic structure inference.

The measurement of biodiversity at all levels of organization is an essential first step to understand the ecological and evolutionary processes that drive spatial patterns of biodiversity. Ecologists have explored the use of a large range of different summary statistics and have come to the view that information-based summary statistics, and in particular so-called Hill numbers, are a useful tool to measure biodiversity. Population geneticists, on the other hand, have focused largely on summary statistics based on heterozygosity and measures of allelic richness. However, recent studies proposed the adoption of information-based summary statistics in population genetics studies. Here, we performed a comprehensive assessment of the power of this family of summary statistics to inform regarding spatial patterns of genetic diversity and we compared it with that of traditional population genetics approaches, namely measures based on allelic richness and heterozygosity. To give an unbiased evaluation, we used three machine learning methods to test the performance of different sets of summary statistics to discriminate between spatial scenarios. We defined three distinct sets, (i) one based on allelic richness measures which included the Jaccard index, (ii) a set based on heterozygosity that included FST and (iii) a set based on Hill numbers derived from Shannon entropy, which included the recently proposed Shannon differentiation, ΔD. The results showed that the last of these performed as well or, under some specific spatial scenarios, even better than the traditional population genetics measures. Interestingly, we found that a rarely or never used genetic differentiation measure based on allelic richness, Jaccard dissimilarity (J), showed the highest discriminatory power to discriminate among spatial scenarios, followed by Shannon differentiation ΔD. We concluded, therefore, that information-based measures as well as Jaccard dissimilarity represent excellent additions to the population genetics toolkit.

Crystal Chemical Relations in the Shchurovskyite Family: Synthesis and Crystal Structures of K2Cu[Cu3O]2(PO4)4 and K2.35Cu0.825[Cu3O]2(PO4)4

Single crystals of two novel shchurovskyite-related compounds, K2Cu[Cu3O]2(PO4)4 (1) and K2.35Cu0.825[Cu3O]2(PO4)4 (2), were synthesized by crystallization from gaseous phase and structurally characterized using single-crystal X-ray diffraction analysis. The crystal structures of both compounds are based upon similar Cu-based layers, formed by rods of the [O2Cu6] dimers of oxocentered (OCu4) tetrahedra. The topologies of the layers show both similarities and differences from the shchurovskyite-type layers. The layers are connected in different fashions via additional Cu atoms located in the interlayer, in contrast to shchurovskyite, where the layers are linked by Ca2+ cations. The structures of the shchurovskyite family are characterized using information-based structural complexity measures, which demonstrate that the crystal structure of 1 is the simplest one, whereas that of 2 is the most complex in the family.

On the determinants and prediction of corporate financial distress in India

PurposeThe main aim of the study is to identify some critical microeconomic determinants of financial distress and to design a parsimonious distress prediction model for an emerging economy like India. In doing so, the authors also attempt to compare the forecasting accuracy of alternative distress prediction techniques.Design/methodology/approachIn this study, the authors use two alternatives accounting information-based definitions of financial distress to construct a measure of financial distress. The authors then use the binomial logit model and two other popular machine learning–based models, namely artificial neural network and support vector machine, to compare the distress prediction accuracy rate of these alternative techniques for the Indian corporate sector.FindingsThe study’s empirical results suggest that five financial ratios, namely return on capital employed, cash flows to total liability, asset turnover ratio, fixed assets to total assets, debt to equity ratio and a measure of firm size (log total assets), play a highly significant role in distress prediction. The study’s findings suggest that machine learning-based models, namely support vector machine (SVM) and artificial neural network (ANN), are superior in terms of their prediction accuracy compared to the simple binomial logit model. Results also suggest that one-year-ahead forecasts are relatively better than the two-year-ahead forecasts.Practical implicationsThe findings of the study have some important practical implications for creditors, policymakers, regulators and other stakeholders. First, rather than monitoring and collecting information on a list of predictor variables, only six most important accounting ratios may be monitored to track the transition of a healthy firm into financial distress. Second, our six-factor model can be used to devise a sound early warning system for corporate financial distress. Three, machine learning–based distress prediction models have prediction accuracy superiority over the commonly used time series model in the available literature for distress prediction involving a binary dependent variable.Originality/valueThis study is one of the first comprehensive attempts to investigate and design a parsimonious distress prediction model for the emerging Indian economy which is currently facing high levels of corporate financial distress. Unlike the previous studies, the authors use two different accounting information-based measures of financial distress in order to identify an effective way of measuring financial distress. Some of the determinants of financial distress identified in this study are different from the popular distress prediction models used in the literature. Our distress prediction model can be useful for the other emerging markets for distress prediction.

Automatic analysis of artistic paintings using information-based measures

The artistic community is increasingly relying on automatic computational analysis for authentication and classification of artistic paintings. In this paper, we identify hidden patterns and relationships present in artistic paintings by analysing their complexity, a measure that quantifies the sum of characteristics of an object. Specifically, we apply Normalized Compression (NC) and the Block Decomposition Method (BDM) to a dataset of 4,266 paintings from 91 authors and examine the potential of these information-based measures as descriptors of artistic paintings. Both measures consistently described the equivalent types of paintings, authors, and artistic movements. Moreover, combining the NC with a measure of the roughness of the paintings creates an efficient stylistic descriptor. Furthermore, by quantifying the local information of each painting, we define a fingerprint that describes critical information regarding the artists’ style, their artistic influences, and shared techniques. More fundamentally, this information describes how each author typically composes and distributes the elements across the canvas and, therefore, how their work is perceived. Finally, we demonstrate that regional complexity and two-point height difference correlation function are useful auxiliary features that improve current methodologies in style and author classification of artistic paintings. The whole study is supported by an extensive website (http://panther.web.ua.pt) for fast author characterization and authentication.