Entropy Metrics Research Articles

Identifying the 'inorganic gene' for high-temperature piezoelectric perovskites through statistical learning.

This paper develops a statistical learning approach to identify potentially new high-temperature ferroelectric piezoelectric perovskite compounds. Unlike most computational studies on crystal chemistry, where the starting point is some form of electronic structure calculation, we use a data-driven approach to initiate our search. This is accomplished by identifying patterns of behaviour between discrete scalar descriptors associated with crystal and electronic structure and the reported Curie temperature (TC) of known compounds; extracting design rules that govern critical structure–property relationships; and discovering in a quantitative fashion the exact role of these materials descriptors. Our approach applies linear manifold methods for data dimensionality reduction to discover the dominant descriptors governing structure–property correlations (the ‘genes’) and Shannon entropy metrics coupled to recursive partitioning methods to quantitatively assess the specific combination of descriptors that govern the link between crystal chemistry and TC (their ‘sequencing’). We use this information to develop predictive models that can suggest new structure/chemistries and/or properties. In this manner, BiTmO3–PbTiO3 and BiLuO3–PbTiO3 are predicted to have a TC of 730°C and 705°C, respectively. A quantitative structure–property relationship model similar to those used in biology and drug discovery not only predicts our new chemistries but also validates published reports.

Identity Attributes Mining, Metrics Composition and Information Fusion Implementation Using Fuzzy Inference System

Term weight a technique in text mining and entropy from Shannon’s information theory are both used to quantify information. In this paper they are used to develop an identity attribute metrics model. Using term weight and entropy metrics values, Sugeno-style fuzzy inference system is envisaged in the implementation of information fusion in a multimode authentication system. This is in an effort to provide a solution to the cases of identity theft and fraud. Three corpora are used to mine the identity attributes and generate the statistical information required to compose a metric model from a set of questionnaires and the application forms for the various services. Triangular and sigmoidally shaped membership functions are used in the fuzzification of the three inputs categorised as biometrics, pseudo metrics and device metrics.

Metrics suite for maintainability of eXtensible Markup Language web services

The eXtensible Markup Language (XML) web services are emerging as the de-facto mechanism for exchanging structured information between applications. The large popularity and acceptance of web services led the developers to adopt the best practices of web service implementation and to find the ways for managing and maintaining web services more effectively. Maintainability, one of the important factors, which affects the quality of XML web services, can be controlled by the proper software metrics that are specifically designed and developed for it. In this paper, we present a suite of metrics to evaluate the quality of the XML web service in terms of its maintainability. The present suite of metrics includes: data weight of a web service description language, distinct message ratio metric, message entropy metric and message repetition scale metric. All the proposed metrics have been evaluated theoretically and validated empirically. A comparative study with similar measures proves the worth of the metric suite.

An entropy approach to disclosure risk assessment: Lessons from real applications and simulated domains

We live in an increasingly mobile world, which leads to the duplication of information across domains. Though organizations attempt to obscure the identities of their constituents when sharing information for worthwhile purposes, such as basic research, the uncoordinated nature of such environment can lead to privacy vulnerabilities. For instance, disparate healthcare providers can collect information on the same patient. Federal policy requires that such providers share “de-identified” sensitive data, such as biomedical (e.g., clinical and genomic) records. But at the same time, such providers can share identified information, devoid of sensitive biomedical data, for administrative functions. On a provider-by-provider basis, the biomedical and identified records appear unrelated, however, links can be established when multiple providers' databases are studied jointly. The problem, known as trail disclosure, is a generalized phenomenon and occurs because an individual's location access pattern can be matched across the shared databases. Due to technical and legal constraints, it is often difficult to coordinate between providers and thus it is critical to assess the disclosure risk in distributed environments, so that we can develop techniques to mitigate such risks. Research on privacy protection has so far focused on developing technologies to suppress or encrypt identifiers associated with sensitive information. There is a growing body of work on the formal assessment of the disclosure risk of database entries in publicly shared databases, but less attention has been paid to the distributed setting. In this research, we review the trail disclosure problem in several domains with known vulnerabilities and show that disclosure risk is influenced by the distribution of how people visit service providers. Based on empirical evidence, we propose an entropy metric for assessing such risk in shared databases prior to their release. This metric assesses risk by leveraging the statistical characteristics of a visit distribution, as opposed to person-level data. It is computationally efficient and superior to existing risk assessment methods, which rely on ad hoc assessment that are often computationally expensive and unreliable. We evaluate our approach on a range of location access patterns in simulated environments. Our results demonstrate that the approach is effective at estimating trail disclosure risks and the amount of self-information contained in a distributed system is one of the main driving factors.

Bid mark‐up selection using artificial neural networks and an entropy metric

PurposeThe purpose of the paper is to perform bid mark‐up optimisation through the use of artificial neural networks (ANN) and a metric of the selected bid mark‐up's derived entropy. The scope is to provide an alternative, entropy‐based method for bid mark‐up optimisation that improves on the analytical models of Friedman and Gates.Design/methodology/approachThe proposed method enables the incorporation of bid parameters through the use of ANN's pattern recognition capabilities and the integration of these parameters with a mark‐up selection process that relies on the entropy produced by possible mark‐up values. The entropy metric used is the product of the probability of winning over the bidder's competitors multiplied by the natural logarithm of the inverse of this probability.FindingsThe case study results show that the proposed entropy‐based bidding model compares favourably with the prevailing competitive bidding models of Friedman and Gates, resulting in higher optimisation with regards to the number of jobs won, the monetary value of contracts awarded and the value of “money left on the table”. Furthermore, the method allows for the incorporation of several objective and subjective bid parameters, in contrast to Friedman's and Gates's models, which are based solely on the bid mark‐up history of a bidder's competitors.Research limitations/implicationsWhile the proposed method is a useful tool for the selection of optimal bid mark‐up values, it requires historical data on the bidding behaviour of key competitors, much like the classic bidding models of Friedman and Gates.Originality/valueThe method is suitable for quantifying objective and subjective competitive bidding parameters and for optimising bid mark‐up values.

A Metabolic Entropy Approach for Measurements of Systemic Metabolic Disruptions in Patho-Physiological States

Multicellular organisms maintain the stability of their internal environment using metabolic and physiological regulatory mechanisms that are disrupted during disease. The loss of homeostatic control results in a complex set of disordered states that may lead to metabolic network failure and irreversible system damage. We have applied a new statistical entropy-based approach to quantify temporal systemic disorder (divergence of metabolic responses) in experimental patho-physiological states, via NMR-spectroscopy generated metabolic profiles of urine. A recovery (R-) potential metric has also been developed to evaluate the relative extent to which defined metabolic processes are perturbed in the context of a global system in terms of multiple changes in concentrations of biofluid components accompanying the disrupted functional activity. This approach is sensitive to physiological as well as pathological interventions. We show that global disruptions of metabolic processes, lesion reversibility, and disorder in metabolic responses to a stressor can be visualized via metabolic entropy metrics, giving insights into biological robustness and thus providing a new tool for assessing deviation from homeostatic regulation.

Nonlinear statistical coupling

By considering a nonlinear combination of the probabilities of a system, a physical interpretation of Tsallis statistics as representing the nonlinear coupling or decoupling of statistical states is proposed. The escort probability is interpreted as the coupled probability, with Q = 1 − q defined as the degree of nonlinear coupling between the statistical states. Positive values of Q have coupled statistical states, a larger entropy metric, and a maximum coupled-entropy distribution of compact-support coupled-Gaussians. Negative values of Q have decoupled statistical states and for − 2 < Q < 0 a maximum coupled-entropy distribution of heavy-tail coupled-Gaussians. The conjugate transformation between the heavy-tail and compact-support domains is shown to be Q ˆ = − 2 Q 2 + Q for coupled-Gaussian distributions. This conjugate relationship has been used to extend the generalized Fourier transform to the compact-support domain and to define a scale-invariant correlation structure with heavy-tail limit distribution. In the present paper, we show that the conjugate is a mapping between the source of nonlinearity in non-stationary stochastic processes and the nonlinear coupling which defines the coupled-Gaussian limit distribution. The effects of additive and multiplicative noise are shown to be separable into the coupled- variance and the coupling parameter Q, providing further evidence of the importance of the generalized moments.

Robust GRAPPA-accelerated diffusion-weighted readout-segmented (RS)-EPI.

Readout segmentation (RS-EPI) has been suggested as a promising variant to echo-planar imaging (EPI) for high-resolution imaging, particularly when combined with parallel imaging. This work details some of the technical aspects of diffusion-weighted (DW)-RS-EPI, outlining a set of reconstruction methods and imaging parameters that can both minimize the scan time and afford high-resolution diffusion imaging with reduced distortions. These methods include an efficient generalized autocalibrating partially parallel acquisition (GRAPPA) calibration for DW-RS-EPI data without scan time penalty, together with a variant for the phase correction of partial Fourier RS-EPI data. In addition, the role of pulsatile and rigid-body brain motion in DW-RS-EPI was assessed. Corrupt DW-RS-EPI data arising from pulsatile nonlinear brain motion had a prevalence of approximately 7% and were robustly identified via k-space entropy metrics. For DW-RS-EPI data corrupted by rigid-body motion, we showed that no blind overlap was required. The robustness of RS-EPI toward phase errors and motion, together with its minimized distortions compared with EPI, enables the acquisition of exquisite 3 T DW images with matrix sizes close to 512(2).

Characterizing European high speed train stations using intermodal time and entropy metrics

This paper presents a quantitative method for characterizing high speed train stations in terms of passenger intermodality. The aim of the procedure is two-fold: firstly, the method is to be used as an objective measure for comparing stations in order to detect suboptimal points and improve the performance of stations as nodal points; secondly, the method provides a means to embed intermodality into regional accessibility models, allowing comprehensive modelling at lower scales. The empirical base of the work comprises data from 27 European high speed rail stations, which is deemed an appropriate statistical sample for the whole European rail network. Using the entropy metric, we found that several different patterns emerged: there was a clear hierarchy of stations which was linked to their respective roles within the system, while strong constraints impeded stations from performing optimally as true intermodal nodes.

Visual Entropy Metric Reveals Differences in Drivers' Eye Gaze Complexity across Variations in Age and Subsidiary Task Load

The purpose of this study was to evaluate the global complexity of drivers' eye movement behavior as a function of advanced adult age and variations in information processing resource demands. Global complexity was operationalized using an information theory metric commonly known as entropy. Fourteen young (mean age = 27) and 14 older (mean age = 75) participants drove on a 2-lane rural highway while simultaneously performing a subsidiary loading task designed to increase cognitive demands drawing upon either verbal-memory resources or visual-spatial resources (within the context of Wickens' (1980) multiple resource model of attention). Compared to traditional distributional measures of eye movement behavior such as mean saccade amplitude and fixation dwell time, the entropy measure of global complexity was better able to distinguish between young and old drivers as well as between baseline driving and conditions of high visual-spatial task load. These results strongly suggest that global measures of eye movement behavior hold significant potential for increasing our understanding of the interacting effects of normal adult aging and task-induced cognitive demands within the context of real-world driving.

Land surface response to precipitation events using MODIS and NEXRAD data

We have developed a wavelet‐based information theoretic approach to examine the interaction between precipitation (PPT) forcing events and the land surface response. Combining Next Generation Weather Radar (NEXRAD) PPT with Moderate Resolution Imaging Spectroradiometer (MODIS) vegetation (NDVI) and surface temperature (T s) data over the Missouri Basin in the north‐central USA, we are able to address the spatial and temporal fluctuations surrounding the hydrometeorology of grassland ecosystems. Information theory metrics of entropy and mutual information content are combined with a wavelet multi‐resolution analysis to examine to what extent the observed PPT signal directly determines the spatial distribution of the land surface temperature and vegetation and how this relationship varies with spatial scale. Results indicate that (1) there is a reduction in the temporal variance of the wavelet coefficients as the signal is transferred from the PPT into the surface temperature and finally the vegetation signal, (2) there are significant correlations as a function of spatial resolution between PPT–NDVI and PPT–T s signals which generally increase with spatial resolution, while there is little correlation between the NDVI and T s signals as a function of resolution, and (3) the scale‐wise entropy and the mutual information content of the signals increase for all fields as the spatial resolution increases. This provides a methodology for determining the relative impact of regional climatology and local land–atmosphere interactions as a function of spatial scale.

Mutual information applied to anomaly detection

Anomaly detection systems play a significant role in protection mechanism against attacks launched on a network. The greatest challenge in designing systems detecting anomalous exploits is defining what to measure. Effective yet simple, Shannon entropy metrics have been successfully used to detect specific types of malicious traffic in a number of commercially available IDS's. We believe that Renyi entropy measures can also adequately describe the characteristics of a network as a whole as well as detect abnormal traces in the observed traffic. In addition, Renyi entropy metrics might boost sensitivity of the methods when disambiguating certain anomalous patterns. In this paper we describe our efforts to understand how Renyi mutual information can be applied to anomaly detection as an offline computation. An initial analysis has been performed to determine how well fast spreading worms (Slammer, Code Red, and Welchia) can be detected using our technique. We use both synthetic and real data audits to illustrate the potentials of our method and provide a tentative explanation of the results.

Analysis of neuromuscular disorders using statistical and entropy metrics on surface EMG

This paper introduces the surface electromyogram (EMG) classification system based on statistical and entropy metrics. The system is intended for diagnostic use and enables classification of examined subject as normal, myopathic or neuropathic, regarding to the acquired EMG signals. 39 subjects in total participated in the experiment, 19 normal, 11 myopathic and 9 neuropathic. Surface EMG was recorded using 4-channel surface electrodes on the biceps brachii muscle at isometric voluntary contractions. The recording time was only 5 seconds long to avoid muscle fatigue, and contractions at five force levels were performed, i.e. 10, 30, 50, 70 and 100 % of maximal voluntary contraction. The feature extraction routine deployed the wavelet transform and calculation of the Shannon entropy across all the scales in order to obtain a feature set for each subject. Subjects were classified regarding the extracted features using three machine learning techniques, i.e. decision trees, support vector machines and ensembles of support vector machines. Four 2-class classifications and a 3-class classification were performed. The scored classification rates were the following: 64±11% for normal/abnormal, 74±7% for normal/myopathic, 79±8% for normal/neuropathic, 49±20% for myopathic/neuropathic, and 63±8% for normal/myopathic/neuropathic.

Modelling of urban growth using spatial analysis techniques: a case study of Ajmer city (India)

The concentration of people in densely populated urban areas, especially in developing countries like India and China, calls for the use of sophisticated monitoring systems, like remote sensing and Geographical Information Systems (GIS). Time series of land use/cover changes can easily be generated using sequential satellite images, which are required for the prediction of urban growth, verification of growth model outputs, estimation of impervious area, parameterization of various hydrological models, water resources planning and management and environmental studies. In the present work, urban growth of Ajmer city (India) in the last 29 years has been studied at mid‐scale level (5–25 m). Remote sensing and GIS have been used to extract the information related to urban growth, impervious area and its spatial and temporal variation. Statistical classification approaches have been used to derive the land use information from satellite images of eight years (1977–2005). The Shannon's entropy and landscape metrics (patchiness and map density) are computed in order to quantify the urban form (impervious area) in terms of spatial phenomena. Further, multivariate statistical techniques have been used to establish the relationship between the urban growth and its causative factors. Results reveal that land development (200%) in Ajmer is more than three times the population growth (59%). Shannon's entropy and landscape metrics has revealed the spatial distribution of the sprawl.

Sensitive ultrasonic delineation of steroid treatment in living dystrophic mice with energy-based and entropy-based radio frequency signal processing.

Duchenne muscular dystrophy is a severe wasting disease, involving replacement of necrotic muscle tissue by fibrous material and fatty infiltrates. One primary animal model of this human disease is the X chromosome-linked mdx strain of mice. The goals of the present work were to validate and quantify the capability of both energy and entropy metrics of radio-frequency ultrasonic backscatter to differentiate among normal, dystrophic, and steroid-treated skeletal muscle in the mdx model. Thirteen 12-month-old mice were blocked into three groups: 4 treated mdx-dystrophic that received daily subcutaneous steroid (prednisolone) treatment for 14 days, 4 positive-control mdx-dystrophic that received saline injections for 14 days, and 5 negative-control animals. Biceps muscle of each animal was imaged in vivo using a 40-MHz center frequency transducer in conjunction with a Vevo-660 ultrasound system. Radio-frequency data were acquired (1 GHz, 8 bits) corresponding to a sequence of transverse images, advancing the transducer from "shoulder" to "elbow" in 100-micron steps. Data were processed to generate both "integrated backscatter" (log energy), and "entropy" (information theoretic receiver, H(f)) representations. Analyses of the integrated-backscatter values delineated both treated-and untreated-mdx biceps from normal controls (p < 0.01). Complementary analyses of the entropy images differentiated the steroid-treated and positive-control mdx groups (p < 0.01). To our knowledge, this study represents the first reported use of quantitative ultrasonic characterization of skeletal muscle in mdx mice. Successful differentiation among dystrophic, steroid-treated, and normal tissues suggests the potential for local noninvasive monitoring of disease severity and therapeutic effects.

Monitoring and modelling of urban sprawl using remote sensing and GIS techniques

The concentration of people in densely populated urban areas, especially in developing countries, calls for the use of monitoring systems like remote sensing. Such systems along with spatial analysis techniques like digital image processing and geographical information system (GIS) can be used for the monitoring and planning purposes as these enable the reporting of overall sprawl at a detailed level.In the present work, urban sprawl of the Ajmer city (situated in Rajasthan State of India) has been studied at a mid scale level, over a period of 25 years (1977–2002), to extract the information related to sprawl, area of impervious surfaces and their spatial and temporal variability. Statistical classification approaches have been used for the classification of the remotely sensed images obtained from various sensors viz. Landsat MSS, TM, ETM+ and IRS LISS-III. Urban sprawl and its spatial and temporal characteristics have been derived from the classified satellite images. The Shannon's entropy and landscape metrics (patchiness and map density) have been computed in terms of spatial phenomenon, in order to quantify the urban form (impervious area). Further, multivariate statistical techniques have been used to establish the relationship between the urban sprawl and its causative factors. Results reveal that land development (160.8%) in Ajmer is more than three times the population growth (50.1%). Shannon's entropy and landscape metrics has revealed the spatial distribution of the urban sprawl over a period of last 25 years.

Bayesian Networks for Knowledge-Based Authentication

Knowledge-based authentication (KBA) has gained prominence as a user authentication method for electronic transactions. This paper presents a Bayesian network model of KBA grounded in probabilistic reasoning and information theory. The probabilistic semantics of the model parameters naturally lead to the definitions of two key KBA metrics-guessability and memorability. The statistical modeling approach allows parameter estimation using methods such as the maximum likelihood estimator (MLE). The information-theoretic view helps to derive the closed-form solutions to estimating the guessability and guessing entropy metrics. The results related to KBA metrics and the models under different attacking strategies and factoid distributions are unified under a game-theoretic framework that yields lower and upper bounds of optimal guessability. The paper also proposes a methodology for implementing a Bayesian network-based KBA system. Further, an empirical evaluation of the relative merits of two Bayesian network structures for KBA, the Naive Bayes (NB) and the Tree Augmented Naive Bayes (TAN), confirms the hypothesis that the TAN structure is superior in terms of authentication accuracy and error rates. The results of the theoretical analysis and the empirical study provide insights into the KBA design problem and establish a foundation for future research in the KBA area.

Join the IEEE Computer Society

Knowledge-based authentication (KBA) has gained prominence as a user authentication method for electronic transactions. This paper presents a Bayesian network model of KBA grounded in probabilistic reasoning and information theory. The probabilistic semantics of the model parameters naturally lead to the definitions of two key KBA metrics - guessability and memorability. The statistical modeling approach allows parameter estimation using methods such as the maximum likelihood estimator (MLE). The information-theoretic view helps to derive the closed-form solutions to estimating the guessability and guessing entropy metrics. The results related to KBA metrics and the models under different attacking strategies and factoid distributions are unified under a game-theoretic framework that yields lower and upper bounds of optimal guessability. The paper also proposes a methodology for implementing a Bayesian network-based KBA system. Further, an empirical evaluation of the relative merits of two Bayesian network structures for KBA, the naive Bayes (NB) and the tree augmented naive Bayes (TAN), confirms the hypothesis that the TAN structure is superior in terms of authentication accuracy and error rates. The results of the theoretical analysis and the empirical study provide insights into the KBA design problem and establish a foundation for future research in the KBA area

A Review of Concentration, Diversity or Entropy Metrics in Economics, Finance, Ecology and Communication Science

A Review of Concentration, Diversity or Entropy Metrics in Economics, Finance, Ecology and Communication Science

Intelligent unmanned ground vehicle navigation via information evaluation

Sensor-centric navigation of unmanned ground vehicles (UGVs) operating in rugged and expansive terrains requires the competency to evaluate the utility of sensor information such that it results in intelligent behavior of the vehicles. Highly imperfect, inconsistent information and incomplete a priori knowledge introduce uncertainty in such unmanned navigation systems. Understanding and quantifying uncertainty yields a measure of useful information that plays a critical role in several robotic navigation tasks such as sensor fusion, mapping, localization, path planning and control. In this article, within a probabilistic framework, the utility of estimation and information-theoretic concepts towards quantifying uncertainty using entropy and mutual information metrics in various contexts of UGV navigation via experimental results is demonstrated.