Measure Of Information Content Research Articles

NETWORK-ENSEMBLE COMPARISONS WITH STOCHASTIC REWIRING AND VON NEUMANN ENTROPY.

Assessing whether a given network is typical or atypical for a random-network ensemble (i.e., network-ensemble comparison) has widespread applications ranging from null-model selection and hypothesis testing to clustering and classifying networks. We develop a framework for network-ensemble comparison by subjecting the network to stochastic rewiring. We study two rewiring processes-uniform and degree-preserved rewiring-which yield random-network ensembles that converge to the Erdős-Rényi and configuration-model ensembles, respectively. We study convergence through von Neumann entropy (VNE)-a network summary statistic measuring information content based on the spectra of a Laplacian matrix-and develop a perturbation analysis for the expected effect of rewiring on VNE. Our analysis yields an estimate for how many rewires are required for a given network to resemble a typical network from an ensemble, offering a computationally efficient quantity for network-ensemble comparison that does not require simulation of the corresponding rewiring process.

Analyzing the Effect of Bin-width on the Computed Entropy

The Shannon entropy is a mathematical expression for quantifying the amount of randomness which can be used to measure information content. It is used in objective function. Mutual Information (MI) uses Shannon entropy in order to determine shared information content of two images. The Shannon entropy, which was originally derived by Shannon in the context of lossless encoding of messages, is also used to define an optimum message length used in the Minimum Description Length (MDL) principle for groupwise registration. We first derived the Shannon entropy from the integral of probability density function ( pdf ), and thenfound that Gaussian has maximum entropy over all possible distribution. We also show that the entropy of the flat distribution is less than the entropy of the Gaussian distribution with the same variance.We then investigated the effect of bin-width on the computed entropy. Weanalyzed the relationship between the computed entropy and the integral entropy when we vary bin-width, but fix variance and the number of samples. We then found that the value of the computed entropy lies within the theoretical predictions at small and large bin-widths. Wealso show two types of bias in entropy estimators.

Measuring ecological characteristics of environmental building performance: Suggestion of an information-network model and indices to quantify complexity, power, and sustainability of energetic organization

The authors present preliminary study in pursuance of developing a flow network-based methodology of building performance evaluation, as current efficiency-centered methods do not fully account for the complex building performance in which nature, economy, and humans are inseparably involved. Based on the principle of entropy, this study defines building as a thermodynamic system that networks useful resources―energy, material and information―through close interconnection with the global environment. Measures of information content in energy-flow networking and ecological performance indicators from Shannon’s information theory, Ulanowicz’s ascendency principle, and Odum’s maximum empower principle are discussed and integratively applied to developing a generic building performance evaluation model. For the holistic indication of building sustainability, this work attempts to reconcile Ulanowicz’s and Odum’s statements about ecosystem development and also integrates emergy (spelled with an “m”) and information metrics. Environmental behaviour of the building model was tested with simulation to validate consistency with system-level principles. Results reveal that network complexity corresponds to system power and resilience (L) and fitness (F) tend to peak at an intermediate level of efficiency. This finding demonstrates applicability of Odum’s maximum power principle to building study, suggesting that increasing complexity (and power) of emergy-flow networking be a fundamental characteristic of sustainable building performance.

A multivariate extension of mutual information for growing neural networks

Recordings of neural network activity in vitro are increasingly being used to assess the development of neural network activity and the effects of drugs, chemicals and disease states on neural network function. The high-content nature of the data derived from such recordings can be used to infer effects of compounds or disease states on a variety of important neural functions, including network synchrony. Historically, synchrony of networks in vitro has been assessed either by determination of correlation coefficients (e.g. Pearson’s correlation), by statistics estimated from cross-correlation histograms between pairs of active electrodes, and/or by pairwise mutual information and related measures. The present study examines the application of Normalized Multiinformation (NMI) as a scalar measure of shared information content in a multivariate network that is robust with respect to changes in network size. Theoretical simulations are designed to investigate NMI as a measure of complexity and synchrony in a developing network relative to several alternative approaches. The NMI approach is applied to these simulations and also to data collected during exposure of in vitro neural networks to neuroactive compounds during the first 12 days in vitro, and compared to other common measures, including correlation coefficients and mean firing rates of neurons. NMI is shown to be more sensitive to developmental effects than first order synchronous and nonsynchronous measures of network complexity. Finally, NMI is a scalar measure of global (rather than pairwise) mutual information in a multivariate network, and hence relies on less assumptions for cross-network comparisons than historical approaches.

Angle-resolved RABBITT: theory and numerics

Angle-resolved (AR) RABBITT measurements offer a high information content measurement scheme, due to the presence of multiple, interfering, ionization channels combined with a phase-sensitive observable in the form of angle and time-resolved photoelectron interferograms. In order to explore the characteristics and potentials of AR-RABBITT, a perturbative 2-photon model is developed; based on this model, example AR-RABBITT results are computed for model and real systems, for a range of RABBITT schemes. These results indicate some of the phenomena to be expected in AR-RABBITT measurements, and suggest various applications of the technique in photoionization metrology.

Designing a calibration set in spectral space for efficient development of an NIR method for tablet analysis

Designing a calibration set is the first step in developing a multivariate spectroscopic calibration method for quantitative analysis of pharmaceutical tablets. This step is critical because successful model development depends on the suitability of the calibration data. For spectroscopic-based methods, traditional concentration based techniques for designing calibration sets are prone to have redundant information while simultaneously lacking necessary information for a successful calibration model. A method for designing a calibration set in spectral space was developed. The pure component spectra of a tablet formulation were used to define the spectral space of that formulation. This method maximizes the information content of measurements and minimizes sample requirements to provide an efficient means for developing multivariate spectroscopic calibration. A comparative study was conducted between a commonly employed full factorial approach to calibration development and the newly developed technique. The comparison was based on a system to quantify a model drug, acetaminophen, in pharmaceutical compacts using near infrared spectroscopy. A 2-factor full factorial design (acetaminophen with 5 levels and MCC:Lactose with 3 levels) was used for calibration development. Three replicates at each design point resulted in a total of 45 tablets for the calibration set. Using the newly developed spectral based method, 11 tablets were prepared for the calibration set. Partial least square (PLS) models were developed from respective calibration sets. Model performance was comprehensively assessed based on the ability to predict acetaminophen concentrations in multiple prediction sets. One prediction set contained similar information to calibration set while the other prediction sets contained different information from calibration set in order to assess the model accuracy and robustness. Similar prediction performance was achieved using the 11-tablet design (spectral space), compared to the 45-tablet full factorial approach. This work demonstrates that a calibration set designed in spectral space provided an efficient means of developing spectroscopic multivariate calibration.

Interpretation and Analysis of Transient-Sandface- and Wellbore-Temperature Data

Summary This paper presents new analytical and semianalytical solutions derived from a coupled transient-wellbore/reservoir thermal model to investigate the information content of transient-temperature measurement made within the vertical wellbore across from the producing horizon or at a gauge depth above it during drawdown and buildup tests. The solutions consider flow of a slightly compressible, single-phase fluid in a homogeneous infinite-acting reservoir system with skin modeled as a composite zone adjacent to the wellbore and account for the Joule-Thomson (J-T) heating/cooling, adiabatic-fluid expansion, conduction and convection effects both in the wellbore and reservoir. They are developed depending on the assumption that the effects of temperature changes on wellbore and reservoir-pressure-transient data can be neglected so that the mass-, momentum-, and energy-balance equations in the wellbore and reservoir can be decoupled. The semianalytical solution for predicting sandface temperatures is verified by use of a general-purpose thermal simulator. Wellbore temperatures at a certain gauge depth are evaluated through the analytical steady-state and transient-wellbore-temperature equations coupled with a semianalytical reservoir-temperature model accounting for conservation of momentum in the wellbore. Results show that drawdown- and buildup-sandface-temperature data may exhibit two semilog straight lines: one at early times reflecting the effects of adiabatic-fluid expansion in the skin zone near the wellbore, and the other, the late-time semilog straight line, reflecting the J-T effects and exhibiting the nonskin-zone properties. However, the wellbore-temperature measurements made at locations above the producing horizon may not exhibit these semilog straight lines because they are strongly dependent upon distance above the producing horizon, geothermal gradient, and radial-heat losses from the wellbore fluid to the formation on the way to gauge. It is found that the skin-zone properties are very difficult to be estimated from drawdown- and buildup-wellbore temperatures unless the gauge location is not far from the producing zone. Specifically, we found that buildup-wellbore temperature is mostly dominated by wellbore-heat losses compared with drawdown-wellbore-temperature data, and hence may not be useful to estimate the formation properties, including skin-zone properties.

Bayesian Retrievals of Vertically Resolved Cloud Particle Size Distribution Properties

AbstractRetrievals of liquid cloud properties from remote sensing observations by necessity assume sufficient information is contained in the measurements, and in the prior knowledge of the cloudy state, to uniquely determine a solution. Bayesian algorithms produce a retrieval that consists of the joint probability distribution function (PDF) of cloud properties given the measurements and prior knowledge. The Bayesian posterior PDF provides the maximum likelihood estimate, the information content in specific measurements, the effect of observation and forward model uncertainties, and quantitative error estimates. It also provides a test of whether, and in which contexts, a set of observations is able to provide a unique solution. In this work, a Bayesian Markov chain Monte Carlo (MCMC) algorithm is used to sample the joint posterior PDF for retrieved cloud properties in shallow liquid clouds over the remote Southern Ocean. Combined active and passive observations from spaceborne W-band cloud radar and visible and near-infrared reflectance are used to retrieve the parameters of a gamma particle size distribution (PSD) for cloud droplets and drizzle. Combined active and passive measurements are able to distinguish between clouds with and without precipitation; however, unique retrieval of PSD properties requires specification of a scene-appropriate prior estimate. While much of the uncertainty in an unconstrained retrieval can be mitigated by use of information from 94-GHz passive brightness temperature measurements, simply increasing measurement accuracy does not render a unique solution. The results demonstrate the robustness of a Bayesian retrieval methodology and highlight the importance of an appropriately scene-consistent prior constraint in underdetermined remote sensing retrievals.

An algorithm for hyperspectral remote sensing of aerosols: 2. Information content analysis for aerosol parameters and principal components of surface spectra

This paper describes the second part of a series of investigation to develop algorithms for simultaneous retrieval of aerosol parameters and surface reflectance from the future hyperspectral and geostationary satellite sensors such as Tropospheric Emissions: Monitoring of POllution (TEMPO). The information content in these hyperspectral measurements is analyzed for 6 principal components (PCs) of surface spectra and a total of 14 aerosol parameters that describe the columnar aerosol volume Vtotal, fine-mode aerosol volume fraction, and the size distribution and wavelength-dependent index of refraction in both coarse and fine mode aerosols. Forward simulations of atmospheric radiative transfer are conducted for 5 surface types (green vegetation, bare soil, rangeland, concrete and mixed surface case) and a wide range of aerosol mixtures. It is shown that the PCs of surface spectra in the atmospheric window channel could be derived from the top-of-the-atmosphere reflectance in the conditions of low aerosol optical depth (AOD≤ 0.2 at 550nm), with a relative error of 1%. With degree freedom for signal analysis and the sequential forward selection method, the common bands for different aerosol mixture types and surface types can be selected for aerosol retrieval. The first 20% of our selected bands accounts for more than 90% of information content for aerosols, and only 4 PCs are needed to reconstruct surface reflectance. However, the information content in these common bands from each TEMPO individual observation is insufficient for the simultaneous retrieval of surface’s PC weight coefficients and multiple aerosol parameters (other than Vtotal). In contrast, with multiple observations for the same location from TEMPO in multiple consecutive days, 1–3 additional aerosol parameters could be retrieved. Consequently, a self-adjustable aerosol retrieval algorithm to account for surface types, AOD conditions, and multiple-consecutive observations is recommended to derive aerosol parameters and surface reflectance simultaneously from TEMPO.

A Sampling Method for Quantifying the Information Content of IASI Channels

AbstractThere is a vast amount of information about the atmosphere available from instruments on board satellites. One example is the Infrared Atmospheric Sounding Interferometer (IASI) instrument, which measures radiances emitted from Earth’s atmosphere and surface in 8461 channels. It is difficult to transmit, store, and assimilate such a large amount of data. A practical solution to this has been to select a subset of a few hundred channels based on those that contain the most useful information.Different measures of information content for objective channel selection have been suggested for application to variational data assimilation. These include mutual information and the degrees of freedom for signal. To date, the calculation of these measures of information content has been based on the linear theory that is at the heart of operational variational data assimilation. However, the retrieval of information about the atmosphere from the satellite radiances can be highly nonlinear.Here, a sampling method for calculating the mutual information that is free from assumptions about the linearity of the relationship between the observed radiances and the state variables is examined. It is found that large linearization errors can indeed lead to large discrepancies in the value of mutual information. How this new estimate of information content can be used in channel selection is addressed, with particular attention given to the efficiency of the new method. It is anticipated that accounting for the nonlinearity in the channel selection will be beneficial when using nonlinear data assimilation methods currently in development.

Novel methodology for accurate resolution of fluid signatures from multi-dimensional NMR well-logging measurements

A novel methodology for accurate fluid characterization from multi-dimensional nuclear magnetic resonance (NMR) well-logging measurements is introduced. This methodology overcomes a fundamental challenge of poor resolution of features in multi-dimensional NMR distributions due to low signal-to-noise ratio (SNR) of well-logging measurements. Based on an unsupervised machine-learning concept of blind source separation, the methodology resolves fluid responses from simultaneous analysis of large quantities of well-logging data. The multi-dimensional NMR distributions from a well log are arranged in a database matrix that is expressed as the product of two non-negative matrices. The first matrix contains the unique fluid signatures, and the second matrix contains the relative contributions of the signatures for each measurement sample. No a priori information or subjective assumptions about the underlying features in the data are required. Furthermore, the dimensionality of the data is reduced by several orders of magnitude, which greatly simplifies the visualization and interpretation of the fluid signatures. Compared to traditional methods of NMR fluid characterization which only use the information content of a single measurement, the new methodology uses the orders-of-magnitude higher information content of the entire well log. Simulations show that the methodology can resolve accurate fluid responses in challenging SNR conditions. The application of the methodology to well-logging data from a heavy oil reservoir shows that individual fluid signatures of heavy oil, water associated with clays and water in interstitial pores can be accurately obtained.

The Information Content of Corridor Volatility Measures During Calm and Turmoil Periods

Measurement of volatility is of paramount importance in finance because of the effects on risk measurement and risk management. Corridor implied volatility measures allow us to disentangle the volatility of positive returns from that of negative returns, providing investors with additional information beyond standard market volatility. The aim of the paper is twofold. First, to propose different types of corridor implied volatility and some combinations of them as risk indicators, in order to provide useful information about investors’ sentiment and future market returns. Second, to investigate their usefulness in prediction of market returns under different market conditions (with a particular focus on the subprime crisis and the European debt crisis). The data set consists of daily index options traded on the Italian market and covers the 2005–2014 period. We find that upside corridor implied volatility measure embeds the highest information content about contemporaneous market returns, claiming the superiority of call options in measuring current sentiment in the market. Moreover, both upside and downside volatilities can be considered as barometers of investors’ fear. The volatility measures proposed have forecasting power on future returns only during high volatility periods and in particular during the European debt crisis. The explanatory power on future market returns improves when two of the proposed volatility measures are combined together in the same model.

MeSH-based disambiguation method using an intrinsic information content measure of semantic similarity

Word Sense Disambiguation represents a crucial task in many natural language processing applications such as information extraction, information retrieval and text summarization. It intends to identify the correct sense of an ambiguous term (target word) according to its context. In this paper, we present a biomedical knowledge-based disambiguation method for determining the adequate domain or sub-domain (sense) of an ambiguous biomedical term. This method uses the MeSH thesaurus as a knowledge resource and an intrinsic information content-based semantic similarity to measure the likeness between their different senses of the ambiguous term MeSH and its context. We define this latter as the set of unambiguous MeSH terms characterizing a document. We aim on one hand to minimize the time and computational complexity and on the other hand to increase the accuracy of the disambiguation by using such context. To evaluate this method, we involved it into a document indexing and retrieval system. The results of experiments, carried out on a sub-set of the OHSUMED collection, in terms of Mean Average Precision show that our method performs well.

A study of integrated experiment design for NMPC applied to the Droop model

Nonlinear model predictive control (NMPC) has become an important tool for optimization based control of many (bio)chemical systems. A requirement for a well-performing NMPC implementation is obtaining and maintaining an appropriate mathematical process model. To cope with model degradation in view of plant changes and/or system evolution, developments have been made for linear systems to incorporate the information content of future measurements in the closed loop objective. However, formulations for integrated experiment design in nonlinear systems (iED-NMPC) remain scarce. Two different formulations are studied in this paper and applied to a bioprocess, namely, algae growth as described by the Droop model. First, a formulation for the integration of experiment design in linear dynamic systems is extended to nonlinear dynamic systems resulting in an NMPC formulation with integrated experiment design. In a second approach, the notion of economic optimal experiment design is incorporated within the NMPC formulation. Here, an economic loss function related to inaccurate parameter estimates is minimized instead of a measure of the parameter variances, resulting in improved control performance. The advantage of the proposed techniques over a naive experiment design integration approach is illustrated with Monte Carlo simulations.

Using Fisher information to track stability in multivariate systems.

With the current proliferation of data, the proficient use of statistical and mining techniques offer substantial benefits to capture useful information from any dataset. As numerous approaches make use of information theory concepts, here, we discuss how Fisher information (FI) can be applied to sustainability science problems and used in data mining applications by analysing patterns in data. FI was developed as a measure of information content in data, and it has been adapted to assess order in complex system behaviour. The main advantage of the approach is the ability to collapse multiple variables into an index that can be used to assess stability and track overall trends in a system, including its regimes and regime shifts. Here, we provide a brief overview of FI theory, followed by a simple step-by-step numerical example on how to compute FI. Furthermore, we introduce an open source Python library that can be freely downloaded from GitHub and we use it in a simple case study to evaluate the evolution of FI for the global-mean temperature from 1880 to 2015. Results indicate significant declines in FI starting in 1978, suggesting a possible regime shift.

Optimal input design for multibody systems by using an extended adjoint approach

We present a method for optimizing inputs of multibody systems for a subsequently performed parameter identification. Herein, optimality with respect to identifiability is attained by maximizing the information content in measurements described by the Fisher information matrix. For solving the resulting optimization problem, the adjoint system of the sensitivity differential equation system is employed. The proposed approach combines these two well-established methods and can be applied to multibody systems in a systematic, automated manner. Furthermore, additional optimization goals can be added and used to find inputs satisfying, for example, end conditions or state constraints.

An ontological approach to engineering requirement representation and analysis

AbstractRequirement planning is one of the most critical tasks in the product development process. Despite its significant impact on the outcomes of the design process, engineering requirement planning is often conducted in an ad hoc manner without much structure. In particular, the requirement planning phase suffers from a lack of quantifiable measures for evaluating the quality of the generated requirements and also a lack of structure and formality in representing engineering requirements. The main objective of this research is to develop a formal Web Ontology Language ontology for standard representation of engineering requirements. The proposed ontology uses explicit semantics that makes the ontology amenable to automated reasoning. To demonstrate how the proposed ontology can support requirement analysis and evaluation in engineering design, three possible services enabled by the ontology are introduced in this paper. These services are information content measurement, specificity and completeness analysis, and requirement classification. The proposed ontology and its associated algorithms and tools are validated experimentally in this work.

The Impact of Vertical Measurement Depth on the Information Content of Soil Moisture for Latent Heat Flux Estimation

Abstract This study aims to identify the impact of vertical support on the information content of soil moisture (SM) for latent heat flux estimation. This objective is achieved via calculation of the mutual information (MI) content between multiple soil moisture variables (with different vertical supports) and current/future evaporative fraction (EF) using ground-based soil moisture and latent/sensible heat flux observations acquired from the AmeriFlux network within the contiguous United States. Through the intercomparison of MI results from different SM–EF pairs, the general value (for latent heat flux estimation) of superficial soil moisture observations , vertically integrated soil moisture observations , and vertically extrapolated soil moisture time series [soil wetness index (SWI) from a simple low-pass transformation of ] are examined. Results suggest that, contrary to expectations, 2-day averages of and have comparable mutual information with regards to EF. That is, there is no clear evidence that the information content for flux estimation is enhanced via deepening the vertical support of superficial soil moisture observations. In addition, the utility of SWI in monitoring and forecasting EF is partially dependent on the adopted parameterization of time-scale parameter T in the exponential filter. Similar results are obtained when analyses are conducted at the monthly time scale, only with larger error bars. The contrast between the results of this paper and past work focusing on utilizing soil moisture to predict vegetation condition demonstrates that the particular application should be considered when characterizing the information content of soil moisture time series measurements.

Bank accounting disclosure, information content in stock prices, and stock crash risk

Purpose This paper aims to examine how banks’ accounting disclosure policies affect information content in stock prices and stock crash risk. Design/methodology/approach This paper uses 1996-2013 as the sample period. The final sample includes 10,045 observations in 37 countries. This paper uses stock return synchronicity to measure information content in stock prices. This study uses the frequency difference between extremely negative and positive stock returns to measure stock crash risk. To measure the level of bank accounting disclosure, this research follows Nier and Baumann (2006) to construct an aggregate disclosure index based on inclusions and omissions of a series of items in a bank’s annual accounting reports. Findings This paper finds that banks’ stocks have lower stock return synchronicity and fewer extremely negative returns if banks have higher levels of financial statement disclosure. These results suggest that banks’ stocks have higher information content and lower crash risk if banks’ information environment is more transparent. Originality/value Overall, this paper provides new insight about how to increase banks’ transparency and the safety of the banking industry, which is beneficial to economic growth. To increase banks’ transparency and reduce the possibility of extremely negative stock returns, one way to regulate banks is to increase their accounting disclosure. In addition, the extant literature (Chen et al., 2006, Durnev et al., 2003, 2004; Wurgler, 2000) demonstrates that firms with lower stock return synchronicity have more transparent information environments and higher investment efficiency. Thus, this paper finds that higher levels of bank accounting disclosure are associated with lower stock return synchronicity, which further reduces banks’ opacity and increases banks’ investment efficiency. Finally, compared to business firms, stock crash risk has much direr consequences because one bank’s stock crash will affect overall financial stability. Thus, it is important for authorities to know the effects of accounting disclosure on bank stock crash risk.

Multistatic aerosol–cloud lidar in space: A theoretical perspective

Accurate aerosol and cloud retrievals from space remain quite challenging and typically involve solving a severely ill-posed inverse scattering problem. In this Perspective, we formulate in general terms an aerosol and aerosol–cloud interaction space mission concept intended to provide detailed horizontal and vertical profiles of aerosol physical characteristics as well as identify mutually induced changes in the properties of aerosols and clouds. We argue that a natural and feasible way of addressing the ill-posedness of the inverse scattering problem while having an exquisite vertical-profiling capability is to fly a multistatic (including bistatic) lidar system. We analyze theoretically the capabilities of a formation-flying constellation of a primary satellite equipped with a conventional monostatic (backscattering) lidar and one or more additional platforms each hosting a receiver of the scattered laser light. If successfully implemented, this concept would combine the measurement capabilities of a passive multi-angle multi-spectral polarimeter with the vertical profiling capability of a lidar; address the ill-posedness of the inverse problem caused by the highly limited information content of monostatic lidar measurements; address the ill-posedness of the inverse problem caused by vertical integration and surface reflection in passive photopolarimetric measurements; help relax polarization accuracy requirements; eliminate the need for exquisite radiative-transfer modeling of the atmosphere–surface system in data analyses; yield the day-and-night observation capability; provide direct characterization of ground-level aerosols as atmospheric pollutants; and yield direct measurements of polarized bidirectional surface reflectance. We demonstrate, in particular, that supplementing the conventional backscattering lidar with just one additional receiver flown in formation at a scattering angle close to 170° can dramatically increase the information content of the measurements. Although the specific subject of this Perspective is the multistatic lidar concept, all our conclusions equally apply to a multistatic radar system intended to study from space the global distribution of cloud and precipitation characteristics.