Maximum Information Gain Research Articles

Optimal Multi-Type Sensor Placement for Structural Identification by Static-Load Testing.

Assessing ageing infrastructure is a critical challenge for civil engineers due to the difficulty in the estimation and integration of uncertainties in structural models. Field measurements are increasingly used to improve knowledge of the real behavior of a structure; this activity is called structural identification. Error-domain model falsification (EDMF) is an easy-to-use model-based structural-identification methodology which robustly accommodates systematic uncertainties originating from sources such as boundary conditions, numerical modelling and model fidelity, as well as aleatory uncertainties from sources such as measurement error and material parameter-value estimations. In most practical applications of structural identification, sensors are placed using engineering judgment and experience. However, since sensor placement is fundamental to the success of structural identification, a more rational and systematic method is justified. This study presents a measurement system design methodology to identify the best sensor locations and sensor types using information from static-load tests. More specifically, three static-load tests were studied for the sensor system design using three types of sensors for a performance evaluation of a full-scale bridge in Singapore. Several sensor placement strategies are compared using joint entropy as an information-gain metric. A modified version of the hierarchical algorithm for sensor placement is proposed to take into account mutual information between load tests. It is shown that a carefully-configured measurement strategy that includes multiple sensor types and several load tests maximizes information gain.

Active Search for Sparse Signals with Region Sensing

Autonomous systems can be used to search for sparse signals in a large space; e.g., aerial robots can be deployed to localize threats, detect gas leaks, or respond to distress calls. Intuitively, search algorithms may increase efficiency by collecting aggregate measurements summarizing large contiguous regions. However, most existing search methods either ignore the possibility of such region observations (e.g., Bayesian optimization and multi-armed bandits) or make strong assumptions about the sensing mechanism that allow each measurement to arbitrarily encode all signals in the entire environment (e.g., compressive sensing). We propose an algorithm that actively collects data to search for sparse signals using only noisy measurements of the average values on rectangular regions (including single points), based on the greedy maximization of information gain. We analyze our algorithm in 1d and show that it requires $\tilde{O}(\frac{n}{\mu^2}+k^2)$ measurements to recover all of $k$ signal locations with small Bayes error, where $\mu$ and $n$ are the signal strength and the size of the search space, respectively. We also show that active designs can be fundamentally more efficient than passive designs with region sensing, contrasting with the results of Arias-Castro, Candes, and Davenport (2013). We demonstrate the empirical performance of our algorithm on a search problem using satellite image data and in high dimensions.

Adaptive Measurement and Modelling Methodology for In-line 3D Surface Metrology Scanners

3D surface metrology scanners are primarily used for surface reconstruction in reverse engineering and for off-line inspection of parts but, their application potential for in-line process control remains largely unexplored. The lack of usage of 3D surface metrology scanners for in-line process control can be attributed to the large processing time of metrology equipment compared to the cycle time of automotive assembly systems. To overcome this challenge, a novel methodology for adaptive measurement and modelling of data captured by in-line 3D surface measurement systems is proposed in this paper. The methodology models part deviations by augmenting the scanner data with spatio-temporal correlations in deviation field and enables efficient implementation of in-line part shape measurement for process control. The proposed methodology consists of two steps: (i) modelling deviations to enable prediction of entire part deviations with partial measurements, and; (ii) adaptively selecting part regions to be measured by taking into account the measurement data available from the scanner up to current time-step. The partial measurement strategy based on deviation modeling and adaptive region selection results in maximum information gain within the given assembly system cycle time. The prediction of entire part surface deviation with higher confidence leads to effective identification of part-to-part defect variation patterns. The proposed methodology is demonstrated on an automotive door component.

An Evaluation of Information Sharing Parking Guidance Policies Using a Bayesian Approach

Real-time parking occupancy information is critical for a parking management system to facilitate drivers to park more efficiently. Recent advances in connected and automated vehicle technologies enable sensor-equipped cars (probe cars) to detect and broadcast available parking spaces when driving through parking lots. In this paper, we evaluate the impact of market penetration of probe cars on the system performance, and investigate different parking guidance policies to improve the data acquisition process. We adopt a simulation-based approach to impose four policies on an offstreet parking lot influencing the behavior of probe cars to park in assigned parking spaces. This in turn effects the scanning route and the parking space occupancy estimations. The last policy we propose is a near-optimal guidance strategy that maximizes the information gain of posteriors. The results suggest that an efficient information gathering policy can compensate for low penetration of connected and automated vehicles. We also highlight the policy trade-off that occur while attempting to maximize information gain through explorations and improve assignment accuracy through exploitations. Our results can assist urban policy makers in designing and managing smart parking systems.

Inefficient search strategies in simulated hemianopia.

We investigated whether healthy participants can spontaneously adopt effective eye movement strategies to compensate for information loss similar to that experienced by patients with damage to visual cortex (hemianopia). Visual information in 1 hemifield was removed or degraded while participants searched for an emotional face among neutral faces or a line tilted 45° to the right among lines of varying degree of tilt. A bias to direct saccades toward the sighted field was observed across all 4 experiments. The proportion of saccades directed toward the "blind" field increased with the amount of information available in that field, suggesting fixations are driven toward salient visual stimuli rather than toward locations that maximize information gain. In Experiments 1 and 2, the sighted-field bias had a minimal impact on search efficiency, because the target was difficult to find. However, the sighted-field bias persisted even when the target was visually distinct from the distractors and could easily be detected in the periphery (Experiments 3 and 4). This surprisingly inefficient search behavior suggests that eye movements are biased to salient visual stimuli even when it comes at a clear cost to search efficiency, and efficient strategies to compensate for visual deficits are not spontaneously adopted by healthy participants. (PsycINFO Database Record

A multi-objective optimization approach to optimal sensor location problem in IGCC power plants

Integrated Gasification Combined Cycle (IGCC) power plants provide a cleaner and more efficient way to obtain energy from coal. In order to operate an IGCC power plant in a safe and stable manner, several input and output process parameters need to be monitored. However, due to economic and operational constraints, it is infeasible to place sensors at every input and output process parameter location. Hence, it becomes important to select the most effective sensor locations which lead to maximum information gain about the plant conditions. Practical issues present in an IGCC power plant, such as harsh physical conditions and variability in process parameters, make the optimal sensor location problem an especially complicated one. Further, sensors can have multiple objectives and they can produce uncertainties due to measurement errors. This work considers hybrid hardware and virtual sensing for advanced power systems with multiple objectives. In order to solve this real world large scale problem, we use a novel algorithm called Better Optimization of Nonlinear Uncertain Systems (BONUS). BONUS works in probability distribution space and avoids sampling for each optimization and derivative calculations iterations. A new algorithm for multi-objective optimization is also developed specifically for problem. The result of this nonlinear stochastic multi-objective problem is the non-dominated, or Pareto set, which provides trade-offs between various objectives like observability, cost, and thermal efficiency. This is the first attempt at solving the problem of optimal sensor deployment in advanced power plants, with consideration of hybrid hardware and virtual sensing and incorporation of uncertainty with multiple objectives.

Asymptotic optimality of myopic information-based strategies for Bayesian adaptive estimation

This paper presents a general asymptotic theory of sequential Bayesian estimation giving results for the strongest, almost sure convergence. We show that under certain smoothness conditions on the probability model, the greedy information gain maximization algorithm for adaptive Bayesian estimation is asymptotically optimal in the sense that the determinant of the posterior covariance in a certain neighborhood of the true parameter value is asymptotically minimal. Using this result, we also obtain an asymptotic expression for the posterior entropy based on a novel definition of almost sure convergence on “most trials” (meaning that the convergence holds on a fraction of trials that converges to one). Then, we extend the results to a recently published framework, which generalizes the usual adaptive estimation setting by allowing different trial placements to be associated with different, random costs of observation. For this setting, the author has proposed the heuristic of maximizing the expected information gain divided by the expected cost of that placement. In this paper, we show that this myopic strategy satisfies an analogous asymptotic optimality result when the convergence of the posterior distribution is considered as a function of the total cost (as opposed to the number of observations).

Eavesdropping on quantum secure direct communication in quantum channels with arbitrarily low loss rate

Quantum attacks that provide an undetectable eavesdropping of the ping-pong protocol operating over lossy quantum channels have already been demonstrated by Wojcik (Phys Rev Lett 90(15):157901, 2003) and Zhang et al. (Phys Lett A 333(12):46---50, 2004). These attacks provide a maximum information gain of 0.311 bits per protocol cycle as long as the induced loss rate remains acceptable. Otherwise, the skipping of some protocol cycles is advised to stay within an accepted loss limit. Such policy leads to a reduction in information gain proportional to the number of skipped cycles. The attack transformation parametrized by the induced loss ratio is proposed. It provides smaller reduction in information gain when the losses accepted by the communicating parties are too low to mount the most effective attack. Other properties of the attack remain the same.

Active classification using belief functions and information gain maximization

Obtaining reliable estimates of the parameters of a probabilistic classification model is often a challenging problem because the amount of available training data is limited. In this paper, we present a classification approach based on belief functions that makes the uncertainty resulting from limited amounts of training data explicit and thereby improves classification performance. In addition, we model classification as an active information acquisition problem where features are sequentially selected by maximizing the expected information gain with respect to the current belief distribution, thus reducing uncertainty as quickly as possible. For this, we consider different measures of uncertainty for belief functions and provide efficient algorithms for computing them. As a result, only a small subset of features need to be extracted without negatively impacting the recognition rate. We evaluate our approach on an object recognition task where we compare different evidential and Bayesian methods for obtaining likelihoods from training data and we investigate the influence of different uncertainty measures on the feature selection process.

Lattice Index Coding

The index coding problem involves a sender with K messages to be transmitted across a broadcast channel, and a set of receivers each of which demands a subset of the K messages while having prior knowledge of a different subset as side information. We consider the specific case of noisy index coding where the broadcast channel is Gaussian and every receiver demands all the messages from the source. Instances of this communication problem arise in wireless relay networks, sensor networks, and retransmissions in broadcast channels. We construct 'lattice index codes' for this channel by encoding the K messages individually using K modulo lattice constellations and transmitting their sum modulo a coarse lattice. We introduce a design metric called 'side information gain' that measures the advantage of a code in utilizing the side information at the receivers, and hence its goodness as an index code. Based on the Chinese remainder theorem, we then construct lattice index codes with large side information gains using lattices over the following principal ideal domains: rational integers, Gaussian integers, Eisenstein integers, and the Hurwitz quaternions. Among all lattice index codes constructed using any densest lattice of a given dimension, our codes achieve the maximum side information gain. Finally, using an example, we illustrate how the proposed lattice index codes can benefit Gaussian broadcast channels with more general message demands.

Learning Perceptual Causality from Video

Perceptual causality is the perception of causal relationships from observation. Humans, even as infants, form such models from observation of the world around them [Saxe and Carey 2006]. For a deeper understanding, the computer must make similar models through the analogous form of observation: video. In this article, we provide a framework for the unsupervised learning of this perceptual causal structure from video. Our method takes action and object status detections as input and uses heuristics suggested by cognitive science research to produce the causal links perceived between them. We greedily modify an initial distribution featuring independence between potential causes and effects by adding dependencies that maximize information gain. We compile the learned causal relationships into a Causal And-Or Graph, a probabilistic and-or representation of causality that adds a prior to causality. Validated against human perception, experiments show that our method correctly learns causal relations, attributing status changes of objects to causing actions amid irrelevant actions. Our method outperforms Hellinger’s χ 2 -statistic by considering hierarchical action selection, and outperforms the treatment effect by discounting coincidental relationships.

Autoscanning for coupled scene reconstruction and proactive object analysis

Detailed scanning of indoor scenes is tedious for humans. We propose autonomous scene scanning by a robot to relieve humans from such a laborious task. In an autonomous setting, detailed scene acquisition is inevitably coupled with scene analysis at the required level of detail. We develop a framework for object-level scene reconstruction coupled with object-centric scene analysis. As a result, the autoscanning and reconstruction will be object-aware , guided by the object analysis. The analysis is, in turn, gradually improved with progressively increased object-wise data fidelity. In realizing such a framework, we drive the robot to execute an iterative analyze-and-validate algorithm which interleaves between object analysis and guided validations. The object analysis incorporates online learning into a robust graph-cut based segmentation framework, achieving a global update of object-level segmentation based on the knowledge gained from robot-operated local validation. Based on the current analysis, the robot performs proactive validation over the scene with physical push and scan refinement, aiming at reducing the uncertainty of both object-level segmentation and object-wise reconstruction. We propose a joint entropy to measure such uncertainty based on segmentation confidence and reconstruction quality, and formulate the selection of validation actions as a maximum information gain problem. The output of our system is a reconstructed scene with both object extraction and object-wise geometry fidelity.

Exact and Heuristic Approaches for Directional Sensor Control

Directional sensors are gaining importance due to applications, including surveillance, detection, and tracking. Such sensors have a limited field of view and a discrete set of directions they can be pointed to. The directional sensor control problem (DSCP) consists in assigning a direction of view to each sensor. The location of the targets is known with uncertainty given by a joint a priori Gaussian distribution, while the sensor locations are known exactly. In this paper, we study the exact and heuristic approaches for the DSCP with the goal of maximizing information gain on the location of a given set of immobile target objects. In particular, we propose an exact mixed integer convex programming (MICP) formulation to be solved by a black-box MICP solver and several metaheuristic approaches based on local search. A computational evaluation shows the very good performance of both methods.

Rational design of digital assays.

Optimum algorithm for digital assays treats chemical compartments as bits of probabilistic information and arranges these bits in a fractional positional system. Maximization of information gain reduces, by orders of magnitude, the number of partitions required to achieve the requested dynamic range and precision of the assay. The method simplifies the execution of digital analytical methods providing for more accessible use of absolute quantization in research and in diagnostics.

Active inference and epistemic value

We offer a formal treatment of choice behavior based on the premise that agents minimize the expected free energy of future outcomes. Crucially, the negative free energy or quality of a policy can be decomposed into extrinsic and epistemic (or intrinsic) value. Minimizing expected free energy is therefore equivalent to maximizing extrinsic value or expected utility (defined in terms of prior preferences or goals), while maximizing information gain or intrinsic value (or reducing uncertainty about the causes of valuable outcomes). The resulting scheme resolves the exploration-exploitation dilemma: Epistemic value is maximized until there is no further information gain, after which exploitation is assured through maximization of extrinsic value. This is formally consistent with the Infomax principle, generalizing formulations of active vision based upon salience (Bayesian surprise) and optimal decisions based on expected utility and risk-sensitive (Kullback-Leibler) control. Furthermore, as with previous active inference formulations of discrete (Markovian) problems, ad hoc softmax parameters become the expected (Bayes-optimal) precision of beliefs about, or confidence in, policies. This article focuses on the basic theory, illustrating the ideas with simulations. A key aspect of these simulations is the similarity between precision updates and dopaminergic discharges observed in conditioning paradigms.

HICC: an entropy splitting-based framework for hierarchical co-clustering

Two-dimensional contingency tables or co-occurrence matrices arise frequently in various important applications such as text analysis and web-log mining. As a fundamental research topic, co-clustering aims to generate a meaningful partition of the contingency table to reveal hidden relationships between rows and columns. Traditional co-clustering algorithms usually produce a predefined number of flat partition of both rows and columns, which do not reveal relationship among clusters. To address this limitation, hierarchical co-clustering algorithms have attracted a lot of research interests recently. Although successful in various applications, the existing hierarchical co-clustering algorithms are usually based on certain heuristics and do not have solid theoretical background. In this paper, we present a new co-clustering algorithm, HICC, with solid theoretical background. It simultaneously constructs a hierarchical structure of both row and column clusters, which retains sufficient mutual information between rows and columns of the contingency table. An efficient and effective greedy algorithm is developed, which grows a co-cluster hierarchy by successively performing row-wise or column-wise splits that lead to the maximal mutual information gain. Extensive experiments on both synthetic and real datasets demonstrate that our algorithm can reveal essential relationships of row (and column) clusters and has better clustering precision than existing algorithms. Moreover, the experiments on real dataset show that HICC can effectively reveal hidden relationships between rows and columns in the contingency table.

Communication-aware information gathering with dynamic information flow

We are interested in the problem of how to improve estimation in multi-robot information gathering systems by actively controlling the rate of communication between robots. Communication is essential in such systems for decentralized data fusion and decision-making, but wireless networks impose capacity constraints that are frequently overlooked. In order to make efficient use of available capacity, it is necessary to consider a fundamental trade-off between communication cost, computation cost and information value. We introduce a new problem, dynamic information flow, that formalizes this trade-off in terms of decentralized constrained optimization. We propose algorithms that dynamically adjust the data rate of each communication link to maximize an information gain metric subject to constraints on communication and computation resources. The metric is balanced against the communication resources required to transmit data and the computation cost of processing sensor data to form observations. The optimization process selectively routes raw sensor data or processed observation data to zero, one or many robots. Our algorithms therefore allow large systems with many different types of sensors and computational resources to maximize information gain performance while satisfying realistic communication constraints. We also present experimental results with multiple ground robots and multiple sensor types that demonstrate the benefit of dynamic information flow in comparison to simpler bandwidth-limiting methods.

Bayesian compressive sensing for ultrawideband inverse scattering in random media

We develop an ultrawideband inverse scattering technique for reconstructing continuous random media on the basis of Bayesian compressive sensing. In addition to providing maximum a posteriori estimates of the unknown weights, Bayesian inversion provides an estimate of the confidence level of the solution, as well as a systematic approach for optimizing subsequent measurement(s) to maximize information gain. We impose sparsity priors directly on spatial harmonics to exploit the spatial correlation exhibited by continuous media and then efficiently solve for their posterior probability density functions by using a fast relevance vector machine. We linearize the problem by using the first-order Born approximation, which enables us to combine, in a single inversion, measurements from multiple transmitters and ultrawideband frequencies. To extend the method to high-contrast media, we also introduce a Bayesian compressive sensing distorted-Born iterative approach. Further, we illustrate the use of time-reversal strategies to adaptively focus the inversion effort onto subdomains of interest and hence reduce the overall inversion cost. The proposed techniques are applied to a number of scenarios including crosshole and borehole sensing.

Optimizing taxonomic resolution and sampling effort to design cost‐effective ecological models for environmental assessment

Summary Predictive models relating ecological assemblages to environmental conditions are widely used in environmental impact assessment and biomonitoring. Such models are often parameterized using comprehensive ecological sampling and taxonomic identification efforts. Limited resources mean that expensive sampling and analytical procedures should be planned to maximize information gain and minimize unnecessary expense. However, there has been little consideration of cost‐effectiveness in parameterizing predictive models using ecological assemblages and no explicit consideration of cost‐effectiveness in balancing investment in the crucial aspects of sample size and taxonomic resolution. Using lacustrine diatom (Bacillariophyceae) assemblages from four large‐scale (c. 77 000–1·3 million km2) data sets containing between 207 and 493 lakes, we address the following questions: (1) how does taxonomic resolution affect information content; (2) how does sample size affect information content for different taxonomic resolutions; and (3) what are the most cost‐effective strategies for constructing environmental assessment models using diatom assemblages across a range of budgets? We use weighted averaging regression models for pH, phosphorus, salinity and lake depth and realistic data collection costs to examine the relationship between cost and model information content (R2 and root mean squared error of prediction). For diatom‐based models, finer taxonomic resolutions almost always provide more cost‐effective information content than collecting more samples, with (morpho)species being the most appropriate taxonomic resolution for nearly all budget scenarios. Information content exhibits an asymptotic relationship with sample size and budget, with greatest information gain during initial sample size increases, and little gain beyond c. 100 samples. Smaller sample sizes can also achieve surprising predictive power in some cases, suggesting low‐cost regional models may be achievable. However, caution is necessary in such an approach, because spatial dependencies in predictions may be missed and analogues with predicted assemblages may be poor. Synthesis and applications. We demonstrate the utility of explicitly considering cost estimates to determine optimal sampling effort and taxonomic resolution for ecological assemblage models. For large, regional biomonitoring programmes, cost‐effective sampling could save millions of dollars. Our framework for determining optimal trade‐offs in ecological assemblage models is easily adaptable to other taxa and analytical techniques used in biomonitoring and environmental assessment.

A Myopic Approach to Ordering Nodes for Parameter Elicitation in Bayesian Belief Networks

Building Bayesian belief networks in the absence of data involves the challenging task of eliciting conditional probabilities from experts to parameterize the model. In this paper, we develop an analytical method for determining the optimal order for eliciting these probabilities. Our method uses prior distributions on network parameters and a novel expected proximity criteria, to propose an order that maximizes information gain per unit elicitation time. We present analytical results when priors are uniform Dirichlet; for other priors, we find through experiments that the optimal order is strongly affected by which variables are of primary interest to the analyst. Our results should prove useful to researchers and practitioners involved in belief network model building and elicitation.