Linear Opinion Pool Research Articles

Characterising information gains and losses when collecting multiple epidemic model outputs

BackgroundCollaborative comparisons and combinations of epidemic models are used as policy-relevant evidence during epidemic outbreaks. In the process of collecting multiple model projections, such collaborations may gain or lose relevant information. Typically, modellers contribute a probabilistic summary at each time-step. We compared this to directly collecting simulated trajectories. We aimed to explore information on key epidemic quantities; ensemble uncertainty; and performance against data, investigating potential to continuously gain information from a single cross-sectional collection of model results. MethodsWe compared projections from the European COVID-19 Scenario Modelling Hub. Five teams modelled incidence in Belgium, the Netherlands, and Spain. We compared July 2022 projections by incidence, peaks, and cumulative totals. We created a probabilistic ensemble drawn from all trajectories, and compared to ensembles from a median across each model’s quantiles, or a linear opinion pool. We measured the predictive accuracy of individual trajectories against observations, using this in a weighted ensemble. We repeated this sequentially against increasing weeks of observed data. We evaluated these ensembles to reflect performance with varying observed data. ResultsBy collecting modelled trajectories, we showed policy-relevant epidemic characteristics. Trajectories contained a right-skewed distribution well represented by an ensemble of trajectories or a linear opinion pool, but not models’ quantile intervals. Ensembles weighted by performance typically retained the range of plausible incidence over time, and in some cases narrowed this by excluding some epidemic shapes. ConclusionsWe observed several information gains from collecting modelled trajectories rather than quantile distributions, including potential for continuously updated information from a single model collection. The value of information gains and losses may vary with each collaborative effort’s aims, depending on the needs of projection users. Understanding the differing information potential of methods to collect model projections can support the accuracy, sustainability, and communication of collaborative infectious disease modelling efforts.

Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty

Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single model in periods of valid scenario assumptions, while projection interval coverage was near target levels. SMH projections were used to guide pandemic response, illustrating the value of collaborative hubs for longer-term scenario projections.

Context-dependent representation of within- and between-model uncertainty: aggregating probabilistic predictions in infectious disease epidemiology.

Probabilistic predictions support public health planning and decision making, especially in infectious disease emergencies. Aggregating outputs from multiple models yields more robust predictions of outcomes and associated uncertainty. While the selection of an aggregation method can be guided by retrospective performance evaluations, this is not always possible. For example, if predictions are conditional on assumptions about how the future will unfold (e.g. possible interventions), these assumptions may never materialize, precluding any direct comparison between predictions and observations. Here, we summarize literature on aggregating probabilistic predictions, illustrate various methods for infectious disease predictions via simulation, and present a strategy for choosing an aggregation method when empirical validation cannot be used. We focus on the linear opinion pool (LOP) and Vincent average, common methods that make different assumptions about between-prediction uncertainty. We contend that assumptions of the aggregation method should align with a hypothesis about how uncertainty is expressed within and between predictions from different sources. The LOP assumes that between-prediction uncertainty is meaningful and should be retained, while the Vincent average assumes that between-prediction uncertainty is akin to sampling error and should not be preserved. We provide an R package for implementation. Given the rising importance of multi-model infectious disease hubs, our work provides useful guidance on aggregation and a deeper understanding of the benefits and risks of different approaches.

Systematic Quantification of Waste Compositions: A Case Study for Waste of Electric and Electronic Equipment Plastics in the European Union

Waste Electric and Electronic Equipment (WEEE) is a prominent and increasing waste stream for which the Commission of the European Union has put in place ambitious recycling targets. However, these targets can only be achieved by ensuring that both industry and governments develop adequate infrastructure and policies for recycling plastics in an economically and technically optimal manner. Unfortunately, a quantitative overview of WEEE plastics covering the composition of waste streams down to the product component level and describing polymer and additive concentrations, is currently lacking. This hinders policymakers and recyclers in making strategic decisions regarding WEEE plastics recycling. Therefore, a novel method is proposed in this paper combining experimental results with findings from prior literature in order to provide sound quantitative insights into the volume and characteristics of the plastics content of WEEE collected in the European Union. The provided overview was obtained through a combination of proprietary experimental data and a statistical data integration method. More specifically, over 3800 samples awere analysed through manual composition analysis, FTIR, and XRF. The obtained results were integrated with data from prior literature through a novel data integration methodology based on linear opinion pools. The obtained results confirm that distinct plastic types can be found in different product categories and that flame retardants are only found in high concentrations in specific waste streams or components thereof. Hence, the presented analysis provides a quantitative substantiation for the separate collection and treatment of specific waste streams in order to reduce the complexity of the mix of plastic types and allow for the more cost-efficient and higher quality recycling of plastics.

Empirically-transformed linear opinion pools

The linear opinion pool (LOP) produces potentially non-Gaussian combination forecast densities. In this paper, we propose a computationally convenient transformation for the LOP to mirror the non-Gaussianity exhibited by the target variable. Our methodology involves a Smirnov transform to reshape the LOP combination forecasts using the empirical cumulative distribution function. We illustrate our empirically transformed opinion pool (EtLOP) approach with an application examining quarterly real-time forecasts for U.S. inflation evaluated on a sample from 1990:1 to 2020:2. EtLOP improves performance by approximately 10% to 30% in terms of the continuous ranked probability score across forecasting horizons.

An Interpretation of Weights in Linear Opinion Pooling

AbstractThis paper explores the fact that linear opinion pooling can be represented as a Bayesian update on the opinions of others. It uses this fact to propose a new interpretation of the pooling weights. Relative to certain modelling assumptions the weights can be equated with the so-called truth-conduciveness known from the context of Condorcet's jury theorem. This suggests a novel way to elicit the weights.

Classification of hyperspectral remote sensing image via rotation-invariant local binary pattern-based weighted generalized closest neighbor

In this article, the authors suggested a rotation-invariant local binary pattern-based weighted generalized closest neighbor (RILBP-WGCN) method for HSI classification. The proposed RILBP is an enhanced texture-based classification paradigm that utilizes local binary pattern filter for some designated bands to generate a broad sketch of spatial texture information. Likewise, the proposed WGCN technique efficiently maintains the spatial uniformity between the nearby pixels via utilizing a local weight scheme and point-to-set distance. Also, as a postprocessing step, a label enhancement method is included for additional enhancement of the label uniformity as well as increases the performance of classification method. The color composite remotely sensed image of the initial three subsequent bands is segmented into several consistent regions by utilizing the graph-based superpixel segmentation technique. Then, extracted super pixels have been made extra homogeneous by utilizing a segment grouping process. Finally, advanced decision-level fusion is also applied on the retrieved local LBP features and unique spectral features, where linear opinion pool executes a serious role for concatenating the probabilistic outcomes of numerous spectral as well as texture features. The authors evaluated the proposed technique by comparing them with the seven competing methods on numerous datasets related to HSI classification. Evaluation results confirmed that the classification effects of proposed RILBP-WGCN algorithm are significantly better in contrast to other competing classification schemes.

A Review of Probabilistic Opinion Pooling Algorithms with Application to Insider Threat Detection

We retrospectively explore the effectiveness of various probabilistic opinion pools against a set of insider threat detection modeling data from a recently completed, multiyear, sponsored research effort. We explored four opinion pools: the linear opinion pool (likely the most popular), the beta-transformed linear opinion pool, the geometric opinion pool, and a multiplicative method based on odds called Bordley’s formula. The data for our study came from our recent work in the inference of insider threats for our research sponsor. In this work, we created a multimodeling inference enterprise modeling (MIEM) process to either predict threats within a population or, given the threats, predict how well the enterprise system can detect those threats. As part of larger research challenges designed by the research sponsor, we applied the MIEM process quarterly to respond to a sequence of varying challenge problems (CPs). Via MIEM, we developed multiple, independent computation forecast models. These models generated certainty intervals to answer CP questions. These intervals were fused into a single interval for each question via an expert panel prior to submission. The sponsors scored the responses against ground truth. In this paper, we (a) ask which pooling functions work best on these data and consider why, and (b) compare this performance to the actual submissions to determine if one of the pooling functions performed better than our judgment-based fusion.

Multiobject Fusion With Minimum Information Loss

Generalized covariance intersection (GCI) has been effective in fusing multiobject densities from multiple agents for multitarget tracking and mapping purposes. From an information-theoretic viewpoint, it has been shown that GCI fusion essentially minimizes the weighted information gain (WIG) from local densities to the fused one. In this paper, the interest is in the fusion rule that dually minimizes the weighted information loss (WIL) and it turns out that such a fusion rule is consistent with the so-called linear opinion pool (LOP). However, the LOP cannot be directly applied to multiobject fusion since the resulting fused multiobject density (FMD), in general, no longer belongs to the same family of the local ones, thus it cannot be utilized as prior information for the next recursion in the context of Bayesian multiobject filtering. In order to overcome such a difficulty, the principle of minimizing WIL is further exploited in that the optimal FMD in the same family of the local ones is looked for. Implementation issues relative to the proposed minimum WIL (MWIL) fusion rule are discussed. Finally, the performance of the MWIL rule is assessed via simulation experiments concerning distributed multitarget tracking over a wireless sensor network.

Resolving some contradictions in the theory of linear opinion pools

Bradley (Theory Decis 85:5–20, 2018) develops some theory of the linear opinion pool, in apparent contradiction to results of Dawid et al. (Test 4:263–314, 1995). We investigate the sources of these contradictions, and in particular identify a mathematical error in Bradley (2018) that invalidates his main result.

Empirically-Transformed Linear Opinion Pools

Many studies have found that combining density forecasts improves predictive accuracy for macroeconomic variables. A prevalent approach known as the Linear Opinion Pool (LOP) combines forecast densities from “experts”; see, among others, Stone (1961), Geweke and Amisano (2011), Kascha and Ravazzolo (2011), Ranjan and Gneiting (2010) and Gneiting and Ranjan (2013). Since the LOP approach averages the experts’ probabilistic assessments, the distribution of the combination generally differs from the marginal distributions of the experts. As a result, the LOP combination forecasts sometimes fail to match salient features of the sample data, including asymmetries in risk. In this paper, we propose a computationally convenient transformation for a target macroeconomic variable with an asymmetric marginal distribution. Our methodology involves a Smirnov transform to reshape the LOP combination forecasts using a nonparametric kernel-smoothed empirical cumulative distribution function. We illustrate our methodology with an application examining quarterly real-time forecasts for US inflation based on multiple output gap measures over an evaluation sample from 1990:1 to 2017:2. Our proposed methodology improves combination forecast performance by approximately 10% in terms of both the root mean squared forecast error and the continuous ranked probability score. We find that our methodology delivers a similar performance gain for the Logarithmic Opinion Pool (LogOP), a commonly-used alternative to the LOP.

A probabilistic fusion of a support vector machine and a joint sparsity model for hyperspectral imagery classification

Hyperspectral imagery (HSI) is now in use for a wide range of applications such as land cover classification, climate change studies and environmental monitoring; however, the acquisition of HSI is still costly, and the curse of dimensionality i.e. the phenomenon that the amount of required training samples increases exponentially as the dimensionality increases linearly, makes it difficult to exploit the full potential of machine learning in HSI classification. To resolve the problem, we propose a novel framework for spatial-spectral HSI classification in this article. A soft support vector machine (SVM) and a probabilistic joint sparsity model (JSM) are proposed to compute a posteriori probabilities of the test pixels, respectively; and the probability scores are then fused by a linear opinion pool. Furthermore, a Markov random field (MRF) model is used as a maximum a posteriori (MAP) segmentation method for further regularization of the neighbor information to derive the labels for pixels. Extensive experiments conducted on three commonly-used benchmarking data sets show that the proposed probabilistic fusion method outperforms a number of well-known spatial-spectral HSI classification techniques.

Data fusion methods for improving resolution from a gamma-ray scintillator network

Abstract Enhancements in gamma-ray spectral resolution have been realized through hardware development, including low-defect scintillation materials, solid-state detector technology, and low-noise read-out electronics. Enhanced resolution has also been provided by deconvoluting hardware response through the application of spectral unfolding techniques. However, fusing data from multiple independent measurements, both before and after spectral unfolding, to obtain more information than exists from a single measurement represents a largely unexplored path. This work provides a statistical foundation from which data acquired from an arbitrary number of independent gamma-ray spectra can be fused to yield higher resolution than the constituent measurements yield in isolation. Specifically, three data fusion techniques are considered: (i) pulse-height spectrum fusion, (ii) unfolded fluence linear opinion pool, and (iii) unfolded fluence logarithmic opinion pool. We explore the qualitative characteristics of these methods. We then apply them to spectra of thallium-doped sodium iodide [NaI(Tl)] and bismuth germanate (BGO), where all spectra are provided by MCNP simulations. Our results demonstrate improved resolution, thus establishing the methods as an inexpensive alternative to hardware upgrades. Moreover, our results demonstrate that these results have the potential to expand the capability of cutting-edge technology to provide currently unachievable resolution.

A weighted normalized likelihood procedure for empirical land change modeling

A critical foundation for empirical land change modeling is the mapping of transition potentials—quantitative evaluations of the readiness or suitability of land to go through a transition. This paper presents a procedure based on empirically determined normalized likelihoods of transition. It shows that these normalized likelihoods equate to posterior probabilities if case–control sampling is carried out among historical instances of change and persistence. The posterior probabilities can then be aggregated at the pixel level across multiple covariates using linear opinion pooling where the pixel-specific weight for each covariate is determined locally by its ability to distinguish between the alternatives of change or persistence. Thus, covariates with spatially varying diagnostic ability can be productively incorporated. The resulting algorithm is shown to have a skill comparable to that of a multi-layer perceptron approach with the advantage of high efficiency and amenability to distributed processing in a cloud environment.

Parametric methods for probabilistic forecasting of solar irradiance

Abstract This paper proposes two parametric probabilistic forecast methods using beta and two-sided power distributions to predict solar irradiance. It also evaluates their performance. To improve their performance metrics a hybrid procedure based on the beta transformed linear opinion pool is utilized. Our simulations show that these methods – despite their simple structure – can effectively forecast solar irradiance and accurately describe its stochastic characteristics. The proposed approach is flexible and could be extended to many different point forecast methods which otherwise minimize RMSE or MSE.

Multiple features fusion for facial expression recognition based on ELM

Traditional facial expression recognition includes a feature extractor and a classifier. In this paper, multiple features fusion approaches for facial expression recognition are proposed to improve the recognition accuracy. We consider a feature level fusion method, serial feature fusion, and decision level fusion, linear opinion pool, to combine multiple features. Local binary patterns, local directional number pattern and edge orientation histograms are used to extract features. Then, extreme learning machine is used as the classifier for expression classification. Experiments on JAFFE and CK+ show the method achieves better results.

Measuring the output gap in Switzerland with linear opinion pools

We use the recently proposed linear opinion pool methodology of Garratt et al. (2014) to construct real-time output gap estimates for Switzerland over the out-of-sample period from 2003:Q1 to 2015:Q4. The model space consists of a large number of bivariate VAR specifications for the output gap and inflation, with each VAR specification using a different estimate of the output gap, lag order, and structural break information. We find that the linear opinion pool performs rather poorly. Real-time estimates of the output gap are no more accurate than those from some simple benchmark models, no more robust to ex post revisions than the real-time estimates of the individual univariate output gaps, and do not produce more accurate forecasts of inflation. The key driver of ‘good’ forecast performance is structural break information. Once the same structural break information is conditioned upon in all prediction models, the gain from averaging over many different pools of models that utilize various output gap estimates or lag structures in the VAR specification is of negligible magnitude.

Probabilistic opinion pooling generalized. Part two: the premise-based approach

How can several individuals’ probability functions on a given sigma -algebra of events be aggregated into a collective probability function? Classic approaches to this problem usually require ‘event-wise independence’: the collective probability for each event should depend only on the individuals’ probabilities for that event. In practice, however, some events may be ‘basic’ and others ‘derivative’, so that it makes sense first to aggregate the probabilities for the former and then to let these constrain the probabilities for the latter. We formalize this idea by introducing a ‘premise-based’ approach to probabilistic opinion pooling, and show that, under a variety of assumptions, it leads to linear or neutral opinion pooling on the ‘premises’.

Deriving the probability of a linear opinion pooling method being superior to a set of alternatives

Linear opinion pools are a common method for combining a set of distinct opinions into a single succinct opinion, often to be used in a decision making task. In this paper we consider a method, termed the Plug-in approach, for determining the weights to be assigned in this linear pool, in a manner that can be deemed as rational in some sense, while incorporating multiple forms of learning over time into its process. The environment that we consider is one in which every source in the pool is herself a decision maker (DM), in contrast to the more common setting in which expert judgments are amalgamated for use by a single DM. We discuss a simulation study that was conducted to show the merits of our technique, and demonstrate how theoretical probabilistic arguments can be used to exactly quantify the probability of this technique being superior (in terms of a probability density metric) to a set of alternatives. Illustrations are given of simulated proportions converging to these true probabilities in a range of commonly used distributional cases.

A non‐biased trust model for wireless mesh networks

AbstractTrust models that rely on recommendation trusts are vulnerable to badmouthing and ballot‐stuffing attacks. To cope with these attacks, existing trust models use different trust aggregation techniques to process the recommendation trusts and combine them with the direct trust values to form a combined trust value. However, these trust models are biased as recommendation trusts that deviate too much from one's own opinion are discarded. In this paper, we propose a non‐biased trust model that considers every recommendation trusts available regardless they are good or bad. Our trust model is based on a combination of 2 techniques: the dissimilarity test and the Dempster‐Shafer Theory. The dissimilarity test determines the amount of conflict between 2 trust records, whereas the Dempster‐Shafer Theory assigns belief functions based on the results of the dissimilarity test. Numerical results show that our trust model is robust against reputation‐based attacks when compared to trust aggregation techniques such as the linear opinion pooling, subjective logic model, entropy‐based probability model, and regression analysis. In addition, our model has been extensively tested using network simulator NS‐3 in an Infrastructure‐based wireless mesh networks and a Hybrid‐based wireless mesh networks to demonstrate that it can mitigate blackhole and grayhole attacks.