Original Ensemble Research Articles

Extending the forecast range of the UK storm surge ensemble

AbstractFlooding due to coastal storm surges presents a significant threat to life and property. The UK has long had a storm surge forecasting system based on a single ‘deterministic’ simulation. This was augmented with an operational storm surge ensemble in December 2009. By producing several simulations sampling the forecast uncertainty, the ensemble estimates the probability of reaching critical water levels and thus supports a more risk‐based approach to civil protection.The original storm surge ensemble provided forecasts out to T + 54 h, limited by the forecast range of the driving MOGREPS‐R atmospheric ensemble. Longer‐range forecasts could provide advance notice of the potential for a significant event, allowing suitable preparatory actions to be taken. This study investigates the possibility of extending the storm surge ensemble to between 5 and 7 days using atmospheric data from the lower‐resolution Met Office 15‐day ensemble (MOGREPS‐15).Both case studies and statistical verification indicate the potential for useful forecasts out to the full 7.25 days tested. The best performance is obtained by extending the existing surge ensemble products with output from separate runs of the storm surge model, which have been driven by MOGREPS‐15 meteorology from T + 0 h. An attempt to create a single surge history for each member by switching from MOGREPS‐R to MOGREPS‐15 input at T + 54 h led to spurious oscillations in some cases, and poorer performance on several statistical measures. These issues might be improved by smoothing the discontinuity in atmospheric forcing.Following this successful trial, the separate‐runs extension to the surge ensemble was implemented operationally in summer 2011. The study also demonstrates the benefit of online bias correction and ‘dressing’ the forecast members to account for errors which the system does not otherwise sample. The operational implementation of these features is left for future work. Copyright © 2012 British Crown copyright, the Met Office Published by John Wiley & Sons Ltd.

Rank Histograms of Stratified Monte Carlo Ensembles

Abstract The application of forecast ensembles to probabilistic weather prediction has spurred considerable interest in their evaluation. Such ensembles are commonly interpreted as Monte Carlo ensembles meaning that the ensemble members are perceived as random draws from a distribution. Under this interpretation, a reasonable property to ask for is statistical consistency, which demands that the ensemble members and the verification behave like draws from the same distribution. A widely used technique to assess statistical consistency of a historical dataset is the rank histogram, which uses as a criterion the number of times that the verification falls between pairs of members of the ordered ensemble. Ensemble evaluation is rendered more specific by stratification, which means that ensembles that satisfy a certain condition (e.g., a certain meteorological regime) are evaluated separately. Fundamental relationships between Monte Carlo ensembles, their rank histograms, and random sampling from the probability simplex according to the Dirichlet distribution are pointed out. Furthermore, the possible benefits and complications of ensemble stratification are discussed. The main conclusion is that a stratified Monte Carlo ensemble might appear inconsistent with the verification even though the original (unstratified) ensemble is consistent. The apparent inconsistency is merely a result of stratification. Stratified rank histograms are thus not necessarily flat. This result is demonstrated by perfect ensemble simulations and supplemented by mathematical arguments. Possible methods to avoid or remove artifacts that stratification induces in the rank histogram are suggested.

Incorporation of a Regularization Term to Control Negative Correlation in Mixture of Experts

Combining accurate neural networks (NN) in the ensemble with negative error correlation greatly improves the generalization ability. Mixture of experts (ME) is a popular combining method which employs special error function for the simultaneous training of NN experts to produce negatively correlated NN experts. Although ME can produce negatively correlated experts, it does not include a control parameter like negative correlation learning (NCL) method to adjust this parameter explicitly. In this study, an approach is proposed to introduce this advantage of NCL into the training algorithm of ME, i.e., mixture of negatively correlated experts (MNCE). In this proposed method, the capability of a control parameter for NCL is incorporated in the error function of ME, which enables its training algorithm to establish better balance in bias-variance-covariance trade-off and thus improves the generalization ability. The proposed hybrid ensemble method, MNCE, is compared with their constituent methods, ME and NCL, in solving several benchmark problems. The experimental results show that our proposed ensemble method significantly improves the performance over the original ensemble methods.

Not all G-quadruplexes are created equally: an investigation of the structural polymorphism of the c-Myc G-quadruplex-forming sequence and its interaction with the porphyrin TMPyP4.

G-quadruplexes, DNA tertiary structures highly localized to functionally important sites within the human genome, have emerged as important new drug targets. The putative G-quadruplex-forming sequence (Pu27) in the NHE-III(1) promoter region of the c-Myc gene is of particular interest as stabilization of this G-quadruplex with TMPyP4 has been shown to repress c-Myc transcription. In this study, we examine the Pu27 G-quadruplex-forming sequence and its interaction with TMPyP4. We report that the Pu27 sequence exists as a heterogeneous mixture of monomeric and higher-order G-quadruplex species in vitro and that this mixture can be partially resolved by size exclusion chromatography (SEC) separation. Within this ensemble of configurations, the equilibrium can be altered by modifying the buffer composition, annealing procedure, and dialysis protocol thereby affecting the distribution of G-quadruplex species formed. TMPyP4 was found to bind preferentially to higher-order G-quadruplex species suggesting the possibility of stabilization of the junctions of the c-Myc G-quadruplex multimers by porphyrin end-stacking. We also examined four modified c-Myc sequences that have been previously reported and found a narrower distribution of G-quadruplex configurations compared to the parent Pu27 sequence. We could not definitively conclude whether these G-quadruplex structures were selected from the original ensemble or if they are new G-quadruplex structures. Since these sequences differ considerably from the wild-type promoter sequence, it is unclear whether their structures have any actual biological relevance. Additional studies are needed to examine how the polymorphic nature of G-quadruplexes affects the interpretation of in vitro data for c-Myc and other G-quadruplexes. The findings reported here demonstrate that experimental conditions contribute significantly to G-quadruplex formation and should be carefully considered, controlled, and reported in detail.

Dynamics of Large Ensembles of Coupled Active and Inactive Oscillators

Here briefly reviewed are our studies of the dynamics of large ensembles composed of oscillators with qualitatively different behaviors (active: oscillatory and inactive: non-self-oscillatory). Such studies not only give insight into the robustness of the dynamic activity of the original ensemble in which all units are active against deterioration of some of them, but also suggest a new way of controlling the system's behavior.

Empirical analysis and evaluation of approximate techniques for pruning regression bagging ensembles

Identifying the optimal subset of regressors in a regression bagging ensemble is a difficult task that has exponential cost in the size of the ensemble. In this article we analyze two approximate techniques especially devised to address this problem. The first strategy constructs a relaxed version of the problem that can be solved using semidefinite programming. The second one is based on modifying the order of aggregation of the regressors. Ordered aggregation is a simple forward selection algorithm that incorporates at each step the regressor that reduces the training error of the current subensemble the most. Both techniques can be used to identify subensembles that are close to the optimal ones, which can be obtained by exhaustive search at a larger computational cost. Experiments in a wide variety of synthetic and real-world regression problems show that pruned ensembles composed of only 20% of the initial regressors often have better generalization performance than the original bagging ensembles. These improvements are due to a reduction in the bias and the covariance components of the generalization error. Subensembles obtained using either SDP or ordered aggregation generally outperform subensembles obtained by other ensemble pruning methods and ensembles generated by the Adaboost.R2 algorithm, negative correlation learning or regularized linear stacked generalization. Ordered aggregation has a slightly better overall performance than SDP in the problems investigated. However, the difference is not statistically significant. Ordered aggregation has the further advantage that it produces a nested sequence of near-optimal subensembles of increasing size with no additional computational cost.

Nondestructive identification of the Bell diagonal state

We propose a scheme for identifying an unknown Bell diagonal state. In our scheme the measurements are performed on the probe qubits instead of the Bell diagonal state. The distinct advantage is that the quantum state of the evolved Bell diagonal state ensemble plus probe states will still collapse on the original Bell diagonal state ensemble after the measurement on probe states; i.e., our identification is quantum state nondestructive. How to realize our scheme in the framework of cavity electrodynamics is also shown.

A hydrologic post-processor for ensemble streamflow predictions

Abstract. This paper evaluates the performance of a statistical post-processor for imperfect hydrologic model forecasts. Assuming that the meteorological forecasts are well-calibrated, we employ a "General Linear Model (GLM)" to post-process simulations produced by a hydrologic model. For a particular forecast date, the observations and simulations from an "analysis window" and hydrologic model forecasts for a "forecast window", the GLM Post-Processor (GLMPP) is used to produce an ensemble of predictions of the streamflow observations that will occur during the "forecast window". The objectives of the GLMPP are to: (1) preserve any skill in the original hydrologic ensemble forecast; (2) correct systematic model biases; (3) retain the equal-likelihood assumption for the ensemble; (4) preserve temporal scale dependency relationships in streamflow hydrographs and the uncertainty in the predictions; and, (5) produce reliable ensemble predictions. Observed and simulated daily streamflow data from the Second Workshop on Model Parameter Estimation Experiment (MOPEX) are used to test how well these objectives are met when the GLMPP is applied to ensemble hydrologic forecasts driven by well calibrated meteorological forecasts. A 39-year hydrologic dataset from the French Broad basin is split into calibration and verification periods. The results show that the GLMPP built using data from the calibration period removes the mean bias when applied to hydrologic model simulations from both the calibration and verification periods. Probability distributions of the post-processed model simulations are shown to be closer to the climatological probability distributions of observed streamflow than the distributions of the unadjusted simulated flows. A number of experiments with different GLMPP configurations were also conducted to examine the effects of different configurations for forecast and analysis window lengths on the robustness of the results.

In the heavenly horizons three Christian temples in the region of Matka near Skopje

The article deals with the problem of chronological determination of three sacral edifices located in the middle, as well as in the highest cultural horizon of the region of Matka near Skopje. Due to the lack of historical sources, as well as preserved portions of the original fresco ensembles, the buildings are dated on the ground of their architectural features and according to the archeological finds discovered in their vicinity. In that regard, the church of St. Kyriake on the highest mountain post of Matka is dated in the first half of the fourteenth century, the temple of St. Archangel Michael at the site of Peter’s Rock in the late twelfth, while the church dedicated to St. Saviour, as a restoration of the later, was repeatedly built in the period between the late thirteenth and the second half of the twentieth century.

Partition Maps

Tree ensembles, notably Random Forests, have been shown to deliver very accurate predictions on a wide range of regression and classification tasks. A common, yet maybe unjustified, criticism is that they operate as black boxes and provide very little understanding of the data beyond accurate predictions. We focus on multiclass classification and show that Homogeneity Analysis, a technique mostly used in psychometrics, can be leveraged to provide interesting and meaningful visualizations of tree ensemble predictions. Observations and nodes of the tree ensemble are placed in a bipartite graph, connecting each observation to all nodes it belongs to. The graph layout is then chosen by minimizing the sum of the squared edge lengths under certain constraints. We propose a variation of Homogeneity Analysis, called Partition Maps, and analyze advantages and shortcomings compared with multidimensional scaling of proximity matrices. Partition Maps have as potential advantages that (a) the influence of the original nodes and variables is visible in the low-dimensional embedding, similar to biplots, (b) new test observations can be added very easily, and (c) the test error is very similar to the original tree ensemble when using simple nearest-neighbor classification in the two-dimensional Partition Map embedding. Class boundaries, as found by the original tree ensemble algorithm, are thus reflected accurately in Partition Maps. Subgroups and outliers can furthermore be identified in the low-dimensional visualizations, allowing meaningful exploratory analysis of tree ensembles. An R-package partitionMap is provided as a supplementary material.

REDUCING DECISION TREE ENSEMBLE SIZE USING PARALLEL DECISION DAGS

This research presents a new learning model, the Parallel Decision DAG (PDDAG), and shows how to use it to represent an ensemble of decision trees while using significantly less storage. Ensembles such as Bagging and Boosting have a high probability of encoding redundant data structures, and PDDAGs provide a way to remove this redundancy in decision tree based ensembles. When trained by encoding an ensemble, the new model behaves similar to the original ensemble, and can be made to perform identically to it. The reduced storage requirements allow an ensemble approach to be used in cases where storage requirements would normally be exceeded, and the smaller model can potentially execute faster by reducing redundant computation.

Two alternative implementations of the ensemble-transform (ET) analysis-perturbation scheme: The ET with extended cycling intervals, and the ET without cycling

Four alternative approximate bases for the ensemble transform (ET) are obtained by extending the cycling interval to 24, 48, 72 and 96 h. Another alternative basis is obtained by foregoing cycling and instead drawing randomly generated perturbations from an archive. Experiments based upon 16-member global ensembles and a diagonal estimate of analysis-error covariance indicate that the alternative bases are effective at reducing the discrepancy between the ET analysis-perturbation variance and the estimated analysis-error variance. Forecast ensembles associated with the alternative bases maintain considerably more energy in the tropics and subtropics than the forecast ensemble associated with the original basis. Forecast ensembles associated with the alternative bases also outperform the original forecast ensemble in terms of the ensemble forecast-error covariance-matrix eigenvalue spectrum, the relationship between ensemble variance and observed squared error and the Brier score. The performance gains facilitated by the alternative approximate bases are substantial in some instances, especially in the tropics. The randomly sampled basis is superior to the original basis in most respects. Published in 2009 by John Wiley & Sons, Ltd.

COCO: A simple tool to enrich the representation of conformational variability in NMR structures

NMR structures are typically deposited in databases such as the PDB in the form of an ensemble of structures. Generally, each of the models in such an ensemble satisfies the experimental data and is equally valid. No unique solution can be calculated because the experimental NMR data is insufficient, in part because it reflects the conformational variability and dynamical behavior of the molecule in solution. Even for relatively rigid molecules, the limited number of structures that are typically deposited cannot completely encompass the structural diversity allowed by the observed NMR data, but they can be chosen to try and maximize its representation. We describe here the adaptation and application of techniques more commonly used to examine large ensembles from molecular dynamics simulations, to the analysis of NMR ensembles. The approach, which is based on principal component analysis, we call COCO ("Complementary Coordinates"). The COCO approach analyses the distribution of an NMR ensemble in conformational space, and generates a new ensemble that fills "gaps" in the distribution. The method is very rapid, and analysis of a 25-member ensemble and generation of a new 25 member ensemble typically takes 1-2 min on a conventional workstation. Applied to the 545 structures in the RECOORD database, we find that COCO generates new ensembles that are as structurally diverse-both from each other and from the original ensemble-as are the structures within the original ensemble. The COCO approach does not explicitly take into account the NMR restraint data, yet in tests on selected structures from the RECOORD database, the COCO ensembles are frequently good matches to this data, and certainly are structures that can be rapidly refined against the restraints to yield high-quality, novel solutions. COCO should therefore be a useful aid in NMR structure refinement and in other situations where a richer representation of conformational variability is desired-for example in docking studies. COCO is freely accessible via the website www.ccpb.ac.uk/COCO.

Forest Garrote

Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of ℓ1-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these regularized linear models are often slightly inferior to machine learning procedures like tree ensembles. Tree ensembles, on the other hand, lack usually a formal way of variable selection and are difficult to visualize. A Garrote-style convex penalty for trees ensembles, in particular Random Forests, is proposed. The penalty selects functional groups of nodes in the trees. These could be as simple as monotone functions of individual predictor variables. This yields a parsimonious function fit, which lends itself easily to visualization and interpretation. The predictive power is maintained at least at the same level as the original tree ensemble. A key feature of the method is that, once a tree ensemble is fitted, no further tuning parameter needs to be selected. The empirical performance is demonstrated on a wide array of datasets.

Collective-agreement-based pruning of ensembles

Ensemble methods combine several individual pattern classifiers in order to achieve better classification. The challenge is to choose the minimal number of classifiers that achieve the best performance. An ensemble that contains too many members might incur large storage requirements and even reduce the classification performance. The goal of ensemble pruning is to identify a subset of ensemble members that performs at least as good as the original ensemble and discard any other members. In this paper, we introduce the Collective-Agreement-based Pruning (CAP) method. Rather than ranking individual members, CAP ranks subsets by considering the individual predictive ability of each member along with the degree of redundancy among them. Subsets whose members highly agree with the class while having low inter-agreement are preferred.

Martingale dynamics in quantum discrimination

Conclusive discrimination of quantum states is expressible as a sequence of optimal weak measurements. The sequence of probabilities of successful discrimination realized in this process is shown to be a martingale, a model of a fair game. For discriminating between two pure states, this martingale is constant; for at least one strictly mixed state or more than two pure states, it is constant just on average. In all these latter cases, therefore, the realized discrimination probability conditioned on prior measurements may be greater than the optimal success probability of the original ensemble. So in these cases superoptimal discrimination is conditionally achievable. This phenomenon is demonstrated by examples.

On the Freezing of Variables in Random Constraint Satisfaction Problems

The set of solutions of random constraint satisfaction problems (zero energy groundstates of mean-field diluted spin glasses) undergoes several structural phase transitions as the amount of constraints is increased. This set first breaks down into a large number of well separated clusters. At the freezing transition, which is in general distinct from the clustering one, some variables (spins) take the same value in all solutions of a given cluster. In this paper we introduce and study a message passing procedure that allows to compute, for generic constraint satisfaction problems, the sizes of the rearrangements induced in response to the modification of a variable. These sizes diverge at the freezing transition, with a critical behavior which is also investigated in details. We apply the generic formalism in particular to the random satisfiability of boolean formulas and to the coloring of random graphs. The computation is first performed in random tree ensembles, for which we underline a connection with percolation models and with the reconstruction problem of information theory. The validity of these results for the original random ensembles is then discussed in the framework of the cavity method.

Biosensors based on gold nanoelectrode ensembles and screen printed electrodes

Gold nanowires were synthesized within polycarbonate membranes according to an electroless deposition method, obtaining nanoelectrode ensembles (NEEs) with special electrochemical features. NEEs were coupled with home-produced carbon graphite screen printed electrodes and the electrochemical properties of the original nanoelectrode ensemble on screen printed substrate (NEE/SPS) assembly has been tested for sensors application. Glucose oxidase has been used as model enzyme in order to verify the feasibility of disposable gold NEE/SPS biosensors. Finally, different immobilisation and electrochemical deposition techniques based on either self assembled monolayers of cysteamine (CYS) or amino-propyl-triethoxysilane (APTES) and conductive polyaniline (PANI) molecular wires were used. Spatial patterning of the enzyme on the polycarbonate surface and of PANI wires on gold nanoelectrodes was obtained. Possible direct electron transfer between the enzyme and the PANI modified gold nanoelectrodes has been evaluated.

Multi-model ensemble hydrologic prediction using Bayesian model averaging

Multi-model ensemble strategy is a means to exploit the diversity of skillful predictions from different models. This paper studies the use of Bayesian model averaging (BMA) scheme to develop more skillful and reliable probabilistic hydrologic predictions from multiple competing predictions made by several hydrologic models. BMA is a statistical procedure that infers consensus predictions by weighing individual predictions based on their probabilistic likelihood measures, with the better performing predictions receiving higher weights than the worse performing ones. Furthermore, BMA provides a more reliable description of the total predictive uncertainty than the original ensemble, leading to a sharper and better calibrated probability density function (PDF) for the probabilistic predictions. In this study, a nine-member ensemble of hydrologic predictions was used to test and evaluate the BMA scheme. This ensemble was generated by calibrating three different hydrologic models using three distinct objective functions. These objective functions were chosen in a way that forces the models to capture certain aspects of the hydrograph well (e.g., peaks, mid-flows and low flows). Two sets of numerical experiments were carried out on three test basins in the US to explore the best way of using the BMA scheme. In the first set, a single set of BMA weights was computed to obtain BMA predictions, while the second set employed multiple sets of weights, with distinct sets corresponding to different flow intervals. In both sets, the streamflow values were transformed using Box–Cox transformation to ensure that the probability distribution of the prediction errors is approximately Gaussian. A split sample approach was used to obtain and validate the BMA predictions. The test results showed that BMA scheme has the advantage of generating more skillful and equally reliable probabilistic predictions than original ensemble. The performance of the expected BMA predictions in terms of daily root mean square error (DRMS) and daily absolute mean error (DABS) is generally superior to that of the best individual predictions. Furthermore, the BMA predictions employing multiple sets of weights are generally better than those using single set of weights.

Ensemble Pruning Via Semi-definite Programming

An ensemble is a group of learning models that jointly solve a problem. However, the ensembles generated by existing techniques are sometimes unnecessarily large, which can lead to extra memory usage, computational costs, and occasional decreases in effectiveness. The purpose of ensemble pruning is to search for a good subset of ensemble members that performs as well as, or better than, the original ensemble. This subset selection problem is a combinatorial optimization problem and thus finding the exact optimal solution is computationally prohibitive. Various heuristic methods have been developed to obtain an approximate solution. However, most of the existing heuristics use simple greedy search as the optimization method, which lacks either theoretical or empirical quality guarantees. In this paper, the ensemble subset selection problem is formulated as a quadratic integer programming problem. By applying semi-definite programming (SDP) as a solution technique, we are able to get better approximate solutions. Computational experiments show that this SDP-based pruning algorithm outperforms other heuristics in the literature. Its application in a classifier-sharing study also demonstrates the effectiveness of the method.