Differences In Predictive Performance Research Articles

Reliable Generation of Native-Like Decoys Limits Predictive Ability in Fragment-Based Protein Structure Prediction.

Our previous work with fragment-assembly methods has demonstrated specific deficiencies in conformational sampling behaviour that, when addressed through improved sampling algorithms, can lead to more reliable prediction of tertiary protein structure when good fragments are available, and when score values can be relied upon to guide the search to the native basin. In this paper, we present preliminary investigations into two important questions arising from more difficult prediction problems. First, we investigated the extent to which native-like conformational states are generated during multiple runs of our search protocols. We determined that, in cases of difficult prediction, native-like decoys are rarely or never generated. Second, we developed a scheme for decoy retention that balances the objectives of retaining low-scoring structures and retaining conformationally diverse structures sampled during the course of the search. Our method succeeds at retaining more diverse sets of structures, and, for a few targets, more native-like solutions are retained as compared to our original, energy-based retention scheme. However, in general, we found that the rate at which native-like structural states are generated has a much stronger effect on eventual distributions of predictive accuracy in the decoy sets, as compared to the specific decoy retention strategy used. We found that our protocols show differences in their ability to access native-like states for some targets, and this may explain some of the differences in predictive performance seen between these methods. There appears to be an interaction between fragment sets and move operators, which influences the accessibility of native-like structures for given targets. Our results point to clear directions for further improvements in fragment-based methods, which are likely to enable higher accuracy predictions.

Macrocycle modeling in ICM: benchmarking and evaluation in D3R Grand Challenge 4.

Macrocycles represent a potentially vast extension of drug chemical space still largely untapped by synthetic compounds. Sampling of flexible rings is incorporated in the ICM-dock protocol. We tested the ability of ICM-dock to reproduce macrocyclic ligand-protein receptor complexes, first in a large retrospective benchmark (246 complexes), and next, in context of the D3R Grand Challenge 4 (GC4), where we modeled bound complexes and predicted activities for a series of macrocyclic BACE inhibitors. Sub-angstrom accuracy was achieved in ligand pose prediction both in cross-docking (D3R Challenge Stage 1A) and cognate (Stage 1B) setup. Stage 1B submission was top ranked by mean and average RMSDs, even though no ligand knowledge was used in our simulations on this Stage. Furthermore, we demonstrate successful receptor conformational selection in Stage 1A, aided by the enhanced '4D' multiple receptor conformation docking protocol with optimized scoring offsets. In the activity 3D QSAR modeling, predictivity of the BACE pKd model was modest, while for the second target (Cathepsin-S), leading performance was achieved. Difference in activity prediction performance between the targets is likely explained by the amount of available and relevant training data.

Degree and centrality-based approaches in network-based variable selection: Insights from the Singapore Longitudinal Aging Study.

We describe a network-based method to obtain a subset of representative variables from clinical data of subjects of the second Singapore Longitudinal Aging Study (SLAS-2), while preserving to a good extent the predictive performance of the full set with regards to a multi-faceted index of successful aging, SAGE. To examine differences in predictive performance of high-degree nodes (“hubs”) and high-centrality ones (“cores”), we implement four subsetting strategies (two degree-based, two centrality-based) and obtain four surrogate sets of variables, which we use as input features for machine learning models to predict the SAGE index of subjects. All four models have variables belonging to the physical, cardiovascular, cognitive and immunological domains among their fifteen most important predictors. A fifth domain (leisure-time activities, LTA) is also present in some form. From a comparison of the surrogate sets’ size and predictive performance, a centrality-based approach (selection of the most central variable-nodes within each cluster) yielded the smallest-sized surrogate set, while having high prediction accuracy (measured by its model’s area-under-curve, AUC) in comparison to its analogous degree-based strategy (selection of the highest-degree nodes per cluster). Inclusion of the next most-central variables yielded negligible changes in predictive performance while more than doubling the surrogate set size. The centrality-based approach thus yields a surrogate set which offers a good balance between number of variables and prediction performance, and can act as a representative subset of the SLAS-2 clinical dataset.

Accuracy of Gene Expression Prediction From Genotype Data With PrediXcan Varies Across and Within Continental Populations.

Using genetic data to predict gene expression has garnered significant attention in recent years. PrediXcan has become one of the most widely used gene-based methods for testing associations between predicted gene expression values and a phenotype, which has facilitated novel insights into the relationship between complex traits and the component of gene expression that can be attributed to genetic variation. The gene expression prediction models for PrediXcan were developed using supervised machine learning methods and training data from the Depression Genes and Networks (DGN) study and the Genotype-Tissue Expression (GTEx) project, where the majority of subjects are of European descent. Many genetic studies, however, include samples from multi-ethnic populations, and in this paper we evaluate the accuracy of PrediXcan for predicting gene expression in diverse populations. Using transcriptomic data from the GEUVADIS (Genetic European Variation in Disease) RNA sequencing project and whole genome sequencing data from the 1000 Genomes project, we evaluate and compare the predictive performance of PrediXcan in an African population (Yoruban) and four European ancestry populations for thousands of genes. We evaluate a range of models from the PrediXcan weight databases and use Pearson's correlation coefficient to assess gene expression prediction accuracy with PrediXcan. From our evaluation, we find that the predictive performance of PrediXcan varies substantially among populations from different continents (F-test p-value < 2.2 × 10−16), where prediction accuracy is lower in the Yoruban population from West Africa compared to the European-ancestry populations. Moreover, not only do we find differences in predictive performance between populations from different continents, we also find highly significant differences in prediction accuracy among the four European ancestry populations considered (F-test p-value < 2.2 × 10−16). Finally, while there is variability in prediction accuracy across different PrediXcan weight databases, we also find consistency in the qualitative performance of PrediXcan for the five populations considered, with the African ancestry population having the lowest accuracy across databases.

Operationally Structured Model for Strategic Runway Configuration Predictions

This paper presents a model for predicting runway configurations for congested airports by hour, over a full-day look-ahead horizon. Airport departure and arrival capacity is primarily determined by the set of runways in use and the terminal weather, which can limit the volume of operations. Predicting airport runway configuration and the resulting operational capacity, especially for high-volume airports, can therefore provide valuable information for strategic planning. The model developed in this paper encapsulates the multiscale operational procedures involved in selecting runway configurations, while also accounting for the terminal-area weather forecast data. A detailed case study for two airports is presented and the model’s performance is evaluated for all of the Federal Aviation Administration’s 35 Operational Evolution Partnership airports, using both recorded weather and Terminal Aerodrome Forecasts. The differences in prediction performance between weather sources and among airports are examined.

Predicting Pulmonary Function Testing from Quantified Computed Tomography Using Machine Learning Algorithms in Patients with COPD.

Introduction: Quantitative computed tomography (qCT) is an emergent technique for diagnostics and research in patients with chronic obstructive pulmonary disease (COPD). qCT parameters demonstrate a correlation with pulmonary function tests and symptoms. However, qCT only provides anatomical, not functional, information. We evaluated five distinct, partial-machine learning-based mathematical models to predict lung function parameters from qCT values in comparison with pulmonary function tests. Methods: 75 patients with diagnosed COPD underwent body plethysmography and a dose-optimized qCT examination on a third-generation, dual-source CT with inspiration and expiration. Delta values (inspiration—expiration) were calculated afterwards. Four parameters were quantified: mean lung density, lung volume low-attenuated volume, and full width at half maximum. Five models were evaluated for best prediction: average prediction, median prediction, k-nearest neighbours (kNN), gradient boosting, and multilayer perceptron. Results: The lowest mean relative error (MRE) was calculated for the kNN model with 16%. Similar low MREs were found for polynomial regression as well as gradient boosting-based prediction. Other models led to higher MREs and thereby worse predictive performance. Beyond the sole MRE, distinct differences in prediction performance, dependent on the initial dataset (expiration, inspiration, delta), were found. Conclusion: Different, partially machine learning-based models allow the prediction of lung function values from static qCT parameters within a reasonable margin of error. Therefore, qCT parameters may contain more information than we currently utilize and can potentially augment standard functional lung testing.

Predicting response to somatostatin analogues in acromegaly: machine learning-based high-dimensional quantitative texture analysis on T2-weighted MRI.

To investigate the value of machine learning (ML)-based high-dimensional quantitative texture analysis (qTA) on T2-weighted magnetic resonance imaging (MRI) in predicting response to somatostatin analogues (SA) in acromegaly patients with growth hormone (GH)-secreting pituitary macroadenoma, and to compare the qTA with quantitative and qualitative T2-weighted relative signal intensity (rSI) and immunohistochemical evaluation. Forty-seven patients (24 responsive; 23 resistant patients to SA) were eligible for this retrospective study. Coronal T2-weighted images were used for qTA and rSI evaluation. The immunohistochemical evaluation was based on the granulation pattern of the adenomas. Dimension reduction was carried out by reproducibility analysis and wrapper-based algorithm. ML classifiers were k-nearest neighbours (k-NN) and C4.5 algorithm. The reference standard was the biochemical response status. Predictive performance of qTA was compared with those of the quantitative and qualitative rSI and immunohistochemical evaluation. Five hundred thirty-five out of 828 texture features had excellent reproducibility. For the qTA, k-NN correctly classified 85.1% of the macroadenomas regarding response to SAs with an area under the receiver operating characteristic curve (AUC-ROC) of 0.847. The accuracy and AUC-ROC ranges of the other methods were 57.4-70.2% and 0.575-0.704, respectively. Differences in predictive performance between qTA-based classification and the other methods were significant (p < 0.05). The ML-based qTA of T2-weighted MRI is a potential non-invasive tool in predicting response to SAs in patients with acromegaly and GH-secreting pituitary macroadenoma. The method performed better than the qualitative and quantitative rSI and immunohistochemical evaluation. • Machine learning-based texture analysis of T2-weighted MRI can correctly classify response to somatostatin analogues in more than four fifths of the patients. • Machine learning-based texture analysis performs better than qualitative and quantitative evaluation of relative T2 signal intensity and immunohistochemical evaluation. • About one third of the texture features may not be excellently reproducible, indicating that a reliability analysis is necessary before model development.

Evaluation and Optimization of Pharmacokinetic Models for in Vitro to in Vivo Extrapolation of Estrogenic Activity for Environmental Chemicals.

Background:To effectively incorporate in vitro data into regulatory use, confidence must be established in the quantitative extrapolation of in vitro activity to relevant end points in animals or humans.Objective:Our goal was to evaluate and optimize in vitro to in vivo extrapolation (IVIVE) approaches using in vitro estrogen receptor (ER) activity to predict estrogenic effects measured in rodent uterotrophic studies.Methods:We evaluated three pharmacokinetic (PK) models with varying complexities to extrapolate in vitro to in vivo dosimetry for a group of 29 ER agonists, using data from validated in vitro [U.S. Environmental Protection Agency (U.S. EPA) ToxCast™ ER model] and in vivo (uterotrophic) methods. In vitro activity values were adjusted using mass-balance equations to estimate intracellular exposure via an enrichment factor (EF), and steady-state model calculations were adjusted using fraction of unbound chemical in the plasma () to approximate bioavailability. Accuracy of each model-adjustment combination was assessed by comparing model predictions with lowest effect levels (LELs) from guideline uterotrophic studies.Results:We found little difference in model predictive performance based on complexity or route-specific modifications. Simple adjustments, applied to account for in vitro intracellular exposure (EF) or chemical bioavailability (), resulted in significant improvements in the predictive performance of all models.Conclusion:Computational IVIVE approaches accurately estimate chemical exposure levels that elicit positive responses in the rodent uterotrophic bioassay. The simplest model had the best overall performance for predicting both oral (PPK_EF) and injection (PPK_) LELs from guideline uterotrophic studies, is freely available, and can be parameterized entirely using freely available in silico tools. https://doi.org/10.1289/EHP1655

Genetic architecture of gene expression traits across diverse populations.

For many complex traits, gene regulation is likely to play a crucial mechanistic role. How the genetic architectures of complex traits vary between populations and subsequent effects on genetic prediction are not well understood, in part due to the historical paucity of GWAS in populations of non-European ancestry. We used data from the MESA (Multi-Ethnic Study of Atherosclerosis) cohort to characterize the genetic architecture of gene expression within and between diverse populations. Genotype and monocyte gene expression were available in individuals with African American (AFA, n = 233), Hispanic (HIS, n = 352), and European (CAU, n = 578) ancestry. We performed expression quantitative trait loci (eQTL) mapping in each population and show genetic correlation of gene expression depends on shared ancestry proportions. Using elastic net modeling with cross validation to optimize genotypic predictors of gene expression in each population, we show the genetic architecture of gene expression for most predictable genes is sparse. We found the best predicted gene in each population, TACSTD2 in AFA and CHURC1 in CAU and HIS, had similar prediction performance across populations with R2 > 0.8 in each population. However, we identified a subset of genes that are well-predicted in one population, but poorly predicted in another. We show these differences in predictive performance are due to allele frequency differences between populations. Using genotype weights trained in MESA to predict gene expression in independent populations showed that a training set with ancestry similar to the test set is better at predicting gene expression in test populations, demonstrating an urgent need for diverse population sampling in genomics. Our predictive models and performance statistics in diverse cohorts are made publicly available for use in transcriptome mapping methods at https://github.com/WheelerLab/DivPop.

Handling missing values: A study of popular imputation packages in R

In real world data are often plagued by missing values which adversely affects the final outcome of the analysis based on such data. The missing values can be handled using various techniques like deletion or imputation. Of late, R has become one of the most preferred platform for carrying out data analysis, and its popularity is growing further. R provides various packages for handling missing values through imputation. The presence of multiple packages however, calls for an analysis of their comparative performance and examine their suitability for handling a given set of data. The performance of different R packages may differ for different datasets and may depend on the size of the dataset and richness of the missing values in the datasets. In this paper, the authors perform comparative study of the performance of the common R packages, namely VIM, MICE, MissForest, and HMISC, used for missing value imputation. The authors measured the performances of the said packages in terms of their imputation time, imputation efficiency and the effect on the variance. The imputation efficiency was measured in terms of the difference in predictive performance of a model built using original dataset vis-à-vis a dataset with imputed values. Similarly, the variance of the variables in the original dataset was compared that of corresponding variables in the imputed dataset. A missing value imputation package can be considered to be better if it consumes less imputation time and provides high imputation accuracy. Also in terms of variance, one would like to have the imputation package maintain the original variance of the variables. On analysing the four imputation packages on two datasets over three predictive algorithms–Logistic Regression, Support Vector Machines, and Artificial Neural Networks–it was observed that the performances varies depending on the size of the dataset, and the missing values present in them. The study highlights that certain missing value package used in conjunction with a given predictive algorithm provides better performance, which is again a function of the dataset characteristics.

Comparison of the performance of mental health, drug and alcohol comorbidities based on ICD-10-AM and medical records for predicting 12-month outcomes in trauma patients

BackgroundMany outcome studies capture the presence of mental health, drug and alcohol comorbidities from administrative datasets and medical records. How these sources compare as predictors of patient outcomes has not been determined. The purpose of the present study was to compare mental health, drug and alcohol comorbidities based on ICD-10-AM coding and medical record documentation for predicting longer-term outcomes in injured patients.MethodsA random sample of patients (n = 500) captured by the Victorian State Trauma Registry was selected for the study. Retrospective medical record reviews were conducted to collect data about documented mental health, drug and alcohol comorbidities while ICD-10-AM codes were obtained from routinely collected hospital data. Outcomes at 12-months post-injury were the Glasgow Outcome Scale – Extended (GOS-E), European Quality of Life Five Dimensions (EQ-5D-3L), and return to work. Linear and logistic regression models, adjusted for age and gender, using medical record derived comorbidity and ICD-10-AM were compared using measures of calibration (Hosmer-Lemeshow statistic) and discrimination (C-statistic and R2).ResultsThere was no demonstrable difference in predictive performance between the medical record and ICD-10-AM models for predicting the GOS-E, EQ-5D-3L utility sore and EQ-5D-3L mobility, self-care, usual activities and pain/discomfort items. The area under the receiver operating characteristic (AUC) for models using medical record derived comorbidity (AUC 0.68, 95% CI: 0.63, 0.73) was higher than the model using ICD-10-AM data (AUC 0.62, 95% CI: 0.57, 0.67) for predicting the EQ-5D-3L anxiety/depression item. The discrimination of the model for predicting return to work was higher with inclusion of the medical record data (AUC 0.69, 95% CI: 0.63, 0.76) than the ICD-10-AM data (AUC 0.59, 95% CL: 0.52, 0.65).ConclusionsMental health, drug and alcohol comorbidity information derived from medical record review was not clearly superior for predicting the majority of the outcomes assessed when compared to ICD-10-AM. While information available in medical records may be more comprehensive than in the ICD-10-AM, there appears to be little difference in the discriminative capacity of comorbidities coded in the two sources.

Method for Automatic Selection of Parameters in Normal Tissue Complication Probability Modeling

Topresent a fully automatic method to generate multiparameter normal tissue complication probability (NTCP) models and compare its results with those of a published model, using the same patient cohort. Data were analyzed from 345 rectal cancer patients treated with external radiation therapy to predict the risk of patients developing grade 1 or ≥2 cystitis. In total, 23 clinical factors were included in the analysis as candidate predictors of cystitis. Principal component analysis was used to decompose the bladder dose-volume histogram into 8 principal components, explaining more than 95% of the variance. The data set of clinical factors and principal components was divided into training (70%) and test (30%) data sets, with the training data set used by the algorithm to compute an NTCP model. The first step of the algorithm was to obtain a bootstrap sample, followed by multicollinearity reduction using the variance inflation factor and genetic algorithm optimization to determine an ordinal logistic regression model that minimizes the Bayesian information criterion. The process was repeated 100 times, and the model with the minimum Bayesian information criterion was recorded on each iteration. The most frequent model was selected as the final "automatically generated model" (AGM). The published model and AGM were fitted on the training data sets, and the risk of cystitis was calculated. The 2 models had no significant differences in predictive performance, both for the training and test data sets (P value>.05) and found similar clinical and dosimetric factors as predictors. Both models exhibited good explanatory performance on the training data set (P values>.44), which was reduced on the test data sets (Pvalues<.05). The predictive value of the AGM is equivalent to that of the expert-derived published model. It demonstrates potential in saving time, tackling problems with a large number of parameters, and standardizing variable selection in NTCP modeling.

Evaluation of digital soil mapping approaches with large sets of environmental covariates

Abstract. The spatial assessment of soil functions requires maps of basic soil properties. Unfortunately, these are either missing for many regions or are not available at the desired spatial resolution or down to the required soil depth. The field-based generation of large soil datasets and conventional soil maps remains costly. Meanwhile, legacy soil data and comprehensive sets of spatial environmental data are available for many regions.Digital soil mapping (DSM) approaches relating soil data (responses) to environmental data (covariates) face the challenge of building statistical models from large sets of covariates originating, for example, from airborne imaging spectroscopy or multi-scale terrain analysis. We evaluated six approaches for DSM in three study regions in Switzerland (Berne, Greifensee, ZH forest) by mapping the effective soil depth available to plants (SD), pH, soil organic matter (SOM), effective cation exchange capacity (ECEC), clay, silt, gravel content and fine fraction bulk density for four soil depths (totalling 48 responses). Models were built from 300–500 environmental covariates by selecting linear models through (1) grouped lasso and (2) an ad hoc stepwise procedure for robust external-drift kriging (georob). For (3) geoadditive models we selected penalized smoothing spline terms by component-wise gradient boosting (geoGAM). We further used two tree-based methods: (4) boosted regression trees (BRTs) and (5) random forest (RF). Lastly, we computed (6) weighted model averages (MAs) from the predictions obtained from methods 1–5.Lasso, georob and geoGAM successfully selected strongly reduced sets of covariates (subsets of 3–6 % of all covariates). Differences in predictive performance, tested on independent validation data, were mostly small and did not reveal a single best method for 48 responses. Nevertheless, RF was often the best among methods 1–5 (28 of 48 responses), but was outcompeted by MA for 14 of these 28 responses. RF tended to over-fit the data. The performance of BRT was slightly worse than RF. GeoGAM performed poorly on some responses and was the best only for 7 of 48 responses. The prediction accuracy of lasso was intermediate. All models generally had small bias. Only the computationally very efficient lasso had slightly larger bias because it tended to under-fit the data. Summarizing, although differences were small, the frequencies of the best and worst performance clearly favoured RF if a single method is applied and MA if multiple prediction models can be developed.

Modelling field scale spatial variation in water run-off, soil moisture, N2O emissions and herbage biomass of a grazed pasture using the SPACSYS model

In this study, we evaluated the ability of the SPACSYS model to simulate water run-off, soil moisture, N2O fluxes and grass growth using data generated from a field of the North Wyke Farm Platform. The field-scale model is adapted via a linked and grid-based approach (grid-to-grid) to account for not only temporal dynamics but also the within-field spatial variation in these key ecosystem indicators. Spatial variability in nutrient and water presence at the field-scale is a key source of uncertainty when quantifying nutrient cycling and water movement in an agricultural system. Results demonstrated that the new spatially distributed version of SPACSYS provided a worthy improvement in accuracy over the standard (single-point) version for biomass productivity. No difference in model prediction performance was observed for water run-off, reflecting the closed-system nature of this variable. Similarly, no difference in model prediction performance was found for N2O fluxes, but here the N2O predictions were noticeably poor in both cases. Further developmental work, informed by this study's findings, is proposed to improve model predictions for N2O. Soil moisture results with the spatially distributed version appeared promising but this promise could not be objectively verified.

Demystifying Multitask Deep Neural Networks for Quantitative Structure-Activity Relationships.

Deep neural networks (DNNs) are complex computational models that have found great success in many artificial intelligence applications, such as computer vision1,2 and natural language processing.3,4 In the past four years, DNNs have also generated promising results for quantitative structure-activity relationship (QSAR) tasks.5,6 Previous work showed that DNNs can routinely make better predictions than traditional methods, such as random forests, on a diverse collection of QSAR data sets. It was also found that multitask DNN models-those trained on and predicting multiple QSAR properties simultaneously-outperform DNNs trained separately on the individual data sets in many, but not all, tasks. To date there has been no satisfactory explanation of why the QSAR of one task embedded in a multitask DNN can borrow information from other unrelated QSAR tasks. Thus, using multitask DNNs in a way that consistently provides a predictive advantage becomes a challenge. In this work, we explored why multitask DNNs make a difference in predictive performance. Our results show that during prediction a multitask DNN does borrow "signal" from molecules with similar structures in the training sets of the other tasks. However, whether this borrowing leads to better or worse predictive performance depends on whether the activities are correlated. On the basis of this, we have developed a strategy to use multitask DNNs that incorporate prior domain knowledge to select training sets with correlated activities, and we demonstrate its effectiveness on several examples.

Performance of risk prediction for inflammatory bowel disease based on genotyping platform and genomic risk score method

BackgroundPredicting risk of disease from genotypes is being increasingly proposed for a variety of diagnostic and prognostic purposes. Genome-wide association studies (GWAS) have identified a large number of genome-wide significant susceptibility loci for Crohn’s disease (CD) and ulcerative colitis (UC), two subtypes of inflammatory bowel disease (IBD). Recent studies have demonstrated that including only loci that are significantly associated with disease in the prediction model has low predictive power and that power can substantially be improved using a polygenic approach.MethodsWe performed a comprehensive analysis of risk prediction models using large case-control cohorts genotyped for 909,763 GWAS SNPs or 123,437 SNPs on the custom designed Immunochip using four prediction methods (polygenic score, best linear genomic prediction, elastic-net regularization and a Bayesian mixture model). We used the area under the curve (AUC) to assess prediction performance for discovery populations with different sample sizes and number of SNPs within cross-validation.ResultsOn average, the Bayesian mixture approach had the best prediction performance. Using cross-validation we found little differences in prediction performance between GWAS and Immunochip, despite the GWAS array providing a 10 times larger effective genome-wide coverage. The prediction performance using Immunochip is largely due to the power of the initial GWAS for its marker selection and its low cost that enabled larger sample sizes. The predictive ability of the genomic risk score based on Immunochip was replicated in external data, with AUC of 0.75 for CD and 0.70 for UC. CD patients with higher risk scores demonstrated clinical characteristics typically associated with a more severe disease course including ileal location and earlier age at diagnosis.ConclusionsOur analyses demonstrate that the power of genomic risk prediction for IBD is mainly due to strongly associated SNPs with considerable effect sizes. Additional SNPs that are only tagged by high-density GWAS arrays and low or rare-variants over-represented in the high-density region on the Immunochip contribute little to prediction accuracy. Although a quantitative assessment of IBD risk for an individual is not currently possible, we show sufficient power of genomic risk scores to stratify IBD risk among individuals at diagnosis.

Predicting soil properties in the Canadian boreal forest with limited data: Comparison of spatial and non-spatial statistical approaches

Digital soil mapping (DSM) involves the use of georeferenced information and statistical models to map predictions and uncertainties related to soil properties. Many remote regions of the globe, such as boreal forest ecosystems, are characterized by low sampling efforts and limited availability of field soil data. Although DSM is an expanding topic in soil science, little guidance currently exists to select the appropriate combination of statistical methods and model formulation in the context of limited data availability. Using the Canadian managed forest as a case study, the main objective of this study was to investigate to which extent the choice of statistical method and model specification could improve the spatial prediction of soil properties with limited data. More specifically, we compared the cross-product performance of eight statistical approaches (linear, additive and geostatistical models, and four machine-learning techniques) and three model formulations (“covariates only”: a suite of environmental covariates only; “spatial only”: a function of geographic coordinates only; and “covariates+spatial”: a combination of both covariates and spatial functions) to predict five key forest soil properties in the organic layer (thickness and C:N ratio) and in the top 15cm of the mineral horizon (carbon concentration, percentage of sand, and bulk density). Our results show that 1) although strong differences in predictive performance occurred across all statistical approaches and model formulations, spatially explicit models consistently had higher R2 and lower RMSE values than non-spatial models for all soil properties, except for the C:N ratio; 2) Bayesian geostatistical models were among the best methods, followed by ordinary kriging and machine-learning methods; and 3) comparative analyses made it possible to identify the more performant models and statistical methods to predict specific soil properties. We make modeling tools and code available (e.g., Bayesian geostastical models) that increase DSM capabilities and support existing efforts toward the production of improved digital soil products with limited data.

Comparison of epileptic seizure prediction performance for different EEG derivation schemes

Introduction. The influence of EEG derivation scheme on performance of epileptic seizure prediction is considered in the paper. Comparison of epileptic seizure prediction performance for different EEG derivation schemes is conducted. Database of EEG recordings from 20 patients (between 1 and 25 years old) suffering epilepsy was used. Correlation coefficients between channels of signal extracted with different combination of window and preictal lengths were used as features. Support vector machine was used to classify EEG data into interictal and preictal classes. Epileptic seizures prediction performance was evaluated using area under the ROC-curve for two types of derivation schemes: monopolar scheme with reference electrode on ipsilateral ear and scheme with averaged reference. Conclusions. It was found that there is a difference in epileptic seizures prediction performance for different schemes and patients. For subgroup of patients (9 of 20) usage of scheme with reference electrode in ipsilateral ear shows higher performance for window length between 2 and 10 seconds and in entire range of preictal lengths. Scheme with averaged reference shows higher performance when window length is in the range between 90 and 300 seconds.

Allometric Equations for Estimating Tree Aboveground Biomass in Tropical Dipterocarp Forests of Vietnam

There are few allometric equations available for dipterocarp forests, despite the fact that this forest type covers extensive areas in tropical Southeast Asia. This study aims to develop a set of equations to estimate tree aboveground biomass (AGB) in dipterocarp forests in Vietnam and to validate and compare their predictive performance with allometric equations used for dipterocarps in Indonesia and pantropical areas. Diameter at breast height (DBH), total tree height (H), and wood density (WD) were used as input variables of the nonlinear weighted least square models. Akaike information criterion (AIC) and residual plots were used to select the best models; while percent bias, root mean square percentage error, and mean absolute percent error were used to compare their performance to published models. For mixed-species, the best equation was AGB = 0.06203 × DBH 2.26430 × H 0.51415 × WD 0.79456 . When applied to a random independent validation dataset, the predicted values from the generic equations and the dipterocarp equations in Indonesia overestimated the AGB for different sites, indicating the need for region-specific equations. At the genus level, the selected equations were AGB = 0.03713 × DBH 2.73813 and AGB = 0.07483 × DBH 2.54496 for two genera, Dipterocarpus and Shorea, respectively, in Vietnam. Compared to the mixed-species equations, the genus-specific equations improved the accuracy of the AGB estimates. Additionally, the genus-specific equations showed no significant differences in predictive performance in different regions (e.g., Indonesia, Vietnam) of Southeast Asia.

Validation of a national hydrological model

Nationwide predictions of flow time-series are valuable for development of policies relating to environmental flows, calculating reliability of supply to water users, or assessing risk of floods or droughts. This breadth of model utility is possible because various hydrological signatures can be derived from simulated flow time-series. However, producing national hydrological simulations can be challenging due to strong environmental diversity across catchments and a lack of data available to aid model parameterisation. A comprehensive and consistent suite of test procedures to quantify spatial and temporal patterns in performance across various parts of the hydrograph is described and applied to quantify the performance of an uncalibrated national rainfall-runoff model of New Zealand. Flow time-series observed at 485 gauging stations were used to calculate Nash-Sutcliffe efficiency and percent bias when simulating between-site differences in daily series, between-year differences in annual series, and between-site differences in hydrological signatures. The procedures were used to assess the benefit of applying a correction to the modelled flow duration curve based on an independent statistical analysis. They were used to aid understanding of climatological, hydrological and model-based causes of differences in predictive performance by assessing multiple hypotheses that describe where and when the model was expected to perform best. As the procedures produce quantitative measures of performance, they provide an objective basis for model assessment that could be applied when comparing observed daily flow series with competing simulated flow series from any region-wide or nationwide hydrological model. Model performance varied in space and time with better scores in larger and medium-wet catchments, and in catchments with smaller seasonal variations. Surprisingly, model performance was not sensitive to aquifer fraction or rain gauge density.