Multi-site Validation Research Articles

Performance of the Tinnitus Functional Index as a diagnostic instrument in a UK clinical population

ObjectivesThe Tinnitus Functional Index (TFI) has been optimised as a diagnostic tool for quantifying the functional impact of tinnitus in US veteran and civilian groups. However, the TFI has not been fully evaluated for use in other English-speaking clinical populations despite its increasingly popular uptake. Here, a prospective multi-site longitudinal validation study was conducted to evaluate psychometric properties relevant to the UK clinical population. Guided by quality criteria for the measurement properties of health-related questionnaires, we specifically evaluated three diagnostic properties relating to the degree to which the TFI (i) covers the eight dimensions proposed to be important for diagnosis, (ii) reliably distinguishes individual differences in severity of tinnitus, and (iii) reliably measures the functional impact of tinnitus. We also examine whether clinically meaningful interpretations of the scores can be produced for the UK population. MethodsTwelve National Health Service audiology clinics across the UK recruited 255 tinnitus patients to complete questionnaires at four time-intervals, from initial clinical assessment and then over a nine-month period. Patients completed the TFI, the Tinnitus Handicap Inventory (THI), tinnitus case history questions, a Global rating of Perceived Problem with tinnitus and a Clinical Global Impression of perceived change in tinnitus. Baseline TFI data were used to examine the factor structure, construct validity and interpretability of the TFI. Follow-up TFI data were used to examine reliability. ResultsConfirmatory factor analysis suggested that of the eight subscales (factors) initially established for the TFI, the ‘Auditory’ subscale did not contribute to the overall construct ‘functional impact of tinnitus’, and a modified seven-factor model (TFI-22) better fit the variance in the patient scores. Both the global 25-item TFI and the global TFI-22 scores showed exceptionally high internal consistency (α ≥ 0.95), high construct validity with the THI (r = 0.80) and high test-retest reliability (ICC = 0.87). Test-retest agreement however was only deemed to be borderline acceptable (89%). Receiver Operator Characteristic analysis indicated the 25-item TFI and TFI-22 has excellent ability to distinguish between different levels of impact (Area under the curve > 0.7). ConclusionThe TFI was confirmed to cover multiple symptom domains, measuring a multi-domain construct of tinnitus, and satisfies a range of psychometric requirements for a good clinical measure, including having excellent reliability, stability over time and sensitivity to individual differences in tinnitus severity. However, a modified seven-factor structure without the Auditory subscale (TFI-22) is recommended for calculating a global composite score for UK patients. Using patients' experience and Receiver Operator Characteristic analysis, a grading system was presented which identifies the distinct grades of tinnitus impact in the UK clinical population that is broadly comparable to the US-based system.

Social cognition and African American men: The roles of perceived discrimination and experimenter race on task performance

The Social Cognition Psychometric Evaluation (SCOPE) study consists of a battery of eight tasks selected to measure social-cognitive deficits in individuals with schizophrenia. The battery is currently in a multisite validation process. While the SCOPE study collects basic demographic data, more nuanced race-related factors might artificially inflate cross-cultural differences in social cognition. As an initial step, we investigated whether race, independent of mental illness status, affects performance on the SCOPE battery. Thus, we examined the effects of perceived discrimination and experimenter race on the performance of 51 non-clinical African American men on the SCOPE battery. Results revealed that these factors impacted social cognitive task performance. Specifically, participants performed better on a skills-based task factor in the presence of Black experimenters, and frequency of perceived racism predicted increased perception of hostility in negative interpersonal situations with accidental causes. Thus, race-related factors are important to identify and explore in the measurement of social cognition in African Americans.

A Brain Electrical Activity Electroencephalographic-Based Biomarker of Functional Impairment in Traumatic Brain Injury: A Multi-Site Validation Trial

The potential clinical utility of a novel quantitative electroencephalographic (EEG)-based Brain Function Index (BFI) as a measure of the presence and severity of functional brain injury was studied as part of an independent prospective validation trial. The BFI was derived using quantitative EEG (QEEG) features associated with functional brain impairment reflecting current consensus on the physiology of concussive injury. Seven hundred and twenty adult patients (18-85 years of age) evaluated within 72 h of sustaining a closed head injury were enrolled at 11 U.S. emergency departments (EDs). Glasgow Coma Scale (GCS) score was 15 in 97%. Standard clinical evaluations were conducted and 5 to 10 min of EEG acquired from frontal locations. Clinical utility of the BFI was assessed for raw scores and percentile values. A multinomial logistic regression analysis demonstrated that the odds ratios (computed against controls) of the mild and moderate functionally impaired groups were significantly different from the odds ratio of the computed tomography (CT) postive (CT+, structural injury visible on CT) group (p = 0.0009 and p = 0.0026, respectively). However, no significant differences were observed between the odds ratios of the mild and moderately functionally impaired groups. Analysis of variance (ANOVA) demonstrated significant differences in BFI among normal (16.8%), mild TBI (mTBI)/concussed with mild or moderate functional impairment, (61.3%), and CT+ (21.9%) patients (p < 0.0001). Regression slopes of the odds ratios for likelihood of group membership suggest a relationship between the BFI and severity of impairment. Findings support the BFI as a quantitative marker of brain function impairment, which scaled with severity of functional impairment in mTBI patients. When integrated into the clinical assessment, the BFI has the potential to aid in early diagnosis and thereby potential to impact the sequelae of TBI by providing an objective marker that is available at the point of care, hand-held, non-invasive, and rapid to obtain.

Multi-site calibration and validation of a net ecosystem carbon exchange model for croplands

Croplands play an important role in the carbon budget of many regions. However, the estimation of their carbon balance remains difficult due to diversity and complexity of the processes involved. We report the coupling of a one-dimensional soil water, heat, and CO2 flux model (SOILCO2), a pool concept of soil carbon turnover (RothC), and a crop growth module (SUCROS) to predict the net ecosystem exchange (NEE) of carbon. The coupled model, further referred to as AgroC, was extended with routines for managed grassland as well as for root exudation and root decay. In a first step, the coupled model was applied to two winter wheat sites and one upland grassland site in Germany. The model was calibrated based on soil water content, soil temperature, biometric, and soil respiration measurements for each site, and validated in terms of hourly NEE measured with the eddy covariance technique. The overall model performance of AgroC was sufficient with a model efficiency above 0.78 and a correlation coefficient above 0.91 for NEE. In a second step, AgroC was optimized with eddy covariance NEE measurements to examine the effect of different objective functions, constraints, and data-transformations on estimated NEE. It was found that NEE showed a distinct sensitivity to the choice of objective function and the inclusion of soil respiration data in the optimization process. In particular, both positive and negative day- and nighttime fluxes were found to be sensitive to the selected optimization strategy. Additional consideration of soil respiration measurements improved the simulation of small positive fluxes remarkably. Even though the model performance of the selected optimization strategies did not diverge substantially, the resulting cumulative NEE over simulation time period differed substantially. Therefore, it is concluded that data-transformations, definitions of objective functions, and data sources have to be considered cautiously when a terrestrial ecosystem model is used to determine NEE by means of eddy covariance measurements.

Individual contextual factors in the validation of the Bernese pain scale for neonates: protocol for a prospective observational study

BackgroundThe Bernese Pain Scale for Neonates (BPSN) is a multidimensional pain assessment tool that is already widely used in clinical settings in the German speaking areas of Europe. Recent findings indicate that pain responses in preterm neonates are influenced by individual contextual factors, such as gestational age (GA), gender and the number of painful procedures experienced. Currently, the BPSN does not consider individual contextual factors. Therefore, the aim of this study is the validation of the BPSN using a large sample of neonates with different GAs. Furthermore, the influence of individual contextual factors on the variability in pain reactions across GA groups will be explored. The results will be used for a modification of the BPSN to account for individual contextual factors in future clinical pain assessment in neonates.Methods and designThis prospective multisite validation study with a repeated measures design will take place in three university hospital neonatal intensive care units (NICUs) in Switzerland (Bern, Basel and Zurich). To examine the impact of GA on pain responses and their variability, the infants will be stratified into six GA groups ranging from 24 0/7 to 42 0/7. Among preterm infants, 2–5 routine capillary heel sticks within the first 14 days of life, and among full-term infants, two heel sticks during the first days of life will be documented. For each heel stick, measurements will be video recorded for each of three phases: baseline, heel stick, and recovery. The infants’ pain responses will be rated according to the BPSN by five nurses who are blinded as to the number of each heel stick and as to the measurement phases. Individual contextual factors of interest will be extracted from patient charts.DiscussionUnderstanding and considering the influence of individual contextual factors on pain responses in a revised version of the BPSN will help the clinical staff to more appropriately assess pain in neonates, particularly preterm neonates hospitalized in NICUs. Pain assessment is a first step toward appropriate and efficient pain management, which itself is an important factor in later motor and cognitive development in this vulnerable patient population.Trial registrationThe study is registered in the database of Clinical Trial gov. Study ID-number: NCT 02749461. Registration date: 12 April 2016.

DSM-5 substance use disorders among adult primary care patients: Results from a multisite study

BackgroundThere are limited data about the extent of DSM-5 substance use disorders (SUDs) among primary care patients. MethodsThis study analyzed data from a multisite validation study of a substance use screening instrument conducted in a diverse sample of 2000 adults aged ≥18 years recruited from five primary care practices in four states. Prevalence and correlates of 12-month DSM-5 SUDs were examined. ResultsOverall, 75.5% of the sample used any substance, including alcohol (62.0%), tobacco (44.1%), or illicit drugs/nonmedical medications (27.9%) in the past 12 months (marijuana 20.8%, cocaine 7.3%, opioids 4.8%, sedatives 4.1%, heroin 3.9%). The prevalence of any 12-month SUD was 36.0% (mild disorder 14.2%, moderate/severe disorder 21.8%): tobacco 25.3% (mild 11.5%, moderate/severe 13.8%); alcohol 13.9% (mild 6.9%, moderate/severe 7.0%); and any illicit/nonmedical drug 14.0% (mild 4.0%, moderate/severe 10.0%). Among past 12-month users, a high proportion of tobacco or drug users met criteria for a disorder: tobacco use disorder 57.4% (26.1% mild, 31.3% moderate/severe) and any drug use disorder 50.2% (14.3% mild, 35.8% moderate/severe); a lower proportion of alcohol users (22.4%) met criteria for alcohol use disorder (11.1% mild, 11.3% moderate/severe). Over 80% of adults with opioid/heroin use disorder met criteria for a moderate/severe disorder. Younger ages, male sex, and low education were associated with increased odds of having SUD. ConclusionThese findings reveal the high prevalence of SUDs in primary care and underscore the need to identify and address them.

Multisite external validation of a risk prediction model for the diagnosis of blood stream infections in febrile pediatric oncology patients without severe neutropenia.

Pediatric oncology patients are at an increased risk of invasive bacterial infection due to immunosuppression. The risk of such infection in the absence of severe neutropenia (absolute neutrophil count ≥ 500/μL) is not well established and a validated prediction model for blood stream infection (BSI) risk offers clinical usefulness. A 6-site retrospective external validation was conducted using a previously published risk prediction model for BSI in febrile pediatric oncology patients without severe neutropenia: the Esbenshade/Vanderbilt (EsVan) model. A reduced model (EsVan2) excluding 2 less clinically reliable variables also was created using the initial EsVan model derivative cohort, and was validated using all 5 external validation cohorts. One data set was used only in sensitivity analyses due to missing some variables. From the 5 primary data sets, there were a total of 1197 febrile episodes and 76 episodes of bacteremia. The overall C statistic for predicting bacteremia was 0.695, with a calibration slope of 0.50 for the original model and a calibration slope of 1.0 when recalibration was applied to the model. The model performed better in predicting high-risk bacteremia (gram-negative or Staphylococcus aureus infection) versus BSI alone, with a C statistic of 0.801 and a calibration slope of 0.65. The EsVan2 model outperformed the EsVan model across data sets with a C statistic of 0.733 for predicting BSI and a C statistic of 0.841 for high-risk BSI. The results of this external validation demonstrated that the EsVan and EsVan2 models are able to predict BSI across multiple performance sites and, once validated and implemented prospectively, could assist in decision making in clinical practice. Cancer 2017;123:3781-3790. © 2017 American Cancer Society.

A combination of computer extracted measurements of prostate capsule shape and tumor texture on MRI to predict biochemical recurrence post treatment.

e16554 Background: Prostate cancer (PCa) biochemical recurrence (BCR) occurs in a significant proportion of men after treatment and is associated with increased mortality. Identifying PCa patients at risk of BCR following definitive therapy may help identify patients who might benefit from adjuvant therapy. Multi-parametric MRI (mpMRI) is being increasingly used in pre-treatment staging and risk stratification of PCa. In this work, we sought to explore whether a combination of computer extracted features relating to prostate capsule shape and tumor texture from pre-treatment mpMRI is predictive of BCR post treatment. Methods: This single center and retrospective study included 80 men with PCa who underwent pre-treatment 3T mpMRI and were followed for &gt; 3 years post treatment. These men were grouped into a training set D1 (N = 50, 25 each of BCR+ and BCR-) and an independent validation set D2 (N = 30, 10 BCR+ and 20 BCR-). The prostate capsule and cancer region of interest (ROI) were annotated on mpMRI by a single experienced radiologist. Shape features (curvature and orientation) were extracted from a surface of interest where statistically significant differences were observed between BCR+ and BCR- patients in the training set. Radiomic features for capturing tumor textural patterns (including 1st and 2nd order statistics, Gabor and Haralick) were extracted from within the radiologist annotated cancer ROIs. Features from D1 were used to train random forest classifiers, one each with shape (Cs) and radiomic (CR) features. A fused Bayesian classifier (CR+S) was created by integrating decisions from both Cs and CR. Results: The classifiers Cs, CR and CR+S were evaluated on independent validation set D2, resulting in area under the curve (AUCs) of 0.71, 0.81 and 0.84 respectively. Cs added value in patients who had extra prostatic spread of PCa, but suffered from mpMRI intensity artefacts that affected performance of CR. Conclusions: Integrating prostate capsule shape and tumor radiomic features from pre-treatment mpMRI enabled prediction of PCa BCR after treatment. Multi-site validation is needed to establish the robustness of the approach.

Changes in computer extracted features of vessel tortuosity on CT scans post-treatment in responders compared to non-responders for non-small cell lung cancer on immunotherapy.

11518 Background: Immune-checkpoint blockade treatments demonstrate promising clinical efficacy in patients with non-small cell lung cancer (NSCLC). Nivolumab is a PD-1 inhibitor that is FDA approved for treatment of patients with chemotherapy refractory advanced NSCLC. The current standard clinical approach to evaluating tumor response is sub-optimal in defining clinical benefit from immunotherapy drugs. We sought to evaluate whether computer extracted measurements of vessel tortuosity significantly and differentially change post treatment between NSCLC patients who do and do not respond to immunotherapy. Methods: A total of 50 NSCLC patients including pre- and post- treatment CT scans were included in this study. The patients were either responders or non-responders to Nivolumab. Patients who did not receive Nivolumab after 2 cycles due to lack of response or progression as per RECIST were classified as ‘non-responders’. A total of 35 tortuosity features of the vessels around the lung nodules were investigated. In the training cohort (N = 25), the features were ranked based on the degree of change between pre- and post- treatment CT. The top 4 features were used for training a Support Vector Machine (SVM) classifier to identify which patients did and did not respond to immunotherapy on a validation cohort of N = 25 patients. Results: The top features identified were the ones associated with the curvature of the vessel branches. The AUC for the SVM classifier was 0.75 for the training and 0.79 for the test set. Conclusions: Changes in specific vessel tortuosity features between baseline and post-treatment CT scans following nivolumab were different between NSCLC patients who did and did not respond. Multi-site validation of the vessel tortuosity features is needed to establish it as a predictive biomarker for NSCLC patients treated with immunotherapy.

Computer extracted nuclear features from tumor and benign regions of Feulgen and H&amp;E images to help predict recurrence in prostate cancer patients following radical prostatectomy.

e16556 Background: Following radical prostatectomy, around 30% of prostate cancer (PCa) patients experience biochemical recurrence (BCR). H&amp;E highlights nuclear morphology and Feulgen reflects nuclear DNA content, a feature linked to PCa presence and aggressiveness. In this work we sought to explore whether computer extracted measurements of tumor morphology and tumor adjacent benign regions on H&amp;E and Feulgen tissue images could predict BCR. Methods: We used 108 patients (59 BCR and 49 non-recurrence (NR)) and each patient had 242 QH features calculated from both the tumor and benign region of stained TMA core images. Feature selection was performed on a training set (30 BCR, 24 NR) to select the 10 most discriminating tumor and tumor adjacent benign features of each stain. A random forest classifier was trained with features so identified and validated on a test set (29 BCR, 25 NR) to predict BCR. Predictions were displayed using Kaplan-Meier analysis and area under the ROC curve (AUC). Results: The most discriminating feature from the tumor regions of the H&amp;E stain was Fourier descriptors of nuclear shape and from the Feulgen stain was texture intensity while from the benign regions it was invariant moments of nuclear shape and texture contrast energy. Combining the significant features from tumor and tumor adjacent benign regions from H&amp;E and Feulgen resulted in the highest accuracy and a statistically significant difference (p &lt; 0.05) via a log-rank test (Table 1). Gleason score did not show statistically significant differences and had the lowest AUC. Conclusions: Combining nuclear morphology and DNA related features of the tumor and tumor adjacent benign regions enabled accurate prediction of BCR. With additional multi-site validation, the combined H&amp;E + Feulgen classifier could allow better risk stratification and post-surgical patient management. [Table: see text]

Computer extracted measurements of vessel tortuosity on baseline CT scans to predict response to nivolumab immunotherapy for non-small cell lung cancer.

11566 Background: Immune-checkpoint blockade treatments, particularly drugs targeting the programmed death-1 (PD-1) receptor, demonstrate promising clinical efficacy in patients with non-small cell lung cancer (NSCLC). We sought to evaluate whether computer extracted measurements of tortuosity of vessels in lung nodules on baseline CT scans in NSCLC patients(pts) treated with a PD-1 inhibitor, nivolumab could distinguish responders and non-responders. Methods: A total of 61 NSCLC pts who underwent treatment with nivolumab were included in this study. Pts who did not receive nivolumab after 2 cycles due to lack of response or progression per RECIST were classified as ‘non-responders’, patients who had radiological response per RECIST or had clinical benefit (defined as stable disease &gt;10 cycles) were classified as ‘responders’. A total of 35 quantitative tortuosity features of the vessels associated with lung nodule were investigated. In the training cohort (N=33), the features were ranked in their ability to identify responders to nivolumab using a support vector machine (SVM) classifier. The three most informative features were then used for training the SVM, which was then validated on a cohort of N=28 pts. Results: The maximum curvature ( f1), standard deviation of the torsion ( f2) and mean curvature ( f3) were identified as the most discriminating features. The area under Receiver operating characteristic (ROC) curve (AUC) of the SVM was 0.84 for the training and 0.72 for the validation cohort. Conclusions: Vessel tortuosity features were able to distinguish responders from non-responders for patients with NSCLC treated with nivolumab. Large scale multi-site validation will need to be done to establish vessel tortuosity as a predictive biomarker for immunotherapy. [Table: see text]

Computer extracted measurements of intra-tumoral heterogeneity on H&amp;E stained tissue images to distinguish short term and long term survivors in patients with non-small cell lung carcinoma.

e20052 Background: Molecular and morphologic heterogeneity is an important characteristic of cancer. This spatial and temporal tumor heterogeneity has important implications on tumor behavior and response to therapies. This study aims to evaluate the role of computer extracted features of intra-tumoral heterogeneity (ITH) from digitized whole slide H&amp;E stained images of early stage NSCLC patients treated with surgery as a prognostic marker for survival. Methods: A cohort of 89 early stage NSCLC patients treated with surgery with long term survival data were identified. 28 patients had OS&gt; 3 years from the date of definitive surgery and were defined as long term survivors and 61 patients had OS &lt; 3 years, and were defined as short term survivors. Corresponding H&amp;E stained whole mounted lung tissue images was digitally scanned and a thoracic pathologist marked the primary tumor margins on these images. Our computational approach involved determining the variance in measurements relating to nuclear size, shape, and texture across the tissue section; Each feature was then assigned a morphologic diversity score (MDS) based off the variance; the top predictive MDS features were identified via Wilcoxon Rank Sum Test and then evaluated via a quadratic classifier using 3-fold cross validation. Kaplan-Meier (KM) survival analysis was performed for the ITH features, as well as T- and N-stage. Results: The top ranked MDS features yielded a mean area under the receiver operating characteristic curve (AUC) of 0.66 in discriminating short term from long term survivors. A p=0.00657 (see Table) was obtained for KM-analysis of the ITH features. Conclusions: Computer extracted image features of ITH enabled differentiation of NSCLC patients with short and longer term survival. Large scale multi-site validation will need to be done to establish ITH measurements as a prognostic biomarker for NSCLC patients. [Table: see text]

A comparison of single- and multi-site calibration and validation: a case study of SWAT in the Miyun Reservoir watershed, China

An essential task in evaluating global water resource and pollution problems is to obtain the optimum set of parameters in hydrological models through calibration and validation. For a large-scale watershed, single-site calibration and validation may ignore spatial heterogeneity and may not meet the needs of the entire watershed. The goal of this study is to apply a multi-site calibration and validation of the Soil andWater Assessment Tool (SWAT), using the observed flow data at three monitoring sites within the Baihe watershed of the Miyun Reservoir watershed, China. Our results indicate that the multi-site calibration parameter values are more reasonable than those obtained from single-site calibrations. These results are mainly due to significant differences in the topographic factors over the large-scale area, human activities and climate variability. The multi-site method involves the division of the large watershed into smaller watersheds, and applying the calibrated parameters of the multi-site calibration to the entire watershed. It was anticipated that this case study could provide experience of multi-site calibration in a large-scale basin, and provide a good foundation for the simulation of other pollutants in followup work in the Miyun Reservoir watershed and other similar large areas.

Erratum: Resting-state connectivity biomarkers define neurophysiological subtypes of depression.

Biomarkers have transformed modern medicine but remain largely elusive in psychiatry, partly because there is a weak correspondence between diagnostic labels and their neurobiological substrates. Like other neuropsychiatric disorders, depression is not a unitary disease, but rather a heterogeneous syndrome that encompasses varied, co-occurring symptoms and divergent responses to treatment. By using functional magnetic resonance imaging (fMRI) in a large multisite sample (n = 1,188), we show here that patients with depression can be subdivided into four neurophysiological subtypes (‘biotypes’) defined by distinct patterns of dysfunctional connectivity in limbic and frontostriatal networks. Clustering patients on this basis enabled the development of diagnostic classifiers (biomarkers) with high (82–93%) sensitivity and specificity for depression subtypes in multisite validation (n = 711) and out-of-sample replication (n = 477) data sets. These biotypes cannot be differentiated solely on the basis of clinical features, but they are associated with differing clinical-symptom profiles. They also predict responsiveness to transcranial magnetic stimulation therapy (n = 154). Our results define novel subtypes of depression that transcend current diagnostic boundaries and may be useful for identifying the individuals who are most likely to benefit from targeted neurostimulation therapies.

Resting-state connectivity biomarkers define neurophysiological subtypes of depression.

Biomarkers have transformed modern medicine but remain largely elusive in psychiatry, partly because there is a weak correspondence between diagnostic labels and their neurobiological substrates. Like other neuropsychiatric disorders, depression is not a unitary disease, but rather a heterogeneous syndrome that encompasses varied, co-occurring symptoms and divergent responses to treatment. By using functional magnetic resonance imaging (fMRI) in a large multisite sample (n = 1,188), we show here that patients with depression can be subdivided into four neurophysiological subtypes ('biotypes') defined by distinct patterns of dysfunctional connectivity in limbic and frontostriatal networks. Clustering patients on this basis enabled the development of diagnostic classifiers (biomarkers) with high (82-93%) sensitivity and specificity for depression subtypes in multisite validation (n = 711) and out-of-sample replication (n = 477) data sets. These biotypes cannot be differentiated solely on the basis of clinical features, but they are associated with differing clinical-symptom profiles. They also predict responsiveness to transcranial magnetic stimulation therapy (n = 154). Our results define novel subtypes of depression that transcend current diagnostic boundaries and may be useful for identifying the individuals who are most likely to benefit from targeted neurostimulation therapies.

Challenges in modelling of water quantity and quality in two contrasting meso-scale catchments in Poland

AbstractThis study presents an application of the SWAT model (Soil and Water Assessment Tool) in two meso-scale catchments in Poland (Upper Narew and Barycz), contrasting in terms of human pressures on water quantity and quality. The main objective was multi-variable and multi-site calibration and validation of the model against daily discharge, sediment and nutrient loads as well as discussion of challenges encountered in calibration phase. Multi-site calibration and validation gave varied results ranging from very good (daily discharge) to acceptable (sediment, nitrogen and phosphorus loads in most of gauges) and rather poor (individual gauges for all variables) in both catchments. The calibrated models enabled spatial quantification of water yield, sediment and nutrient loads, indicating areas of special concern in terms of pollution, as well as estimation of contribution of pollution from different sources, indicating agriculture as the most important source in both catchments. During the calibration process a number of significant issues were encountered: (i) global vs. local parametrization, (ii) simulation of different pools of water quality parameters in reservoirs and streams and (iii) underestimation of NO3-N loads in winter due to farmers practices. Discussion of these issues is hoped to aid SWAT model users in Poland in a deeper understanding of mechanisms of multi-variable and multi-site calibration.

Comprehensive Translational Assessment of Human-Induced Pluripotent Stem Cell Derived Cardiomyocytes for Evaluating Drug-Induced Arrhythmias.

Induced pluripotent stem cell-derived cardiomyocytes (iPSC-CM) hold promise for assessment of drug-induced arrhythmias and are being considered for use under the comprehensive in vitro proarrhythmia assay (CiPA). We studied the effects of 26 drugs and 3 drug combinations on 2 commercially available iPSC-CM types using high-throughput voltage-sensitive dye and microelectrode-array assays being studied for the CiPA initiative and compared the results with clinical QT prolongation and torsade de pointes (TdP) risk. Concentration-dependent analysis comparing iPSC-CMs to clinical trial results demonstrated good correlation between drug-induced rate-corrected action potential duration and field potential duration (APDc and FPDc) prolongation and clinical trial QTc prolongation. Of 20 drugs studied that exhibit clinical QTc prolongation, 17 caused APDc prolongation (16 in Cor.4U and 13 in iCell cardiomyocytes) and 16 caused FPDc prolongation (16 in Cor.4U and 10 in iCell cardiomyocytes). Of 14 drugs that cause TdP, arrhythmias occurred with 10 drugs. Lack of arrhythmic beating in iPSC-CMs for the four remaining drugs could be due to differences in relative levels of expression of individual ion channels. iPSC-CMs responded consistently to human ether-a-go-go potassium channel blocking drugs (APD prolongation and arrhythmias) and calcium channel blocking drugs (APD shortening and prevention of arrhythmias), with a more variable response to late sodium current blocking drugs. Current results confirm the potential of iPSC-CMs for proarrhythmia prediction under CiPA, where iPSC-CM results would serve as a check to ion channel and in silico modeling prediction of proarrhythmic risk. A multi-site validation study is warranted.

Multi-Site Validation of the SWAT Model on the Bani Catchment: Model Performance and Predictive Uncertainty

The objective of this study was to assess the performance and predictive uncertainty of the Soil and Water Assessment Tool (SWAT) model on the Bani River Basin, at catchment and subcatchment levels. The SWAT model was calibrated using the Generalized Likelihood Uncertainty Estimation (GLUE) approach. Potential Evapotranspiration (PET) and biomass were considered in the verification of model outputs accuracy. Global Sensitivity Analysis (GSA) was used for identifying important model parameters. Results indicated a good performance of the global model at daily as well as monthly time steps with adequate predictive uncertainty. PET was found to be overestimated but biomass was better predicted in agricultural land and forest. Surface runoff represents the dominant process on streamflow generation in that region. Individual calibration at subcatchment scale yielded better performance than when the global parameter sets were applied. These results are very useful and provide a support to further studies on regionalization to make prediction in ungauged basins.

A multisite validation of whole slide imaging for primary diagnosis using standardized data collection and analysis

Context: Text-based reporting and manual arbitration for whole slide imaging (WSI) validation studies are labor intensive and do not allow for consistent, scalable, and repeatable data collection or analysis. Objective: The objective of this study was to establish a method of data capture and analysis using standardized codified checklists and predetermined synoptic discordance tables and to use these methods in a pilot multisite validation study. Methods and Study Design: Fifteen case report form checklists were generated from the College of American Pathology cancer protocols. Prior to data collection, all hypothetical pairwise comparisons were generated, and a level of harm was determined for each possible discordance. Four sites with four pathologists each generated 264 independent reads of 33 cases. Preestablished discordance tables were applied to determine site by site and pooled accuracy, intrareader/intramodality, and interreader intramodality error rates. Results: Over 10,000 hypothetical pair-wise comparisons were evaluated and assigned harm in discordance tables. The average difference in error rates between WSI and glass, as compared to ground truth, was 0.75% with a lower bound of 3.23% (95% confidence interval). Major discordances occurred on challenging cases, regardless of modality. The average inter-reader agreement across sites for glass was 76.5% (weighted kappa of 0.68) and for digital it was 79.1% (weighted kappa of 0.72). Conclusion: These results demonstrate the feasibility and utility of employing standardized synoptic checklists and predetermined discordance tables to gather consistent, comprehensive diagnostic data for WSI validation studies. This method of data capture and analysis can be applied in large-scale multisite WSI validations.

Applying SWAT to predict ortho-phosphate loads and trophic status in four reservoirs in the upper Olifants catchment, South Africa

Abstract. Excessive phosphate loading in the upper Olifants River, South Africa has resulted in widespread eutrophication and associated ecosystem health impacts. The majority of sewage treatment works (STWs) are operating poorly and are likely an important source of bioavailable ortho-phosphate (OP) in the catchment. The Soil Water Assessment Tool (SWAT) was used to identify important sources of OP loading in the catchment and to predict changes in the trophic status of four reservoirs associated with three STW effluent OP standards (the current average of 4 mg L−1 and standards of 1 and 0.1 mg L−1). Nash–Sutcliffe and PBIAS statistics showed good model performance for simulations of flow and OP loading using a multi-site calibration and validation approach. Key sub-catchments responsible for high OP loading were identified. Three of the four reservoirs are heavily impacted by OP loading originating from STWs. Two reservoirs show mean annual OP concentrations indicative of eutrophic conditions and a 1 mg L−1 effluent standard would reduce concentrations to well within the mesotrophic range. There was little difference in OP loads and concentrations associated with the 1 and 0.1 mg L−1 effluent standards, indicating that attention to nonpoint sources would be required to realise any benefit associated with the strictest effluent standard. Regression analyses and associated 95% prediction limits between simulated OP loads and concentrations for all effluent treatment scenarios from 2002 to 2010 allowed for the quantification of OP loading that would ensure a specific trophic status in each reservoir. This study is one of the first to apply SWAT in simulating OP loading and concentrations in large reservoirs, and its application in South Africa provides further support for its utility throughout a wide geographical area.