Quantitative Comparison Of Models Research Articles

Optimization and uncertainty quantification of in situ combustion chemical reaction models

Heavy oil, deep, and waterflooded reservoirs contain significant petroleum resources but present many unique challenges for oil recovery. In situ combustion (ISC) is a thermal enhanced oil recovery method that has been proposed for such reservoirs. A persistent challenge in understanding ISC is modeling the underlying chemical reactions. While many different reaction models have been proposed, these models are typically developed on an ad-hoc basis for individual crude oil samples and manually calibrated to experimental data. While some work exists on applying optimization-based approaches to reaction calibration, existing approaches are not fully-generalized nor do they quantify uncertainty in the estimated parameters. Here we present such an approach for calibrating and quantifying uncertainty in ISC chemical reaction parameters. The workflow presented is generalized to any hydrocarbon pseudocomponent-based reaction model of ISC, provides a framework for quantification of uncertainty in estimated parameters, and is fully automated. Thus, no parameter initialization or manual calibration is needed. We apply this workflow to characterization of a heavy crude oil sample and demonstrate the capability of this workflow to predict ISC oxidation kinetics and upscale to a combustion tube simulation. Overall, this work provides an algorithmic approach to calibrating ISC kinetics that enables quantitative analysis and comparison of oil types, reaction models, and reservoir conditions for ISC.

Development of an open framework for a qualitative and quantitative comparison of power system and electricity grid models for Europe

The ongoing needs to develop power systems towards more environmentally friendly technologies with respect to climate change in conjunction with the continuous evolution of the respective market conditions is leading to a transition away from the traditional system operation. The upcoming challenges have motivated the development of an increasing number of models for transmission grids. Nevertheless, the high complexity of such models renders it exceedingly difficult to compare their results as well as any corresponding conclusions.In this paper, we develop an open framework to compare a variety of pan-European transmission grid models with a strong focus on the German power system. The comparison is performed in both a qualitative and quantitative manner, depending on the investigated modeling aspect including input data, methods, system boundaries and results. The quantitative model comparison is done by performing harmonized model experiments, one for 2016 as back testing and one for 2030 for analyzing the future system.Core elements of our comparison framework are:We proved that our comparison framework is suitable to make similarities and differences between the different model results visible, e.g. using quadratic heat maps. To ensure transparency and to support the open modeling community, the fact sheets with the model specifications and the database with selected model results are uploaded on the open energy platform.

Natural Language Processing and Computational Linguistics

Natural Language Processing and Computational Linguistics

A proposed framework for the development and qualitative evaluation of West Nile virus models and their application to local public health decision-making.

West Nile virus (WNV) is a globally distributed mosquito-borne virus of great public health concern. The number of WNV human cases and mosquito infection patterns vary in space and time. Many statistical models have been developed to understand and predict WNV geographic and temporal dynamics. However, these modeling efforts have been disjointed with little model comparison and inconsistent validation. In this paper, we describe a framework to unify and standardize WNV modeling efforts nationwide. WNV risk, detection, or warning models for this review were solicited from active research groups working in different regions of the United States. A total of 13 models were selected and described. The spatial and temporal scales of each model were compared to guide the timing and the locations for mosquito and virus surveillance, to support mosquito vector control decisions, and to assist in conducting public health outreach campaigns at multiple scales of decision-making. Our overarching goal is to bridge the existing gap between model development, which is usually conducted as an academic exercise, and practical model applications, which occur at state, tribal, local, or territorial public health and mosquito control agency levels. The proposed model assessment and comparison framework helps clarify the value of individual models for decision-making and identifies the appropriate temporal and spatial scope of each model. This qualitative evaluation clearly identifies gaps in linking models to applied decisions and sets the stage for a quantitative comparison of models. Specifically, whereas many coarse-grained models (county resolution or greater) have been developed, the greatest need is for fine-grained, short-term planning models (m–km, days–weeks) that remain scarce. We further recommend quantifying the value of information for each decision to identify decisions that would benefit most from model input.

Machine Translation Vs. Multilingual Dictionaries Assessing Two Strategies for the Topic Modeling of Multilingual Text Collections

ABSTRACT The goal of this paper is to evaluate two methods for the topic modeling of multilingual document collections: (1) machine translation (MT), and (2) the coding of semantic concepts using a multilingual dictionary (MD) prior to topic modeling. We empirically assess the consequences of these approaches based on both a quantitative comparison of models and a qualitative validation of each method’s potentials and weaknesses. Our case study uses two text collections (of tweets and news articles) in three languages (English, Hebrew, Arabic), covering the ongoing local conflicts between Israeli authorities, settlers, and Palestinian Bedouins in the West Bank. We find that both methods produce a large share of equivalent topics, especially in the context of fairly homogenous news discourse, yet show limited but systematic differences when applied to highly heterogenous social media discourse. While the MD model delivers a more nuanced picture of conflict-related topics, it misses several more peripheral topics, especially those unrelated to the dictionary’s focus, which are picked up by the MT model. Our study is a first step toward instrument validation, indicating that both methods yield valid, comparable results, while method-specific differences remain.

Comparing global seismic tomography models using varimax principal component analysis

Abstract. Global seismic tomography has greatly progressed in the past decades, with many global Earth models being produced by different research groups. Objective, statistical methods are crucial for the quantitative interpretation of the large amount of information encapsulated by the models and for unbiased model comparisons. Here we propose using a rotated version of principal component analysis (PCA) to compress the information in order to ease the geological interpretation and model comparison. The method generates between 7 and 15 principal components (PCs) for each of the seven tested global tomography models, capturing more than 97 % of the total variance of the model. Each PC consists of a vertical profile, with which a horizontal pattern is associated by projection. The depth profiles and the horizontal patterns enable examining the key characteristics of the main components of the models. Most of the information in the models is associated with a few features: large low-shear-velocity provinces (LLSVPs) in the lowermost mantle, subduction signals and low-velocity anomalies likely associated with mantle plumes in the upper and lower mantle, and ridges and cratons in the uppermost mantle. Importantly, all models highlight several independent components in the lower mantle that make between 36 % and 69 % of the total variance, depending on the model, which suggests that the lower mantle is more complex than traditionally assumed. Overall, we find that varimax PCA is a useful additional tool for the quantitative comparison and interpretation of tomography models.

Toward a formal theory of proactivity

Beyond merely reacting to their environment and impulses, people have the remarkable capacity to proactively set and pursue their own goals. The extent to which they leverage this capacity varies widely across people and situations. The goal of this article is to propose and evaluate a model of proactivity and reactivity. We proceed in three steps. First, we model proactivity in a widely used cognitive control task known as the AX Continuous Performance Task (AX-CPT). Our theory formalizes an important aspect of proactivity as meta-control over proactive and reactive control. Second, we perform a quantitative model comparison to identify the number and nature of meta-control decisions that are involved in the regulation of proactive behavior. Our findings suggest that individual differences in proactivity are governed by two independent meta-control decisions, namely deciding whether to set an intention for what to do in a future situation and deciding whether to recall one’s intentions when the situation occurs. Third, we test the assumptions and qualitative predictions of the winning model against data from numerous experiments varying the incentives, cognitive load, and statistical structure of the task. Our results suggest that proactivity can be understood in terms of computational models of meta-control. Future work will extend our models from proactive control in the AX-CPT to proactive goal creation and goal pursuit in the real world.

Measurements of the time-dependent cosmic-ray Sun shadow with seven years of IceCube data: Comparison with the Solar cycle and magnetic field models

Observations of the time-dependent cosmic-ray Sun shadow have been proven as a valuable diagnostic for the assessment of solar magnetic field models. In this paper, seven years of IceCube data are compared to solar activity and solar magnetic field models. A quantitative comparison of solar magnetic field models with IceCube data on the event rate level is performed for the first time. Additionally, a first energy-dependent analysis is presented and compared to recent predictions. We use seven years of IceCube data for the moon and the Sun and compare them to simulations on data rate level. The simulations are performed for the geometrical shadow hypothesis for the moon and the Sun and for a cosmic-ray propagation model governed by the solar magnetic field for the case of the Sun. We find that a linearly decreasing relationship between Sun shadow strength and solar activity is preferred over a constant relationship at the 6.4σ level. We test two commonly used models of the coronal magnetic field, both combined with a Parker spiral, by modeling cosmic-ray propagation in the solar magnetic field. Both models predict a weakening of the shadow in times of high solar activity as it is also visible in the data. We find tensions with the data on the order of 3σ for both models, assuming only statistical uncertainties. The magnetic field model CSSS fits the data slightly better than the PFSS model. This is generally consistent with what is found previously by the Tibet AS−γ Experiment; a deviation of the data from the two models is, however, not significant at this point. Regarding the energy dependence of the Sun shadow, we find indications that the shadowing effect increases with energy during times of high solar activity, in agreement with theoretical predictions.7 MoreReceived 26 June 2020Accepted 11 December 2020DOI:https://doi.org/10.1103/PhysRevD.103.042005© 2021 American Physical SocietyPhysics Subject Headings (PhySH)Research AreasCosmic ray propagationMagnetic field generation & plasma dynamoSpace & astrophysical plasmaWave-wave, wave-particle interactionsPropertiesInterplanetary magnetic fieldTechniquesCherenkov detectorsCosmic ray & astroparticle detectorsCosmic rays & astroparticlesNeutrino detectorsTest-particle methodsGravitation, Cosmology & AstrophysicsPlasma PhysicsParticles & Fields

Interrelation between polycrystalline structure and time-dependent magnetic anisotropies in exchange-biased bilayers

For polycrystalline exchange-biased thin films the macroscopic magnetic properties evolve from a complex interplay of different individual magnetic anisotropies which are directly connected to the grain volume distribution and to the crystal and interface structure of the layer system. A quantitative comparison of models describing a macroscopically observed exchange bias by the contributing anisotropies has in most cases been hampered by neglecting their time dependence with respect to data acquisition and storage times. Using a recently developed model we are able to connect time-dependent parts of the prevailing anisotropies with parameters describing the polycrystalline structure of the layer system. The model will be compared to experiments on a prototypical IrMn/CoFe bilayer, where structural and magnetic parameters have been systematically altered by varying the deposition rate of the antiferromagnetic layer and the field-cooling temperature. The combination of angular-resolved measurements obtained by vectorial magneto-optic Kerr magnetometry and a systematic analysis of the polycrystalline structure enables the disentanglement of the different anisotropy contributions to the macroscopic exchange bias and coercive fields and serves as a verification of the utilized model.

An extended radar relative calibration adjustment (eRCA) technique for higher-frequency radars and range–height indicator (RHI) scans

Abstract. This study extends the relative calibration adjustment technique for calibration of weather radars to higher-frequency radars as well as range–height indicator (RHI) scans. The calibration of weather radars represents one of the most dominant sources of error for their use in a variety of fields including quantitative precipitation estimation and model comparisons. While most weather radars are routinely calibrated, the frequency of calibration is often less than required, resulting in miscalibrated time periods. While full absolute calibration techniques often require the radar to be taken offline for a period of time, there have been online calibration techniques discussed in the literature. The relative calibration adjustment (RCA) technique uses the statistics of the ground clutter surrounding the radar as a monitoring source for the stability of calibration but has only been demonstrated to work at S- and C-band for plan-position indicator (PPI) scans at a constant elevation. In this work the RCA technique is modified to work with higher-frequency radars, including Ka-band cloud radars. At higher frequencies the properties of clutter can be much more variable. This work introduces an extended clutter selection procedure that incorporates the temporal stability of clutter and helps to improve the operational stability of RCA for relatively higher-frequency radars. The technique is also extended to utilize range–height scans from radars where the elevation is varied rather than the azimuth. These types of scans are often utilized with research radars to examine the vertical structure of clouds. The newly extended technique (eRCA) is applied to four Department of Energy Atmospheric Radiation Measurement (DOE ARM) weather radars ranging in frequency from C- to Ka-band. Cross comparisons of three co-located radars with frequencies C, X, and Ka at the ARM Cloud, Aerosol, and Complex Terrain Interactions (CACTI) site show that the technique can determine changes in calibration. Using an X-band radar at the ARM Eastern North Atlantic (ENA) site, we show how the technique can be modified to be more resilient to clutter fields that show increased variability, in this case due to sea clutter. The results show that this technique is promising for a posteriori data calibration and monitoring.

Onset of Sliding of Elastomer Multicontacts: Failure of a Model of Independent Asperities to Match Experiments

Modelling of rough frictional interfaces is often based on asperity models, in which the individual behaviour of individual microjunctions is assumed. In the absence of local measurements at the microjunction scale, quantitative comparison of such models with experiments is usually based only on macroscopic quantities, like the total tangential load resisted by the interface. Recently however, a new experimental dataset was presented on the onset of sliding of rough elastomeric interfaces, which includes local measurements of the contact area of the individual microjunctions. Here, we use this more comprehensive dataset to test the possibility of quantitatively matching the measurements with a model of independent asperities, enriched with experimental information about the area of microjunctions and its evolution under shear. We show that, despite using parameter values and behaviour laws constrained and inspired by experiments, our model does not quantitatively match the macroscopic measurements. We discuss the possible origins of this failure .

A Quantitative Comparison of Multiple Population Mortality Model on Some East Asian Countries and Regions

This paper reviews the progress of the multiple population mortality model and the defects in parameter estimation and proposes an effective method to improve the performance of the mortality model. We set up a multiple population group, using the data of mainland China, Hong Kong (China), and Japan, to test fitting performance and forecasting performance. Using the TSWLS and TSSVD methods in a multiple population stochastic mortality model has advantages in fitting performance and robustness. In addition, the forecasting value of mortality ratio between any two populations can converge to a fixed constant in a certain time period which obeys the regular of human biological characteristics.

Quantitative comparison of cascading failure models for risk-based decision making in power systems

The accurate allocation and prediction of risk in power systems is vital for reliable operations of the electrical infrastructure. Several models with varying degrees of accuracy and computational cost are available. Relying on less computationally intensive methods increases the efficiency of risk assessment provided that the output does not impair control actions and decision making. This study focuses on comparing two established cascading failure models for determining their consistency to risk-based decision making. Effects of ambient conditions are captured via temperature-dependent dynamic transmission line ratings. The investigations on the IEEE 24-Bus reliability test system highlight that, when the power grid is subjected to elevated temperature and demand levels, the deviations between Manchester and OPA model can be significant. However, both models show the same general trends, namely, that the demand not served generally increases with increasing temperature and demand. Further similarities are found in terms of the most critical lines and the most heavily loaded generators, providing useful information for power system expansion planning. The OPA model displays a much larger area of elevated risk across the input space that also includes almost the entire area found by the Manchester model, providing conservative estimates in highly stressed power systems.

Turbulent shear-layer mixing: initial conditions, and direct-numerical and large-eddy simulations

Aspects of turbulent shear-layer mixing are investigated over a range of shear-layer Reynolds numbers, $Re_{\unicode[STIX]{x1D6FF}}=\unicode[STIX]{x0394}U\unicode[STIX]{x1D6FF}/\unicode[STIX]{x1D708}$, based on the shear-layer free-stream velocity difference, $\unicode[STIX]{x0394}U$, and mixing-zone thickness, $\unicode[STIX]{x1D6FF}$, to probe the role of initial conditions in mixing stages and the evolution of the scalar-field probability density function (p.d.f.) and variance. Scalar transport is calculated for unity Schmidt numbers, approximating gas-phase diffusion. The study is based on direct-numerical simulation (DNS) and large-eddy simulation (LES), comparing different subgrid-scale (SGS) models for incompressible, uniform-density, temporally evolving forced shear-layer flows. Moderate-Reynolds-number DNS results help assess and validate LES SGS models in terms of scalar-spectrum and mixing estimates, as well as other metrics, to $Re_{\unicode[STIX]{x1D6FF}}\lesssim 3.3\times 10^{4}$. High-Reynolds-number LES investigations to $Re_{\unicode[STIX]{x1D6FF}}\lesssim 5\times 10^{5}$ help identify flow parameters and conditions that influence the evolution of scalar variance and p.d.f., e.g. marching versus non-marching. Initial conditions that generate shear flows with different mixing behaviour elucidate flow characteristics in each flow regime and identify elements that induce p.d.f. transition and scalar-variance behaviour. P.d.f. transition is found to be largely insensitive to local flow parameters, such as $Re_{\unicode[STIX]{x1D6FF}}$, or a previously proposed vortex-pairing parameter based on downstream distance, or other equivalent criteria. The present study also allows a quantitative comparison of LES SGS models in moderate- and high-$Re_{\unicode[STIX]{x1D6FF}}$ forced shear-layer flows.

A quantitative comparison of phase-averaged models for bubbly, cavitating flows

We compare the computational performance of two modeling approaches for the flow of dilute cavitation bubbles in a liquid. The first approach is a deterministic model, for which bubbles are represented in a Lagrangian framework as advected features, each sampled from a distribution of equilibrium bubble sizes. The dynamic coupling to the liquid phase is modeled through local volume averaging. The second approach is stochastic; ensemble-phase averaging is used to derive mixture-averaged equations and field equations for the associated bubble properties are evolved in an Eulerian reference frame. For polydisperse mixtures, the probability density function of the equilibrium bubble radii is discretized and bubble properties are solved for each representative bin. In both cases, the equations are closed by solving Rayleigh–Plesset-like equations for the bubble dynamics as forced by the local or mixture-averaged pressure, respectively. An acoustically excited dilute bubble screen is used as a case study for comparisons. We show that observables of ensemble- and volume-averaged simulations match closely and that their convergence is first order under grid refinement. Guidelines are established for phase-averaged simulations by comparing the computational costs of methods. The primary costs are shown to be associated with stochastic closure; polydisperse ensemble-averaging requires many samples of the underlying PDF and volume-averaging requires repeated, randomized simulations to accurately represent a homogeneous bubble population. The relative sensitivities of these costs to spatial resolution and bubble void fraction are presented.

Morpho-MNIST: Quantitative Assessment and Diagnostics for Representation Learning

Revealing latent structure in data is an active field of research, having introduced exciting technologies such as variational autoencoders and adversarial networks, and is essential push machine learning towards unsupervised knowledge discovery. However, a major challenge is the lack of suitable benchmarks for an objective and quantitative evaluation of learned representations. To address this issue we introduce Morpho-MNIST, a framework that aims answer: to what extent has my model learned represent specific factors of variation in the data? We extend the popular MNIST dataset by adding a morphometric analysis enabling quantitative comparison of trained models, identification of the roles of latent variables, and characterisation of sample diversity. We further propose a set of quantifiable perturbations assess the performance of unsupervised and supervised methods on challenging tasks such as outlier detection and domain adaptation. Data and code are available at this https URL.

Comparison of Quantitative Exposure Models for Occupational Exposure to Organic Solvents in Korea.

This study aims to evaluate the accuracy, precision and conservatism of three models (ECETOC TRA 3.1, Stoffenmanager 7.0 and ART 1.5) by comparing model predictions and repeated exposure measurements in Korea. We collected the exposure measurements (n = 390) and detailed contextual information extracted from 10 survey reports published by Ministry of Employment and Labour in the mid-2000s. Using these three models, seven occupational health professionals predicted inhalation exposure to 10 solvents used for cleaning tasks in 51 situations at 33 companies in 15 industries. We applied four comparison approaches previously used by several European studies and calculated the lack of agreement (bias, relative bias, precision), Pearson correlation coefficients (r), level of conservatism, and residuals between the log-transformed predicted estimates and measured exposures on both individual and situation levels. The overall bias and precision were -0.53 ± 2.11 with ART, -1.32 ± 5.88 with Stoffenmanager, and -1.03 ± 8.88 with ECETOC TRA. Pearson correlation coefficients were significantly high in ART (r = 0.95) and Stoffenmanager (r = 0.82), but moderate in ECETOC TRA (r = 0.58). We found ART to be the most accurate model, and Stoffenmanager was the most balanced model in terms of good accuracy, high correlation, and medium conservatism in the model predictions. However, ECETOC TRA showed less accurate outcomes and lower level of conservatism but still had moderate correlations. We observed a systematic tendency to overestimate low exposures and underestimate higher exposures in all models, similar to previous studies. Therefore, our findings suggest that these European models can be used to predict occupational exposure to solvents in Korea.

Limitations of Proposed Signatures of Bayesian Confidence.

The Bayesian model of confidence posits that confidence reflects the observer's posterior probability that the decision is correct. Hangya, Sanders, and Kepecs (2016) have proposed that researchers can test the Bayesian model by deriving qualitative signatures of Bayesian confidence (i.e., patterns that one would expect to see if an observer were Bayesian) and looking for those signatures in human or animal data. We examine two proposed signatures, showing that their derivations contain hidden assumptions that limit their applicability and that they are neither necessary nor sufficient conditions for Bayesian confidence. One signature is an average confidence of 0.75 on trials with neutral evidence. This signature holds only when class-conditioned stimulus distributions do not overlap and when internal noise is very low. Another signature is that as stimulus magnitude increases, confidence increases on correct trials but decreases on incorrect trials. This divergence signature holds only when stimulus distributions do not overlap or when noise is high. Navajas et al. (2017) have proposed an alternative form of this signature; we find no indication that this alternative form is expected under Bayesian confidence. Our observations give us pause about the usefulness of the qualitative signatures of Bayesian confidence. To determine the nature of the computations underlying confidence reports, there may be no shortcut to quantitative model comparison.

Comparison of Quantitative and Qualitative (Q)SAR Models Created for the Prediction of Ki and IC50 Values of Antitarget Inhibitors.

Estimation of interaction of drug-like compounds with antitargets is important for the assessment of possible toxic effects during drug development. Publicly available online databases provide data on the experimental results of chemical interactions with antitargets, which can be used for the creation of (Q)SAR models. The structures and experimental Ki and IC50 values for compounds tested on the inhibition of 30 antitargets from the ChEMBL 20 database were used. Data sets with Ki and IC50 values including more than 100 compounds were created for each antitarget. The (Q)SAR models were created by GUSAR software using quantitative neighborhoods of atoms (QNA), multilevel neighborhoods of atoms (MNA) descriptors, and self-consistent regression. The accuracy of (Q)SAR models was validated by the fivefold cross-validation procedure. The balanced accuracy was higher for qualitative SAR models (0.80 and 0.81 for Ki and IC50 values, respectively) than for quantitative QSAR models (0.73 and 0.76 for Ki and IC50 values, respectively). In most cases, sensitivity was higher for SAR models than for QSAR models, but specificity was higher for QSAR models. The mean R2 and RMSE were 0.64 and 0.77 for Ki values and 0.59 and 0.73 for IC50 values, respectively. The number of compounds falling within the applicability domain was higher for SAR models than for the test sets.

Detection and Geometrical Characterization of a Buried Landfill Site by Integrating Land Use Historical Analysis, Digital Photogrammetry and Airborne Lidar Data

Abandoned quarries are frequently used as sites of illegal dumping of solid urban waste. These sites often occur nearby or within urban areas so that their detection may turn out to be quite difficult from the surface. This study focuses on the detection and geometrical characterization of a hidden landfill site located along the coastline of the Campi Flegrei, near Naples, Italy. Our approach is based on the analysis of historical topographic maps and aerial photographs, coupled with quantitative comparison of multitemporal digital elevation models obtained by digital photogrammetry and lidar techniques. The comparative analysis of topographic maps and aerial photos clearly shows modifications of the landscape associated with the urban development and quarrying activity, as well as the later filling of the quarry. The change detection analysis reveals that remarkable elevation changes occurred in the study area between 1956 and 2008. The average thickness of the landfill deposits is ca. 8 m, whereas the average volume is ca. 100,000 m3. The results of this work confirm the suitability of the used methodological approach that combines both qualitative and quantitative techniques for the detection of buried landfill sites. The geometric characterization of a landfill represents a fitting starting point for the further planning of geophysical site surveys and direct investigations aimed at the assessment of environmental hazards.