Framework For Model Comparison Research Articles

Modeling regional occupancy of fishes using acoustic telemetry: a model comparison framework applied to lake trout

Acoustic telemetry is a common tool used in fisheries management to estimate fish space use (i.e., occupancy) from a local habitat scale to entire systems. Numerous analytical models have been developed to estimate different aspects of fish movement from telemetry datasets, yet evaluations of model performance and comparisons among models are limited. Here, we develop a framework to evaluate model estimates of regional occupancy in large and fragmented systems using an acoustic receiver array in Lake Champlain. We simulated the tracks of 100 acoustically tagged fish using a random walk function and created detection events based on receiver positions and distance-based detection probability. Regional occupancy for the simulated data was estimated by six movement models that ranged in analytical complexity, and results were compared to the true distributions for each simulated track to evaluate model error. The six movement models included: (1) a basic residency index using detections alone; (2) a residency index using last-observation-carried-forward; (3) a centers of activity model; (4) linear and non-linear interpolations (i.e., least-cost paths); and (5 and 6) two dynamic Brownian bridge movement models generated using separate packages in R. We developed a model selection process to compare model performance and select the optimal analysis based on simulation error. This process showed significant differences in model performance among the six movement models based on model error. Overall, the model generating least-cost paths using linear and non-linear interpolations consistently provided the most accurate regional occupancy estimates. Based on these simulation results, we applied this model to a case study that evaluated patterns in the regional distribution of stocked lake trout (Salvelinus namaycush) in Lake Champlain, which demonstrated distinct regional occupancy of two stocked lake trout groups. These results demonstrate potential for large variability in interpretation of acoustic telemetry data for describing regional fish distribution dependent on the analytical method used.

Could this be next for corpus linguistics? Methods of semi-automatic data annotation with contextualized word embeddings

Abstract This paper explores how linguistic data annotation can be made (semi-)automatic by means of machine learning. More specifically, we focus on the use of “contextualized word embeddings” (i.e. vectorized representations of the meaning of word tokens based on the sentential context in which they appear) extracted by large language models (LLMs). In three example case studies, we assess how the contextualized embeddings generated by LLMs can be combined with different machine learning approaches to serve as a flexible, adaptable semi-automated data annotation tool for corpus linguists. Subsequently, to evaluate which approach is most reliable across the different case studies, we use a Bayesian framework for model comparison, which estimates the probability that the performance of a given classification approach is stronger than that of an alternative approach. Our results indicate that combining contextualized word embeddings with metric fine-tuning yield highly accurate automatic annotations.

Generalized Bayes approach to inverse problems with model misspecification

We propose a general framework for obtaining probabilistic solutions to PDE-based inverse problems. Bayesian methods are attractive for uncertainty quantification but assume knowledge of the likelihood model or data generation process. This assumption is difficult to justify in many inverse problems, where the specification of the data generation process is not obvious. We adopt a Gibbs posterior framework that directly posits a regularized variational problem on the space of probability distributions of the parameter. We propose a novel model comparison framework that evaluates the optimality of a given loss based on its ‘predictive performance’. We provide cross-validation procedures to calibrate the regularization parameter of the variational objective and compare multiple loss functions. Some novel theoretical properties of Gibbs posteriors are also presented. We illustrate the utility of our framework via a simulated example, motivated by dispersion-based wave models used to characterize arterial vessels in ultrasound vibrometry.

Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework.

With the fast-growing number of classification models being produced every day, numerous model interpretation and comparison solutions have also been introduced. For example, LIME and SHAP can interpret what input features contribute more to a classifier's output predictions. Different numerical metrics (e.g., accuracy) can be used to easily compare two classifiers. However, few works can interpret the contribution of a data feature to a classifier in comparison with its contribution to another classifier. This comparative interpretation can help to disclose the fundamental difference between two classifiers, select classifiers in different feature conditions, and better ensemble two classifiers. To accomplish it, we propose a learning-from-disagreement (LFD) framework to visually compare two classification models. Specifically, LFD identifies data instances with disagreed predictions from two compared classifiers and trains a discriminator to learn from the disagreed instances. As the two classifiers' training features may not be available, we train the discriminator through a set of meta-features proposed based on certain hypotheses of the classifiers to probe their behaviors. Interpreting the trained discriminator with the SHAP values of different meta-features, we provide actionable insights into the compared classifiers. Also, we introduce multiple metrics to profile the importance of meta-features from different perspectives. With these metrics, one can easily identify meta-features with the most complementary behaviors in two classifiers, and use them to better ensemble the classifiers. We focus on binary classification models in the financial services and advertising industry to demonstrate the efficacy of our proposed framework and visualizations.

Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds

Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7 T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl’s gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl’s gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.

Different Traits, Different Evolutionary Pathways: Insights from Salamandrina (Amphibia, Caudata).

Species delimitation is often based on a single or very few genetic or phenetic traits, something which leads to misinterpretations and often does not provide information about evolutionary processes. Here, we investigated the diversity pattern of multiple phenetic traits of the two extant species of Salamandrina, a genus split only after molecular traits had been studied but the two species of which are phenetically very similar. The phenetic traits we studied are size, external body shape and head colour pattern, in a model comparison framework using non-linear mixed models and unsupervised and supervised clustering. Overall, we found high levels of intra-specific variability for body size and shape, depending on population belonging and habitat, while differences between species were generally lower. The habitat the salamanders dwell in also seems important for colour pattern. Basing on our findings, from the methodological point of view, we suggest (i) to take into account the variability at population level when testing for higher level variability, and (ii) a semi-supervised learning approach to high dimensional data. We also showed that different phenotypic traits of the same organism could result from different evolutionary routes. Local adaptation is likely responsible for body size and shape variability, with selective pressures more similar across species than within them. Head colour pattern also depends on habitat, differently from ventral colour pattern (not studied in this paper) which likely evolved under genetic drift.

Data driven analysis of the desired speed in ordinary differential equation based pedestrian simulation models

In this paper, the desired speed is figured out as an important parameter for ordinary differential equation (ODE) based pedestrian simulation models. However, there lacks a thorough study on the optimal way of assigning the desired speed for ODE based pedestrian simulation models to achieve reliable performance. To gain a deep understanding of the role of the desired speed and its influence on the performances of the ODE based pedestrian simulation models, a total of nine assigning strategies of the desired speed are conducted a comparison study. Specifically, the performances of the ODE based models equipped with such desired speed assigning strategies are compared with regard to reference data from both the uni- and bi-directional pedestrian flow scenarios using a fair model comparison framework. Then, the performances are compared across several dimensions, including different assigning strategies of the desired speed, the different pedestrian simulation models, different experiments for the same motion base, and different motion base scenarios. Based on these results, four findings are summarized. Firstly, the desired speed is quantitatively verified to be an important role in the performance of ODE based pedestrian simulation models. Secondly, pedestrian density level is verified to be an effective indicator for the complexity of pedestrian crowd dynamics. Thirdly, the performance of the ODE based models is dependent on the extent to which the heterogeneity of desired speed is considered. Lastly, the performance of the social force model (SF) is suggested to be more predictable than optimal reciprocal collision avoidance model (ORCA) regarding the stability and optimality. Finally, based on the above findings, recommendations on assigning the desired speed for ODE based pedestrian simulation models are given.

A group theoretic approach to model comparison with simplicial representations

The complexity of biological systems, and the increasingly large amount of associated experimental data, necessitates that we develop mathematical models to further our understanding of these systems. Because biological systems are generally not well understood, most mathematical models of these systems are based on experimental data, resulting in a seemingly heterogeneous collection of models that ostensibly represent the same system. To understand the system we therefore need to understand how the different models are related to each other, with a view to obtaining a unified mathematical description. This goal is complicated by the fact that a number of distinct mathematical formalisms may be employed to represent the same system, making direct comparison of the models very difficult. A methodology for comparing mathematical models based on their underlying conceptual structure is therefore required. In previous work we developed an appropriate framework for model comparison where we represent models, specifically the conceptual structure of the models, as labelled simplicial complexes and compare them with the two general methodologies of comparison by distance and comparison by equivalence. In this article we continue the development of our model comparison methodology in two directions. First, we present a rigorous and automatable methodology for the core process of comparison by equivalence, namely determining the vertices in a simplicial representation, corresponding to model components, that are conceptually related and the identification of these vertices via simplicial operations. Our methodology is based on considerations of vertex symmetry in the simplicial representation, for which we develop the required mathematical theory of group actions on simplicial complexes. This methodology greatly simplifies and expedites the process of determining model equivalence. Second, we provide an alternative mathematical framework for our model-comparison methodology by representing models as groups, which allows for the direct application of group-theoretic techniques within our model-comparison methodology.

New Directions in L2 Self-efficacy Research: Comparing Bifactor and Second-Order Models in the English Public Speaking Domain.

Reflecting an alternative model comparison framework and grounded in Bandura's self-efficacy theory and in western and Chinese English Public Speaking (EPS) competences literature, we examine the possibility of a hierarchical structure of the EPS self-efficacy construct. In particular, we contrast an established first-order model of the EPS self-efficacy with two higher-order rival models, testing for subordinate (second-order model) versus nested (bifactor model) relationships in a sample of 203 Chinese EFL college students. From a theoretical perspective, the results supported the bifactor model as a strong conceptual foundation for understanding the EPS self-efficacy construct as nested. From a practical perspective, this conceptualization would allow for gauging EPS self-efficacy development both holistically and by subdomains. From a research methodology perspective, this study is among the first to bring the bifactor model into the broader field of L2 research. Other implications are discussed.

Elucidating the influence of temperature and strain rate on the mechanics of AFS-D through a combined experimental and computational approach

In this study, the temperature and strain rate dependent physics of Additive Friction Stir Deposition (AFS-D) are elucidated for the first time. Candidate constitutive models are selected to account for the severe plastic deformation and wide ranges of temperature and strain rate inherent to AFS-D. Torsion mechanical tests are performed to capture AA6061 stress dependency on temperature and strain rate. Four constitutive models, including a low fidelity-calibrated one, are applied to single-layer AFS-D simulations. The robust meshfree method, smoothed particle hydrodynamics (SPH), provided the framework for constitutive model comparison. Utilizing the SPH simulations, peak values, temperature contours, effective stress contours, effective plastic strain contours, build profiles, and simulation run time are all compared to reveal hidden mechanisms of AFS-D processing. This work highlights the importance of constitutive model selection, which can lead to improved simulation results by accurately capturing plastic stress dependence on strain, strain rate, and temperature.

MagmaFOAM-1.0: a modular framework for the simulation of magmatic systems

Abstract. Numerical simulations of volcanic processes play a fundamental role in understanding the dynamics of magma storage, ascent, and eruption. The recent extraordinary progress in computer performance and improvements in numerical modeling techniques allow simulating multiphase systems in mechanical and thermodynamical disequilibrium. Nonetheless, the growing complexity of these simulations requires the development of flexible computational tools that can easily switch between sub-models and solution techniques. In this work we present MagmaFOAM, a library based on the open-source computational fluid dynamics software OpenFOAM that incorporates models for solving the dynamics of multiphase, multicomponent magmatic systems. Retaining the modular structure of OpenFOAM, MagmaFOAM allows runtime selection of the solution technique depending on the physics of the specific process and sets a solid framework for in-house and community model development, testing, and comparison. MagmaFOAM models thermomechanical nonequilibrium phase coupling and phase change, and it implements state-of-the-art multiple volatile saturation models and constitutive equations with composition-dependent and space–time local computation of thermodynamic and transport properties. Code testing is performed using different multiphase modeling approaches for processes relevant to magmatic systems: Rayleigh–Taylor instability for buoyancy-driven magmatic processes, multiphase shock tube simulations propaedeutical to conduit dynamics studies, and bubble growth and breakage in basaltic melts. Benchmark simulations illustrate the capabilities and potential of MagmaFOAM to account for the variety of nonlinear physical and thermodynamical processes characterizing the dynamics of volcanic systems.

Intermittent Hormone Therapy Models Analysis and Bayesian Model Comparison for Prostate Cancer

The prostate is an exocrine gland of the male reproductive system dependent on androgens (testosterone and dihydrotestosterone) for development and maintenance. First-line therapy for prostate cancer includes androgen deprivation therapy (ADT), depriving both the normal and malignant prostate cells of androgens required for proliferation and survival. A significant problem with continuous ADT at the maximum tolerable dose is the insurgence of cancer cell resistance. In recent years, intermittent ADT has been proposed as an alternative to continuous ADT, limiting toxicities and delaying time-to-progression. Several mathematical models with different biological resistance mechanisms have been considered to simulate intermittent ADT response dynamics. We present a comparison between 13 of these intermittent dynamical models and assess their ability to describe prostate-specific antigen (PSA) dynamics. The models are calibrated to longitudinal PSA data from the Canadian Prospective Phase II Trial of intermittent ADT for locally advanced prostate cancer. We perform Bayesian inference and model analysis over the models’ space of parameters on- and off-treatment to determine each model’s strength and weakness in describing the patient-specific PSA dynamics. Additionally, we carry out a classical Bayesian model comparison on the models’ evidence to determine the models with the highest likelihood to simulate the clinically observed dynamics. Our analysis identifies several models with critical abilities to disentangle between relapsing and not relapsing patients, together with parameter intervals where the critical points’ basin of attraction might be exploited for clinical purposes. Finally, within the Bayesian model comparison framework, we identify the most compelling models in the description of the clinical data.

A proposed framework for the development and qualitative evaluation of West Nile virus models and their application to local public health decision-making.

West Nile virus (WNV) is a globally distributed mosquito-borne virus of great public health concern. The number of WNV human cases and mosquito infection patterns vary in space and time. Many statistical models have been developed to understand and predict WNV geographic and temporal dynamics. However, these modeling efforts have been disjointed with little model comparison and inconsistent validation. In this paper, we describe a framework to unify and standardize WNV modeling efforts nationwide. WNV risk, detection, or warning models for this review were solicited from active research groups working in different regions of the United States. A total of 13 models were selected and described. The spatial and temporal scales of each model were compared to guide the timing and the locations for mosquito and virus surveillance, to support mosquito vector control decisions, and to assist in conducting public health outreach campaigns at multiple scales of decision-making. Our overarching goal is to bridge the existing gap between model development, which is usually conducted as an academic exercise, and practical model applications, which occur at state, tribal, local, or territorial public health and mosquito control agency levels. The proposed model assessment and comparison framework helps clarify the value of individual models for decision-making and identifies the appropriate temporal and spatial scope of each model. This qualitative evaluation clearly identifies gaps in linking models to applied decisions and sets the stage for a quantitative comparison of models. Specifically, whereas many coarse-grained models (county resolution or greater) have been developed, the greatest need is for fine-grained, short-term planning models (m–km, days–weeks) that remain scarce. We further recommend quantifying the value of information for each decision to identify decisions that would benefit most from model input.

Agriculture Model Comparison Framework and MyGeoHub Hosting: Case of Soil Nitrogen

To be able to compare many agricultural models, a general framework for model comparison when field data may limit direct comparison of models is proposed, developed, and also demonstrated. The framework first calibrates the benchmark model against the field data, and next it calibrates the test model against the data generated by the calibrated benchmark model. The framework is validated for the modeling of the soil nutrient nitrogen (N), a critical component in the overall agriculture system modeling effort. The nitrogen dynamics and related carbon (C) dynamics, as captured in advanced agricultural modeling such as RZWQM, are highly complex, involving numerous states (pools) and parameters. Calibrating many parameters requires more time and data to avoid underfitting. The execution time of a complex model is higher as well. A study of tradeoff among modeling complexities vs. speed-up, and the corresponding impact on modeling accuracy, is desirable. This paper surveys soil nitrogen models and lists those by their complexity in terms of the number of parameters, and C-N pools. This paper also examines a lean soil N and C dynamics model and compares it with an advanced model, RZWQM. Since nitrate and ammonia are not directly measured in this study, we first calibrate RZWQM using the available data from an experimental field in Greeley, CO, and next use the daily nitrate and ammonia data generated from RZWQM as ground truth, against which the lean model’s N dynamics parameters are calibrated. In both cases, the crop growth was removed to zero out the plant uptake, to compare only the soil N-dynamics. The comparison results showed good accuracy with a coefficient of determination (R2) match of 0.99 and 0.62 for nitrate and ammonia, respectively, while affording significant speed-up in simulation time. The lean model is also hosted in MyGeoHub cyberinfrastructure for universal online access.

Representation Learning for the Clustering of Multi-Omics Data.

The integration of several sources of data for the identification of subtypes of diseases has gained attention over the past few years. The heterogeneity and the high dimensions of the data sets calls for an adequate representation of the data. We summarize the field of representation learning for the multi-omics clustering problem and we investigate several techniques to learn relevant combined representations, using methods from group factor analysis (PCA, MFA and extensions) and from machine learning with autoencoders. We highlight the importance of appropriately designing and training the latter, notably with a novel combination of a disjointed deep autoencoder (DDAE) architecture and a layer-wise reconstruction loss. These different representations can then be clustered to identify biologically meaningful clusters of patients. We provide a unifying framework for model comparison between statistical and deep learning approaches with the introduction of a new weighted internal clustering index that evaluates how well the clustering information is retained from each source, favoring contributions from all data sets. We apply our methodology to two case studies for which previous works of integrative clustering exist, TCGA Breast Cancer and TARGET Neuroblastoma, and show how our method can yield good and well-balanced clusters across the different data sources.

Bayesian analysis of turbulent transport coefficients in 2D interchange dominated ExB turbulence involving flow shear

While turbulent transport is known to dominate the radial particle and energy transport in the plasma edge, a self-consistent description of turbulent transport in mean-field transport codes remains lacking. Mean-field closure models based on the turbulent kinetic energy (k ⊥) and turbulent enstrophy (ζ⊥) have recently been proposed to self-consistently model this transport. This contribution analyses the diffusive particle transport relations of these models by means of the Bayesian framework for parameter estimation and model comparison. The reference data includes a set of isothermal simulations that not only include the scrape-off layer (SOL) but also the outer core region (in which a drift wave-like model is activated) and a set of anisothermal SOL simulations, both obtained with the TOKAM2D turbulence code. This analysis shows that the k ⊥(–ζ⊥) model does not appropriately capture the diffusion coefficients for these new data sets, presumably due to the strong flows in the diamagnetic direction that appear in these new cases. While flow shear is expected to quench the turbulence and the turbulent transport, its effect was not explicitly taken into account in the earlier k ⊥(–ζ⊥) transport models. As flow shear provides a new mechanism for the decorrelation of the turbulence, we propose to introduce an additional time scale in the diffusive transport relation as . Inspiration is drawn from shear decorrelation times reported in literature to propose several new candidate models, which are then analysed in a Bayesian setting. This allowed identifying irrelevant terms for certain models and to rank all models according to the Bayesian evidence. While the new models accounting for shear do improve the match to the data, significant errors still remain. Also, no single model could be identified that performs best for all data sets.

Buying versus renting: On the environmental friendliness of item-sharing

Item-sharing is a concept of the sharing economy where items such as tools or leisure equipment are made available to a community and are accessed need-based. This allows to serve demand with a few highly utilized items, which is why item-sharing connotes resource efficiency and environmental friendliness. However, a shared use also necessitates to (repeatedly) transport single items between consecutive consumers. This kind of transports are relatively inefficient and may significantly affect the environmental efficiency of item-sharing. We investigate to what extent a few shared items allow to serve demand and under which circumstances the environmental benefit of providing fewer items is larger than the negative effects of a higher transportation effort. To this end, we develop a business model comparison framework to analyze and compare alternative ways of serving requests regarding the degree of request fulfillment and the emissions that are released during item production and transport. Our comparison includes variants of a traditional sales-based approach and two types of item-sharing in which items are either exchanged via stations or transferred directly from one consumer to another. We show by simulation studies for a large metropolitan area that item-sharing with direct transfers has a great emissions-cutting potential and that it becomes even more efficient if item exchanges are facilitated through so-called crowdshippers who offer to conduct transports on trips they would make anyway. The low environmental impact of item-sharing is further demonstrated in experiments with different requesting behaviors, various emission rates, and item replacements due to wear and obsolescence.

The Multispecies Coalescent Model Outperforms Concatenation Across Diverse Phylogenomic Data Sets.

A statistical framework of model comparison and model validation is essential to resolving the debates over concatenation and coalescent models in phylogenomic data analysis. A set of statistical tests are here applied and developed to evaluate and compare the adequacy of substitution, concatenation, and multispecies coalescent (MSC) models across 47 phylogenomic data sets collected across tree of life. Tests for substitution models and the concatenation assumption of topologically congruent gene trees suggest that a poor fit of substitution models, rejected by 44% of loci, and concatenation models, rejected by 38% of loci, is widespread. Logistic regression shows that the proportions of GC content and informative sites are both negatively correlated with the fit of substitution models across loci. Moreover, a substantial violation of the concatenation assumption of congruent gene trees is consistently observed across six major groups (birds, mammals, fish, insects, reptiles, and others, including other invertebrates). In contrast, among those loci adequately described by a given substitution model, the proportion of loci rejecting the MSC model is 11%, significantly lower than those rejecting the substitution and concatenation models. Although conducted on reduced data sets due to computational constraints, Bayesian model validation and comparison both strongly favor the MSC over concatenation across all data sets; the concatenation assumption of congruent gene trees rarely holds for phylogenomic data sets with more than 10 loci. Thus, for large phylogenomic data sets, model comparisons are expected to consistently and more strongly favor the coalescent model over the concatenation model. We also found that loci rejecting the MSC have little effect on species tree estimation. Our study reveals the value of model validation and comparison in phylogenomic data analysis, as well as the need for further improvements of multilocus models and computational tools for phylogenetic inference. [Bayes factor; Bayesian model validation; coalescent prior; congruent gene trees; independent prior; Metazoa; posterior predictive simulation.]

The Thermohaline Modes of the Global Ocean

AbstractThe first 2000 m of the global thermohaline structure of the ocean are statistically decomposed into vertical thermohaline modes, using a multivariate functional principal component analysis (FPCA). This method is applied on the Monthly Isopycnal and Mixed-Layer Ocean Climatology (MIMOC). The first three modes account for 92% of the joint temperature and salinity (T–S) variance, which yields a surprisingly good reduction of dimensionality. The first mode (69% of the variance) is related to the thermocline depth and delineates the subtropical gyres. The second mode (18%) is mostly driven by salinity and mainly displays the asymmetry between the North Pacific and Atlantic basins and the salty circumpolar deep waters in the Southern Ocean. The third mode (5%) identifies the low- and high-salinity intermediate waters, covarying with the freshwater inputs of the upper ocean. The representation of the ocean in the space defined by the first three modes offers a simple visualization of the global thermohaline structure that strikingly emphasizes the role of the Southern Ocean in linking and distributing water masses to the other basins. The vertical thermohaline modes offer a convenient framework for model and observation data comparison. This is illustrated by projecting the repeated Pacific section P16 together with profiles from the Array for Real-Time Geostrophic Oceanography (ARGO) global array of profiling floats on the modes defined with the climatology MIMOC. These thermohaline modes have a potential for water mass identification and robust analysis of heat and salt content.

Quantifying model evidence for yellow fever transmission routes in Africa.

Yellow fever is a vector-borne disease endemic in tropical regions of Africa, where 90% of the global burden occurs, and Latin America. It is notoriously under-reported with uncertainty arising from a complex transmission cycle including a sylvatic reservoir and non-specific symptom set. Resulting estimates of burden, particularly in Africa, are highly uncertain. We examine two established models of yellow fever transmission within a Bayesian model averaging framework in order to assess the relative evidence for each model’s assumptions and to highlight possible data gaps. Our models assume contrasting scenarios of the yellow fever transmission cycle in Africa. The first takes the force of infection in each province to be static across the observation period; this is synonymous with a constant infection pressure from the sylvatic reservoir. The second model assumes the majority of transmission results from the urban cycle; in this case, the force of infection is dynamic and defined through a fixed value of R0 in each province. Both models are coupled to a generalised linear model of yellow fever occurrence which uses environmental covariates to allow us to estimate transmission intensity in areas where data is sparse. We compare these contrasting descriptions of transmission through a Bayesian framework and trans-dimensional Markov chain Monte Carlo sampling in order to assess each model’s evidence given the range of uncertainty in parameter values. The resulting estimates allow us to produce Bayesian model averaged predictions of yellow fever burden across the African endemic region. We find strong support for the static force of infection model which suggests a higher proportion of yellow fever transmission occurs as a result of infection from an external source such as the sylvatic reservoir. However, the model comparison highlights key data gaps in serological surveys across the African endemic region. As such, conclusions concerning the most prevalent transmission routes for yellow fever will be limited by the sparsity of data which is particularly evident in the areas with highest predicted transmission intensity. Our model and estimation approach provides a robust framework for model comparison and predicting yellow fever burden in Africa. However, key data gaps increase uncertainty surrounding estimates of model parameters and evidence. As more mathematical models are developed to address new research questions, it is increasingly important to compare them with established modelling approaches to highlight uncertainty in structures and data.