Increasing Model Complexity Research Articles

Exploring the relationship between simulation model accuracy and complexity

Little is known about the relationship between the accuracy of a discrete-event simulation model and its complexity. Despite this, authors propose that increasing model complexity leads to improved accuracy with diminishing returns. In this paper we explore whether this proposition is correct by applying successive simplifications, some in different sequences, to three simulation models and measuring the impact on the accuracy of the model output. The results show that while increased complexity often leads to improved accuracy, this is not always the case. Also, we see that even when there is a positive correlation between complexity and accuracy, the returns to accuracy from greater complexity can be diminishing, constant or increasing. So, we conclude that the proposition about the relationship between accuracy and complexity is not correct. However, it does remain a useful heuristic for guiding modellers when considering the scope and level of detail at which to model a system.

Balancing structural complexity with ecological insight in Spatio‐temporal species distribution models

Abstract The potential for statistical complexity in species distribution models (SDMs) has greatly increased with advances in computational power. Structurally complex models provide the flexibility to analyse intricate ecological systems and realistically messy data, but can be difficult to interpret, reducing their practical impact. Founding model complexity in ecological theory can improve insight gained from SDMs. Here, we evaluate a marked point process approach, which uses multiple Gaussian random fields to represent population dynamics of the Eurasian crane Grus grus in a spatio‐temporal species distribution model. We discuss the role of model components and their impacts on predictions, in comparison with a simpler binomial presence/absence approach. Inference is carried out using Integrated Nested Laplace Approximation (INLA) with inlabru, an accessible and computationally efficient approach for Bayesian hierarchical modelling, which is not yet widely used in SDMs. Using the marked point process approach, crane distribution was predicted to be dependent on the density of suitable habitat patches, as well as close to observations of the existing population. This demonstrates the advantage of complex model components in accounting for spatio‐temporal population dynamics (such as habitat preferences and dispersal limitations) that are not explained by environmental variables. However, including an AR1 temporal correlation structure in the models resulted in unrealistic predictions of species distribution; highlighting the need for careful consideration when determining the level of model complexity. Increasing model complexity, with careful evaluation of the effects of additional model components, can provide a more realistic representation of a system, which is of particular importance for a practical and impact‐focused discipline such as ecology (though these methods extend to applications for a wide range of systems). Founding complexity in contextual theory is not only fundamental to maintaining model interpretability but can be a useful approach to improving insight gained from model outputs.

An Improved Data-Driven Decision Feedback Receiver via Deep Unfolding

The proposed decision feedback receiver (DFR) is an end-to-end data-driven iterative receiver, and the performance gain is achieved from iterations. However, the mismatch problem between the training set and test set exists in the DFR training, and thus performance degradation, slow convergence speed, and oscillation are introduced. On the other hand, deep unfolding using parameter sharing is a practical method to reduce the model parameter number and improve the training efficiency, but the problem whether the parameter sharing will cause performance degradation is rarely considered. In this work, we generally discuss and analyze these two problems, and solution to solve the problem or conditions that the problem no longer exists is then introduced. We give the improvements to address the mismatch problem in the DFR, and thus we propose an improved-DFR via deep unfolding. The improved-DFRs with and without parameter sharing, namely, DFR-I and DFR-IS, are both developed with low computation complexity and model complexity and can be executed by parallel processing. Besides, the practical training tricks and performance analysis including computation complexity and model complexity are given. In the experiments, the improved-DFRs outperform the DFR in various scenarios, in terms of convergence speed and symbol error rate. The simulation results also show that the DFR-IS is easier to train, and the slight performance loss can be reduced by increasing model complexity, in comparison to DFR-I.

Special Feature: Linking capture-recapture and movement.

Special Feature: Linking capture-recapture and movement.

Topology-Based Initialization for the Optimization-Based Design of Heteroazeotropic Distillation Processes

Distillation-based separation processes, such as extractive or heteroazeotropic distillation, present important processes for separating azeotropic mixtures in the chemical and biochemical industry. However, heteroazeotropic distillation has received much less attention than extractive distillation, which can be attributed to multiple reasons. The phase equilibrium calculations require a correct evaluation of phase stability, while the topology of the heterogeneous mixtures is generally more complex, comprising multiple azeotropes and distillation regions, resulting in an increased modeling complexity. Due to the integration of distillation columns and a decanter, even the simulation of these processes is considered more challenging, while an optimal process design should include the selection of a suitable solvent, considering the performance of the integrated hybrid process. Yet, the intricate mixture topologies largely impede the use of simplified criteria for solvent selection. To overcome these limitations and allow for a process-based screening of potential solvents, the current work presents a topology-based initialization and optimization approach for designing heteroazeotropic distillation processes. The systematic initialization enables an efficient evaluation of different solvents with different mixture topologies, which is further exploited for optimization-based sensitivity analysis and multi-objective optimization. Three case studies are analyzed with about 170 individually optimized process designs, including stage numbers, feed locations, phase ratios, and heat duties.

Prediction of enteric methane production and yield in sheep using a Latin America and Caribbean database

Methane (CH4) produced from enteric fermentation in ruminants has a noticeable impact on climate change. Prediction models are an alternative to current laborious and costly in vivo CH4 measurement techniques. The objectives of this study were to: (1) collate a database of individual sheep records from CH4 emission studies conducted in the Latin America and Caribbean (LAC) region; (2) identify key variables for predicting CH4 production (g/d) and CH4 yield [g/kg of dry matter intake (DMI)]; (3) develop and cross-validate these newly-developed models; and (4) compare models’ predictive ability with equations currently used to support national greenhouse gas (GHG) inventories in the LAC region. After removing outliers, the final database retained 219 individual sheep records from 11 studies, 48.2% of the original database. Models were developed using a sequential approach, by incrementally adding different variables with increasing complexity. Production and yield of CH4 were predicted by fitting mixed-effects models with a random effect of study. The predictive accuracy of fitted CH4 prediction models was evaluated using a leave-one-out cross-validation. Overall, increasing model complexity improved the predictive performance of CH4 production and yield equations. Feed intake was the most important predictor of sheep CH4 production. Our best-developed CH4 production models outperformed Tier 2 equations from the Intergovernmental Panel on Climate Change (IPCC) in the growing lambs and mature sheep subsets, whereas they performed slightly worse in the complete subset. Methane yield can be predicted using dietary forage content only, or with an increased complexity model combining body weight, feeding level, and dietary forage content. The use of the newly-developed models rather than IPCC Tier 2 equations can substantially improve the accuracy of GHG inventories from LAC countries.

Viscoelastoplastic classification of cementitious suspensions: transient and non-linear flow analysis in rotational and oscillatory shear flows

Rheological modeling of special concretes such as ultra-high performance concretes (UHPCs), self-compacting concretes (SCCs), or environmentally friendly concretes with low clinker contents but containing many fine particles increases model complexity because high amounts of colloidal and fine particles and complex chemistry result in strongly non-Newtonian rheological behavior. Straight-forward viscoplastic rheological models such as the well-known Bingham- or Herschel-Bulkley approach do not fit the flow behavior on a large scale at and lead to boundary value problems in numerical flow simulations. Increased viscoelastic properties due to chemical admixtures are not described by pure viscoplasticity analysis methods. To fill this gap, our research presents viscoplastic and viscoelastic linear and non-linear rheological characterization methods for common and strongly non-Newtonian cement pastes and combines the methods for a comprehensive viscoelastoplastic modeling of cementitious building materials. We illustrate and discuss the benefits and boundaries of each measurement technique in dependence of cementitious paste complexity. Three solid volume fractions from phi = 0.45 to 0.55 and three different flowability ranges, adjusted through varying superplasticizer amounts, were investigated in respect of steady-state and transient yield stress, viscosity, and non-linear viscoelastic material properties. Our results reveal that with increasing solid volume fraction and polymer amount, viscoelastoplastic modeling describes rheological flow on a large flow scale more appropriately than common viscoplastic methods. The results show a new approach for a full quantitative and qualitative rheological classification of cementitious building materials and thus improve the understanding of densely packed colloidal cementitious suspensions. The findings serve as prospective guideline to model complex cementitious building materials both at low and high shear rates, and to find an appropriate rheological description at the boundaries of flow.Graphical abstract

Improving Single-Stage Object Detectors for Nighttime Pedestrian Detection

Improving the reliability of nighttime pedestrian detection is a crucial challenge towards the design of robust autonomous systems. Not surprisingly, most pedestrian fatalities occur in low-illumination settings, thus emphasizing the need for new algorithmic advances. This work presents a novel pedestrian detection approach that makes a number of crucial modifications to the state-of-the-art YOLOV5-PANet architecture, in order to improve the reliability of features extracted from nighttime images. More specifically, the proposed architecture systematically incorporates powerful shuffle attention mechanisms and a transformer module to improve the feature learning pipeline. Instead of advocating the use of other sensing modalities that are better suited for nighttime detection, our approach relies only on conventional RGB cameras and is hence broadly applicable. Our empirical studies with nighttime pedestrian detection benchmarks show that with only minimal increase in model complexity, our approach provides significant improvements in detection efficacy over existing solutions. Finally, we explore the impact of post-hoc network pruning on the speed-accuracy trade-off of our approach and demonstrate that it is well suited for reduced memory/compute requirements.

Demographics as Determinants of Building Occupants’ Indoor Environmental Perceptions: Insights from a Machine Learning Incremental Modeling and Analysis Approach

The relationship between the demographical characteristics of building occupants and their perception of indoor comfort is increasingly being studied. However, the added value from accounting for such characteristics when modeling and predicting occupants’ perceptions remains unclear. An incremental machine learning (ML) modeling and analysis approach is proposed to quantify the influence of four demographical factors (gender, age, nationality, and time lived in the environment) on occupants’ perceptions of their indoor environment conditions. A three-step methodology is presented: (1) data collection through sensors and a questionnaire administered on 206 occupants of academic and office buildings in Abu Dhabi, UAE, (2) development of ML models (i.e., support vector machine, random forest, and gradient boosting) to predict occupants’ perceptions under different scenarios of demographical representation (i.e., from no representation to all demographical parameters included), and (3) analysis of the impact of demographical parameters’ inclusion on the performance of the ML models in terms of predictive accuracy, F1-scores, and computing time. Results confirm that including demographical variables could increase prediction accuracy and F1-scores by approximately 19% and 56%, respectively. However, in some instances, the inclusion of these variables reduced model performance while increasing computing time by as much as 50%. A detailed discussion is presented on the comparative performance of the different tested ML algorithms and the need to strike a balance between increasing model complexity and computational costs.

Review on Interpretable Machine Learning in Smart Grid

In recent years, machine learning, especially deep learning, has developed rapidly and has shown remarkable performance in many tasks of the smart grid field. The representation ability of machine learning algorithms is greatly improved, but with the increase of model complexity, the interpretability of machine learning algorithms is worse. The smart grid is a critical infrastructure area, so machine learning models involving it must be interpretable in order to increase user trust and improve system reliability. Unfortunately, the black-box nature of most machine learning models remains unresolved, and many decisions of intelligent systems still lack explanation. In this paper, we elaborate on the definition, motivations, properties, and classification of interpretability. In addition, we review the relevant literature addressing interpretability for smart grid applications. Finally, we discuss the future research directions of interpretable machine learning in the smart grid.

Compensation of nonlinear distortion in hybrid beamforming MIMO transmitters

Compensation of nonlinear distortion in hybrid beamforming MIMO transmitters is an open and challenging field of research. The goal of the study is to find an efficient method for linearization of power amplifiers in fully connected hybrid beamforming MIMO transmitters, which would not additionally increase the complexity of the system too much. In this paper, the polynomial-based compensation method of nonlinear distortion in hybrid beamforming MIMO transmitters is presented. The method is verified through comprehensive Matlab simulations. A 64x64 fully connected hybrid beamforming MIMO system with 2 RF branches on the transmitting side and 2 RF branches on the receiving side was considered. The proposed memory polynomial digital predistortion model is based on direct transmit-end feedback. Simulations were performed for different parameters of the memory polynomial model and analyzed. The obtained results are presented in graphical form, using Power Spectrum Density, and in numerical form, using metrics in time and frequency domain, such as Normalized Mean-Squared Error and Error Vector Magnitude. It is shown that the proposed memory polynomial digital predistortion model can very well compensate nonlinear distortion in fully connected hybrid beamforming MIMO transmitter. It has been also demonstrated that with increasing model complexity, ie with increasing memory depth and nonlinearity order of memory polynomial model, better compensation of nonlinear distortion in hybrid beamforming MIMO transmitters can be achieved.

An efficient algorithm for estimating population history from genetic data

The Legofit statistical package uses genetic data to estimate parameters describing population history. Previous versions used computer simulations to estimate probabilities, an approach that limited both speed and accuracy. This article describes a new deterministic algorithm, which makes Legofit faster and more accurate. The speed of this algorithm declines as model complexity increases. With very complex models, the deterministic algorithm is slower than the stochastic one. In an application to simulated data sets, the estimates produced by the deterministic and stochastic algorithms were essentially identical. Reanalysis of a human data set replicated the findings of a previous study and provided increased support for the hypotheses that (a) early modern humans contributed genes to Neanderthals, and (b) a "superarchaic" population (which separated from all other humans early in the Pleistocene) was either large or deeply subdivided.

Statistical methods for Mendelian models with multiple genes and cancers.

Risk evaluation to identify individuals who are at greater risk of cancer as a result of heritable pathogenic variants is a valuable component of individualized clinical management. Using principles of Mendelian genetics, Bayesian probability theory, and variant-specific knowledge, Mendelian models derive the probability of carrying a pathogenic variant and developing cancer in the future, based on family history. Existing Mendelian models are widely employed, but are generally limited to specific genes and syndromes. However, the upsurge of multigene panel germline testing has spurred the discovery of many new gene-cancer associations that are not presently accounted for in these models. We have developed PanelPRO, a flexible, efficient Mendelian risk prediction framework that can incorporate an arbitrary number of genes and cancers, overcoming the computational challenges that arise because of the increased model complexity. We implement an 11-gene, 11-cancer model, the largest Mendelian model created thus far, based on this framework. Using simulations and a clinical cohort with germline panel testing data, we evaluate model performance, validate the reverse-compatibility of our approach with existing Mendelian models, and illustrate its usage. Our implementation is freely available for research use in the PanelPRO R package.

Annual CO2 Budget Estimation From Chamber-Based Flux Measurements on Intensively Drained Peat Meadows: Effect of Gap-Filling Strategies

Estimating annual CO2 budgets on drained peatlands is important in understanding the significance of CO2 emissions from peatland degradation and evaluating the effectiveness of mitigation techniques. The closed-chamber technique is widely used in combination with gap-filling of CO2 fluxes by parameter fitting empirical models of ecosystem respiration (Reco) and gross primary production (GPP). However, numerous gap-filling strategies are available which are suitable for different circumstances and can result in large variances in annual budget estimates. Therefore, a need for guidance on the selection of gap-filling methodology and its influence on the results exists. Here, we propose a framework of gap-filling methods with four Tiers following increasing model complexity at structural and temporal levels. Tier one is a simple parameter fitting of basic empirical models on an annual basis. Tier two adds structural complexity by including extra environmental factors such as grass height, groundwater level and drought condition. Tier three introduces temporal complexity by separation of annual datasets into seasons. Tier four is a campaign-specific parameter fitting approach, representing highest temporal complexity. The methods were demonstrated on two chamber-based CO2 flux datasets, one of which was previously published. Performance of the empirical models were compared in terms of error statistics. Annual budget estimates were indirectly validated with carbon export values. In conclusion, different gap-filling methodologies gave similar annual estimates but different intra-annual CO2 fluxes, which did not affect the detection of the treatment effects. The campaign-wise gap-filling at Tier four gave the best model performances, while Tier three seasonal gap-filling produced satisfactory results throughout, even under data scarcity. Given the need for more complete carbon balances in drained peatlands, our four-Tier framework can serve as a methodological guidance to the handling of chamber-measured CO2 fluxes, which is fundamental in understanding emissions from degraded peatlands and its mitigation. The performance of models on intra-annual data should be validated in future research with continuous measured CO2 flux data.

Maximization and restoration: Action segmentation through dilation passing and temporal reconstruction

Action segmentation aims to split videos into segments of different actions. Recent work focuses on dealing with long-range dependencies of long, untrimmed videos, but still suffers from over-segmentation and performance saturation due to increased model complexity. This paper addresses the aforementioned issues through a divide-and-conquer strategy that first maximizes the frame-wise classification accuracy of the model and then reduces the over-segmentation errors. This strategy is implemented with the Dilation Passing and Reconstruction Network, composed of the Dilation Passing Network, which primarily aims to increase accuracy by propagating information of different dilations, and the Temporal Reconstruction Network, which reduces over-segmentation errors by temporally encoding and decoding the output features from the Dilation Passing Network. We also propose a weighted temporal mean squared error loss that further reduces over-segmentation. Through evaluations on the 50Salads, GTEA, and Breakfast datasets, we show that our model achieves significant results compared to existing state-of-the-art models.

Using Iterative Pairwise External Validation to Contextualize Prediction Model Performance: A Use Case Predicting 1-Year Heart Failure Risk in Patients with Diabetes Across Five Data Sources.

IntroductionExternal validation of prediction models is increasingly being seen as a minimum requirement for acceptance in clinical practice. However, the lack of interoperability of healthcare databases has been the biggest barrier to this occurring on a large scale. Recent improvements in database interoperability enable a standardized analytical framework for model development and external validation. External validation of a model in a new database lacks context, whereby the external validation can be compared with a benchmark in this database. Iterative pairwise external validation (IPEV) is a framework that uses a rotating model development and validation approach to contextualize the assessment of performance across a network of databases. As a use case, we predicted 1-year risk of heart failure in patients with type 2 diabetes mellitus.MethodsThe method follows a two-step process involving (1) development of baseline and data-driven models in each database according to best practices and (2) validation of these models across the remaining databases. We introduce a heatmap visualization that supports the assessment of the internal and external model performance in all available databases. As a use case, we developed and validated models to predict 1-year risk of heart failure in patients initializing a second pharmacological intervention for type 2 diabetes mellitus. We leveraged the power of the Observational Medical Outcomes Partnership common data model to create an open-source software package to increase the consistency, speed, and transparency of this process.ResultsA total of 403,187 patients from five databases were included in the study. We developed five models that, when assessed internally, had a discriminative performance ranging from 0.73 to 0.81 area under the receiver operating characteristic curve with acceptable calibration. When we externally validated these models in a new database, three models achieved consistent performance and in context often performed similarly to models developed in the database itself. The visualization of IPEV provided valuable insights. From this, we identified the model developed in the Commercial Claims and Encounters (CCAE) database as the best performing model overall.ConclusionUsing IPEV lends weight to the model development process. The rotation of development through multiple databases provides context to model assessment, leading to improved understanding of transportability and generalizability. The inclusion of a baseline model in all modelling steps provides further context to the performance gains of increasing model complexity. The CCAE model was identified as a candidate for clinical use. The use case demonstrates that IPEV provides a huge opportunity in a new era of standardised data and analytics to improve insight into and trust in prediction models at an unprecedented scale.Supplementary InformationThe online version contains supplementary material available at 10.1007/s40264-022-01161-8.

Physiological trend analysis of a novel cardio-pulmonary model during a preload reduction manoeuvre

Background and objective: Mechanical ventilation causes adverse effects on the cardiovascular system. However, the exact nature of the effects on haemodynamic parameters is not fully understood. A recently developed cardio-vascular system model which incorporates cardio-pulmonary interactions is compared to the original 3-chamber cardiovascular model to investigate the exact effects of mechanical ventilation on haemodynamic parameters and to assess the trade-off of model complexity and model reliability between the 2 models.Methods: Both the cardio-pulmonary and three chamber models are used to identify cardiovascular system parameters from aortic pressure, left ventricular volume, airway flow and airway pressure measurements from 4 pigs during a preload reduction manoeuvre. Outputs and parameter estimations from both models are contrasted to assess the relative performance of each model and to further investigate the effects of mechanical ventilation on haemodynamic parameters.Results: Both models tracked measurements accurately as expected. There was no identifiable increase in error from the added complexity of the cardio-pulmonary model, with both models having a mean average error below 0.5% for all pigs. Identified left ventricle and vena cava elastances of the 3-chamber model was found to diverge exponentially with PEEP from identified left ventricle and vena cava elastances of the cardio-pulmonary model. The r2 of the fit for each pig ranged from 0.888 to 0.998 for left ventricle elastance divergence and from 0.905 to 0.999 for vena cava elastance divergence. All other identified parameters showed no significant difference between models.Conclusions: Despite the increase in model complexity, there was no loss in the cardio-pulmonary model’s ability to accurately estimate haemodynamic parameters and reproduce system dynamics. Furthermore, the cardio-pulmonary model was able to demonstrate how mechanical ventilation affected parameter estimations as PEEP was increased. The 3-chamber model was shown to produce parameter estimations which diverged exponentially with PEEP, while the cardiopulmonary model estimations remained more stable, suggesting its ability to produce more physiologically accurate parameter estimations under higher PEEP conditions.

Integrated prediction intervals and specific value predictions for regression problems using neural networks

Improving the robustness of neural nets in regression tasks is key to their application in multiple domains. Deep learning-based approaches aim to achieve this goal either by improving their prediction of specific values (i.e., point prediction), or by producing prediction intervals (PIs) that quantify uncertainty. We present IPIV, a deep neural network for producing both a PI and a value prediction. Our loss function expresses the value prediction as a function of the upper and lower bounds, thus ensuring that it falls within the interval without increasing model complexity. Moreover, our approach makes no assumptions regarding data distribution within the PI, making its value prediction more effective for various real-world problems. Experiments and ablation tests on known benchmarks show that our approach produces tighter uncertainty bounds than the current state-of-the-art approaches for producing PIs, while maintaining comparable performance to the state-of-the-art approach for value-prediction. Additionally, we go beyond previous work and include large image datasets in our evaluation, where IPIV is combined with modern neural nets.

Computationally efficient parameter estimation for spatial individual-level models of infectious disease transmission

Individual-level models incorporate individual-specific covariate information, such as spatial location, to model infectious disease transmission. However, fitting these models with traditional Bayesian methods becomes cumbersome as model complexity or population size increases. We consider a spatial individual-level model with a binary susceptibility covariate. A method for fitting this model to aggregate-level data using traditional Metropolis–Hastings MCMC and then disaggregating the results to obtain individual-level estimates for epidemic metrics is proposed. This so-called “Cluster–Aggregate–Disaggregate” (CAD) method is compared to two approximate Bayesian computation (ABC) algorithms in a simulation study. The methods are also applied to a data set from the 2001 U.K. foot and mouth disease epidemic. While the CAD and ABC methods both performed reasonably well at capturing epidemic metrics, the CAD method was found to be much easier to implement and reduced computation time (relative to the traditional model-fitting method) more consistently than the ABC methods.

Forecasting the Future Sustainability of Technology Choices: Qualitative Predictive Validity of Models as a Complement to Quantitative Uncertainty

To support product and technology choices toward a more sustainable future, diverse assessment methods are used, involving life cycle assessment (LCA). This raises the question of their predictive validity. Whereas, many studies focus on quantitative uncertainty, here the main aim is to address the complementary qualitative aspect of the LCA-related model variants. To that end, we first specify three general influential aspects: (1) future conditions, (2) needed predictivity, and (3) mechanism coverage. These have been translated into a more concrete checklist for qualitative predictive validity. Second, we categorized the model variants into a limited number of basic model types, based on five predefined modeling characteristics. These model types show increasingly complex steps for investigating the future, illustrated with energy systems for transport. Different answers to the same questions may result. With increasing model complexity, the relevant questions may change from analysing specific products, to more general product systems, and next to product-technology domain systems. As a third step, the qualitative predictive validity of the nine modeling types is evaluated using the developed checklist. All have limited predictive validity, increasingly so for longer time horizons, as they lack most causal mechanisms, especially the institutional drivers for development and employment of technologies to emerge. Also, the future is only partially determined. For supporting choices, the conclusion is that the comparative analysis regarding long-term also broader product-technology systems has limited predictive validity. As a solution, conditional statements may show directions for explorative analysis resulting in highly tentative advice on potentially attractive directions.