Emulation Approach Research Articles

An emulation-based approach for interrogating reactive transport models

Abstract. We present an emulation-based approach to understand the interactions among different chemical and biological processes modelled in environmental reactive transport models (RTMs) and explore how the parameterisation of these processes influences the results of multi-component RTMs. We utilise a previously published RTM consisting of 20 primary species, 20 secondary complexes, 17 mineral reactions, and 2 biologically mediated reactions; this RTM describes bio-stimulation using sediment from a contaminated aquifer. We choose a subset of the input parameters to vary over a range of values. The result is the construction of a new dataset that describes the model behaviour over a range of environmental conditions. Using this dataset to train a statistical model creates an emulator of the underlying RTM. This is a condensed representation of the original RTM that facilitates rapid exploration of a broad range of environmental conditions and sensitivities. As an illustration of this approach, we use the emulator to explore how varying the boundary conditions in the RTM describing the aquifer impacts the rates and volumes of mineral precipitation. A key result of this work is the recognition of an unanticipated dependency of pyrite precipitation on pCO2 in the injection fluid due to the stoichiometry of the microbially mediated sulfate reduction reaction. This complex relationship was made apparent by the emulator, while the underlying RTM was not specifically constructed to create such a feedback. We argue that this emulation approach to sensitivity analysis for RTMs may be useful in discovering such new coupled sensitives in geochemical systems and for designing experiments to optimise environmental remediation. Finally, we demonstrate that this approach can maximise specific mineral precipitation or dissolution reactions by using the emulator to find local maxima, which can be widely applied in environmental systems.

OP51 Use Of Real-World Data In Cost-effectiveness Analysis Of Sequential Biologic Treatment For Rheumatoid Arthritis

IntroductionIn health technology assessment (HTA), economic evaluations assessing biologic drugs for rheumatoid arthritis (RA) involve modeling patients’ responses to multiple treatments given sequentially over a lifetime horizon. When data from randomized controlled trials (RCTs) are scarce, data from non-randomized studies (e.g., single-arm trials [SATs] and disease registries) can be used to supplement the evidence base. This research aimed to demonstrate meta-analytic methods for combining effectiveness data from randomized and non-randomized studies and their corresponding impact on cost-effectiveness estimates.MethodsData comparing patients receiving second-line rituximab with continued background non-biologic treatment were extracted from one RCT and six SATs identified in an HTA assessing second-line rituximab for RA, and from the British Society for Rheumatology Biologics Register-Rheumatoid Arthritis, by applying a target trial emulation approach. A binomial meta-analysis model was used to estimate the probabilities of achieving the European League against Rheumatism (EULAR) response criteria by pooling data from the RCT, SATs, and the registry. The probabilities were entered into a decision model from a previous HTA to derive incremental cost-effectiveness ratio (ICER) estimates for treatment strategies with and without biologic drugs.ResultsCompared with the original analysis, the estimated probability of at least a moderate EULAR response on rituximab from combined sources was substantially lower. For example, the probability obtained from an RCT was 0.68 (95% credible interval [CrI]: 0.345, 0.907), but only 0.29 (95% [CrI]: 0.242, 0.333) when using RCT plus registry data and 0.29 (95% CrI: 0.244, 0.336) for combined RCT, registry, and SAT data. In the cost-effectiveness analysis, the median ICERs were higher when including real-world data.ConclusionsSynthesis of all relevant data, including RWD, provides additional information regarding the variability in cost-effectiveness estimates and can be considered in sensitivity analyses for HTA decision-making.

Comparative Effectiveness of Ibrutinib Flexible Dosing Treatment Strategies on Time to Next Treatment in a Largely Community-Based Claims Database: A Target Trial Emulation Study

Introduction: Outside of a clinical trial setting, ibrutinib is the most well-characterized of the covalent Bruton's tyrosine kinase inhibitors used in the treatment of patients with chronic lymphocytic leukemia (CLL)/small lymphocytic lymphoma (SLL) in the first-line (1L) setting. However, the impact of ibrutinib dose adjustment on clinical outcomes remains to be defined. In this real-world effectiveness study, we compared time to next treatment (TTNT) as a proxy for disease progression for dose adjustment and standard dose treatment strategies in patients with CLL/SLL receiving 1L single-agent IBR. Methods: This study utilized the Komodo Health payer-complete dataset, derived from U.S. health insurance claims from >150 private insurers covering >160 million individuals from 2015-2023; 53% of these patients are from community centers. Adult patients with CLL/SLL were included if they initiated 1L single-agent IBR 420 mg/day between March 2016 - April 2023 (IBR initiation date = index date; 1L single-agent IBR was defined as no other CLL/SLL treatment in the 12-month baseline period or within 28 days after index). A target trial emulation (TTE) approach was utilized to compare TTNT, defined as the time between the index date to the initiation of a next line of therapy (LOT), for two treatment strategies: 1) Dose adjustment strategy, defined as a lead-in IBR treatment at 420mg/day for 3-12 months, followed by dose adjustment to <420mg/day until next LOT or IBR discontinuation; 2) Standard dose strategy, defined as a continuous IBR treatment at 420mg/day without any dose adjustment for >12 months until next LOT or IBR discontinuation. The TTE approach minimizes immortal time bias in a naïve observational analysis as patients in the dose adjustment cohort cannot initiate next LOT before dose adjustment occurs [Hernan et al. Am J Epidemiol. 2016]. To explicitly emulate a target trial, two clones were created for each patient, with one clone contributing person-time and event to each treatment strategy of interest, respectively. A clone was artificially censored when the actual treatment path deviated from the assigned treatment strategy. Selection bias due to artificial censoring was corrected using time-varying inverse probability of censoring weights (IPCW), accounting for post-index time-varying clinical factors related to IBR dose adjustment or IBR discontinuation. TTNT hazard ratio (HR; robust 95% confidence interval [CI]) was estimated using an IPCW-weighted pooled logistic regression model. Alternative lead-in periods with IBR standard dosing (3-6 and 3-9 months) were also evaluated. A subgroup analysis was conducted among patients with high cardiovascular (CV) risk, defined as having a preexisting CV comorbidity or a high CHA 2DS 2-Vasc risk score. Results: 3,342 adults initiated 1L single-agent IBR at 420 mg/day. Mean (standard deviation [SD]) age was 67.5 (10.6) years, 62% were male, and mean (SD) Quan-Charlson Comorbidity Index score was 3.0 (1.5). The median (interquartile range) follow-up duration was 19.6 (7.5, 35.0) months. During the study period, 463 patients (14%) initiated a subsequent LOT. The high CV risk subgroup consisted of 2,640 (79%) patients. In the overall study population, 608 (18%) patients experienced a dose adjustment after the index date. Of those, the dose adjustment occurred ≤3 months, ≤6 months, and ≤12 months post-index in 159 (26%), 300 (49%), and 440 (72%) patients, respectively, with dose used at the first reduction of 70 mg, 140 mg, or 280 mg in 4 (1%), 85 (14%), and 519 (85%) patients, respectively. The dose adjustment strategy (420 mg/day for 3-12 months followed by a reduced dose) was not associated with increased risk of initiating next LOT (adjusted HR [95% CI] 0.95 [0.80, 1.14]) compared to the continuous standard dose strategy; results for the alternative lead-in periods and the high CV risk subgroup were consistent with the primary scenario. Conclusions: In this real-world comparative effectiveness study with the majority of patients from community centers, dose adjustment of 1L single-agent IBR treatment for CLL/SLL is not associated with an increased risk of initiating next LOT when compared to continuous standard dose treatment. These findings suggest that a flexible dosing approach with IBR may be an effective option to maintain an optimal balance of clinical therapeutic efficacy and safety for patients with CLL/SLL in the 1L setting.

Impact of special educational needs provision on hospital utilisation, school attainment and absences for children in English primary schools stratified by gestational age at birth: A target trial emulation study protocol.

One third of children in English primary schools have additional learning support called special educational needs (SEN) provision, but children born preterm are more likely to have SEN than those born at term. We aim to assess the impact of SEN provision on health and education outcomes in children grouped by gestational age at birth. We will analyse linked administrative data for England using the Education and Child Health Insights from Linked Data (ECHILD) database. A target trial emulation approach will be used to specify data extraction from ECHILD, comparisons of interest and our analysis plan. Our target population is all children enrolled in year one of state-funded primary school in England who were born in an NHS hospital in England between 2003 and 2008, grouped by gestational age at birth (extremely preterm (24-<28 weeks), very preterm (28-<32 weeks), moderately preterm (32-<34 weeks), late preterm (34-<37 weeks) and full term (37-<42 weeks). The intervention of interest will comprise categories of SEN provision (including none) during year one (age five/six). The outcomes of interest are rates of unplanned hospital utilisation, educational attainment, and absences by the end of primary school education (year six, age 11). We will triangulate results from complementary estimation methods including the naïve estimator, multivariable regression, g-formula, inverse probability weighting, inverse probability weighting with regression adjustment and instrumental variables, along with a variety for a variety of causal contrasts (average treatment effect, overall, and on the treated/not treated). We have existing research ethics approval for analyses of the ECHILD database described in this protocol. We will disseminate our findings to diverse audiences (academics, relevant government departments, service users and providers) through seminars, peer-reviewed publications, short briefing reports and infographics for non-academics (published on the study website).

Ibrutinib Dose Adjustment Does Not Impact Time to Next Treatment in First-Line Patients with Chronic Lymphocytic Leukemia: A Real-World Analysis of Electronic Medical Records from Academic and Non-Teaching Hospitals Using Target Trial Emulation

Introduction: Ibrutinib, a once-daily Bruton's tyrosine kinase inhibitor (BTKi), is recommended as first-line (1L) treatment for patients with chronic lymphocytic leukemia (CLL). Dosing flexibility with ibrutinib allows patients to adjust their daily dose to help prevent recurrence or worsening of adverse events, while maintaining efficacy by allowing patients to stay on treatment and benefit from long-term treatment outcomes. The current study aimed to compare time to next treatment (TTNT) in CLL patients initiating 1L single-agent ibrutinib at 420mg/day and staying on this dose versus those having a dose adjustment (DA) within 3-12 months. Methods: Specialty pharmacy electronic medical records from the Acentrus database, comprised primarily of academic and non-teaching hospital systems in the United States (1/1/2016-4/30/2022), were used to identify adults with CLL who initiated 1L single-agent ibrutinib at a starting dose of 420mg/day (index date) (N = 1,171). A washout period of ≥12 months pre-index (also used to evaluate baseline characteristics) was required to confirm use as 1L therapy. The following treatment strategies were compared: treatment for 3 months post-index at 420mg/day followed by DA to a dose below 420mg/day between 3-12 months post-index versus sustained treatment for ≥12 months post-index at 420mg/day. The first 3 months were required to be at 420mg/day for both strategies, as most DAs occur after this period. TTNT was defined as the time from the index date to the initiation of a next treatment. Administrative censoring included treatment discontinuation (based on treatment gap of ≥365 days), death, participation in a clinical trial, end of data availability, within-class BTKi switch (i.e., switching to another covalent BTKi), anti-CD20 antibody (i.e., obinutuzumab or rituximab), or venetoclax add-on within 180 days post-index. To account for immortal time bias (i.e., patients with DA are required to survive on 1L therapy until DA), as well as overlap between follow-up time and definition of treatment strategies, a target trial emulation approach was used, where all patients were cloned at index date and contributed follow-up to both treatment strategy arms. Hazard ratios (HRs) were approximated from odds ratios obtained from multivariate pooled logistic regression models comparing TTNT between the two treatment strategies. Sensitivity analyses using different time periods to capture DA (3-6, 3-9, and 3-18 months) were also conducted. Results were replicated in the subgroup of patients at high risk for a cardiovascular (CV) event (based on pre-existing CV conditions or a high CV risk score). Results: Among the 1,171 patients who initiated 1L single-agent ibrutinib at a starting dose of 420mg/day, the mean age was 70.4 years, with 34.6% being female. The mean Quan-Charlson Comorbidity Index score was 3.0, and 61.8% were classified as high risk for a CV event. The mean (median) follow-up was 29.8 (30.9) months. Overall, 229 (19.6%) patients had a DA at any time (178 [77.7%] adjusted their dose to 280mg/day), with a mean (median) time to DA of 9.0 (5.5) months. Of these patients, 111 (48.5%) had a DA between 3-12 months post-index date (3-6 months: 66 [28.8%] patients, 3-9 months: 87 [38.0%] patients, 3-18 months: 126 [55.0%] patients). A total of 90 (7.7%) patients initiated a next treatment, out of which 59 (65.6%) patients initiated a venetoclax-based regimen (beyond the first 180 days) as their subsequent therapy. After adjustment in a pooled logistic regression model, DA was not associated with an increased risk of having a next treatment (HR: 1.14, p-value: 0.469) ( Figure 1). In sensitivity analyses evaluating DA over periods of 3-6, 3-9, and 3-18 months, results remained non-significant, with HRs ranging from 1.13 to 1.30 (all p-values &amp;gt;0.05) ( Figure 2). Results were also consistent in the subgroup with high risk for a CV event. Conclusions: This large real-world analysis of CLL patients from academic and non-teaching hospital systems showed that undergoing DA of 1L ibrutinib yielded similar clinical outcomes compared to having no DA, with similar findings for the high CV risk subgroup. These findings, along with those from clinical trials and real-world community oncology practices, suggest that a flexible dosing approach with ibrutinib may be an effective approach in allowing patients to achieve optimal outcomes while remaining on long-term continuous 1L treatment.

Individual participant data from digital sources informed and improved precision in the evaluation of predictive biomarkers in Bayesian network meta-analysis

ObjectivesWe aimed to develop a network meta-analytic model for the evaluation of treatment effectiveness within predictive biomarker subgroups, by combining evidence from individual participant data (IPD) from digital sources (in the absence of randomized controlled trials) and aggregate data (AD). Study Design and SettingA Bayesian framework was developed for modeling time-to-event data to evaluate predictive biomarkers. IPD were sourced from electronic health records, using a target trial emulation approach, or digitized Kaplan-Meier curves. The model is illustrated using two examples: breast cancer with a hormone receptor biomarker, and metastatic colorectal cancer with the Kirsten Rat Sarcoma (KRAS) biomarker. ResultsThe model allowed for the estimation of treatment effects in two subgroups of patients defined by their biomarker status. Effectiveness of taxanes did not differ in hormone receptor positive and negative breast cancer patients. Epidermal growth factor receptor inhibitors were more effective than chemotherapy in KRAS wild type colorectal cancer patients but not in patients with KRAS mutant status. Use of IPD reduced uncertainty of the subgroup-specific treatment effect estimates by up to 49%. ConclusionUtilization of IPD allowed for more detailed evaluation of predictive biomarkers and cancer therapies and improved precision of the estimates compared to use of AD alone.

Taming Simulators: Challenges, Pathways and Vision for the Alignment of Large Language Models

As AI systems continue to advance in power and prevalence, ensuring alignment between humans and AI is crucial to prevent catastrophic outcomes. The greater the capabilities and generality of an AI system, combined with its development of goals and agency, the higher the risks associated with misalignment. While the concept of superhuman artificial general intelligence is still speculative, language models show indications of generality that could extend to generally capable systems. Regarding agency, this paper emphasizes the understanding of prediction-trained models as simulators rather than agents. Nonetheless, agents may emerge accidentally from internal processes, so-called simulacra, or deliberately through fine-tuning with reinforcement learning. As a result, the focus of alignment research shifts towards aligning simulacra, comprehending and mitigating mesa-optimization, and aligning agents derived from prediction-trained models. The paper outlines the challenges of aligning simulators and presents research directions based on this understanding. Additionally, it envisions a future where aligned simulators are critical in fostering successful human-AI collaboration. This vision encompasses exploring emulation approaches and the integration of simulators into cyborg systems to enhance human cognitive abilities. By acknowledging the risks associated with misaligned AI, delving into the concept of simulacra, and presenting strategies for aligning agents and simulacra, this paper contributes to the ongoing efforts to safeguard human values in developing and deploying AI systems.

Implementation of the trial emulation approach in medical research: a scoping review

BackgroundWhen conducting randomised controlled trials is impractical, an alternative is to carry out an observational study. However, making valid causal inferences from observational data is challenging because of the risk of several statistical biases. In 2016 Hernán and Robins put forward the ‘target trial framework’ as a guide to best design and analyse observational studies whilst preventing the most common biases. This framework consists of (1) clearly defining a causal question about an intervention, (2) specifying the protocol of the hypothetical trial, and (3) explaining how the observational data will be used to emulate it.MethodsThe aim of this scoping review was to identify and review all explicit attempts of trial emulation studies across all medical fields. Embase, Medline and Web of Science were searched for trial emulation studies published in English from database inception to February 25, 2021. The following information was extracted from studies that were deemed eligible for review: the subject area, the type of observational data that they leveraged, and the statistical methods they used to address the following biases: (A) confounding bias, (B) immortal time bias, and (C) selection bias.ResultsThe search resulted in 617 studies, 38 of which we deemed eligible for review. Of those 38 studies, most focused on cardiology, infectious diseases or oncology and the majority used electronic health records/electronic medical records data and cohort studies data. Different statistical methods were used to address confounding at baseline and selection bias, predominantly conditioning on the confounders (N = 18/49, 37%) and inverse probability of censoring weighting (N = 7/20, 35%) respectively. Different approaches were used to address immortal time bias, assigning individuals to treatment strategies at start of follow-up based on their data available at that specific time (N = 21, 55%), using the sequential trial emulations approach (N = 11, 29%) or the cloning approach (N = 6, 16%).ConclusionDifferent methods can be leveraged to address (A) confounding bias, (B) immortal time bias, and (C) selection bias. When working with observational data, and if possible, the ‘target trial’ framework should be used as it provides a structured conceptual approach to observational research.

Effectiveness of Specific Health Check-ups in Japan for the primary prevention of obesity-related diseases: a protocol for a target trial emulation

IntroductionConcerns about public health threats have shifted towards obesity-related, non-communicable diseases in both developed and developing countries. Since 2008, Japan has adopted a nationwide universal screening programme called Specific Health...

Semi-global exponential stability of stochastic nonlinear functional sampling systems by emulation approach

In this paper, the semi-global exponential stability is studied for the stochastic nonlinear functional systems by emulation of sampled-data controller. In consideration of the stochastic factor, a novel definition of the right and upper Dini’s derivative of the constructed functional is given. Based on this novel definition, combining with stochastic analysis techniques and Lyapunov method, some sufficient criteria for the semi-global exponential stability of the stochastic nonlinear functional systems by suitably fast sampling are given. As a special case, a class of stochastic nonlinear functional systems with multiple time delays is studied and two coefficient type theorems that lead to the semi-global exponential stability of the sampling system are derived. Finally, a practical application about parallel active suspension systems with time delay and stochastic disturbance is discussed at length and the corresponding numerical example is given to illustrate the feasibility of the theoretical results.

P143 The promise of emulating trials using registry data with an illustrative example: the effect of azithromycin on lung function

Randomised controlled trials (RCTs) are the gold standard for evaluating the effects of treatments. However, many questions relating to treatments are challenging to address in RCTs, particularly when involving multiple treatments. An alternative is to use observational data to ‘emulate’ a RCT. The trial emulation approach involves describing the protocol of a target trial (an RCT we would like to conduct if it were possible) and then emulating it using observational data. It applies the study design principles of RCTs to observational studies, combined with an analysis accounting for confounding. Recent years have seen uptake of trial emulation across many disease areas. Its applicability in CF remains unclear, in part due to the possibility of unmeasured confounding.

Can a Computer “Learn” Nonlinear Chromatography?: Experimental Validation of Physics-Based Deep Neural Networks for the Simulation of Chromatographic Processes

This article presents the capabilities of machine learning in addressing the challenges related to the accurate description of adsorption equilibria in the design of chromatographic processes. Our previously developed physics-based artificial neural network framework for adsorption and chromatography emulation (PANACHE) approach is extended to simulate the dynamics of chromatographic columns without using adsorption isotherms. The incorporation of underlying conservation laws in the form of a physics-constrained loss function during training enables PANACHE models to infer the complex dynamics of chromatographic columns, even without the knowledge of adsorption isotherms. The isotherm-agnostic abilities of PANACHE models are tested by considering two binary systems with complex adsorption equilibria: 1) mixed Langmuir M1-type binary system and 2) Tröger’s base enantiomers on Chiralpak AD that show inflection in the isotherm. For each case study, unique feed-forward deep neural networks with an input layer, five hidden layers, and an output layer are trained for two solute components using the elution data either from simulations or experiments. The results show that PANACHE models are successful in predicting the dynamics of binary solute mixtures in chromatographic columns even in the absence of adsorption isotherms, with prediction errors in the order of 10–2 in the measure of relative L2-norm, for most of the cases. The ability of PANACHE models to infer the adsorption isotherms is also demonstrated.

Comparative Safety of Gout Treatment Strategies on Cardiovascular Outcomes Using Observational Data: Clone-censor-weight Target Trial Emulation Approach.

We sought to examine the cardiovascular safety of intensive treat-to-target serum urate strategies for gout using Medicare claims data linked to electronic health record laboratory data. We selected patients with gout who initiated urate-lowering therapy. We emulated a hypothetical trial comparing the rate of major adverse cardiovascular events (nonfatal myocardial infarction, nonfatal stroke, and cardiovascular death) among seven different strategies over 24 months. Three aspects were considered in defining increasingly intensive strategies: (1) continuation of urate-lowering therapy, (2) serum urate monitoring, and (3) modification of urate-lowering therapy when serum urate >6 mg/dl. We applied the "clone-censor-weight" method to account for baseline and time-varying confounding. We identified 4402 patients with gout who initiated urate-lowering therapy (mean age 77; male 60%). During a total of 6611 person-years (PY) of follow-up under usual care, the rate of major cardiovascular events (first and recurrent) was 4.5/100 PY (95% CI = 4.0, 5.1). The rate ratios (RRs) suggested reductions (RR point estimates 0.88-0.84) compared with usual care. All 95% CIs were imprecise, but their upper bounds excluded substantial increase in RRs. RRs were closer to 1.0 for the analysis focusing on the first major adverse cardiovascular event during follow-up and on comparison to the strategy requiring continuation of urate-lowering therapy (but not necessarily titration). Our treatment strategy trial emulation did not find increased risk of major adverse cardiovascular events with intensive urate-lowering strategies. Results may provide reassurance of the cardiovascular safety of intensive treat-to-target serum urate strategies recommended by rheumatology societies.

Target Trial Emulation Using Hospital-Based Observational Data: Demonstration and Application in COVID-19.

Methodological biases are common in observational studies evaluating treatment effectiveness. The objective of this study is to emulate a target trial in a competing risks setting using hospital-based observational data. We extend established methodology accounting for immortal time bias and time-fixed confounding biases to a setting where no survival information beyond hospital discharge is available: a condition common to coronavirus disease 2019 (COVID-19) research data. This exemplary study includes a cohort of 618 hospitalized patients with COVID-19. We describe methodological opportunities and challenges that cannot be overcome applying traditional statistical methods. We demonstrate the practical implementation of this trial emulation approach via clone-censor-weight techniques. We undertake a competing risk analysis, reporting the cause-specific cumulative hazards and cumulative incidence probabilities. Our analysis demonstrates that a target trial emulation framework can be extended to account for competing risks in COVID-19 hospital studies. In our analysis, we avoid immortal time bias, time-fixed confounding bias, and competing risks bias simultaneously. Choosing the length of the grace period is justified from a clinical perspective and has an important advantage in ensuring reliable results. This extended trial emulation with the competing risk analysis enables an unbiased estimation of treatment effects, along with the ability to interpret the effectiveness of treatment on all clinically important outcomes.

On the Unreliability of Network Simulation Results FROM Mininet and iPerf

The networking research community and industry both use the Mininet network simulator to assess new network architectures and predict how new designs will perform. Mininet employs an emulation approach that uses concurrent processes running in a single computer instead of physical hardware. A user specifies a network consisting of network switches, routers, and host computers that have network interface connections and links connecting all the pieces. Mininet creates software artifacts to represent each of the network devices and allows application programs to send packets across the resulting network. Researchers often use the iPerf application to measure network performance. Many research papers report results from Mininet and iPerf and use the results to validate new designs for Software-Defined Networks (SDNs). However, there has been little investigation into the scenarios where these emulations can perform different than intended. The goal of this paper is to understand the edge cases of these emulation methods and understand the severity of these scenarios. This paper reports surprising anomalies in the results of Mininet and iPerf. We show that the choice of apparently valid configuration options can make the reported throughput completely invalid. Our initial discoveries focused on a complex simulation of a data center network. However, we were able to show that Mininet produces completely invalid results for a basic case: network traffic traveling across a single emulated link between two switches with no other network traffic. The paper makes recommendations for ways to configure Mininet to avoid some of the anomalies.

Comparing emulation methods for a high‐resolution storm surge model

AbstractRealistic simulations of complex systems are fundamental for climate and environmental studies. Large computer systems are often not sufficient to run sophisticated computational models for large numbers of different input settings. Statistical surrogate models, or emulators, are key tools enabling fast exploration of the simulator input space. Gaussian processes have become standard for computer simulator emulation. However, they require careful implementation to scale appropriately, motivating alternative methods more recently introduced. We present a comparison study of surrogates of the Sea, Lake, and Overland Surges from Hurricanes (SLOSH) simulator—the simulator of choice for government agencies—using four emulation approaches: BASS; BART; SEPIA; and RobustGaSP. SEPIA and RobustGaSP use Gaussian processes, BASS implements adaptive splines, and BART is based on ensembles of regression trees. We describe the four models and compare them in terms of computation time and predictive metrics. These surrogates use proven and distinct methodologies, are available through accessible software, and quantify prediction uncertainty. Our data cover millions of response values. We find that SEPIA and RobustGaSP provide exceptional predictive power, but cannot scale to emulate experiments as large as the one considered in this paper as effectively as BASS and BART.

Flexible and Low-Cost Emulation of Control Behaviors for Testing and Teaching of AC Microgrid

The fast development of distributed generations enables the microgrid a popular solution for the construction of the modern power grid, where the control behaviors of power electronics converters play a crucial role. Under this scenario, the emulation of microgrid control behaviors is becoming an emerging need for the testing and teaching of the AC microgrid. However, conventional approaches, such as the dynamic simulation test and the Power-Hardware-In-Loop, are still costly or bulky to flexibly recreate the correct characteristics of microgrid including different layers of controls and the interactions among multiple converters. The dynamic simulation test is bulky and costly to emulate various types of control behaviors since all physical components in the test system may need to be adjusted. The high cost of Power-Hardware-In-Loop is mainly caused by the high-performance real-time simulator and power amplifier. In this paper, a novel emulation system is proposed for the testing of the AC microgrid. A low-cost circuit configuration, which includes two face-to-face connected DC–AC converters and some passive loads, is introduced with the possibility of flexibly emulating most of the typical control schemes in an AC microgrid. In addition, a user interface for the real-time operation and measurement of the hardware platform is introduced on a host computer to further facilitate the testing process. Finally, various control schemes in microgrids, including the voltage control, current control, droop control, and secondary control, are validated in the experiment setup based on the proposed emulation approach.

Toward Effective Evaluation of Cyber Defense: Threat Based Adversary Emulation Approach

Attackers compromise organizations with increasingly sophisticated ways, such as Advanced Persistent Threat (APT) attackers. Usually, such attacks have the intention to exploit endpoints to gain access to critical data. For security controls and defense evaluation, organizations may employ offensive security activities. The most important one is penetration testing and red teaming, but such operations are usually resource exhaustive and extend over a longer period of time. Furthermore, traditional Venerability Assessment and Penetration Testing (VAPT) works effectively in the mitigation of known attacks but did not prove to be effective against stealthy attacks. VAPT considers the whole offsec as an acting problem but in reality, an attacker has to deal with uncertainty while conducting real-world attacks. In this paper, we are presenting an adversary emulation approach based on MITRE ATT&CK adversary emulation plan with consideration of planning as a major part of each attack phase. The approach utilizes stealthy attack vectors and paths to emulate adversary for defense evaluation. For effective defense evaluation, we picked more than 40 techniques from ATT&CK, deployed their mitigation on target machines, and then launched attacks against all those techniques. We show that attack paths and payloads generated using our approach are strong enough to evade security controls at endpoints. This approach provides a special environment for cyber defenders to think like adversary, and create new attack vectors and paths to evaluate organizational security preparedness. This process constructs a special environment to expand the attack landscape view and defense evaluation with minimal resources for the organization.

A Flexible Web Traffic Generator for the dimensioning of a 5G backhaul in NPN

Web-based services have nowadays aggregated the majority of applications, ranging from multimedia delivery to instant messaging, embracing requirements from different classes of users (i.e., gamers, streamers, professionals, etc.). The common factor is the need for connectivity everywhere and every time at high performance and quality, while the services traffic profiles and the users’ behaviour can be drastically different. This need is fully satisfied by the recent 5G mobile networks, where the coexistence of heterogeneous applications with different Quality of Service (QoS) levels is possible, defining new vertical markets and cutting-edge applications. In this context, we propose a Flexible Web Traffic Generator (FWTG) tool to model the dynamics of recent applications and user interactions in a realistic way, allowing the generation of real-time HTTP traffic and the possibility to inject it into real networks. FTWG represents a valid in-laboratory solution for the rapid configuration and applications performance evaluation of network slices in the context of 5G Non-Public Networks (NPN), pursuing an effective emulation approach. The advantage of this approach is that it allows the evaluation of real traffic response to different network configurations (and vice-versa), using components, equipment and network functions of the target network under test. After the description of the rationale and the methodology for defining FWTG models and their implementation, we defined a set of Key Performance Indicators (KPI) generated by the tool (e.g., number of requested objects and size, request rate, throughput, etc.). Then, we present initial validation tests on a Linux-based testbed representing the 5G-based scenarios of interest, emulating also a satellite backhaul as representative of a use-case requiring cost-effective dimensioning. Numerical outcomes of running FWTG in different configurations are then provided, considering a variation of the number of users sharing the same resources, their composition and traffic type, showing with concrete examples the usefulness and flexibility of FWTG in supporting 5G network configuration optimizations. Definitively, the tool is released as Open Source software on Gitlab.

A muon tracking algorithm for the Level 1 trigger in the CMS barrel muon chambers during HL-LHC

This contribution presents results on the Analytical Method (AM) algorithm for trigger primitive (TP) generation in the CMS Drift Tube (DT) chambers during the High Luminosity LHC operation (HL-LHC or LHC Phase II). The algorithm has been developed and validated both in software with an emulation approach, and through hardware implementation tests. The obtained performance on Phase II simulated data shows timing and position resolutions close to the ultimate performance of the DT chambers, with resilience to potential ageing situations. The firmware version has been implemented in the so-called AB7 (TwinMux), spare μTCA boards from the present DT system which host Xilinx Virtex 7 FPGAs, and included in a prototype chain of the HL-LHC electronics operated with real DT chambers during cosmic data taking. Agreement between the software emulation and the firmware implementation has been verified using different data samples, including a sample of real muons collected during 2016 data taking.