Uncertainty Metrics Research Articles

Discerning the Management-Relevant Ecology and Distribution of Sea Pens (Cnidaria: Pennatulacea) in Norway and Beyond

Sea pens are considered to be of conservation relevance according to multiple international legislations and agreements. Consequently, any information about their ecology and distribution should be of use to management decision makers. This study aims to provide such information about six taxa of sea pen in Norwegian waters [Funiculina quadrangularis (Pallas, 1766), Halipteris spp., Kophobelemnon stelliferum (Müller, 1776), Pennatulidae spp., Umbellula spp., and Virgulariidae spp.]. Data exploration techniques and ensembled species distribution modelling (SDM) are applied to video observations obtained by the MAREANO project between 2006 and 2020. Norway-based ecological profiles and predicted distributions are provided and discussed. External validations and uncertainty metrics highlight model weaknesses (overfitting, limited training/external observations) and consistencies relevant to marine management. Comparison to international literature further identifies globally relevant findings: (a) disparities in the environmental profile of F. quadrangularis suggest differing “realised niches” in different locations, potentially highlighting this taxon as particularly vulnerable to impact, (b) none of the six sea pen taxa were found to consistently co-occur, instead partially overlapping environmental profiles suggests that grouping taxa as “sea pens and burrowing megafauna” should be done with caution post-analyses only, (c) higher taxonomic level groupings, while sometimes necessary due to identification issues, result in poorer quality predictive models and may mask the occurrence of rarer species. Community-based groupings are therefore preferable due to confirmed shared ecological niches while greater value should be placed on accurate species ID to support management efforts.

Decision making of autonomous vehicles in lane change scenarios: Deep reinforcement learning approaches with risk awareness

Driving safety is the most important element that needs to be considered for autonomous vehicles (AVs). To ensure driving safety, we proposed a lane change decision-making framework based on deep reinforcement learning to find a risk-aware driving decision strategy with the minimum expected risk for autonomous driving. Firstly, a probabilistic-model based risk assessment method was proposed to assess the driving risk using position uncertainty and distance-based safety metrics. Then, a risk aware decision making algorithm was proposed to find a strategy with the minimum expected risk using deep reinforcement learning. Finally, our proposed methods were evaluated in CARLA in two scenarios (one with static obstacles and one with dynamically moving vehicles). The results show that our proposed methods can generate robust safe driving strategies and achieve better driving performances than previous methods.

Towards targeted ultrasound-guided prostate biopsy by incorporating model and label uncertainty in cancer detection.

Systematic prostate biopsy is widely used for cancer diagnosis. The procedure is blind to underlying prostate tissue micro-structure; hence, it can lead to a high rate of false negatives. Development of a machine-learning model that can reliably identify suspicious cancer regions is highly desirable. However, the models proposed to-date do not consider the uncertainty present in their output or the data to benefit clinical decision making for targeting biopsy. We propose a deep network for improved detection of prostate cancer in systematic biopsy considering both the label and model uncertainty. The architecture of our model is based on U-Net, trained with temporal enhanced ultrasound (TeUS) data. We estimate cancer detection uncertainty using test-time augmentation and test-time dropout. We then use uncertainty metrics to report the cancer probability for regions with high confidence to help the clinical decision making during the biopsy procedure. Experiments for prostate cancer classification includes data from 183 prostate biopsy cores of 41 patients. We achieve an area under the curve, sensitivity, specificity and balanced accuracy of 0.79, 0.78, 0.71 and 0.75, respectively. Our key contribution is to automatically estimate model and label uncertainty towards enabling targeted ultrasound-guided prostate biopsy. We anticipate that such information about uncertainty can decrease the number of unnecessary biopsy with a higher rate of cancer yield.

Inferring Bias and Uncertainty in Camera Calibration

Accurate camera calibration is a precondition for many computer vision applications. Calibration errors, such as wrong model assumptions or imprecise parameter estimation, can deteriorate a system’s overall performance, making the reliable detection and quantification of these errors critical. In this work, we introduce an evaluation scheme to capture the fundamental error sources in camera calibration: systematic errors (biases) and uncertainty (variance). The proposed bias detection method uncovers smallest systematic errors and thereby reveals imperfections of the calibration setup and provides the basis for camera model selection. A novel resampling-based uncertainty estimator enables uncertainty estimation under non-ideal conditions and thereby extends the classical covariance estimator. Furthermore, we derive a simple uncertainty metric that is independent of the camera model. In combination, the proposed methods can be used to assess the accuracy of individual calibrations, but also to benchmark new calibration algorithms, camera models, or calibration setups. We evaluate the proposed methods with simulations and real cameras.

Refined protocols to mitigate user-induced uncertainty for ADCP moving-boat discharge measurement in irrigation canals

A Teledyne RD Instruments StreamPro ADCP was used to quantify flow rates between 0.5 and 8 m3/s in man-made irrigation canals with the aim of determining best practices for the method of deployment of an ADCP using a moving boat when computing mean steady discharge. Prescription for total sampling duration is refined here as the product of the duration of a single measurement transect and the total number of transects taken in reciprocal directions. The purpose of this enhancement is to lessen measurement uncertainty arising from insufficient resolution of the turbulent fluctuating velocity signal as determined by the sampling duration chosen by the user. These guidelines also estimate the point of diminishing returns for additional data collection in order to optimize time efficiency for measurement of mean steady discharge. Suggested protocols developed from field experimentation indicate that a significant reduction in the values of uncertainty metrics can be accomplished utilizing a remotely-operated tagline deployment method with a minimum transect duration of 240 s, equivalent to 240 velocity ensembles of an ADCP measuring at 1 Hz. Concurrently, at least four total transects in reciprocal directions should be measured. This expands and refines, in application to small-to-moderate sized irrigation canals, the currently held 720 s minimum total sampling duration guideline established by the USGS. These findings will equip ADCP users across a wide range of experience with further guidance to achieve more reliable direct flow quantification and hydraulic structure calibration.

Combining distribution‐based neural networks to predict weather forecast probabilities

AbstractThe success of deep learning techniques over the last decades has opened up a new avenue of research for weather forecasting. Here, we take the novel approach of using a neural network to predict full probability density functions at each point in space and time rather than a single output value, thus producing a probabilistic weather forecast. This enables the calculation of both uncertainty and skill metrics for the neural network predictions, and overcomes the common difficulty of inferring uncertainty from these predictions. This approach is data‐driven and the neural network is trained on the WeatherBench dataset (processed ERA5 data) to forecast geopotential and temperature 3 and 5 days ahead. Data exploration leads to the identification of the most important input variables. In order to increase computational efficiency, several neural networks are trained on small subsets of these variables. The outputs are then combined through a stacked neural network, the first time such a technique has been applied to weather data. Our approach is found to be more accurate than some coarse numerical weather prediction models and as accurate as more complex alternative neural networks, with the added benefit of providing key probabilistic information necessary for making informed weather forecasts.

Bayesian U-Net: Estimating Uncertainty in Semantic Segmentation of Earth Observation Images

In recent years, numerous deep learning techniques have been proposed to tackle the semantic segmentation of aerial and satellite images, increase trust in the leaderboards of main scientific contests and represent the current state-of-the-art. Nevertheless, despite their promising results, these state-of-the-art techniques are still unable to provide results with the level of accuracy sought in real applications, i.e., in operational settings. Thus, it is mandatory to qualify these segmentation results and estimate the uncertainty brought about by a deep network. In this work, we address uncertainty estimations in semantic segmentation. To do this, we relied on a Bayesian deep learning method, based on Monte Carlo Dropout, which allows us to derive uncertainty metrics along with the semantic segmentation. Built on the most widespread U-Net architecture, our model achieves semantic segmentation with high accuracy on several state-of-the-art datasets. More importantly, uncertainty maps are also derived from our model. While they allow for the performance of a sounder qualitative evaluation of the segmentation results, they also include valuable information to improve the reference databases.

AVA: A Financial Service Chatbot Based on Deep Bidirectional Transformers

We develop a chatbot using deep bidirectional transformer (BERT) models to handle client questions in financial investment customer service. The bot can recognize 381 intents, decides when to say I don’t know, and escalate escalation/uncertain questions to human operators. Our main novel contribution is the discussion about the uncertainty measure for BERT, where three different approaches are systematically compared with real problems. We investigated two uncertainty metrics, information entropy and variance of dropout sampling, in BERT, followed by mixed-integer programming to optimize decision thresholds. Another novel contribution is the usage of BERT as a language model in automatic spelling correction. Inputs with accidental spelling errors can significantly decrease intent classification performance. The proposed approach combines probabilities from masked language model and word edit distances to find the best corrections for misspelled words. The chatbot and the entire conversational AI system are developed using open-source tools and deployed within our company’s intranet. The proposed approach can be useful for industries seeking similar in-house solutions in their specific business domains. We share all our code and a sample chatbot built on a public data set on GitHub.

Evaluating two land surface models for Brazil using a full carbon cycle benchmark with uncertainties

AbstractForecasts of tropical ecosystem C cycling diverge among models due to differences in simulation of internal processes such as turnover, or transit times, of carbon pools. Estimates of these processes for the recent past are needed to test model representations, and so build confidence in model forecasts within and across biomes. Here, we evaluate carbon cycle process representation in two land surface models [Joint UK Land Environment Simulator (JULES) and Integrated Model of Land Surface Processes (INLAND)] for the period 2001–10 across Brazilian biomes. Model outputs are evaluated using the ILAMB system. Probabilistic benchmarking data were created using the carbon data model framework that assimilates observational times series of leaf area index and maps of woody biomass and soil C. New custom uncertainty metrics assess if models are within benchmark uncertainties. Simulations are better in homogeneous areas of vegetation type, and are less robust at ecotones between biomes, likely due to disturbance effects and parameter errors. Gross biosphere‐atmosphere fluxes are robustly modelled across Brazil. However, benchmark uncertainty is too high on net ecosystem exchange to provide an accurate evaluation of the models. The LSMs have significant differences in internal carbon allocation and the dynamics of the different C pools. JULES models dead C stocks more accurately while living C stocks are best resolved for INLAND. JULES' over‐estimate of the C wood pool results from over‐estimation of both inputs to wood and the transit time of wood. INLAND's under‐estimate of dead C stocks arises from an under‐estimate of the transit time of dead organic matter. The models are better at simulating annual averages than seasonal variation of fluxes. Analyses of monthly net C exchanges show that INLAND correctly simulates seasonality, but over‐estimates amplitudes, whereas JULES correctly simulates the annual amplitudes, but is out of phase with the benchmark.

Applicability of the M5 to Forecasting at Walmart

The M5 Forecasting Competition, the fifth in the series of forecasting competitions organized by Professor Spyros Makridakis and the Makridakis Open Forecasting Center at the University of Nicosia, was an extremely successful event. This competition focused on both the accuracy and uncertainty of forecasts and leveraged actual historical sales data provided by Walmart. This has led to the M5 being a unique competition that closely parallels the difficulties and challenges associated with industrial applications of forecasting. Like its precursor the M4, many interesting ideas came from the results of the M5 competition which will continue to push forecasting in new directions.In this article we discuss four topics around the practitioners view of the application of the competition and its results to the actual problems we face. First, we examine the data provided and how it relates to common difficulties practitioners must overcome. Secondly, we review the relevance of the accuracy and uncertainty metrics associated with the competition. Third, we discuss the leading solutions and their implications to forecasting at a company like Walmart. We then close with thoughts about a future M6 competition and further enhancements that can be explored.

Evaluation of Three Low-Cost Particulate Matter (PM2.5) Sensors for Ambient and High Exposure Conditions in Arba Minch, Ethiopia

The burden of disease from ambient and indoor air pollution is highest in low-income countries, while their resources for monitoring air pollutants are the lowest. PM2.5 is the primary indicator of air pollution. Reference monitors of PM2.5 are expensive, but there is an increased use of low-cost sensors (LCS). Three LCS, the UCB-PATS+ (PATS), Airvisual Pro (IQAV) and Sensirion SPS30 (SPSA) are being used in Arba Minch, Ethiopia, but their quality has not yet been evaluated under circumstances common to low-income countries, and the variety of metrics used in evaluation studies make comparisons difficult. This study aims to evaluate the three LCS under circumstances encountered in Arba Minch, with metrics commonly used and officially prescribed. Measurements were conducted with the LCS at 2 ambient and 4 high exposure (kitchen) concentrations, and at four of those locations with the gravimetric reference method as well. The quality of the three LCS was evaluated within identical, with reference, and between different types, with commonly reported (regression slope and R2) and officially prescribed (Pearson correlation, bias, accuracy, expanded uncertainty) metrics. The SPSA has low within variation in both ambient and high-exposure situations, meets official requirements compared to the reference, and shows a stable bias across different time and concentration levels. The IQAV and PATS within variations are not up to official standards but show strong linear associations. The IQAVs as a group, and PATSs individually, meet official reference requirements at daily level. Between comparison reveals that all LCS show strong linear associations even at 10-minute average level. For SPSA the association is similar across all ranges, and for the others the association is strong when different ranges are taken into account. Generally, all LCS are a good alternative for expensive reference methods. The strong linear associations suggest the possibility of correcting LCS measurement data based on other studies’ results and based on other LCS, across different concentration ranges. Projects with a budget of $600 can already supply 10 measurement locations. Higher-budget projects can contribute to the quality of low-budget projects when they do not only use expensive monitors, but also LCS at the same location.

Detect, Consolidate, Delineate: Scalable Mapping of Field Boundaries Using Satellite Images

Digital agriculture services can greatly assist growers to monitor their fields and optimize their use throughout the growing season. Thus, knowing the exact location of fields and their boundaries is a prerequisite. Unlike property boundaries, which are recorded in local council or title records, field boundaries are not historically recorded. As a result, digital services currently ask their users to manually draw their field, which is time-consuming and creates disincentives. Here, we present a generalized method, hereafter referred to as DECODE (DEtect, COnsolidate, and DElinetate), that automatically extracts accurate field boundary data from satellite imagery using deep learning based on spatial, spectral, and temporal cues. We introduce a new convolutional neural network (FracTAL ResUNet) as well as two uncertainty metrics to characterize the confidence of the field detection and field delineation processes. We finally propose a new methodology to compare and summarize field-based accuracy metrics. To demonstrate the performance and scalability of our method, we extracted fields across the Australian grains zone with a pixel-based accuracy of 0.87 and a field-based accuracy of up to 0.88 depending on the metric. We also trained a model on data from South Africa instead of Australia and found it transferred well to unseen Australian landscapes. We conclude that the accuracy, scalability and transferability of DECODE shows that large-scale field boundary extraction based on deep learning has reached operational maturity. This opens the door to new agricultural services that provide routine, near-real time field-based analytics.

Detecting Climate Signals Using Explainable AI With Single‐Forcing Large Ensembles

AbstractIt remains difficult to disentangle the relative influences of aerosols and greenhouse gases on regional surface temperature trends in the context of global climate change. To address this issue, we use a new collection of initial‐condition large ensembles from the Community Earth System Model version 1 that are prescribed with different combinations of industrial aerosol and greenhouse gas forcing. To compare the climate response to these external forcings, we adopt an artificial neural network (ANN) architecture from previous work that predicts the year by training on maps of near‐surface temperature. We then utilize layer‐wise relevance propagation (LRP) to visualize the regional temperature signals that are important for the ANN's prediction in each climate model experiment. To mask noise when extracting only the most robust climate patterns from LRP, we introduce a simple uncertainty metric that can be adopted to other explainable artificial intelligence (AI) problems. We find that the North Atlantic, Southern Ocean, and Southeast Asia are key regions of importance for the neural network to make its prediction, especially prior to the early‐21st century. Notably, we also find that the ANN predictions based on maps of observations correlate higher to the actual year after training on the large ensemble experiment with industrial aerosols held fixed to 1920 levels. This work illustrates the sensitivity of regional temperature signals to changes in aerosol forcing in historical simulations. By using explainable AI methods, we have the opportunity to improve our understanding of (non)linear combinations of anthropogenic forcings in state‐of‐the‐art global climate models.

Uncertainty-Aware Unsupervised Domain Adaptation in Object Detection

Unsupervised domain adaptive object detection aims to adapt detectors from a labelled source domain to an unlabelled target domain. Most existing works take a two-stage strategy that first generates region proposals and then detects objects of interest, where adversarial learning is widely adopted to mitigate the inter-domain discrepancy in both stages. However, adversarial learning may impair the alignment of well-aligned samples as it merely aligns the global distributions across domains. To address this issue, we design an uncertainty-aware domain adaptation network (UaDAN) that introduces conditional adversarial learning to align well-aligned and poorly-aligned samples separately in different manners. Specifically, we design an uncertainty metric that assesses the alignment of each sample and adjusts the strength of adversarial learning for well-aligned and poorly-aligned samples adaptively. In addition, we exploit the uncertainty metric to achieve curriculum learning that first performs easier image-level alignment and then more difficult instance-level alignment progressively. Extensive experiments over four challenging domain adaptive object detection datasets show that UaDAN achieves superior performance as compared with state-of-the-art methods.

Experimental Study of Thermal Management Characteristics of Mass via Arrays

Mass via arrays (MVAs) are intersubstrate thermal management structures that utilize thermal meta-material design principles to target localized hot spots and extreme variation in the temperature profile of electronic systems. MVAs have shown promise for integration into electronic systems as passive thermal management techniques. Theoretical analysis has shown that, when properly designed, MVAs and MVA-like structures provide control over how the heat is transferred in an electronic substrate. While the theoretical confirmation of this behavior is promising, experimental results are important to prove and strengthen MVA design principles. In this work, MVAs are implemented using an industry-standard printed circuit board (PCB) fabrication technique with Rogers 4350B (RO4350) material. The design structures are first theoretically modeled using a multilayer RO4350 stack-up in which equivalent thermal properties are studied for a variety of MVA designs. Uncertainty metrics are integrated into this model, and an iterative, Monte Carlo process is used to simulate variability in MVA performance. Next, these theoretical structures are implemented in conventional PCBs. Sample devices are chosen at random, and the heat spreading characteristics of the devices are measured using a thermal imaging camera. The results of these measurements confirm the theoretical baseline that an MVA structure provides improved thermal management characteristics relative to conventional thermal via array (TVA) structures.

Sparsity Increases Uncertainty Estimation in Deep Ensemble

Deep neural networks have achieved almost human-level results in various tasks and have become popular in the broad artificial intelligence domains. Uncertainty estimation is an on-demand task caused by the black-box point estimation behavior of deep learning. The deep ensemble provides increased accuracy and estimated uncertainty; however, linearly increasing the size makes the deep ensemble unfeasible for memory-intensive tasks. To address this problem, we used model pruning and quantization with a deep ensemble and analyzed the effect in the context of uncertainty metrics. We empirically showed that the ensemble members’ disagreement increases with pruning, making models sparser by zeroing irrelevant parameters. Increased disagreement im-plies increased uncertainty, which helps in making more robust predictions. Accordingly, an energy-efficient compressed deep ensemble is appropriate for memory-intensive and uncertainty-aware tasks.

Strain Energy Decay Predicts Elastic Registration Accuracy From Intraoperative Data Constraints

Image-guided intervention for soft tissue organs depends on the accuracy of deformable registration methods to achieve effective results. While registration techniques based on elastic theory are prevalent, no methods yet exist that can prospectively estimate registration uncertainty to regulate sources and mitigate consequences of localization error in deforming organs. This paper introduces registration uncertainty metrics based on dispersion of strain energy from boundary constraints to predict the proportion of target registration error (TRE) remaining after nonrigid elastic registration. These uncertainty metrics depend on the spatial distribution of intraoperative constraints provided to registration with relation to patient-specific organ geometry. Predictive linear and bivariate gamma models are fit and cross-validated using an existing dataset of 6291 simulated registration examples, plus 699 novel simulated registrations withheld for independent validation. Average uncertainty and average proportion of TRE remaining after elastic registration are strongly correlated ( r = 0.78 ), with mean absolute difference in predicted TRE equivalent to 0.9 ± 0.6 mm (cross-validation) and 0.9 ± 0.5 mm (independent validation). Spatial uncertainty maps also permit localized TRE estimates accurate to an equivalent of 3.0 ± 3.1 mm (cross-validation) and 1.6 ± 1.2 mm (independent validation). Additional clinical evaluation of vascular features yields localized TRE estimates accurate to 3.4 ± 3.2 mm. This work formalizes a lower bound for the inherent uncertainty of nonrigid elastic registrations given coverage of intraoperative data constraints, and demonstrates a relation to TRE that can be predictively leveraged to inform data collection and provide a measure of registration confidence for elastic methods.

Regional ensemble modeling reduces uncertainty for digital soil mapping

Recent country and continental-scale digital soil mapping efforts have used a single model to predict soil properties across large regions. However, different ecophysiographic regions within large-extent areas are likely to have different soil-landscape relationships so models built specifically for these regions may more accurately capture these relationships relative to a ‘global’ model. We ask the question: Is a single ‘global’ model sufficient or are regionally-specific models useful for accurate digital soil mapping? We test this question by modeling soil depth classes across the 432,000 km2 upper Colorado River Basin in the Western USA using a single global model, multiple ecophysiographic models, and ensembles of the ecophysiographic models.Effective soil depth class observations (n = 12,194) were derived from multiple soil databases. Fifty-seven environmental covariates were derived from a 30 m digital elevation model, climate data, satellite imagery, and aeroradiometric data. Three independent land classifications were used to stratify the area. Two expert-derived land classifications, USDA Major Land Resource Areas (MLRA) and US-EPA Level III ecoregions, divided the study area into multiple ecophysiographic regions based on vegetation and broad-scale physiographic differences. The third land classification divided the study area into broad landforms.Soil depth observations were split into separate training (n = 10,470) and validation (n = 1,724) datasets. First, a ‘global’ random forest model was used to model soil depth classes using all training observations and covariates. ‘Global’ denotes a model built with all training data across the extent of the area, not a model at world extent. Second, the land classifications were used to subset the observations into ecophysiographic sub-datasets and random forest models were refit for each region. Models fit by ecophysiographic region are referred to as regional models. Thirdly, predictions from each regional model were fused into regional-ensemble models. Accuracy, Brier scores, and Shannon’s entropy were used to compare model accuracy and uncertainty. Regional ecophysiographic models were also compared to models built for geographic areas that were defined solely to be approximately equal in area. Training dataset density and the imbalance ratio were investigated to determine if data characteristics influenced regional accuracy/uncertainty metrics.Accuracy for the global model using the validation set was 62.8%. Regional model accuracies ranged between 56.1% and 75.0%. We found: 1) useful inter-regional differences in global model accuracy were revealed when the global model was validated by region, 2) no consistent relationship between training observation density and accuracy/uncertainty metrics, 3) no meaningful differences in accuracy and uncertainty metrics between physiographic and geographic regions, 4) ensembles of regionally-specific models were approximately as accurate as global models, and 5) both region-specific models and ensembles of regional models were less uncertain than the global model. Overall, we recommend the use of soil depth class predictions made from MLRA regional ensemble models because this prediction had higher accuracy than the ecoregion ensemble model prediction, but lower uncertainty than both the global model and the landform ensemble model predictions. We answer our question: Ensembles of regionally-specific models are approximately as accurate as global models, but result in less uncertainty.

A nearest neighbor-based active learning method and its application to time series classification

• Locally estimated prediction uncertainty and sampling utility metrics are introduced for active learning. • An effective batch-mode active learning algorithm is introduced based on the proposed metrics. • Experimental results on two time series data demonstrate the effectiveness of the proposed method. Although the one nearest neighbor approach is widely used in time series classification , its successful performance requires enough labeled data, which is often difficult to obtain due to a high labeling cost. This article considers a practical classification scenario in which labeled data are scant but unlabeled data are plenty, and a limited budget for the annotating task is provided. For an effective classification with limited resources, we propose a nearest neighbor-based sampling strategy for active learning. The proposed approach uses highly local information to measure the uncertainty and utility of an unlabeled instance and is applicable to extremely sparse labeled data. Furthermore, we extend the proposed approach to batch mode active learning to select a batch of informative samples at each sampling iteration. Experimental results on the WAFER and ECG5000 data sets demonstrate the effectiveness of the proposed algorithm as compared with other nearest neighbor-based approaches.

Uncertainty in runup predictions on natural beaches using XBeach nonhydrostatic

The wave time series that forces phase-resolving models is a source of model uncertainty that can propagate into wave runup predictions when the wave phase information is unknown. The effect of beach morphology on the propagation of this intrinsic uncertainty related to wave randomness (Ui) is largely unexplored. Here, we quantify the importance of uncertainty in wave runup at a dissipative (TS), intermediate (DU) and reflective (DE) beach using the phase-resolving model XBeach nonhydrostatic. Intrinsic uncertainty is evaluated with respect to the effect of free model parameters and of an evolving bed. Uncertainty contributions due to model equations and numerical methods are not assessed. The model is forced at each beach with 100 different wave time series corresponding to the same storm condition. Computed uncertainty metrics reveal that Ui is most important at TS, where extreme runup (R2%) of a single simulation can deviate up to 31% from the 100-simulation ensemble average. Intrasite differences in R2% can be attributed to the predominance of low-frequency swash (SLF), which is more sensitive to Ui than high-frequency swash and setup. Additional simulations using the suggested range of breaking and bed friction coefficients demonstrate that Ui produces more variability in SLF than the two most important model parameters, whereas the parametric range is more important for high-frequency wave height. Furthermore, morphodynamic simulations show that Ui produces a larger variability in R2% and SLF than an evolving bed under a 7-hr storm. Not surprisingly, Ui causes larger variability in morphologic than hydrodynamic predictions due to morphologic feedback. Trends can reverse from swash zone accretion to erosion (TS) by using a wave time series with differently distributed phase. Ensemble simulations are recommended for forecasts based on wave time series with randomly distributed phase.