Uncertainty Visualization Research Articles

Graphical uncertainty representations for ensemble predictions

We investigated how different graphical representations convey the underlying uncertainty distribution in ensemble predictions. In ensemble predictions, a set of forecasts is produced, indicating the range of possible future states. Adopting a use case from life sciences, we asked non-expert participants to compare ensemble predictions of the growth distribution of individual children to that of the normal population. For each individual child, the historical growth data of a set of 20 of its best matching peers was adopted as the ensemble prediction of the child’s growth curve. The ensemble growth predictions were plotted in seven different graphical formats (an ensemble plot, depicting all 20 forecasts and six summary representations, depicting the peer group mean and standard deviation). These graphs were plotted on a population chart with a given mean and variance. For comparison, we included a representation showing only the initial part of the growth curve without any future predictions. For 3 months old children that were measured at four occasions since birth, participants predicted their length at the age of 2 years. They compared their prediction to either (1) the population mean or to (2) a “normal” population range (the mean ± 2(standard deviation)). Our results show that the interpretation of a given uncertainty visualization depends on its visual characteristics, on the type of estimate required and on the user’s numeracy. Numeracy correlates negatively with bias (mean response error) magnitude (i.e. people with lower numeracy show larger response bias). Compared to the summary plots that yield a substantial overestimation of probabilities, and the No-prediction representation that results in quite variable predictions, the Ensemble representation consistently shows a lower probability estimation, resulting in the smallest overall response bias. The current results suggest that an Ensemble or “spaghetti plot” representation may be the best choice for communicating the uncertainty in ensemble predictions to non-expert users.

Toward the better modeling and visualization of uncertainty for streaming data

Streaming data can be found in many different scenarios, in which data are generated and arriving continuously. Sampling approaches have been proven as an effective means to cope with the sheer volume of the streaming data. However, sampling methods also introduce uncertainty, which can affect the reliability of subsequent analysis and visualization. In this paper, we propose a novel model called PDm and visualization named uncertainty tree to present uncertainty that arises from sampling streaming data. PDm is first introduced to characterize uncertainty of streaming data, and an optimization method is then proposed to minimize uncertainty. Uncertainty tree is further developed to enhance data understanding by visualizing uncertainty and revealing temporal patterns of streaming data. Lastly, a quantitative evaluation and real-world examples have been conducted to demonstrate the effectiveness and efficacy of the proposed techniques.

In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation.

Understanding and accounting for uncertainty is critical to effectively reasoning about visualized data. However, evaluating the impact of an uncertainty visualization is complex due to the difficulties that people have interpreting uncertainty and the challenge of defining correct behavior with uncertainty information. Currently, evaluators of uncertainty visualization must rely on general purpose visualization evaluation frameworks which can be ill-equipped to provide guidance with the unique difficulties of assessing judgments under uncertainty. To help evaluators navigate these complexities, we present a taxonomy for characterizing decisions made in designing an evaluation of an uncertainty visualization. Our taxonomy differentiates six levels of decisions that comprise an uncertainty visualization evaluation: the behavioral targets of the study, expected effects from an uncertainty visualization, evaluation goals, measures, elicitation techniques, and analysis approaches. Applying our taxonomy to 86 user studies of uncertainty visualizations, we find that existing evaluation practice, particularly in visualization research, focuses on Performance and Satisfaction-based measures that assume more predictable and statistically-driven judgment behavior than is suggested by research on human judgment and decision making. We reflect on common themes in evaluation practice concerning the interpretation and semantics of uncertainty, the use of confidence reporting, and a bias toward evaluating performance as accuracy rather than decision quality. We conclude with a concrete set of recommendations for evaluators designed to reduce the mismatch between the conceptualization of uncertainty in visualization versus other fields.

An Interactive Framework for Visualization of Weather Forecast Ensembles.

Numerical Weather Prediction (NWP) ensembles are commonly used to assess the uncertainty and confidence in weather forecasts. Spaghetti plots are conventional tools for meteorologists to directly examine the uncertainty exhibited by ensembles, where they simultaneously visualize isocontours of all ensemble members. To avoid visual clutter in practical usages, one needs to select a small number of informative isovalues for visual analysis. Moreover, due to the complex topology and variation of ensemble isocontours, it is often a challenging task to interpret the spaghetti plot for even a single isovalue in large ensembles. In this paper, we propose an interactive framework for uncertainty visualization of weather forecast ensembles that significantly improves and expands the utility of spaghetti plots in ensemble analysis. Complementary to state-of-the-art methods, our approach provides a complete framework for visual exploration of ensemble isocontours, including isovalue selection, interactive isocontour variability exploration, and interactive sub-region selection and re-analysis. Our framework is built upon the high-density clustering paradigm, where the mode structure of the density function is represented as a hierarchy of nested subsets of the data. We generalize the high-density clustering for isocontours and propose a bandwidth selection method for estimating the density function of ensemble isocontours. We present novel visualizations based on high-density clustering results, called the mode plot and the simplified spaghetti plot. The proposed mode plot visually encodes the structure provided by the high-density clustering result and summarizes the distribution of ensemble isocontours. It also enables the selection of subsets of interesting isocontours, which are interactively highlighted in a linked spaghetti plot for providing spatial context. To provide an interpretable overview of the positional variability of isocontours, our system allows for selection of informative isovalues from the simplified spaghetti plot. Due to the spatial variability of ensemble isocontours, the system allows for interactive selection and focus on sub-regions for local uncertainty and clustering re-analysis. We examine a number of ensemble datasets to establish the utility of our approach and discuss its advantages over state-of-the-art visual analysis tools for ensemble data.

A Hydraulic MultiModel Ensemble Framework for Visualizing Flood Inundation Uncertainty

AbstractWhile deterministic forecasts provide a single realization of potential inundation, the inherent uncertainty associated with forecasts also needs to be conveyed for improved decision support. The objective of this study was to develop an ensemble framework for the quantification and visualization of uncertainty associated with flood inundation forecast maps. An 11‐member ensemble streamflow forecast at lead times from 0 to 48 hr was used to force two hydraulic models to produce a multimodel ensemble. The hydraulic models used are (1) the International River Interface Cooperative along with Flow and Sediment Transport with Morphological Evolution of Channels solver and (2) the two‐dimensional Hydrologic Engineering Center‐River Analysis System. Uncertainty was quantified and augmented onto flood inundation maps by calculating statistical spread among the ensemble members. For visualization, a series of probability flood maps conveying the uncertainty in forecasted water extent, water depth, and flow velocity was disseminated through a web‐based decision support tool. The results from this study offer a framework for quantifying and visualizing model uncertainty in forecasted flood inundation maps.

Uncertainty in Phylogenetic Tree Estimates

ABSTRACTEstimating phylogenetic trees is an important problem in evolutionary biology, environmental policy, and medicine. Although trees are estimated, their uncertainties are generally discarded in statistical models for tree-valued data. Here, we explicitly model the multivariate uncertainty of tree estimates. We consider both the cases where uncertainty information arises extrinsically (through covariate information) and intrinsically (through the tree estimates themselves). The latter case is applicable to any procedure for tree estimation, and thus has broad relevance to the entire field of phylogenetics. The importance of accounting for tree uncertainty in tree space is demonstrated in two case studies. In the first instance, differences between gene trees are small relative to their uncertainties, while in the second, the differences are relatively large. Our main goal is visualization of tree uncertainty, and we demonstrate advantages of our method with respect to reproducibility, speed, and preservation of topological differences compared to visualization based on multidimensional scaling. The proposal highlights that phylogenetic trees are estimated in an extremely high-dimensional space, resulting in uncertainty information that cannot be discarded. Most importantly, it is a method that allows biologists to diagnose whether differences between gene trees are biologically meaningful or due to uncertainty in estimation.

Uncertainty visualization of gaze estimation to support operator-controlled calibration

In this paper, we investigate how visualization assets can support the qualitative evaluation of gaze estimation uncertainty. Although eye tracking data are commonly available, little has been done to visually investigate the uncertainty of recorded gaze information. This paper tries to fill this gap by using innovative uncertainty computation and visualization. Given a gaze processing pipeline, we estimate the location of this gaze position in the world camera. To do so we developed our own gaze data processing which give us access to every stage of the data transformation and thus the uncertainty computation. To validate our gaze estimation pipeline, we designed an experiment with 12 participants and showed that the correction methods we proposed reduced the Mean Angular Error by about 1.32 cm, aggregating all 12 participants’ results. The Mean Angular Error is 0.25° (SD=0.15°) after correction of the estimated gaze. Next, to support the qualitative assessment of this data, we provide a map which codes the actual uncertainty in the user point of view.

Uncertainty visualization of gaze estimation to support operator-controlled calibration.

In this paper, we investigate how visualization assets can support the qualitative evaluation of gaze estimation uncertainty. Although eye tracking data are commonly available, little has been done to visually investigate the uncertainty of recorded gaze information. This paper tries to fill this gap by using innovative uncertainty computation and visualization. Given a gaze processing pipeline, we estimate the location of this gaze position in the world camera. To do so we developed our own gaze data processing which give us access to every stage of the data transformation and thus the uncertainty computation. To validate our gaze estimation pipeline, we designed an experiment with 12 participants and showed that the correction methods we proposed reduced the Mean Angular Error by about 1.32 cm, aggregating all 12 participants' results. The Mean Angular Error is 0.25° (SD=0.15°) after correction of the estimated gaze. Next, to support the qualitative assessment of this data, we provide a map which codes the actual uncertainty in the user point of view.

Visualization of Task Time Schedule Uncertainty in Project Management

Visualization of Task Time Schedule Uncertainty in Project Management

Geovisualizing attribute uncertainty of interval and ratio variables: A framework and an implementation for vector data

Geovisualization of attribute uncertainty helps users to recognize underlying processes of spatial data. However, it still lacks an availability of uncertainty visualization tools in a standard GIS environment. This paper proposes a framework for attribute uncertainty visualization by extending bivariate mapping techniques. Specifically, this framework utilizes two cartographic techniques, choropleth mapping and proportional symbol mapping based on the types of attributes. This framework is implemented as an extension of ArcGIS in which three types of visualization tools are available: overlaid symbols on a choropleth map, coloring properties to a proportional symbol map, and composite symbols.

Depiction of uncertainty in the visually interpreted land cover data

Remote sensing data analysis to infer land cover and the subsequent modelling of land use change are subject to uncertainties, which may have an impact on the accuracy of future land-use predictions. Part of these uncertainties come from the visual interpretation of remote sensing data as during this phase a specific knowledge and expertise are crucial. Visual interpretation includes the meaning of the image content but also goes beyond what can be seen on the image to recognize spatial and landscape patterns. The quality of recognition depends on the expertise in image interpretation and visual perception. Based on this statement, it is necessary to visualize also the information about uncertainty. This paper describes different perspectives of uncertainty visualization on the example of visually interpreted aerial photographs in a mining area. The study is focused on visualization techniques for uncertainty awareness analysis of land cover data. This approach should contribute to better understanding, assessment and potential spreading of visual information of uncertainty and help to appraise the data critically. The goal of this paper is to raise uncertainty awareness among practitioners who deal with land cover data and stress the visualizations techniques as a more reliable means to assess the quality, and hence the uncertainty, of these data.

Uncertainty visualization for interactive assessment of stenotic regions in vascular structures

Stenosis refers to the thinning of the inner surface (lumen) of vascular structures. Detecting stenoses and correctly estimating their degree is crucial in clinical settings for proper treatment planning. Such a planning involves a visual assessment, which in case of vascular structures is frequently based on 3D visual representations of the vessels. However, since vessel segmentation is affected by various sources of errors and noise in the imaging and image processing pipeline, it is crucial to capture and visually convey the uncertainty in a 3D visual representation. Moreover, it is crucial to quantify how much this uncertainty affects the calculated stenotic degree, since different severities lead to different treatments. We propose a novel approach for visualizing the shape deviation of different probability levels in vascular data, where the probability levels are computed from a probabilistic segmentation approach. Our non-obstructive visual encoding is based on rendering a single opaque surface representing a probability level of the cumulative distribution function around the vessels’ centerline. The surface rendering is enhanced with cumulative information about other levels. To do so, we traverse the probability space by applying an iterative projection method both inwards and outward until we reach surface variability within a given margin. We capture the shape variability between the different probability levels using the lengths of the projection lines, the change in angular directions, and the distortion of a parametrization. They are visually encoded using color and texture mapping. Furthermore, we allow for an interactive selection of a region of interest that automatically calculates the stenotic degree and how much the uncertainty affects the most likely result. We analyze our approach in comparison to state-of-the-art methods with medical experts in a study using both real magnetic resonance (MR) and computed tomography (CT) angiography data of vertebral arteries with stenoses as well as on MR angiography data with synthetically added stenoses and stenotic uncertainties. We evaluate how well our approach can guide medical experts in their assessment of the uncertainty in vertebral stenoses.

Bayesian Methods for Quantifying and Reducing Uncertainty and Error in Forest Models

Forest models are tools for analysis and prediction of productivity and other services. Model outputs can only be useful if possible errors in inputs and model structure are recognized. However, errors cannot be quantified directly, making uncertainty inevitable. In this paper, we aim to clarify terminological confusion around the concepts of error and uncertainty and review current methods for addressing uncertainty in forest modelling. Modellers increasingly recognize that all uncertainties—in data, model inputs and model structure—can be represented using probability distributions. This has stimulated the use of Bayesian methods for quantifying and reducing uncertainty and error in models of forests and other vegetation. The Achilles’ heel of Bayesian methods has always been their computational demand, but solutions are being found. We conclude that future work will likely include (1) more use of Bayesian methods, (2) more use of hierarchical modelling, (3) replacement of model spin-up by Bayesian calibration, (4) more use of ensemble modelling and Bayesian model averaging, (5) new ways to account for model structural error in calibration, (6) better software for Bayesian calibration of complex models, (7) faster Markov chain Monte Carlo algorithms, (8) more use of model emulators, (9) novel uncertainty visualization techniques, (10) more use of graphical modelling and (11) more use of risk analysis.

Developing A Theoretical Framework of Task Complexity for Research on Visualization in Support of Decision Making Under Uncertainty

This paper describes initial validation of a theoretical framework to support research on the visualization of uncertainty. Two experiments replicated and extended this framework, illustrating how the manipulation of task complexity produces differences in performance. Additionally, using a combinatory metric of workload and performance, this framework provides a new metric for assessing uncertainty visualization. We describe how this work acts as a theoretical scaffold for examining differing forms of visualizations of uncertainty by providing a means for systematic variations in task context.

Visualization of Uncertainty Aids Spatial Judgments but Fails to Improve Metacognitive Efficiency

Uncertainty is an element of many decision-making tasks and inherently compromises performance. Research has found only equivocal evidence that uncertainty representations—displays that explicitly denote data quality—offset the performance costs of uncertainty. As yet, though, no work has examined the potential benefits of uncertainty displays to metacognition, display readers’ ability to assess the quality of their own decision-making processes. The current study examined the benefits of uncertainty visualization to first-order (Type 1) and metacognitive (Type 2) sensitivity in a spatial judgment task. Data revealed only small improvements in Type 1 and Type 2 sensitivity with visualized uncertainty displays, and gave no evidence of disproportionate gains to metacognition.

Bubble Treemaps for Uncertainty Visualization.

We present a novel type of circular treemap, where we intentionally allocate extra space for additional visual variables. With this extended visual design space, we encode hierarchically structured data along with their uncertainties in a combined diagram. We introduce a hierarchical and force-based circle-packing algorithm to compute Bubble Treemaps, where each node is visualized using nested contour arcs. Bubble Treemaps do not require any color or shading, which offers additional design choices. We explore uncertainty visualization as an application of our treemaps using standard error and Monte Carlo-based statistical models. To this end, we discuss how uncertainty propagates within hierarchies. Furthermore, we show the effectiveness of our visualization using three different examples: the package structure of Flare, the S&P 500 index, and the US consumer expenditure survey.

Color map design for visualization in flood risk assessment

ABSTRACTVisualizations of flood maps from simulation models are widely used for assessing the likelihood of flood hazards in spatial planning. The choice of a suitable type of visualization as well as efficient color maps is critical to avoid errors or bias when interpreting the data. Based on a review of previous flood uncertainty visualization techniques, this paper identifies areas of improvements and suggests criteria for the design of a task-specific color scale in flood map visualization. We contribute a novel color map design for visualizing probabilities and uncertainties from flood simulation ensembles. A user study encompassing 83 participants was carried out to evaluate the effects of this new color map on user’s decisions in a spatial planning task. We found that the type of visualization makes a difference when it comes to identification of non-hazardous sites in the flood risk map and when accepting risks in more uncertain areas. In comparison with two other existing visualization techniques, we observed that the new design was superior both in terms of task compliance and efficiency. In regions with uncertain flood statuses, users were biased toward accepting less risky locations with our new color map design.

Visualization of Delay Uncertainty and its Impact on Train Trip Planning: A Design Study

AbstractUncertainty about possible train delays has an impact on train trips, as the exact arrival time is unknown during trip planning. Delays can lead to missing a connecting train at the transfer station, or to coming too late to an appointment at the destination. Facing this uncertainty, the traveler may wish to use an earlier train or a different connection arriving well before the appointment. Currently, train trip planning is based on scheduled times of connections between two stations. Information about approximate delays is only available shortly before train departure. Although several visualization approaches can show temporal uncertainty, we are not aware of any visual design specifically supporting trip planning, which can show delay uncertainty and its impact on the connections. We propose and evaluate a visual design which extends train trip planning with delay uncertainty. It shows the scheduled train connections together with their expected train delays as well as their impacts on both the arrival time, and the potential of missing a transfer. The visualization also includes information about alternative connections in case of these critical transfers. In this way the user is able to judge which train connection is suitable for a trip. We conducted a user study with 76 participants to evaluate our design. We compared it to two alternative presentations that are prominent in Germany. The study showed that our design performs comparably well for tasks concerning train schedules. The additional uncertainty display as well as the visualization of alternative connections was appreciated and well understood. The participants were able to estimate when they would likely arrive at their destination despite possible train delays while they were unable to estimate this with existing presentations. The users would prefer to use the new design for their trip planning.

Visualizing the Uncertainty of Graph‐based 2D Segmentation with Min‐path Stability

AbstractThis paper presents a novel approach to visualize the uncertainty in graph‐based segmentations of scalar data. Segmentation of 2D scalar data has wide application in a variety of scientific and medical domains. Typically, a segmentation is presented as a single unambiguous boundary although the solution is often uncertain due to noise or blur in the underlying data as well as imprecision in user input. Our approach provides insight into this uncertainty by computing the “min‐path stability”, a scalar measure analyzing the stability of the segmentation given a set of input constraints. Our approach is efficient, easy to compute, and can be generally applied to either graph cuts or live‐wire (even partial) segmentations. In addition to its general applicability, our new approach to graph cuts uncertainty visualization improves on the time complexity of the current state‐of‐the‐art with an additional fast approximate solution. We also introduce a novel query enabled by our approach which provides users with alternate segmentations by efficiently extracting local minima of the segmentation optimization. Finally, we evaluate our approach and demonstrate its utility on data from scientific and medical applications.

Visualizing flood risk, enabling participation and supporting climate change adaptation using the Geoweb: the case of coastal communities in Nova Scotia, Canada

Impacts of climate change have been observed in natural systems and are expected to intensify in future decades (IPCC in Contribution of Working Groups I, II and III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change IPPC, Geneva, 2014). Governments are seeking to establish adaptive measures for minimizing the effects of climate change on vulnerable citizen groups, economic sectors and critical infrastructure (Adger et al. in Global Environ Change 15(2):77–86, 2005. doi: 10.1016/j.gloenvcha.2004.12.005 ; Smit and Wandel in Global Environ Change 16(3):282–292, 2006. doi: 10.1016/j.gloenvcha.2006.03.008 ). Coastal areas are particularly vulnerable to changing conditions due to rising sea levels and storm event intensification that produce new flood exposures (Richards and Daigle in Government of Prince Edward Island, Halifax, Nova Scotia, 2011 http://www.gov.pe.ca/photos/original/ccscenarios.pdf ). However, communities oftentimes lack access to locally-relevant climate change information that can support adaptation planning. This research introduces the use of a Geoweb tool for supporting local climate change adaptation efforts in coastal Canadian communities. The Geoweb tool (called “AdaptNS”) is a web-based visualization tool that displays interactive flood exposure maps generated using local climate change projections of sea level rise and storm surge impacts between the years 2000 and 2100. AdaptNS includes participatory features that allow users to identify and share specific locations to protect against present and future coastal flood events. By soliciting feedback from community members, AdaptNS is shown to support local adaptation through the provision of flood exposure visuals, as a platform for identifying adaptation priorities, and as an avenue to communicate local risks to external entities that could facilitate local adaptation initiatives (e.g. upper levels of government). Future Geoweb research directions include improving the visualization of climate change projection uncertainties, the expansion of informational and participation capabilities, and understanding the potential for long-term adoption of Geoweb tools in adaptation decision-making.