Epoch Persistence: Safe, Efficient, On-demand Rendering for Streaming Data

Abstract

Streaming data is always changing. Incremental results are incomplete, but often useful in their own right. Data analysis and rendering compete with each other for computational resources and access to core data structures because they are executing concurrently. This paper presents a set of data structures that ensures a visualization is internally consistent, and therefore interpretable. The data structures are based on persistent data structures, as commonly found in functional programming, but made more efficient by incorporating computational epochs. This paper also provides a definition for consistency that can be applied to visualizations. These data structures are used in the Stencil visualization system, which is used to benchmark the impact of epoch consistency. The net result of employing these data structures is that internally consistent incremental results are displayed as often as hardware allows without significantly impeding data loading.