An empirical characterization of event sourced systems and their schema evolution — Lessons from industry

Michiel Overeem,Marten Spoor,Slinger Jansen,Sjaak Brinkkemper

doi:10.1016/j.jss.2021.110970

Abstract

Event sourced systems are increasing in popularity because they are reliable, flexible, and scalable. In this article, we point a microscope at a software architecture pattern that is rapidly gaining popularity in industry, but has not received as much attention from the scientific community. We do so through constructivist grounded theory, which proves a suitable qualitative method for extracting architectural knowledge from practitioners.Based on the discussion of 19 event sourced systems we explore the rationale for and the context of the event sourcing pattern. A description of the pattern itself and its relation to other patterns as discussed with practitioners is given. The description itself is grounded in the experience of 25 engineers, making it a reliable source for both new practitioners and scientists. We identify five challenges that practitioners experience: event system evolution, the steep learning curve, lack of available technology, rebuilding projections, and data privacy. For the first challenge of event system evolution, we uncover five tactics and solutions that support practitioners in their design choices when developing evolving event sourced systems: versioned events, weak schema, upcasting, in-place transformation, and copy-and-transform.

Highlights

Software systems are increasing in complexity, used in increasingly critical processes, and serve increasing numbers of end-users
event schema evolution (ESE) Technique 3: Upcasting - This technique is well known to event sourcing practitioners and described by Betts et al
ESE Technique 4: In-Place Transformation - This technique updates events to resemble the new schema, and forces event sourced systems (ESSs) to forgo of immutability

Summary

Introduction

Software systems are increasing in complexity, used in increasingly critical processes, and serve increasing numbers of end-users. Architectural patterns enable engineers to built these systems using knowledge acquired by other engineers. Influential books such as Patterns of Enterprise Application Architecture by Fowler (2002) and Enterprise Integration Patterns by Hohpe and Woolf (2004) demonstrate the impact of pattern descriptions on software engineering. The foundational idea of event sourcing is the domain event as described by Evans (2015). His seminal book on Domain-Driven Design (DDD), does not mention the pattern. Vernon (2013) describes event sourcing only briefly in his book on the implementation of various DDD patterns. Recent academic literature (Erb, 2019; Zhong et al, 2019) shows an interest in applying event sourcing for research projects

Objectives

Results

Discussion

Conclusion