PALADIN: A process-based constraint language for data validation

Abstract

In many processes, ranging from medical treatments to supply chains and employee management, there is a growing need to gather information with the objective of enhancing the efficiency of the process in question. Often, the information gathered from different stages of a process resides in disparate storage systems, necessitating an information fusion process. Post-fusion, it is common to encounter data inconsistencies that hinder an accurate analysis. Unfortunately, existing data validation languages lack the capability to model constraints across stages, making it challenging to identify inconsistencies without introducing artificial elements. This paper introduces PALADIN, a language which has been specifically designed to allow the formulation of constraints in the realm of process-based data, i.e., data points that evolve through various stages of a process with constraints that change according to the stage at which a data point is. PALADIN is data model-agnostic, which means it is not specific to any particular data model or format. This paper provides a formalization, together with implementation details of PALADIN validators, and their validation through a use case. Furthermore, PALADIN is subjected to an empirical evaluation across 20 datasets, including 18 synthetically generated ones that are openly shared with the scientific community. The experimentation involves 53 testbeds, and shows that PALADIN reduces the data validation time compared with other languages that are not tailored for process-based data—achieving a speed-up of up to five times. The results also highlight the impact of parameters such as the type of data integration system, the number of integrity constraints, and the dataset size on the validation time of PALADIN shape schemas.