Data Quality Assurance and Analysis for Food Science Data

Abstract

Data is of paramount significance in every information system. Even though data quality might have a different meaning for different users and applications, it is always recognized that the quality of the data in an information system can have a large impact to the quality and value of every other aspect of the system. In this master thesis, we model and quantify data quality in information systems with the use of constraints. We propose a novel concept for a highly flexible and configurable data quality management framework that can easily be adapted to the needs of different users and applications. For this purpose, we introduce the notion of contracts, which are simple, interchangeable files that allow each user and application of the system to specify an individual importance weight for each quality-related constraint, thus enabling the creation of unique quality profiles. Our framework allows the user to validate data during input, as well as analyze, quantify and visualize the overall quality of the data at any time. For both the data input validation and the data quality analysis, the user is able to select the data entities and constraint groups that should be validated and analyzed, as well as the contract (quality profile) that should be used for the validation and analysis. To measure quality, we defined a novel quality calculation scheme and scale. For the definition, validation and management of the constraints in a unified way, we use the bean validation1 specification. For evaluating our approach, we implemented our concept within the FoodCASE food science data management system. FoodCASE is used for the administration of nutrients for food composition studies and contaminants for Total Diet Studies (TDS). Currently, FoodCASE is used to manage the data of the Swiss Food Composition Database as well as the data of TDS-Exposure2 studies for several countries across Europe. Our data quality management framework and its FoodCASE implementation prototype were presented to the participants of the TDS-Exposure workshop that took place during the third TDS-Exposure General Assembly in February 2015 and received positive feedback, which is also discussed in this work. http://beanvalidation.org/ http://www.tds-exposure.eu/