Chapter 10 - Data Quality

Abstract

There are software tools for checking data quality that helps a bit, one can also do simple tests with a basic SQL queries. The schema information tables in many products have statics that one can read. They are used by the optimizer to speed up queries, rather than to check the data quality, so they tend to describe the statistical distribution of the nonkey data elements in a histogram, have the extrema for numeric and temporal data, and perhaps contain some frequency information. Furthermore, finding rules in his also discussed in the chapter. When checking to see if a database is "following the rule," one can first pull out the CHECK() constraint logic and be fairly certain that those rules are enforced. If a CHECK() constraint was added to the schema after bad data was inserted, one needs to know whether database product automatically run a check on the existing data or whether the constraint goes into effect only for future changes.