Generalizable, Extensible Methods to Implement QA/QC Tests on Environmental Data

Abstract

Abstract. As environmental data are increasingly recognized as important, increased attention is focused on data quality. A critical component of quality assurance and quality control (QA/QC) is implementing tests for problems like missing data, outliers, and changes that might be too abrupt or too slow to be real, and most implementations involve coding the combinational logic of tests in if-then-else constructs to arrive at outcomes or actions depending on the inputs. Herein we propose two alternative implementation methods that avoid if-then-else logic and therefore can be used to perform more complex QA/QC screens on data from stations with one or more sensors. The objectives of this technical note are to describe the methods and to demonstrate their performance using realistic examples with existing data. Both new methods use truth tables to express and implement logic; the first method expands a truth table with N tests to all 2N possible combinations and uses that as a lookup table, while the second method matches test patterns in the truth table using its original unexpanded form. The if-then-else method and the two new methods are illustrated with a simple example of air temperature measurement for scenarios with one, two, or three (redundant) sensors. The relative advantages and disadvantages of each method are discussed. Scalability of the new methods is illustrated using measurements from a local weather station. If provided both the test results and either the expanded truth table (for the lookup method) or the original truth table (for the pattern-match method), the core code was general. The flexibility of these methods should increase the power of QA/QC programs for environmental or other time series data. Keywords: Multiple sensors, QA/QC, Quality assurance, Quality control, Truth table, Weather data, Weather station.