Automated quality control of High Frequency radar data

Abstract

There are approximately 300 High Frequency (HF) radars deployed around the globe making real time measurements of the surface currents in the coastal ocean. In the United States, the HF radar network within the Mid Atlantic Regional Association Coastal Ocean Observing System (MARACOOS) became operational with the United States Coast Guard in May 2009. This model was expanded nationally and the Integrated Ocean Observing System (IOOS) National HF Radar Network became operational with the Coast Guard in March 2011. Much of the quality control that is done with the data requires a person in the loop to be inspecting the data. We present several metrics and techniques to automate the quality control process to ensure that the radial and total velocity measurements are accurate. We have used average radial bearing, spectra merged count, radial count and data latency as measurements that are useful in assessing the performance of the network. Some of those techniques include real time comparisons with ADCPs and comparison of the detided total vector fields with nearby wind measurements. We also present metrics to gauge the performance of the network over seasonal and yearly time scales. The goal of the network is to provide surface currents to the Coast Guard over 80% of the spatial region of the Mid Atlantic over 80% of the time. The spatial grid that the network could realistically cover contained all grid points within 150 km of the coast and out beyond the 15 m isobath. We have also developed a user interface for the operators to control what radial sites contribute to the total vector generation. As discussed at the Radiowave Oceanography Working Group (ROWG) meetings it is the responsibility of the region to provide quality-controlled data to the National Network. Currently the National Network has only two checks for the radial data 1) that the radial measurement is over water and 2) that the magnitude of the radial measurement is below a certain threshold based on the region of the measurement. This model where radial data is inspected at the regional level before being sent onto the National Network can be expanded to the other 10 regions of the country. This also keeps the decision of what data is correct where the local knowledge of the current structure is best understood. We saw that these techniques can eliminate errors in the data stream. This also acts as a feedback mechanism to the operators to evaluate their performance in operating and maintaining the radars. The techniques discussed here can serve as data quality checks for the vast number of systems operating today. They will ensure that the data being produced is of the highest quality, which will in turn ensure that the products being generated with this data are sound and reliable.