Quantify Data Quality Research Articles

Quantifying data quality after removing respondents who fail data quality checks

ABSTRACT Check Questions (CQs) are used to scrutinise whether respondents are providing quality data and these are becoming more important to ensure data quality, especially with the use of online panels. Often the proportion of people passing these questions are used as a summary of data quality, but this is flawed and of limited value when assessing the quality of a study. This paper introduces a new statistic, the data quality index Q ^ , to measure the quality of data after respondents who fail CQs are removed. This index is interpretable as the estimated proportion of respondents (after excluding respondents who fail the CQs) who answer the survey conscientiously. The index is derived using simple probability, its properties explored, and directions provided on how it should be used and interpreted. The use of Q ^ in published research will increase transparency and confidence in the quality of research findings.

Electronic health record data quality variability across a multistate clinical research network.

Electronic health record (EHR) data have many quality problems that may affect the outcome of research results and decision support systems. Many methods have been used to evaluate EHR data quality. However, there has yet to be a consensus on the best practice. We used a rule-based approach to assess the variability of EHR data quality across multiple healthcare systems. To quantify data quality concerns across healthcare systems in a PCORnet Clinical Research Network, we used a previously tested rule-based framework tailored to the PCORnet Common Data Model to perform data quality assessment at 13 clinical sites across eight states. Results were compared with the current PCORnet data curation process to explore the differences between both methods. Additional analyses of testosterone therapy prescribing were used to explore clinical care variability and quality. The framework detected discrepancies across sites, revealing evident data quality variability between sites. The detailed requirements encoded the rules captured additional data errors with a specificity that aids in remediation of technical errors compared to the current PCORnet data curation process. Other rules designed to detect logical and clinical inconsistencies may also support clinical care variability and quality programs. Rule-based EHR data quality methods quantify significant discrepancies across all sites. Medication and laboratory sources are causes of data errors.

A Wearable Electrocardiography Armband Resilient Against Artifacts

Electrocardiography (ECG) is an essential technique to assess cardiovascular conditions and monitor physical activities. While the concept is mature, issues surrounding sampling convenience and device adoption as well as maintaining signal quality under artifacts remain a problem. In this article, we present a high-performing wearable ECG armband on the upper left arm. It is equipped with miniaturized hardware, capable of data storage and wireless communication. We evaluate different electrode configurations by conducting ECG measurements both at the static state and under motion and using improved algorithms to quantify data quality and assess the agreement between the proposed new technique and the gold standard. The optimal electrode position is determined by balancing wearable suitability and signal quality. We propose an electronic textile (E-textile) armband with improved design. It offers favorable wearing comfort and a fashionable appearance without sacrificing data quality. Its contact pressure is measured to get a better picture of intimacy and clothing comfort. Our system provides real-time and noise-resilient ECG data without interrupting daily life and can be implemented in use cases that warrant continuous ECG monitoring.

Comparison of a Smartfin with an Infrared Sea Surface Temperature Radiometer in the Atlantic Ocean

The accuracy and precision of satellite sea surface temperature (SST) products in nearshore coastal waters are not well known, owing to a lack of in-situ data available for validation. It has been suggested that recreational watersports enthusiasts, who immerse themselves in nearshore coastal waters, be used as a platform to improve sampling and fill this gap. One tool that has been used worldwide by surfers is the Smartfin, which contains a temperature sensor integrated into a surfboard fin. If tools such as the Smartfin are to be considered for satellite validation work, they must be carefully evaluated against state-of-the-art techniques to quantify data quality. In this study, we developed a Simple Oceanographic floating Device (SOD), designed to float on the ocean surface, and deployed it during the 28th Atlantic Meridional Transect (AMT28) research cruise (September and October 2018). We attached a Smartfin to the underside of the SOD, which measured temperature at a depth of ∼0.1 m, in a manner consistent with how it collects data on a surfboard. Additional temperature sensors (an iButton and a TidbiT v2), shaded and positioned a depth of ∼1 m, were also attached to the SOD at some of the stations. Four laboratory comparisons of the SOD sensors (Smartfin, iButton and TidbiT v2) with an accurate temperature probe (±0.0043 K over a range of 273.15 to 323.15 K) were also conducted during the AMT28 voyage, over a temperature range of 290–309 K in a recirculating water bath. Mean differences (δ), referenced to the temperature probe, were removed from the iButton (δ=0.292 K) and a TidbiT v2 sensors (δ=0.089 K), but not from the Smartfin, as it was found to be in excellent agreement with the temperature probe (δ=0.005 K). The SOD was deployed for 20 min periods at 62 stations (predawn and noon) spanning 100 degrees latitude and a gradient in SST of 19 K. Simultaneous measurements of skin SST were collected using an Infrared Sea surface temperature Autonomous Radiometer (ISAR), a state-of-the-art instrument used for satellite validation. Additionally, we extracted simultaneous SST measurements, collected at slightly different depths, from an underway conductivity, temperature and depth (CTD) system. Over all 62 stations, the mean difference (δ) and mean absolute difference (ϵ) between Smartfin and the underway CTD were −0.01 and 0.06 K respectively (similar results obtained from comparisons between Smartfin and iButton and Smartfin and TidbiT v2), and the δ and ϵ between Smartfin and ISAR were 0.09 and 0.12 K respectively. In both comparisons, statistics varied between noon and predawn stations, with differences related to environmental variability (wind speed and sea-air temperature differences) and depth of sampling. Our results add confidence to the use of Smartfin as a citizen science tool for evaluating satellite SST data, and data collected using the SOD and ISAR were shown to be useful for quantifying near-surface temperature gradients.

Embodied carbon in construction materials: a framework for quantifying data quality in EPDs

Embodied carbon constitutes a significant portion of a building’s greenhouse gas (GHG) emissions and is a key challenge for the construction and real estate sectors. Embodied carbon includes construction product manufacturing, building construction, material replacement and end of life. During the specification and procurement stage, designers and contractors have the opportunity to prioritize products with lower carbon footprints. Environmental product declarations (EPDs) are a growing source of environmental data in the construction products market, and are increasingly being used for (1) environmental performance assessment of buildings and (2) product comparison for procurement decisions during the later stages of building design. An obstacle to identifying and purchasing lower embodied carbon products is a lack of data quality and the transparency of EPDs. However, EPDs vary widely in their data quality and specificity, which can lead to inaccurate and misleading comparisons. A new method is presented to account quantitatively for estimates of variation in underlying data specificity in EPDs to enable fairer comparisons between EPDs and to motivate the reporting of actual variability and uncertainty in EPDs. The application of this approach can help purchasers to assess EPDs quantitatively. Practice relevance Life-cycle assessments (LCAs) and LCA data can be used within the construction sector to evaluate buildings and to assist in design, specification and procurement decision-making. A new method is presented to support the assessment of comparability of functionally equivalent materials and products during the specification and procurement stage. Given the known variation and lack of precision within EPDs, this method provides quantitative metrics that correlate to a qualitative interpretation of EPD precision. This method can be used by anyone who is using EPD data to make product comparisons at the specification and procurement stage: It provides more confidence in choosing low-carbon material or product options when comparing between functionally equivalent options. It can incentivize product manufacturers and LCA practitioners to improve data quality and transparently report known variation in their EPDs. It may also motivate manufacturers to reduce GHGs from their products and processes.

Quantifying data quality in a citizen science monitoring program: False negatives, false positives and occupancy trends

AbstractData collected by volunteers are an important source of information used in species management decisions, yet concerns are often raised over the quality of such data. Two major forms of error exist in occupancy datasets; failing to observe a species when present (imperfect detection—also known as false negatives), and falsely reporting a species as present (false‐positive errors). Estimating these rates allows us to quantify volunteer data quality, and may prevent the inference of erroneous trends. We use a new parameterization of a dynamic occupancy model to estimate and adjust for false‐negative and false‐positive errors, producing accurate estimates of occupancy. We validated this model using simulations and applied it to 12 species datasets collected from a 15‐year, large‐scale volunteer amphibian monitoring program. False‐positive rates were low for most, but not all, species, and accounting for these errors led to quantitative differences in occupancy, although trends remained consistent even when these effects were ignored. We present a model that represents an intuitive way of quantifying the quality of volunteer monitoring datasets, and which can produce unbiased estimates of occupancy despite the presence of multiple types of observation error. Importantly, this allows the quality of volunteer monitoring data to be assessed without relying on comparisons with expert data.

A General Framework for Data Uncertainty and Quality Classification

Abstract It is often assumed that data used to plan operations and supply chain activities is accurate. But in the presence of uncertainty, this assumption is known not to be entirely true. In this context, it becomes relevant to evaluate if a planning decision is appropriate in light of partially accurate data. This paper proposes a general framework for data analysis in order to provide a quality evaluation of the information used in the decision-making process. To this end we propose a process to quantify data quality by comparing “measured” data to “real” data. We use a hybrid approach combining multiple data quality assessment techniques as well as different alternative sources of historic data. A classification phase then rates and «tags» data for proper consideration for decision-making. Such classification provides insights into the level of uncertainty associated with the data. This paper demonstrates the approach developed using a case study from the forest sector. The approach can be adapted to other industrial sectors.

Spatio-temporal quality control: implications and applications for data consumers and aggregators

Quality control for real-time spatio-temporal data is often presented from the perspective of the original owner and provider of the data, and focuses on general techniques for outlier detection or uses domain-specific knowledge and rules to assess quality. The impact of quality control on the data aggregator and redistributor is neglected. As sensor networks proliferate, multiple providers can distribute and redistribute the same original sensor data. Relationships between providers become complex, with data acquired from original and third-party sources. One provider may acquire data from another, and so forth, resulting in larger data sets with value-added components such as quality indicators, but with costs such as increased lag between original observation times and (re)distribution times. The focus of this paper is to define and demonstrate quality control measures for real-time, spatio-temporal data from the perspective of an aggregator to provide tools for evaluation and comparison of overlapping, real-time, spatio-temporal data providers and for assessment and optimization of data acquisition, system operation and data redistribution. We define simple measures that account for temporal completeness and spatial coverage. The measures and methods developed are tested on real-world data and applications. Our results show that these simple measures combine to form methods that are useful in comparing providers and identifying patterns in data which can then be exploited to optimize system performance relative to bandwidth, and to assess the impact of provider quality control mechanisms. The simple measures presented demonstrate the utility of quantifying data quality from the perspective of the data aggregator and redistributor.

INSAT-3D Quality Analysis System (i3dQAS)

Abstract. INSAT-3D, an advanced meteorological satellite of ISRO carrying Imager and Sounder payloads is designed for acquiring data required for detailed climatological applications. At established data centres, Raw (L-0) data is processed to generate systematically corrected (Level-1) and geo-physical (Level-2) products which are further disseminated for use in climate related studies. Data quality evaluation (DQE) of basic data products of INSAT-3D is being operationally carried out at MOSDAC, Ahmedabad to monitor in-orbit sensor health, quantify data quality and provide feedback to various missions teams for corrective actions. DQE methodology and quality parameters have been defined keeping in view repeatability of data acquisition and quality requirements of meteorological data. In this paper, we have described INSAT-3D Quality Analysis System (i3dQAS) designed to facilitate effective data quality monitoring through web based portal. Overall context of i3dQAS in a typical data centre set-up, major objectives and design aspects to highlight the architectural and functional view of system are described. Utility of i3dQAS for mission monitoring is described through analysis scenarios for monitoring payload health and geometric accuracy of Imager.

Quantifying Data Quality for Clinical Trials Using Electronic Data Capture

BackgroundHistorically, only partial assessments of data quality have been performed in clinical trials, for which the most common method of measuring database error rates has been to compare the case report form (CRF) to database entries and count discrepancies. Importantly, errors arising from medical record abstraction and transcription are rarely evaluated as part of such quality assessments. Electronic Data Capture (EDC) technology has had a further impact, as paper CRFs typically leveraged for quality measurement are not used in EDC processes.Methods and Principal FindingsThe National Institute on Drug Abuse Treatment Clinical Trials Network has developed, implemented, and evaluated methodology for holistically assessing data quality on EDC trials. We characterize the average source-to-database error rate (14.3 errors per 10,000 fields) for the first year of use of the new evaluation method. This error rate was significantly lower than the average of published error rates for source-to-database audits, and was similar to CRF-to-database error rates reported in the published literature. We attribute this largely to an absence of medical record abstraction on the trials we examined, and to an outpatient setting characterized by less acute patient conditions.ConclusionsHistorically, medical record abstraction is the most significant source of error by an order of magnitude, and should be measured and managed during the course of clinical trials. Source-to-database error rates are highly dependent on the amount of structured data collection in the clinical setting and on the complexity of the medical record, dependencies that should be considered when developing data quality benchmarks.

Assessing the quality of risk factor survey data: lessons from the WHO MONICA Project

Survey data quality is a combination of the representativeness of the sample, the accuracy and precision of measurements, data processing and management with several subcomponents in each. The purpose of this paper is to show how, in the final risk factor surveys of the WHO MONICA Project, information on data quality were obtained, quantified, and used in the analysis. In the WHO MONICA (Multinational MONItoring of trends and determinants in CArdiovascular disease) Project, the information about the data quality components was documented in retrospective quality assessment reports. On the basis of the documented information and the survey data, the quality of each data component was assessed and summarized using quality scores. The quality scores were used in sensitivity testing of the results both by excluding populations with low quality scores and by weighting the data by its quality scores. Detailed documentation of all survey procedures with standardized protocols, training, and quality control are steps towards optimizing data quality. Quantifying data quality is a further step. Methods used in the WHO MONICA Project could be adopted to improve quality in other health surveys.

Towards Quantifying Data Quality Costs.

Today most organizations run their daily operations using data at their disposal. However, a vast majority of the organizations do not have adequate process and tools to maintain high quality operational data at all times. One of the key reasons for this is the lack of appreciation of the damages that low quality data can bring to an organization, and the cost of ensuring high quality of data. This article provides a basis for quantifying in monetary terms the costs of both low quality data and ensuring high quality data. A comparison of the costs of low quality data and ensuring high quality data can be a simple and compelling basis for an organization to determine the extent of the efforts it must expend to ensure high quality of its operational data.

Geographic Information System-Based Evaluation of Geotechnical Borehole Log Quality

Although standard procedures and methods are available for many aspects of geotechnical site characterization, the quality of the results of such activities can be highly variable because of a variety of factors, including limitations resulting from the test procedures. For initial projects, the nature of these limitations as well as their likely consequences can be reasonably accommodated; however, subsequent use of the same data in a later study may not have the benefit of this project-specific insight, particularly if the data have been archived in a digital database. Consequently, a technique for quantifying data quality is necessary. A geographical information system (GIS)-based methodology has been developed to allow for the evaluation of borehole log quality as a function of its intended use. The methodology uses a flexible hierarchical model to provide a quantitative measure of quality. The quality of the discrete boreholes distributed across a site can then be analyzed spatially within a GIS to provide a more complete assessment of quality for the entire project. This framework also allows for evaluation of the effect of additional borehole logs at a site and can be used to optimize such planning activities in a virtual environment.