Data Quality Evaluation Research Articles

Consistency as a Data Quality Measure for German Corona Consensus Items Mapped from National Pandemic Cohort Network Data Collections.

As a national effort to better understand the current pandemic, three cohorts collect sociodemographic and clinical data from coronavirus disease 2019 (COVID-19) patients from different target populations within the German National Pandemic Cohort Network (NAPKON). Furthermore, the German Corona Consensus Dataset (GECCO) was introduced as a harmonized basic information model for COVID-19 patients in clinical routine. To compare the cohort data with other GECCO-based studies, data items are mapped to GECCO. As mapping from one information model to another is complex, an additional consistency evaluation of the mapped items is recommended to detect possible mapping issues or source data inconsistencies. The goal of this work is to assure high consistency of research data mapped to the GECCO data model. In particular, it aims at identifying contradictions within interdependent GECCO data items of the German national COVID-19 cohorts to allow investigation of possible reasons for identified contradictions. We furthermore aim at enabling other researchers to easily perform data quality evaluation on GECCO-based datasets and adapt to similar data models. All suitable data items from each of the three NAPKON cohorts are mapped to the GECCO items. A consistency assessment tool (dqGecco) is implemented, following the design of an existing quality assessment framework, retaining their-defined consistency taxonomies, including logical and empirical contradictions. Results of the assessment are verified independently on the primary data source. Our consistency assessment tool helped in correcting the mapping procedure and reveals remaining contradictory value combinations within COVID-19 symptoms, vital signs, and COVID-19 severity. Consistency rates differ between the different indicators and cohorts ranging from 95.84% up to 100%. An efficient and portable tool capable of discovering inconsistencies in the COVID-19 domain has been developed and applied to three different cohorts. As the GECCO dataset is employed in different platforms and studies, the tool can be directly applied there or adapted to similar information models.

Discuss Application of Real-world Data from the Boao Lecheng Pilot Zone to Support Premarket Clinical Evaluation of Medical Device

The real-world data of Hainan Boao Lecheng International Tourism Pilot Zone has the advantage of supporting pre-market clinical evaluation of medical devices. Based on the relevant requirements of clinical evaluation of medical devices and based on the practical experience of pilot devices in the early stage, the application of Boao Lecheng real-world data in the pre-market clinical evaluation path of medical devices from the perspective of review is discussed. At the same time, the elements that should be considered in real-world study design and the way of data quality evaluation are proposed. Expect to provide a reference in order to allow registration applicants to use real world data wisely to help declare device registration for marketing.

Frequency of osteoporosis in Chinese patients with rheumatoid arthritis: a meta-analysis.

To evaluate the current frequency of osteoporosis (OP) in Chinese patients with rheumatoid arthritis (RA) through meta-analysis. The databases of PubMed, Web of Science, Cochrane Library, CNKI, Wan Fang, CBM, and VIP were searched for relevant literature regarding the occurrence of OP in Chinese patients with RA from January 1, 2000, to September 15, 2022. The literature was screened using inclusion and exclusion criteria, and qualifying articles were subjected to data extraction, quality evaluation, and meta-analysis using Stata 17.0 software. Altogether, 44 publications were included in this study, with a total sample size of 12,264 RA cases and 4532 patients with OP. Meta-analysis revealed that the frequency of OP in patients with RA was 37.67% [95% CI: 34.38%, 40.97%], while subgroup analysis showed that the frequency in South China was 49.43% [95% CI: 40.53%, 58.32%]. Also, the frequency in men was 29.29% [95% CI: 22.42%, 36.16%], which was significantly lower than in women (41.89% [95% CI: 36.87%, 46.90%]). The incidence rate in pre-menopausal women was only 15.19% [95% CI: 8.79%, 21.59%], much lower than the figure of 54.29% [95% CI: 45.28%, 63.30%] for post-menopausal women. Additionally, the frequency rates in the low, intermediate, and high disease activity groups were 24.52% [95% CI: 11.52%, 37.52%)], 27.67% [95% CI: 13.50%, 41.85%], and 57.96% [95% CI: 37.35%, 78.56%], respectively. The frequency of OP in the Chinese RA population is higher than the world average, and patients with RA should receive standardized anti-RA therapy at an early stage, with emphasis on bone health in postmenopausal women, patients with high disease activity or longer disease duration.

A review of data-driven approaches to predict train delays

Accurate train delay prediction is vital for effective railway traffic planning and management as well as for providing satisfactory passenger service quality. Despite significant advances in data-driven train delay predictions, it lacks of a systematic review of studies and unified modelling development framework. The paper reviews existing studies with an explicit focus on synthesizing a structural framework that could guide effective data-driven train delay prediction model development. The framework consists of three stages including design concept, modelling and evaluation. The study synthesize and discusses six important modules of the framework: (1) Problem scope, (2) Model inputs, (3) Data quality, (4) Methodologies, (5) Model outputs, and (6) Evaluation techniques. For each module, the important problems and techniques reported are synthesized and research gaps are discussed. The review found that most studies focus on developing complex methodologies for the next stop delay predictions that have limited applications in practice. All studies validate the model accuracy, but very few consider other model performance aspects which makes it difficult to assess their usfulness in practical deployment. Future studies need a holistic view on defining the train delay prediction problem considering both application requirements and implementation challenges. Also, the modelling studies should place more attention to data quality and comprehensive model evaluations in representation power, explainability and validity.

The distribution, magnitude, and endemic species of US springs

Freshwater springs and other groundwater-dependent ecosystems represent important natural resources in landscapes, providing consistent, high-quality water to both freshwater and terrestrial organisms. However, spring and GDE conservation does not appear to be a prominent feature on the US conservation radar. Therefore, this study evaluated the distribution and size of springs in the US with the national water dataset: the US Geological Survey National Water Information System. Using all available measurements of spring discharge and the sampling dates for all springs with data in the dataset (10,279 springs), I compared the number and sizes of springs between states and, where time series were available, determined whether the springs were maintaining historic discharges. I evaluated data quality using the date of the last sampling, number of times sampled, and the length of time sampled for each spring. Finally, I searched the literature for spring endemic species and recorded the states in which they occurred. Within the database, springs were most abundant in western states, but average discharges were largest for states in the southeast and for Idaho and Alaska. Very large springs occurred in some western states, but the multitudes of tiny springs reduced the average discharges. The data were poorly resolved as many of the springs had been sampled only once, often 40–50 years ago. Time series were available only for 126 springs and half of these springs exhibited declines in discharge. Endemic species were mentioned in the literature for 24 states, particularly those in the lower half of the country, and so loss of spring integrity would threaten biodiversity in many states. Due to the poor resolution of the data, broad conclusions about the integrity of these important freshwater systems are difficult to impossible to make for most states using the national dataset. Therefore, I call for a concerted national effort to more broadly evaluate spring and GDE resources. Springs and GDEs are likely to become even more important in the future as climate changes and their roles as freshwater refuges, temperature buffers, and bellwethers become even more important.

Comparing Chatbots and Online Surveys for (Longitudinal) Data Collection: An Investigation of Response Characteristics, Data Quality, and User Evaluation

ABSTRACT As chatbots are gaining more popularity than ever, they have recently been considered as interesting tools for survey administration in social science research. To explore this idea, we investigated the extent to which there are differences in response characteristics and data quality between a traditional, web-based survey and a conversational, chatbot-based survey (which we integrated in an instant messaging app). In addition, we zoomed into how respondents evaluate both survey modes. Using a longitudinal design, we also explored how response characteristics evolved over a period of two weeks. Overall, we did not find evidence that chatbots might be better survey administration tools than web surveys. On the contrary, the web survey often seemed to generate more favorable response characteristics and data quality. Finally, when it comes to user perceptions, we found that the chatbot survey was evaluated less favorably in terms of perceived enjoyment, usefulness, and security. Based on these results, we draw conclusions about whether chatbots can be considered as valid alternatives for traditional web survey methods.

Multi-Level Fusion Indoor Positioning Technology Considering Credible Evaluation Analysis

Aiming at the problems of the low robustness and poor reliability of a single positioning source in complex indoor environments, a multi-level fusion indoor positioning technology considering credible evaluation is proposed. A multi-dimensional electromagnetic atlas including pseudolites (PL), Wi-Fi and a geomagnetic field is constructed, and the unsupervised learning model is used to sample in the latent space to achieve a feature-level fusion positioning. A location credibility evaluation method is designed to improve the credibility of the positioning system through a multi-dimensional data quality evaluation and heterogeneous information auxiliary constraints. Finally, a large number of experiments were carried out in the laboratory environment, and, finally, about 90% of the positioning error was better than 1 m, and the average positioning error was 0.56 m. Compared with several relatively advanced positioning methods (Inter-satellite CPDM/Epoch-CPDS/Z-KPI) at present, the average positioning accuracy is improved by about 56%, 83.5% and 82.9%, respectively, which verifies the effectiveness of the algorithm. To verify the effect of the proposed method in a practical application environment, the proposed positioning system is deployed in the 2022 Winter Olympics venues. The results show that the proposed method has a significant improvement in the positioning accuracy and continuity.

Method for Image Quality Evaluation of Satellite-based SAR Data

Method for Image Quality Evaluation of Satellite-based SAR Data

Evaluation of data quality based on Bayesian networks in railway rolling stock monitoring systems

The purpose of the research is to evaluate the quality of data based on the Bayesian network to justify the effectiveness of the Internet of Things technology in monitoring railway rolling stock. To achieve this, performed the following tasks: presented technological schemes for monitoring railway rolling stock; built Bayesian network; created probability tables; determined conditional probabilities of control events. The existing and proposed railway rolling stock monitoring systems in the Republic of Uzbekistan are investigated. To justify the effectiveness in data quality, made an evaluation based on the Bayesian network. Conditionally selected one section with a train that serves intermediate stations. Two systems monitor this train's railcars: the Automated operational transport management system (AOTMS) and the Automatic control system for rolling stock and containers (ACSRSC). Technological operations transformed into probabilistic relations. The model defines three control events for comparison. The results showed that the quality of data of the existing monitoring system is lower than that of the proposed one. It is due to the dependence on the transfer of information among themselves to the AOTMS. In the ACSRSC system, all information, regardless of previous information, will be received individually. In addition, the level of subjective interventions for the transfer of information is also significantly reduced. The capabilities of ASMSCS are significantly superior in terms of data quality. Subsequent applications of the Internet of Things technology will help improve the quality of management decision-making in the organization of train traffic.

FTGAN: A Novel GAN-Based Data Augmentation Method Coupled Time–Frequency Domain for Imbalanced Bearing Fault Diagnosis

For imbalanced bearing fault diagnosis, generative adversarial networks (GANs) are a common data augmentation (DA) approach. Nevertheless, current GAN-based methods cannot update the generator from time–frequency domain simultaneously, downgrading the authenticity of signal time–frequency character. In this article, Fourier-like transform GAN (FTGAN), a novel GAN method, is proposed by introducing a Fourier-like transformer (FLT) based on autoencoder (AE) to improve synthetic data quality. FLT approximates the discrete Fourier transform (DFT) by the neural network, learning a universal map from time to frequency domain during training. FTGAN with FLT can decouple input into a time–frequency domain, fitting the distribution of time and frequency of data simultaneously. Multidomain distribution is manipulated in FTGAN without introducing additional signal transformation means. Furthermore, train on real, test on synthetic (TRTS) and train on synthetic, test on real (TSTR) analyses of 1-D data are introduced to evaluate data quality. Real and synthetic data are applied as training or test sets of diagnostic classifiers by turns so that data quality can be analyzed through diagnosis results. Experiment results show that the proposed method can generate bearing fault signals closer to real data in the time and frequency domains, effectively improving the performance under an imbalanced dataset.

Quality Evaluation of Wind Energy Data with Complete Linkage Clustering

Quality Evaluation of Wind Energy Data with Complete Linkage Clustering

The effect and safety of low-dose Tripterygium wilfordii in patients with type 2 diabetic nephropathy: A meta-analysis.

This review aims to assess the efficacy and safety of low-dose Tripterygium wilfordii Hook F (TWHF) in treating type 2 diabetic nephropathy (DN) and provide high-level evidence supporting its normalized application. Seven electronic databases were queried to locate trials that qualify. Randomized controlled trials (RCTs) about low-dose TWHF long-term treatment of type 2 DN are included. After data extraction and quality evaluation of the clinical studies that met the inclusion criteria, a meta-analysis was performed using RevMan 5.4 and Stata 14. A total of 23 RCTs were included. For the patients in the trial group, the effective rate [confidence interval (CI), odd ratio] [odd ratio = 1.38, 95% CI (1.22-1.56), P < .001], albumin [standard mean difference (SMD) = 0.58, 95% CI (0.18-0.98), P = .004], 24-hour urine total protein [SMD = -1.329, 95% CI = (-1.647 to -1.012), P < .001], serum creatinine [SMD = -0.64, 95% CI = (-0.86 to -0.31), P < .001], and the untoward effect [RR = 2.43 95% CI = (1.23-4.82), P = .01] were significantly higher than those in the control group. However, in white blood cell [Weighted mean difference = -0.27, 95% CI (-0.54 to 0.01), P = .06] and blood urea nitrogen [Weighted mean difference = -0.11, 95% CI (-0.42 to 0.21), z = 0.67, P = .50], none of the differences were significant compared with the control group. This suggests that low-dose TWHF positively affects patients with type 2 DN after a long course of treatment. Although there are some side effects, symptoms can improve after medication suspension or symptomatic treatment. Limited by the methodological quality of the included studies, this conclusion needs to be verified by more large-sample RCTs with rigorous design and long-term follow-up.

Robust Data Augmentation for Neural Machine Translation through EVALNET

Since building Neural Machine Translation (NMT) systems requires a large parallel corpus, various data augmentation techniques have been adopted, especially for low-resource languages. In order to achieve the best performance through data augmentation, the NMT systems should be able to evaluate the quality of augmented data. Several studies have addressed data weighting techniques to assess data quality. The basic idea of data weighting adopted in previous studies is the loss value that a system calculates when learning from training data. The weight derived from the loss value of the data, through simple heuristic rules or neural models, can adjust the loss used in the next step of the learning process. In this study, we propose EvalNet, a data evaluation network, to assess parallel data of NMT. EvalNet exploits a loss value, a cross-attention map, and a semantic similarity between parallel data as its features. The cross-attention map is an encoded representation of cross-attention layers of Transformer, which is a base architecture of an NMT system. The semantic similarity is a cosine distance between two semantic embeddings of a source sentence and a target sentence. Owing to the parallelism of data, the combination of the cross-attention map and the semantic similarity proved to be effective features for data quality evaluation, besides the loss value. EvalNet is the first NMT data evaluator network that introduces the cross-attention map and the semantic similarity as its features. Through various experiments, we conclude that EvalNet is simple yet beneficial for robust training of an NMT system and outperforms the previous studies as a data evaluator.

Modeling the Data Provenance of Relational Databases Supporting Full-Featured SQL and Procedural Languages

Data provenance is information about where data come from (provenance data) and how they transform (provenance transformation). Data provenance is widely used to evaluate data quality, trace errors, audit data, and understand references among data. Current studies on data provenance in relational database management systems (RDBMS) still have limitations in supporting full-featured SQL or procedural languages. With these challenges in mind, we present a formal definition of provenance data and provenance transformation for relational data. Then, we propose a solution to support data provenance in relational databases, including provenance graphs and provenance routes. Our method not only solves the complicated problem of modeling provenance in DBMS but also is capable of extending procedural languages in SQL. We also present ProvPg, a PostgreSQL-based prototype database system supporting data provenance in multiple granularities. ProvPg implements extraction, calculation, query, and visualization of provenance. We perform TPC-H tests for ProvPg and PostgreSQL, respectively. Experimental results show that ProvPg addresses the vision of supporting data provenance with little extra computation overhead for the execution engine, which indicates that our model is applicable to lineage tracing applications.

Evaluation of data quality at the Hungarian National Cancer Registry, 2000–2019

BackgroundThe Hungarian National Cancer Registry (HNCR) was legally established as a population-based cancer registry in 1999, and its operation started in 2000 supporting the planning and development of the Hungarian oncology network as well as informing national cancer control policies. Ensuring comparable, accurate, and complete data on malignant and in situ neoplasms is critical in determining the applicability of the database. The aim of this study was to perform a comprehensive evaluation of the data quality at the HNCR. MethodsBased on qualitative and semiquantitative methods from current international guidelines, we assess the comparability, completeness, validity, and timeliness of the collected data over the diagnostic period 2000–2019, with a focus on the year 2018. ResultsCoding practices and the classification system used at the HNCR are based on the International Classification of Diseases (ICD-10), which differs from the internationally recommended ICD-O. The annual trends in incidence did not indicate major fluctuations, that may have resulted from data collection discrepancies, while comparisons of the mortality-to-incidence ratio (M:I) compared with 1 minus 5-year observed survival indicated some systematic differences requiring further exploration. The age-standardized (European standard) incidence rate per 100 000 measured by the HNCR in 2018 was very high: 647.9 for men and 501.6 for women, 11.6% and 14.6% higher than the International Agency for Research on Cancer (IARC) estimates respectively. Behind the overall differences between the two data sources, we identified that the vast majority were due to ill-defined ICD codes: malignant neoplasm of other and ill-defined sites (C76), and malignant neoplasm without specification of site (C80). Otherwise, there were no major discrepancies by localization. The proportion of morphologically verified cancer cases was 57.8% overall, that of death certificates was 2.3%, and that of unknown primary tumors was 1.4%. ConclusionFurther implementations and interventions are required to ensure that the operations, coding practices, and the classification system used at the national registry are in accordance with international standards, and to increase the completeness and validity of the collected cancer data. In particular, the low morphologically verified proportion questions the overall accuracy of the stated diagnoses within the database. Nevertheless, our examination implies that the data of the HNCR are reasonably comparable, and without doubt fulfill the requirements to support national oncology services and cancer planning. However, most importantly, a review of registry personnel and resource requirements to run the national population-based cancer registry should be an essential part of Hungary’s national cancer strategy.

Tissue registration and exploration user interfaces in support of a human reference atlas

Seventeen international consortia are collaborating on a human reference atlas (HRA), a comprehensive, high-resolution, three-dimensional atlas of all the cells in the healthy human body. Laboratories around the world are collecting tissue specimens from donors varying in sex, age, ethnicity, and body mass index. However, harmonizing tissue data across 25 organs and more than 15 bulk and spatial single-cell assay types poses challenges. Here, we present software tools and user interfaces developed to spatially and semantically annotate (“register”) and explore the tissue data and the evolving HRA. A key part of these tools is a common coordinate framework, providing standard terminologies and data structures for describing specimen, biological structure, and spatial data linked to existing ontologies. As of April 22, 2022, the “registration” user interface has been used to harmonize and publish data on 5,909 tissue blocks collected by the Human Biomolecular Atlas Program (HuBMAP), the Stimulating Peripheral Activity to Relieve Conditions program (SPARC), the Human Cell Atlas (HCA), the Kidney Precision Medicine Project (KPMP), and the Genotype Tissue Expression project (GTEx). Further, 5,856 tissue sections were derived from 506 HuBMAP tissue blocks. The second “exploration” user interface enables consortia to evaluate data quality, explore tissue data spatially within the context of the HRA, and guide data acquisition. A companion website is at https://cns-iu.github.io/HRA-supporting-information/.

Data quality evaluation in wearable monitoring.

Wearable recordings of neurophysiological signals captured from the wrist offer enormous potential for seizure monitoring. Yet, data quality remains one of the most challenging factors that impact data reliability. We suggest a combined data quality assessment tool for the evaluation of multimodal wearable data. We analyzed data from patients with epilepsy from four epilepsy centers. Patients wore wristbands recording accelerometry, electrodermal activity, blood volume pulse, and skin temperature. We calculated data completeness and assessed the time the device was worn (on-body), and modality-specific signal quality scores. We included 37,166h from 632 patients in the inpatient and 90,776h from 39 patients in the outpatient setting. All modalities were affected by artifacts. Data loss was higher when using data streaming (up to 49% among inpatient cohorts, averaged across respective recordings) as compared to onboard device recording and storage (up to 9%). On-body scores, estimating the percentage of time a device was worn on the body, were consistently high across cohorts (more than 80%). Signal quality of some modalities, based on established indices, was higher at night than during the day. A uniformly reported data quality and multimodal signal quality index is feasible, makes study results more comparable, and contributes to the development of devices and evaluation routines necessary for seizure monitoring.

High-quality and robust protein quantification in large clinical/pharmaceutical cohorts with IonStar proteomics investigation.

Robust, reliable quantification of large sample cohorts is often essential for meaningful clinical or pharmaceutical proteomics investigations, but it is technically challenging. When analyzing very large numbers of samples, isotope labeling approaches may suffer from substantial batch effects, and even with label-free methods, it becomes evident that low-abundance proteins are not reliably measured owing to unsufficient reproducibility for quantification. The MS1-based quantitative proteomics pipeline IonStar was designed to address these challenges. IonStar is a label-free approach that takes advantage of the high sensitivity/selectivity attainable by ultrahigh-resolution (UHR)-MS1 acquisition (e.g., 120-240k full width at half maximum at m/z = 200) which is now widely available on ultrahigh-field Orbitrap instruments. By selectively and accurately procuring quantitative features of peptides within precisely defined, very narrow m/z windows corresponding to the UHR-MS1 resolution, the method minimizes co-eluted interferences and substantially enhances signal-to-noise ratio of low-abundance species by decreasing noise level. This feature results in high sensitivity, selectivity, accuracy and precision for quantification of low-abundance proteins, as well as fewer missing data and fewer false positives. This protocol also emphasizes the importance of well-controlled, robust experimental procedures to achieve high-quality quantification across a large cohort. It includes a surfactant cocktail-aided sample preparation procedure that achieves high/reproducible protein/peptide recoveries among many samples, and a trapping nano-liquid chromatography-mass spectrometry strategy for sensitive and reproducible acquisition of UHR-MS1 peptide signal robustly across a large cohort. Data processing and quality evaluation are illustrated using an example dataset ( http://proteomecentral.proteomexchange.org ), and example results from pharmaceutical project and one clinical project (patients with acute respiratory distress syndrome) are shown. The complete IonStar pipeline takes ~1-2 weeks for a sample cohort containing ~50-100 samples.

Data quality evaluation for bridge structural health monitoring based on deep learning and frequency-domain information

Abnormal data recognition is of great importance in structural health monitoring. Most of the existing studies focused on detecting obvious abnormal data, which has obvious abnormal time-domain waveform. Pseudo normal data, which is normally looking in time domain but chaotic in frequency domain, did not receive enough attention and were likely to be misclassified as normal data. As a result, structural performance may be incorrectly evaluated because pseudo normal data are not recognized and eliminated from the monitoring data. This study developed a novel quality evaluation framework for monitoring data of bridge dynamic response. The main novelty of the proposed framework is that the frequency-domain information is used to characterize the quality of the monitoring data so that the normal data, obvious abnormal data, and pseudo normal data can be accurately distinguished. In the framework, the frequency-domain information was obtained by fast Fourier transform (FFT), and the Gramian angular field images converted from FFT results were used to train a designed convolutional neural network (CNN). The cable acceleration data of the Waitan cable-stayed bridge were taken as an example to verify the accuracy of the proposed framework. Compared with the CNN models based on time-domain images and time-frequency stacked images, this framework can better recognize pseudo normal data from the monitoring data. In large-scale testing, the classification accuracy of all channels was more than 96%. Finally, two cable acceleration sensors of another cable-stayed bridge were used to demonstrate the feasibility of the framework in cross-object application. The results show that the framework has good accuracy and robustness in large-scale monitoring data quality evaluation and cross-object application.

Introduction of interactive remote monitoring at SUERC

SUERC has implemented remote control of routine and experimental NEC spectrometer operations.Through secure tunnelling, a dedicated laptop for each spectrometer can execute parts of the AccelNET (Accelerator NETwork control system) suite and communicate remotely with the underlying database: instrument control pages are accessible and changes to the run list can be effected in the dosimetry manager (DMAN). X-forwarding facilitates the use of the abc data reduction software upon the large remote dataset, allowing the operator to evaluate data quality mid-run, while screen-sharing is employed to provide flexible access options. AccelNET’s read-only html output (formerly the predominate form of remote monitoring) remains accessible for ‘casual’ progress checking by non-operators.While there are limitations to each technology used, these changes to our setup and operating procedures have had positive effects both for operators, and upon the smooth delivery of sample data.