Data Quality Assessment Process Research Articles

A clustering approach for data quality results of research information systems

PurposeThe overall goal of the data mining process is to extract information from an extensive data set and make it understandable for further use. When working with large volumes of unstructured data in research information systems, it is necessary to divide the information into logical groupings after examining their quality before attempting to analyze it. On the other hand, data quality results are valuable resources for defining quality excellence programs of any information system. Hence, the purpose of this study is to discover and extract knowledge to evaluate and improve data quality in research information systems.Design/methodology/approachClustering in data analysis and exploiting the outputs allows practitioners to gain an in-depth and extensive look at their information to form some logical structures based on what they have found. In this study, data extracted from an information system are used in the first stage. Then, the data quality results are classified into an organized structure based on data quality dimension standards. Next, clustering algorithms (K-Means), density-based clustering (density-based spatial clustering of applications with noise [DBSCAN]) and hierarchical clustering (balanced iterative reducing and clustering using hierarchies [BIRCH]) are applied to compare and find the most appropriate clustering algorithms in the research information system.FindingsThis paper showed that quality control results of an information system could be categorized through well-known data quality dimensions, including precision, accuracy, completeness, consistency, reputation and timeliness. Furthermore, among different well-known clustering approaches, the BIRCH algorithm of hierarchical clustering methods performs better in data clustering and gives the highest silhouette coefficient value. Next in line is the DBSCAN method, which performs better than the K-Means method.Research limitations/implicationsIn the data quality assessment process, the discrepancies identified and the lack of proper classification for inconsistent data have led to unstructured reports, making the statistical analysis of qualitative metadata problems difficult and thus impossible to root out the observed errors. Therefore, in this study, the evaluation results of data quality have been categorized into various data quality dimensions, based on which multiple analyses have been performed in the form of data mining methods.Originality/valueAlthough several pieces of research have been conducted to assess data quality results of research information systems, knowledge extraction from obtained data quality scores is a crucial work that has rarely been studied in the literature. Besides, clustering in data quality analysis and exploiting the outputs allows practitioners to gain an in-depth and extensive look at their information to form some logical structures based on what they have found.

Multicriteria Based Decision Making of DevOps Data Quality Assessment Challenges Using Fuzzy TOPSIS

In current era, DevOps gain much interaction in software industry as it provides the flexible development environment. To meet the continuous development and operations, DevOps mainly focus, to integrate the data from heterogeneous source. While DevOps adoption, the quality assessment of data integrated from heterogeneous environment, is important and challenging at the same time. This study aims to identify the critical factors that could negatively impact the data quality assessment process in DevOps. We have used the systematic literature review (SLR) approach and identify a total of 13 critical challenging factors. The finding of SLR are further validated with industry experts via questionnaire survey. Finally, we have applied the Fuzzy TOPSIS approach to prioritize the investigated challenging factors with respect to their significance of DevOps data quality assessment process. The results show that analyzing data in real time, visualization of data and missing information and other invalid data are the highest ranked challenging factors which need to be addressed on priority basis, to successfully measure the quality of heterogeneous data in DevOps. We believe that the finding of this study will assist the practitioner to consider the most significant factors for measuring the quality of heterogeneous data in DevOps.

Improving a Secondary Use Health Data Warehouse: Proposing a Multi-Level Data Quality Framework.

Background:Data quality frameworks within information technology and recently within health care have evolved considerably since their inception. When assessing data quality for secondary uses, an area not yet addressed adequately in these frameworks is the context of the intended use of the data.Methods:After review of literature to identify relevant research, an existing data quality framework was refined and expanded to encompass the contextual requirements not present.Results:The result is a two-level framework to address the need to maintain the intrinsic value of the data, as well as the need to indicate whether the data will be able to provide the basis for answers in specific areas of interest or questions.Discussion:Data quality frameworks have always been one dimensional, requiring the implementers of these frameworks to fit the requirements of the data’s use around how the framework is designed to function. Our work has systematically addressed the shortcomings of existing frameworks, through the application of concepts synthesized from the literature to the naturalistic setting of data quality management in an actual health data warehouse.Conclusion:Secondary use of health data relies on contextualized data quality management. Our work is innovative in showing how to apply context around data quality characteristics and how to develop a second level data quality framework, so as to ensure that quality and context are maintained and addressed throughout the health data quality assessment process.

Data Quality Assessment and Multi-Organizational Reporting: Tools to Enhance Network Knowledge.

Objective:Multi-organizational research requires a multi-organizational data quality assessment (DQA) process that combines and compares data across participating organizations. We demonstrate how such a DQA approach complements traditional checks of internal reliability and validity by allowing for assessments of data consistency and the evaluation of data patterns in the absence of an external “gold standard.”Methods:We describe the DQA process employed by the Data Coordinating Center (DCC) for Kaiser Permanente’s (KP) Center for Effectiveness and Safety Research (CESR). We emphasize the CESR DQA reporting system that compares data summaries from the eight KP organizations in a consistent, standardized manner.Results:We provide examples of multi-organization comparisons from DQA to confirm expectations about different aspects of data quality. These include: 1) comparison of direct data extraction from the electronic health records (EHR) and 2) comparison of non-EHR data from disparate sources.Discussion:The CESR DCC has developed codes and procedures for efficiently implementing and reporting DQA. The CESR DCC approach is to 1) distribute DQA tools to empower data managers at each organization to assess their data quality at any time, 2) summarize and disseminate findings to address data shortfalls or document idiosyncrasies, and 3) engage data managers and end-users in an exchange of knowledge about the quality and its fitness for use.Conclusion:The KP CESR DQA model is applicable to networks hoping to improve data quality. The multi-organizational reporting system promotes transparency of DQA, adds to network knowledge about data quality, and informs research.

Detector Monitoring with Artificial Neural Networks at the CMS Experiment at the CERN Large Hadron Collider

Reliable data quality monitoring is a key asset in delivering collision data suitable for physics analysis in any modern large-scale high energy physics experiment. This paper focuses on the use of artificial neural networks for supervised and semi-supervised problems related to the identification of anomalies in the data collected by the CMS muon detectors. We use deep neural networks to analyze LHC collision data, represented as images organized geographically. We train a classifier capable of detecting the known anomalous behaviors with unprecedented efficiency and explore the usage of convolutional autoencoders to extend anomaly detection capabilities to unforeseen failure modes. A generalization of this strategy could pave the way to the automation of the data quality assessment process for present and future high energy physics experiments.

Syndromic Surveillance – Reports of Successes from the Field

Objective: This panel will:● Discuss the importance of identifying and developing success stories● Highlight successes from state and local health departments to show how syndromic surveillance activities enhance situational awareness and address public health concerns● Encourage discussion on how to further efforts for developing and disseminating success storiesIntroduction: Syndromic surveillance uses near-real-time emergency department and other health care data for enhancing public health situational awareness and informing public health activities. In recent years, continued progress has been made in developing and strengthening syndromic surveillance activities. At the national level, syndromic surveillance activities are facilitated by the National Syndromic Surveillance Program (NSSP), a collaboration among state and local health departments, the CDC, other federal organizations, and other organizations that enabled collection of syndromic surveillance data in a timely manner, application of advanced data monitoring and analysis techniques, and sharing of best practices. This panel will highlight the importance of success stories. Examples of successes from state and local health departments will be presented and the audience will be encouraged to provide feedback.Description: ●Success stories – acknowledging and informing syndromic surveillance practiceThis presentation will discuss the importance of success stories for NSSP focused on increasing syndromic surveillance representativeness, improving data quality, and strengthening syndromic surveillance practices among grant recipients and partners. From the beginning of the program, the identification of success stories has been an important part of the efforts to develop knowledge base that better guide syndromic surveillance program activities.●NJ and BioSense – Making The Connection The New Jersey Department of Health (NJDOH) uses Health Monitoring’s EpiCenter as its primary ED data for syndromic surveillance. This data is also submitted to CDC’s NSSP BioSense Platform. In April 2017, a spike in ED Visits of Interest was identified by a CDC NSSP subject matter expert and brought to the attention of NJDOH’s data analyst. Data showed an increase in “Exposure” and “School Exposure” chief complaints in two contiguous counties. News reports showed the visits resulted from a dormitory fire at a university in the area. The NSSP and NJDOH staff collaboration integrated data from both NJDOH’s EpiCenter and CDC’s BioSense Platform for further investigation. This activity shows BioSense Platform’s potential as an additional syndromic surveillance tool because of its different classifications and keyword groupings.●Evaluation and Performance Measures at the Utah Department of HealthSyndromic surveillance related evaluation activities at the Utah Department of Health requires collaboration between subject matter experts and system users from the UT-NSSP workgroup. The progress is examined quarterly and outcomes compared with the short-, mid-, and long-term outcomes listed in the NSSP logic model to ensure activities are in sync with the program’s overall goals. Throughout the budget year, a variety of tools were used to keep track of the progress. During this session, challenges and successes, lessons learned, and effective strategies will be discussed.●NSSP R tool Data Download Useful in NHThe New Hampshire Department of Health and Human Services (NH DHHS) uses the state-wide Automated Hospital Emergency Department Data (AHEDD) system as its primary syndromic surveillance system. A copy of this data is submitted to CDC’s NSSP BioSense Platform. In July of 2017, NH worked with the NSSP vendor, CDC staff, a jurisdictional expert, NH Division of Information Technology staff, and an external vendor to create an “R” software download in CSV format and home-based NSSP Cognos report. This allowed NH DHHS staff to compare these data to the home-based data and ultimately, it proved to be an important step in the NSSP data quality assessment process.●Achieving success to improve data quality through collaborative Community of Practice partnerships The Data Quality Committee is a forum to identify, discuss, and attempt to address syndromic surveillance data quality issues. Maintaining data quality for the chief complaint field is a priority as it can impact the creation and refinement in the successful application of a syndrome definition for one of the fundamental data elements. An issue was observed in the Arizona data in the BioSense Platform, where chief complaint was being truncated at 200 characters. Through efforts to build relationships from the committee in the Community of Practice, Arizona was able to discover the root causes for the issue, assess if it affected other jurisdictions, and work with the partners to find a feasible resolution. This talk will discuss how this collaborative approach helped improve data quality.How the Moderator Intends to Engage the Audience in Discussions on the Topic: The moderator will introduce the session and the panelists, and will invite questions and comments from the audience.

Application of An Ontology for Characterizing Data Quality For a Secondary Use of EHR Data.

The goal of this study is to apply an ontology based assessment process to electronic health record (EHR) data and determine its usefulness in characterizing data quality for calculating an example eMeasure (CMS178). The process uses a data quality ontology that references separate data quality, domain and task ontologies to compute measures based on proportions of constraints that are satisfied. These quantities indicate how well the data conforms to the domain and how well it fits the task. The process was performed on a de-identified 200,000 encounter sample from a hospital EHR. CodingConsistency was poor (44%) but DomainConsistency (97%) and TaskRelevance (95%) were very good. Improvements in the data quality Measures correlated with improvements in the eMeasure. This approach can encourage the development of new detailed Domain ontologies that can be reused for data quality purposes across different organizations' EHR data. Automating the data quality assessment process using this method can enable sharing of data quality metrics that may aid in making research results that use EHR data more transparent and reproducible.

Data Quality Monitoring Display for ATLAS experiment at the LHC

The start of collisions at the LHC brings a new era of particle physics and much improved potential to observe signatures of new physics. Some of these may be evident already from the very beginning of collisions. It's essential at this point in the experiment to be prepared to quickly and efficiently determine the quality of the incoming data. Easy visualization of data for the shift crew and experts is one of the key factors in the data quality assessment process. This paper describes the design and implementation of the Data Quality Monitoring Display and discusses experience from its usage and performance during ATLAS commissioning with cosmic ray and single beam data.

Guidance for evaluating in vivo fish bioaccumulation data

Currently, the laboratory-derived fish bioconcentration factor (BCF) serves as one of the primary data sources used to assess the potential for a chemical to bioaccumulate. Consequently, fish BCF values serve a central role in decision making and provide the basis for development of quantitative structure-property relationships (QSPRs) used to predict the bioaccumulation potential of untested compounds. However, practical guidance for critically reviewing experimental BCF studies is limited. This lack of transparent guidance hinders improvement in predictive models and can lead to uninformed chemical management decisions. To address this concern, a multiple-stakeholder workshop of experts from government, industry, and academia was convened by the International Life Sciences Institute Health and Environmental Sciences Institute to examine the data availability and quality issues associated with in vivo fish bioconcentration and bioaccumulation data. This paper provides guidance for evaluating key aspects of study design and conduct that must be considered when judging the reliability and adequacy of reported laboratory bioaccumulation data. Key criteria identified for judging study reliability include 1) clear specification of test substance and fish species investigated, 2) analysis of test substance in both fish tissue and exposure medium, 3) no significant adverse effects on exposed test fish, and 4) a reported test BCF that reflects steady-state conditions with unambiguous units. This guidance is then applied to 2 data-rich chemicals (anthracene and 2,3,7,8-tetrachlorodibenzo-p-dioxin) to illustrate the critical need for applying a systematic data quality assessment process. Use of these guidelines will foster development of more accurate QSPR models, improve the performance and reporting of future laboratory studies, and strengthen the technical basis for bioaccumulation assessment in chemicals management.