Data Quality Management Framework Research Articles

A data quality management framework for equipment failure risk estimation: Application to the oil and gas industry

In engineering asset management, accurate failure risk estimation is essential for averting equipment breakdowns and optimizing risk-based maintenance strategies. Data quality and model fitting are two critical sources of prediction uncertainty in risk estimation. While much attention has been devoted to model fitting for risk estimation, the critical role of data quality has often been overshadowed. To bridge this research gap, this paper presents a novel data quality management framework tailored for industrial equipment failure risk estimation. The framework covers the steps from data to model. It consists of the following main phases: data development, data quality assessment, data quality requirement decision-making, data quality improvement, and risk estimation model development. The framework provides detailed guidelines that can facilitate data practitioners to build individualized data quality requirement decision-making models for failure risk estimation of their equipment. The decision-making model can measure the adequacy of existing data to build a risk estimation model that meets the specified requirements and further determines the best risk estimation model given the available data. A case study using actual data collected during oil well drilling operations from multiple oil fields demonstrates the practicality and effectiveness of the framework. In this case study, four risk estimation models are compared, including two baseline models (mean time to failure and median time to failure) and two machine-learning models (quantile regression and hidden Markov model). In addition, a decision tree-based decision model is developed to determine whether the data quality meets the requirements and the best risk estimation model in case the data quality meets the requirements.

Designing a Data Quality Management Framework for CRM Platform Delivery and Consultancy

CRM platforms heavily depend on high-quality data, where poor-quality data can negatively influence its adoption. Additionally, these platforms are increasingly interconnected and complex to meet the growing needs of customers. Hence, delivery and consultancy of CRM platforms becomes highly complex. In this study, we propose a CRM data quality management framework that supports CRM delivery and consultancy firms to improve data quality management practices within their CRM projects. We develop the framework by extracting best practices for CRM data quality management by means of a literature study on data quality definition and measurement, data quality challenges, and data quality management methods. In a case study at an IT consultancy company, we investigate how CRM delivery and consultancy projects can benefit from the incorporation of data quality management practices. The results translate into a framework that provides a high-level overview of data quality management practices incorporated in CRM delivery and consultancy projects. It includes the following components: Client profiling, project definition, preparation, migration/integration, data quality definition, assessment, and improvement. The framework is validated by means of confirmatory focus groups and a questionnaire.

(단보)국립공원공단의 전사적 데이터 관리 기반 및 정책에 관한 고찰

Based on the results of the recent〝4th Korea National Park Service Information Strategy Plan〞 establishment and related laws and policies, we want to understand the current level of data management in Korea National Park Service. As a result of confirming the level of enterprise-wide data quality management using the data profiling technique, it is difficult to secure data standard or structural consistency, and the basis for data quality management such as guidelines, manuals, organization and budget is insufficient. The company-wide data of the National Park Service should be managed to ensure accuracy, consistency, timelessness, and completeness, and to realize this, an implementation plan is established according to the data quality management framework.

Data quality assessment in digital score libraries

Sheet music scores have been the traditional way to preserve and disseminate western classical music works for centuries. Nowadays, their content can be encoded in digital formats that yield a very detailed representation of music content expressed in the language of music notation. These digital scores constitute, therefore, an invaluable asset for digital library services such as search, analysis, clustering, recommendations, and synchronization with audio files. Digital scores, like any other published data, may suffer from quality problems. For instance, they can contain incomplete or inaccurate elements. As a “dirty” dataset may be an irrelevant input for some use cases, users need to be able to estimate the quality level of the data they are about to use. This article presents the data quality management framework for digital score libraries (DSL) designed by the GioQoso multi-disciplinary project. It relies on a content model that identifies several information levels that are unfortunately blurred out in digital score encodings. This content model then serves as a foundation to organize the categories of quality issues that can occur in a music score, leading to a quality model. The quality model also positions each issue with respect to potential usage contexts, allowing attachment of a consistent set of indicators that together measure how a given score is fit to a specific usage. We finally report an implementation of these conceptual foundations in an online DSL.

Investments in data quality: Evaluating impacts of faulty data on asset management in power systems

Data play an essential role in asset management decisions. The amount of data is increasing through accumulating historical data records, new measuring devices, and communication technology, notably with the evolution toward smart grids. Consequently, the management of data quantity and quality is becoming even more relevant for asset managers to meet efficiency and reliability requirements for power grids. In this work, we propose an innovative data quality management framework enabling asset managers (i) to quantify the impact of poor data quality, and (ii) to determine the conditions under which an investment in data quality improvement is required. To this end, an algorithm is used to determine the optimal year for component replacement based on three scenarios, a Reference scenario, an Imperfect information scenario, and an Investment in higher data quality scenario. Our results indicate that (i) the impact on the optimal year of replacement is the highest for middle-aged components; (ii) the profitability of investments in data quality improvement depends on various factors, including data quality, and the cost of investment in data quality improvement. Finally, we discuss the implementation of the proposed models to control data quality in practice, while taking into account real-world technological and economic limitations.

Service Data Quality Management Framework to Enable Through-life Engineering Services

With the growth of the product-service system (PSS) in recent years, how to better manage the service data to improve the informed decision-making capability has become an on-going aim among the Through-life Engineering Services (TES) firms. This scenario has led managers, more and more, to turn their attention to the quality of data and information created, gathered and used within the company. Encouraged by this background, a service data quality framework has been developed aiming to provide companies with a set of methods and tools to prioritise relevant service data and assess its quality levels. The process involves four main steps that go through: (1) Mapping out important data for Through-life Support available within the company and its internal and external flows; (2) Application of a multiple criteria decision-making technique to prioritise the relevant data set considering its costs for being collected and maintained, business impact, frequency of use and ease of obtainability; (3) Quality assessment of the prioritised dataset, based on a capability maturity model; (4) Defining strategies to address data quality issues. Validation on an industrial case study demonstrates potential benefits of the process and further work opportunities.

A model-driven framework for data quality management in the Internet of Things

The internet of Things (IoT) is a data stream environment where a large scale deployment of smart things continuously report readings. These data streams are then consumed by pervasive applications, i.e. data consumers, to offer ubiquitous services. The data quality (DQ) is a key criteria for IoT data consumers especially when considering the inherent uncertainty of sensor-enabled data. However, DQ is a highly subjective concept and there is no standard agreement on how to determine “good” data. Moreover, the combinations of considered measured attributes and associated DQ information are as diverse as the needs of data consumers. This introduces expensive overheads for developers tasked with building DQ-aware IoT software systems which are capable of managing their own DQ information. To effectively handle these various perceptions of DQ, we propose a Model-Driven Architecture-based approach that allows each developer to easily and efficiently express, through models and other provided resources, the data consumer’s vision of DQ and its requirements using an easy-to-use graphical model editor. The defined DQ specifications are then automatically transformed to generate an entire infrastructure for DQ management that fits perfectly the data consumer’s requirements. We demonstrate the flexibility and the efficiency of our approach by generating two DQ management infrastructures built on top of different platforms and testing them through a real life data stream environment scenario.

A Mathematical Framework for Data Quality Management in Enterprise Systems

This paper addresses the issue of data quality management in information systems within an enterprise. Motivated by legislative mandates such as the Sarbanes–Oxley Act of 2002 on the reliability and integrity of the data and the enterprise systems from which the data are produced, we propose a process-based modeling framework to assess the impact of data errors in the business process information flow and the resulting data quality metrics. This framework is then integrated with a business control framework in which the placement and effectiveness of control procedures alter the propagation of errors and, ultimately, the quality of the data in the business process. This integrated framework enables mathematical formulations of managerial problems that lead to effective data quality control strategies. We develop a two-stage multiple-choice knapsack model as a special case, and we illustrate the model and analysis through a revenue realization process.

Consistency-driven data quality management of networked sensor systems

With more and more real deployments of wireless sensor network applications, we envision that their success is nonetheless determined by whether the sensor networks can provide a high quality stream of data over a long period. In this paper, we propose a consistency-driven data quality management framework called Orchis that integrates the quality of data into an energy efficient sensor system design. Orchis consists of four components, data consistency models, adaptive data sampling and process protocols, consistency-driven cross-layer protocols and flexible APIs to manage the data quality, to support the goals of high data quality and energy efficiency. We first formally define a consistency model, which not only includes temporal consistency and numerical consistency, but also considers the application-specific requirements of data and data dynamics in the sensing field. Next, we propose an adaptive lazy energy efficient data collection protocol, which adapts the data sampling rate to the data dynamics in the sensing field and keeps lazy when the data consistency is maintained. Finally, we conduct a comprehensive evaluation to the proposed protocol based on both a TOSSIM-based simulation and a real prototype implementation using MICA2 motes. The results from both simulation and prototype show that our protocol reduces the number of delivered messages, improves the quality of collected data, and in turn extends the lifetime of the whole network. Our analysis also implies that a tradeoff should be carefully set between data consistency requirements and energy saving based on the specific requirements of different applications.

Quality Assurance for Environmental Monitoring

Abstract A data quality management framework for ecological monitoring programs is described. A total quality management framework has three key elements: quality management planning, quality control, and quality assessment and audit. The quality management plan establishes the data quality objectives, the protocols and procedure documents to be followed, reporting schedules, training needs and the individuals to be held accountable. Quality control is the systematic set of procedures carried out by each operational unit involved in the measurement process. Quality assessment is the set of procedures designed to provide the overall check on data quality while verifying that the other components of the framework are adequate.