Address Data Quality Issues Research Articles

What’s in a name field?

The National Center for Health Statistics (NCHS) has a data linkage program that combines national survey data with key sources of health outcomes and health care utilization. The overall accuracy and quality of a data linkage depends on the quality of the data fields. This applies in a variety of data linkage methods, including clear text and privacy preserving record linkage. Data pre-processing and cleaning are essential to address data quality issues in most linkage tasks. Automating data cleaning prevents time-consuming manual reviews particularly when linkages involve a large number of records. For some data fields, these activities are relatively straightforward. For example, dates typically have a limited number of plausible values that make checking and cleaning relatively easy and straightforward. Unique identifiers (e.g., social security number) often conform to some set format or have restrictions on the values that would be expected. Other data fields such as first name and last name present greater challenges with respect to automating the cleaning process. For some data sources, first name and last name fields may contain non-name text that needs to be identified and removed. In this project, we examined the use of an artificial intelligence (AI) based large language model (LLM) and a simple rule-based algorithm to identify non-name text in name fields that should be removed prior to linkage. We also investigated the impact of automated name clean-up algorithms on the quality of an example linkage.

Data Integration for a Global Deep-Phenotyping Registry for Pulmonary Hypertension - Lessons Learned.

The integration of data from various healthcare centers into disease registries is pivotal for facilitating collaborative research and enhancing clinical insights. In this study, we investigate the integration process of existing registries into the PVRI GoDeep meta-registry, focusing on the complexities and challenges encountered. We detail the integration process, including data transformation, mapping updates, and feedback mechanisms. Our findings underscore the importance of standardized processes and proactive communication in addressing data quality issues, ultimately enhancing the reliability and trustworthiness of meta-registry data. Through careful harmonization of the data and transparent documentation of data processing, we pave the way for leveraging registry data to drive advancements in pulmonary hypertension research and patient care.

Data quality assurance practices in research data repositories—A systematic literature review

AbstractData quality issues can significantly hinder research reproducibility, data sharing, and reuse. At the forefront of addressing data quality issues are research data repositories (RDRs). This study conducted a systematic analysis of data quality assurance (DQA) practices in RDRs, guided by activity theory and data quality literature, resulting in conceptualizing a data quality assurance model (DQAM) for RDRs. DQAM outlines a DQA process comprising evaluation, intervention, and communication activities and categorizes 17 quality dimensions into intrinsic and product‐level data quality. It also details specific improvement actions for data products and identifies the essential roles, skills, standards, and tools for DQA in RDRs. By comparing DQAM with existing DQA models, the study highlights its potential to improve these models by adding a specific DQA activity structure. The theoretical implication of the study is a systematic conceptualization of DQA work in RDRs that is grounded in a comprehensive analysis of the literature and offers a refined conceptualization of DQA integration into broader frameworks of RDR evaluation. In practice, DQAM can inform the design and development of DQA workflows and tools. As a future research direction, the study suggests applying and evaluating DQAM across various domains to validate and refine this model further.

XRAInet: AI-based decision support for pneumothorax and pleural effusion management.

This study aimed to develop and assess the performance of an artificial intelligence(AI)-driven decision support system, XRAInet, in accurately identifying pediatric patients with pleural effusion or pneumothorax and determining whether tube thoracostomy intervention is warranted. In this diagnostic accuracy study, we retrospectively analyzed a data set containing 510 X-ray images from 170 pediatric patients admitted between 2005 and 2022. Patients were categorized into two groups: Tube (requiring tube thoracostomy) and Conservative (managed conservatively). XRAInet, a deep learning-based algorithm, was trained using this data set. We evaluated its performance using various metrics, including mean Average Precision (mAP), recall, precision, and F1 score. XRAInet, achieved a mAP score of 0.918. This result underscores its ability to accurately identify and localize regions necessitating tube thoracostomy for pediatric patients with pneumothorax and pleural effusion. In an independent testing data set, the model exhibited a sensitivity of 64.00% and specificity of 96.15%. In conclusion, XRAInet presents a promising solution for improving the detection and decision-making process for cases of pneumothorax and pleural effusion in pediatric patients using X-ray images. These findings contribute to the expanding field of AI-driven medical imaging, with potential applications for enhancing patient outcomes. Future research endeavors should explore hybrid models, enhance interpretability, address data quality issues, and align with regulatory requirements to ensure the safe and effective deployment of XRAInet in healthcare settings.

AI in Dermatology: A Systematic Review on Skin Cancer Detection

Skin cancer is the most common type of cancer worldwide and poses a significant public health challenge. Its visible nature often leads individuals to seek medical attention, highlighting the importance of early detection for better patient outcomes. In recent years, Artificial Intelligence (AI) has shown promise in improving the detection and diagnosis of skin cancer, offering the potential to enhance clinical outcomes. A systematic review was conducted, involving a comprehensive literature search to identify studies focused on AI techniques in detecting, diagnosing, or treating skin cancer. Strict inclusion and exclusion criteria were applied to assess the eligibility of scientific articles, resulting in the selection of nine relevant studies. These studies were analyzed to address predefined research questions about the effectiveness of AI in diagnosing skin cancer. The review found that AI-assisted clinicians achieved higher sensitivity and specificity in diagnosing skin cancer than those without assistance. Various AI algorithms demonstrated high sensitivity in detecting skin cancers, highlighting their potential to support primary care clinicians in evaluating suspicious lesions. The analysis also highlighted the effectiveness of smartphone applications designed for skin cancer risk assessment, which could facilitate self-examinations and enhance early detection rates. Despite these promising findings, the field of AI in skin cancer diagnosis is still in its early stages. Challenges remain, including developing robust algorithms, addressing data quality issues, and improving the interpretability of AI-generated results. Collaboration between AI developers and healthcare professionals is crucial to ensure these tools' clinical effectiveness and safety. The review emphasizes the need for continued validation of AI technologies and their integration into clinical practice to improve patient outcomes and alleviate the burden on healthcare systems.

SCSQ: A sample cooperation optimization method with sample quality for recurrent neural networks

Time series forecasting holds significant value across various application scenarios. However, practical implementations often encounter challenges related to low-quality time series data resulting from system failures or external interference. Unfortunately, existing forecasting methods primarily focus on optimizing model architecture for accurate predictions and overlook the importance of addressing data quality issues. In this paper, we propose optimizing data utilization to enhance model performance based on data quality and introduce the Sample Cooperation with Sample Quality (SCSQ) method to facilitate recurrent neural networks training. Firstly, we define sample quality as the matching degree between samples and model, and suggest using the attention entropy to calculate the sample quality through an attention mechanism. Secondly, we optimize the model's gradient vector based on sample quality. To address optimization training involving samples of different qualities effectively, we propose a more reasonable objective function within our proposed sample gradient conflict optimization module and devise a novel algorithm leveraging approximation techniques along with ADMM algorithm implementation. Through experiments conducted on six datasets, the results demonstrate that SCSQ significantly improves LSTM's performance. In certain cases, it even surpasses the state-of-the-art models. Additionally, SCSQ enhances the anti-interference ability of LSTM against low-quality samples.

Electronic Health Records for Predicting Outcomes to Work-Related Musculoskeletal Disorders: A Scoping Review.

Through electronic health records (EHRs), musculoskeletal (MSK) therapists such as chiropractors and physical therapists, as well as occupational medicine physicians could collect data on many variables that can be traditionally challenging to collect in managing work-related musculoskeletal disorders (WMSDs). The review's objectives were to explore the extent of research using EHRs in predicting outcomes of WMSDs by MSK therapists. A systematic search was conducted in Medline, PubMed, CINAHL, and Embase. Grey literature was searched. 2156 unique papers were retrieved, of which 38 were included. Three themes were explored, the use of EHRs to predict outcomes to WMSDs, data sources for predicting outcomes to WMSDs, and adoption of standardised information for managing WMSDs. Predicting outcomes of all MSK disorders using EHRs has been researched in 6 studies, with only 3 focusing on MSK therapists and 4 addressing WMSDs. Similar to all secondary data source research, the challenges include data quality, missing data and unstructured data. There is not yet a standardised or minimum set of data that has been defined for MSK therapists to collect when managing WMSD. Further work based on existing frameworks is required to reduce the documentation burden and increase usability. The review outlines the limited research on using EHRs to predict outcomes of WMSDs. It highlights the need for EHR design to address data quality issues and develop a standardised data set in occupational healthcare that includes known factors that potentially predict outcomes to help regulators, research efforts, and practitioners make better informed clinical decisions.

“Allot?” is “A Lot!” Towards Developing More Generalized Speech Recognition System for Accessible Communication

The proliferation of Automatic Speech Recognition (ASR) systems has revolutionized translation and transcription. However, challenges persist in ensuring inclusive communication for non-native English speakers. This study quantifies the gap between accented and native English speech using Wav2Vec 2.0, a state-of-the-art transformer model. Notably, we found that accented speech exhibits significantly higher word error rates of 30-50%, in contrast to native speakers’ 2-8% (Baevski et al. 2020). Our exploration extends to leveraging accessible online datasets to highlight the potential of enhancing speech recognition by fine-tuning the Wav2Vec 2.0 model. Through experimentation and analysis, we highlight the challenges with training models on accented speech. By refining models and addressing data quality issues, our work presents a pipeline for future investigations aimed at developing an integrated system capable of effectively engaging with a broader range of individuals with diverse backgrounds. Accurate recognition of accented speech is a pivotal step toward democratizing AI-driven communication products.

Improving the Predictive Analytics of Machine-Learning Pipelines for Bridge Infrastructure Asset Management Applications: An Upstream Data Workflow to Address Data Quality Issues in the National Bridge Inventory Database

Improving the Predictive Analytics of Machine-Learning Pipelines for Bridge Infrastructure Asset Management Applications: An Upstream Data Workflow to Address Data Quality Issues in the National Bridge Inventory Database

Data migration strategies in mergers and acquisitions: A case study of banking sector

Data migration is a critical component of mergers and acquisitions (M&A), particularly in the banking sector, where seamless integration of information systems is essential for operational continuity and regulatory compliance. This review examines data migration strategies employed during M&A activities, focusing on a case study in the banking sector. The study explores the challenges, methodologies, and best practices that ensure successful data transfer while minimizing risks and disruptions. In the context of banking, M&A transactions often involve integrating vast amounts of sensitive data, including customer records, financial transactions, and regulatory documents. Effective data migration is crucial to preserving data integrity, maintaining customer trust, and achieving the strategic goals of the merger or acquisition. This case study highlights the importance of a well-structured data migration plan, which includes comprehensive data mapping, validation processes, and robust security measures. The review delves into the various strategies adopted for data migration, such as phased migration, parallel run strategies, and the use of automated tools to streamline the process. It also discusses the significance of collaboration between IT and business teams to address data quality issues, ensure compliance with regulatory standards, and align migration efforts with the overall M&A objectives. Furthermore, the study examines the challenges faced during the data migration process, including legacy system integration, data redundancy, and the need for real-time data synchronization. By analyzing the outcomes of the case study, the review provides insights into the lessons learned and the critical success factors that can guide future M&A activities in the banking sector. In conclusion, this study underscores the pivotal role of effective data migration strategies in ensuring a smooth transition during mergers and acquisitions in the banking sector, ultimately contributing to the long-term success of the consolidated entity. Keywords: Data Migration, Mergers and Acquisitions, Banking Sector, Data Integration, Data Mapping, Regulatory Compliance, Phased Migration, Parallel Run Strategy, Legacy Systems, Data Synchronization.

Setting the stage for Measurement-Based Care (MBC): Practical Lessons in the Implementation and Integration of MBC within Youth Wellness Hubs Ontario

Problem and Context: Measurement-based care (MBC) involves the systematic administration of standardized measures and the use of results to drive clinical decision-making within a therapeutic relationship framework. It is an integral part of youth-centered care and a core standard of integrated youth mental health services. Despite the wealth of evidence for the benefits of MBC in supporting high quality care, the implementation of MBC into routine mental health care is rarely incorporated.
 Who is it for? Youth mental health problems are a growing concern in Canada and globally, causing significant distress, impairment, and negative adult outcomes when untreated. Despite evidence-based interventions, only a minority of youth access and receive adequate treatment, with most youth facing barriers to high quality and effective interventions. To address these gaps, integrated youth service (IYS) models are established globally for youth ages 12-25 and emphasize timely access, youth friendliness, and holistic care, integrate mental and physical health, substance use, education, employment, peer support, and navigation into ‘one-stop shops’. Youth Wellness Hubs Ontario (YWHO) is Ontario’s IYS network with 22 integrated service networks in more than 30 communities across the province. MBC is a core component of the YWHO model and is implemented in 22 sites.
 Who did you involve and engage with? A fundamental contribution of YWHO is the inclusion of meaningful youth and family engagement processes in service design, delivery, and evaluation. YWHO has youth and family advisory councils both at the provincial and local levels who contribute to planning and operations, including service planning, governance, training, evaluation, communications, and funding-related decision-making.
 What did you do? This presentation will present the strategies, challenges, solutions, and subsequent adaptations we used to implement MBC within integrated youth services. We will describe the lessons we learned as we confronted practical obstacles around implementing MBC into integrated care pathways.
 What results did you get? MBC lessons learned included: 1. Leadership, service provider, and youth engagement in MBC; selection of measures and when to use measures, integrated data platform and electronic health records, addressing data quality issues early on, preparing for a demanding process of change, establishing support and coaching mechanisms, adapting MBC to a virtual environment, and adapting MBC in non-dominant cultural contexts.
 
 What is the learning for the international audience? MBC has been shown to improve the quality of care and clinical outcomes but it also requires substantial commitment, time, resources, and change management that can make it difficult to implement well in integrated care settings. Despite these challenges, we provide some creative solutions and adaptations within our model of youth-centered care that will inform audiences of how to effectively use MBC to personalize care, and how best to use MBC to engage service providers and youth in monitoring and management of symptoms. 
 What are the next steps? Future directions include continuous learning and evaluation of the implementation of MBC in integrated care settings and identifying the factors for successful implementation.

Promoting High-Quality Data in OBIS: Insights from the OBIS Data Quality Assessment and Enhancement Project Team

The Ocean Biodiversity Information System (OBIS) (Klein et al. 2019) is a global database of marine biodiversity and associated environmental data, which provides critical information to researchers and policymakers worldwide. Ensuring the accuracy and consistency of the data in OBIS is essential for its usefulness and value, not only to the scientific community but also to the science-policy interface. The OBIS Data Quality Assessment and Enhancement Project Team (QCPT), formed in 2019 by the OBIS steering group, aims to assess and enhance data quality. It has been working on three categories of activities for this purpose: Data quality enhancement and management The OBIS QCPT organized data laundry events to identify and address data quality issues of published OBIS datasets. Furthermore, individual OBIS nodes were invited to give their data-processing presentations in the monthly meetings to foster knowledge sharing and collaborative problem-solving focused on data quality. Data quality issues and solutions highlighted in the presentations and data laundry events were documented in a dedicated GitHub repository as GitHub issues. The solutions for data quality issues and marine-specific pre-publication quality control tools, designed to identify the data quality issues, were provided as feedback to the OBIS Capacity Development Task Team. These inputs were used to create training resources (see OBIS manual, upcoming OBIS training course hosted on OceanTeacher Global Academy) aimed at preventing these issues. Standardization of OBIS data processing pipeline As OBIS uses the Darwin Core standard (Wieczorek et al. 2012), the use of standardized tests and assertions in the data processing pipeline is encouraged. To achieve this, the OBIS QCPT aligned OBIS quality checks with a subset of core tests and assertions (Chapman et al. 2020) developed by the Biodiversity Information Standards (TDWG) Biodiversity Data Quality (BDQ) Task Group 2 (TG2) (Chapman et al. 2020) as tracked in this GitHub issue. Not all default parameters of the core tests and assertions are optimal for marine biodiversity data. The OBIS QCPT met monthly to determine suitable parameters for customizing the tests. The pipeline produces a data quality report for each dataset with quality flags that indicate potential data quality issues, enabling node managers and data providers to review the flagged records. Community engagement The OBIS QCPT led a survey among data users to gather insights into OBIS data quality issues and bridge the gap between the current implementation and user expectations. The survey findings enabled OBIS to prioritize issues to be addressed, as summarized in Section 2.2.2 of the 11th OBIS Steering Group meeting report. In addition to engaging with data users, the OBIS QCPT also served as a platform to discuss questions related to the use of Darwin Core from the nodes and provided feedback for the term discussions. In summary, the OBIS QCPT improves marine species data reliability and usability through transparent and participatory approaches, fostering continuous improvement. Collaborative efforts, standardized procedures, and knowledge sharing advance OBIS' mission of providing high quality biodiversity data for research, conservation, and ocean management.

A new national health system: the opportunity to address data quality issues in maternal immunisation coverage.

Maternal immunisation coverage is suboptimal in Aotearoa New Zealand. Our objective was to highlight discrepancies resulting from how maternal immunisation coverage for pertussis and influenza is measured in Aotearoa New Zealand. A retrospective cohort study of pregnant people was undertaken using administrative datasets. Maternity and immunisation data from three sources (National Immunisation Register [NIR], general practice [GP], and pharmaceutical claims) were linked to determine the proportion of immunisation records not recorded in the NIR but captured in claims data, and to compare this with coverage data available from Te Whatu Ora - Health New Zealand. We found that while increasing numbers of maternal immunisations are being captured in the NIR, around 10% remain unrecorded on the NIR, but within claims datasets. Accurate maternal immunisation coverage data is important for public health action. Implementation of the whole-of-life Aotearoa Immunisation Register (AIR) is an important opportunity to improve completeness and consistency of maternal immunisation coverage reporting.

Integrating statistical and visual analytic methods for bot identification of health-related survey data

ObjectiveIn recent years, we have increasingly observed issues concerning quality of online information due to misinformation and disinformation. Aside from social media, there is growing awareness that questionnaire data collected using online recruitment methods may include suspect data provided by bots. Issues with data quality can be particularly problematic in health and/or biomedical contexts; thus, developing robust methods for suspect data identification and removal is of paramount importance in informatics. In this study, we describe an interactive visual analytics approach to suspect data identification and removal and demonstrate the application of this approach on questionnaire data pertaining to COVID-19 derived from different recruitment venues, including listservs and social media. MethodsWe developed a pipeline for data cleaning, pre-processing, analysis, and automated ranking of data to address data quality issues. We then employed the ranking in conjunction with manual review to identify suspect data and remove them from subsequent analyses. Last, we compared differences in the data before and after removal. ResultsWe performed data cleaning, pre-processing, and exploratory analysis on a survey dataset (N = 4,163) collected using multiple recruitment mechanins using the Qualtrics survey platform. Based on these results, we identified suspect features and used these to generate a suspect feature indicator for each survey response. We excluded survey responses that did not fit the inclusion criteria for the study (n = 29) and then performed manual review of the remaining responses, triangulating with the suspect feature indicator. Based on this review, we excluded 2,921 responses. Additional responses were excluded based on a spam classification by Qualtrics (n=13), and the percentage of survey completion (n=328), resulting in a final sample size of 872. We performed additional analyses to demonstrate the extent to which the suspect feature indicator was congruent with eventual inclusion, as well as compared the characteristics of the included and excluded data. ConclusionOur main contributions are: 1) a proposed framework for data quality assessment, including suspect data identification and removal; 2) the analysis of potential consequences in terms of representation bias in the dataset; and 3) recommendations for implementation of this approach in practice.

Machine Learning-Based Ensemble Classifiers for Anomaly Handling in Smart Home Energy Consumption Data

Addressing data anomalies (e.g., garbage data, outliers, redundant data, and missing data) plays a vital role in performing accurate analytics (billing, forecasting, load profiling, etc.) on smart homes' energy consumption data. From the literature, it has been identified that the data imputation with machine learning (ML)-based single-classifier approaches are used to address data quality issues. However, these approaches are not effective to address the hidden issues of smart home energy consumption data due to the presence of a variety of anomalies. Hence, this paper proposes ML-based ensemble classifiers using random forest (RF), support vector machine (SVM), decision tree (DT), naive Bayes, K-nearest neighbor, and neural networks to handle all the possible anomalies in smart home energy consumption data. The proposed approach initially identifies all anomalies and removes them, and then imputes this removed/missing information. The entire implementation consists of four parts. Part 1 presents anomaly detection and removal, part 2 presents data imputation, part 3 presents single-classifier approaches, and part 4 presents ensemble classifiers approaches. To assess the classifiers' performance, various metrics, namely, accuracy, precision, recall/sensitivity, specificity, and F1 score are computed. From these metrics, it is identified that the ensemble classifier "RF+SVM+DT" has shown superior performance over the conventional single classifiers as well the other ensemble classifiers for anomaly handling.

Enhancing Incident Management in Financial Services through Advanced Data Analytics

This study examines how advanced data analytics can transform incident management in the financial sector by addressing data quality issues, segmenting customer data, and using predictive analytics. It presents a comprehensive approach to improve incident data's accuracy and utility, boosting operational efficiency, customer satisfaction, and regulatory compliance. The paper includes real-time examples to showcase analytics implementation, aiming to enable financial institutions to anticipate and address challenges proactively through data-driven strategies.

Local data commons: the sleeping beauty in the community of data commons

BackgroundPublic Data Commons (PDC) have been highlighted in the scientific literature for their capacity to collect and harmonize big data. On the other hand, local data commons (LDC), located within an institution or organization, have been underrepresented in the scientific literature, even though they are a critical part of research infrastructure. Being closest to the sources of data, LDCs provide the ability to collect and maintain the most up-to-date, high-quality data within an organization, closest to the sources of the data. As a data provider, LDCs have many challenges in both collecting and standardizing data, moreover, as a consumer of PDC, they face problems of data harmonization stemming from the monolithic harmonization pipeline designs commonly adapted by many PDCs. Unfortunately, existing guidelines and resources for building and maintaining data commons exclusively focus on PDC and provide very little information on LDC.ResultsThis article focuses on four important observations. First, there are three different types of LDC service models that are defined based on their roles and requirements. These can be used as guidelines for building new LDC or enhancing the services of existing LDC. Second, the seven core services of LDC are discussed, including cohort identification and facilitation of genomic sequencing, the management of molecular reports and associated infrastructure, quality control, data harmonization, data integration, data sharing, and data access control. Third, instead of commonly developed monolithic systems, we propose a new data sharing method for data harmonization that combines both divide-and-conquer and bottom-up approaches. Finally, an end-to-end LDC implementation is introduced with real-world examples.ConclusionsAlthough LDCs are an optimal place to identify and address data quality issues, they have traditionally been relegated to the role of passive data provider for much larger PDC. Indeed, many LDCs limit their functions to only conducting routine data storage and transmission tasks due to a lack of information on how to design, develop, and improve their services using limited resources. We hope that this work will be the first small step in raising awareness among the LDCs of their expanded utility and to publicize to a wider audience the importance of LDC.

Electronic case reporting (eCR) of COVID-19 to public health: implementation perspectives from the Minnesota Department of Health.

Electronic case reporting (eCR) is the automated generation and transmission of case reports from electronic health records to public health for review and action. These reports (electronic initial case reports: eICRs) adhere to recommended exchange and terminology standards. eCR is a partnership of the Centers for Disease Control and Prevention (CDC), Association of Public Health Laboratories (APHL) and Council of State and Territorial Epidemiologists (CSTE). The Minnesota Department of Health (MDH) received eICRs for COVID-19 from April 2020 (3 sites, manual process), automated eCR implementation in August 2020 (7 sites), and on-boarded ∼1780 clinical units in 460 sites across 6 integrated healthcare systems (through March 2022). Approximately 20 000 eICRs/month were reported to MDH during high-volume timeframes. With increasing provider/health system implementation, the proportion of COVID-19 cases with an eICR increased to 30% (March 2022). Evaluation of data quality for select demographic variables (gender, race, ethnicity, email, phone, language) across the 6 reporting health systems revealed a high proportion of completeness (>80%) for half of variables and less complete data for rest (ethnicity, email, language) along with low ethnicity data (<50%) for one health system. Presently eCR implementation at MDH includes only one EHR vendor. Next steps will focus on onboarding other EHRs, additional eICR data extraction/utilization, detailed analysis, outreach to address data quality issues, and expanding to other reportable conditions.

How Can Interactive Process Discovery Address Data Quality Issues in Real Business Settings? Evidence from a Case Study in Healthcare

The focus of this paper is on how data quality can affect business process discovery in real complex environments, which is a major factor determining the success in any data-driven Business Process Management project. Many real-life event logs, especially healthcare ones, can suffer from several data quality issues, some of which cannot be solved by pre-processing or data cleaning techniques, leading to inaccurate results. We take an innovative Process Mining (PM) approach, termed Interactive Process Discovery (IPD), which combines domain knowledge with available data. This approach can overcome the limitations of noisy and incomplete event logs by putting “humans in the loop”, leading to improved business process modelling. This is particularly valuable in healthcare, where physicians have a tacit domain knowledge not available in the event log, and, thus, difficult to elicit. We conducted a two-step approach based on a controlled experiment and a case study in an Italian hospital. At each step, we compared IPD with traditional PM techniques to assess the extent to which domain knowledge helps to improve the accuracy of process models. The case study tests the effectiveness of IPD to uncover knowledge-intensive processes extracted from noisy real-life event logs. The evaluation has been carried out by exploiting a real dataset of an Italian hospital, involving the medical staff. IPD can produce an accurate process model that is fully compliant with the clinical guidelines by addressing data quality issues. Accurate and reliable process models can support healthcare organizations in detecting process-related issues and in taking decisions related to capacity planning and process re-design.

Data-Driven Techniques for Low-Cost Sensor Selection and Calibration for the Use Case of Air Quality Monitoring.

With the emergence of Low-Cost Sensor (LCS) devices, measuring real-time data on a large scale has become a feasible alternative approach to more costly devices. Over the years, sensor technologies have evolved which has provided the opportunity to have diversity in LCS selection for the same task. However, this diversity in sensor types adds complexity to appropriate sensor selection for monitoring tasks. In addition, LCS devices are often associated with low confidence in terms of sensing accuracy because of the complexities in sensing principles and the interpretation of monitored data. From the data analytics point of view, data quality is a major concern as low-quality data more often leads to low confidence in the monitoring systems. Therefore, any applications on building monitoring systems using LCS devices need to focus on two main techniques: sensor selection and calibration to improve data quality. In this paper, data-driven techniques were presented for sensor calibration techniques. To validate our methodology and techniques, an air quality monitoring case study from the Bradford district, UK, as part of two European Union (EU) funded projects was used. For this case study, the candidate sensors were selected based on the literature and market availability. The candidate sensors were narrowed down into the selected sensors after analysing their consistency. To address data quality issues, four different calibration methods were compared to derive the best-suited calibration method for the LCS devices in our use case system. In the calibration, meteorological parameters temperature and humidity were used in addition to the observed readings. Moreover, we uniquely considered Absolute Humidity (AH) and Relative Humidity (RH) as part of the calibration process. To validate the result of experimentation, the Coefficient of Determination (R2), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE) were compared for both AH and RH. The experimental results showed that calibration with AH has better performance as compared with RH. The experimental results showed the selection and calibration techniques that can be used in designing similar LCS based monitoring systems.