Data Curation Process Research Articles

Electronic health record data quality variability across a multistate clinical research network.

Electronic health record (EHR) data have many quality problems that may affect the outcome of research results and decision support systems. Many methods have been used to evaluate EHR data quality. However, there has yet to be a consensus on the best practice. We used a rule-based approach to assess the variability of EHR data quality across multiple healthcare systems. To quantify data quality concerns across healthcare systems in a PCORnet Clinical Research Network, we used a previously tested rule-based framework tailored to the PCORnet Common Data Model to perform data quality assessment at 13 clinical sites across eight states. Results were compared with the current PCORnet data curation process to explore the differences between both methods. Additional analyses of testosterone therapy prescribing were used to explore clinical care variability and quality. The framework detected discrepancies across sites, revealing evident data quality variability between sites. The detailed requirements encoded the rules captured additional data errors with a specificity that aids in remediation of technical errors compared to the current PCORnet data curation process. Other rules designed to detect logical and clinical inconsistencies may also support clinical care variability and quality programs. Rule-based EHR data quality methods quantify significant discrepancies across all sites. Medication and laboratory sources are causes of data errors.

Deep Learning and Antibiotic Resistance.

Antibiotic resistance (AR) is a naturally occurring phenomenon with the capacity to render useless all known antibiotics in the fight against bacterial infections. Although bacterial resistance appeared before any human life form, this process has accelerated in the past years. Important causes of AR in modern times could be the over-prescription of antibiotics, the presence of faulty infection-prevention strategies, pollution in overcrowded areas, or the use of antibiotics in agriculture and farming, together with a decreased interest from the pharmaceutical industry in researching and testing new antibiotics. The last cause is primarily due to the high costs of developing antibiotics. The aim of the present review is to highlight the techniques that are being developed for the identification of new antibiotics to assist this lengthy process, using artificial intelligence (AI). AI can shorten the preclinical phase by rapidly generating many substances based on algorithms created by machine learning (ML) through techniques such as neural networks (NN) or deep learning (DL). Recently, a text mining system that incorporates DL algorithms was used to help and speed up the data curation process. Moreover, new and old methods are being used to identify new antibiotics, such as the combination of quantitative structure-activity relationship (QSAR) methods with ML or Raman spectroscopy and MALDI-TOF MS combined with NN, offering faster and easier interpretation of results. Thus, AI techniques are important additional tools for researchers and clinicians in the race for new methods of overcoming bacterial resistance.

Physics-informed graph neural networks for predicting cetane number with systematic data quality analysis

Designing alternative fuels for advanced compression ignition engines necessitates a predictive model for cetane number (CN). In this study, the physics-informed graph neural networks are introduced for a reliable CN prediction by considering molecular features pertinent to the physical properties of molecules that affect CN. The reliability of measured data is another key factor to consider for improving the predictive model. Various experimental instruments for measuring CN exist, including standard and non-standard methods. In this regard, a systematic data quality analysis was carried out for the total 630 CNs collected from literature and new measurements in this study using Advanced Fuel Ignition Delay Analyzer (AFIDA). The results from this data curation process were reflected in the model by imposing lower sample weights on the data coming from less reliable measurement techniques. This approach effectively maximized the prediction accuracy while incorporating data from all available sources. Using the sample weights decreased the mean absolute error (MAE) up to 0.8 CN units. The accuracy was also improved by introducing the CN-related physical properties (the number of hydrogen bond donors and acceptors); the test set MAE is 5.74 and 7.01 for the model with and without such properties, respectively. Investigating molecular structural effects on CN was also carried out to gain chemical insights into factors used to design new fuel candidates. The dimensionality reduction analysis of feature vectors showed a clear clustering in terms of functional groups and CN and the structural effect derived from the model was consistent with the physicochemical insights. This physics-informed model and data curation would be helpful for accurate CN prediction and inform rational fuel design.

Incomplete COVID-19 Data: The Curation of Medical Health Data by the Virus Outbreak Data Network-Africa

AbstractThe incompleteness of patient health data is a threat to the management of COVID-19 in Africa and globally. This has become particularly clear with the recent emergence of new variants of concern. The Virus Outbreak Data Network (VODAN)-Africa has studied the curation of patient health data in selected African countries and identified that health information flows often do not involve the use of health data at the point of care, which renders data production largely meaningless to those producing it. This modus operandi leads to disfranchisement over the control of health data, which is extracted to be processed elsewhere. In response to this problem, VODAN-Africa studied whether or not a design that makes local ownership and repositing of data central to the data curation process, would have a greater chance of being adopted. The design team based their work on the legal requirements of the European Union's General Data Protection Regulation (GDPR); the FAIR Guidelines on curating data as Findable, Accessible (under well-defined conditions), Interoperable and Reusable (FAIR); and national regulations applying in the context where the data is produced. The study concluded that the visiting of data curated as machine actionable and reposited in the locale where the data is produced and renders services has great potential for access to a wider variety of data. A condition of such innovation is that the innovation team is intradisciplinary, involving stakeholders and experts from all of the places where the innovation is designed, and employs a methodology of co-creation and capacity-building.

From Green Deal to Cultural Heritage: FAIR Digital Objects and European Common Data Spaces

From Green Deal to Cultural Heritage: FAIR Digital Objects and European Common Data Spaces

A blueprint for enduring human services cross-jurisdictional data linkage, adapted to achieving outcomes for people with disability.

ObjectivesPeople with disability represent 1 in 6 Australians and use services provided across multiple levels of government. Data is siloed between different governments and service systems, limiting visibility and ability to improve outcomes. Our objective was to progress people-centred data sharing across jurisdictions at scale, focused on people with disability. ApproachMore than 140 engagements with the disability community, governments, researchers, and service providers were conducted to ascertain user and data needs; systematically map enablers and barriers to data use; and outline the requisite technical capabilities and social license for a disability asset. Five demonstration projects tested novel linkage approaches, data flows and access models to inform technical aspects of the design. We adopted an iterative try-test-learn approach and forged consensus through extensive dialogue across all stakeholders to collaboratively design a system for enduring data linkage. The final design was reviewed by external experts to ensure best practice and build trust. ResultsA national disability-focused data asset requires linking data across all human services portfolios, as will be required for most national assets seeking to improve outcomes for particular communities or in particular domains. The proposed approach is three building blocks: national data linkage and integration infrastructure, national data system governance, and streamlined data sharing agreements for multiple uses. Current project-by-project approaches to linkage are proposed to be replaced with new enduring national keys to connect disparate state-based infrastructure and enable streamlined, reusable data curation processes. Collaboration between jurisdictions will be embedded through joint governance of the new infrastructure. Finally, a new approach to data sharing agreements between Australian governments will be developed to create safe parameters for streamlined access within existing legislation and authorisation pathways. ConclusionBy designing to meet information needs across human services, this approach can be leveraged for multiple linked, longitudinal enduring data asset, beyond disability. It multiplies the benefits of upfront efforts to share and improve data, reflects a shared understanding among stakeholders, and moves Australia towards scaled use of people-centred data.

Toward a Unified TreeTalker Data Curation Process

The Internet of Things (IoT) development is revolutionizing environmental monitoring and research in macroecology. This technology allows for the deployment of sizeable diffuse sensing networks capable of continuous monitoring. Because of this property, the data collected from IoT networks can provide a testbed for scientific hypotheses across large spatial and temporal scales. Nevertheless, data curation is a necessary step to make large and heterogeneous datasets exploitable for synthesis analyses. This process includes data retrieval, quality assurance, standardized formatting, storage, and documentation. TreeTalkers are an excellent example of IoT applied to ecology. These are smart devices for synchronously measuring trees’ physiological and environmental parameters. A set of devices can be organized in a mesh and permit data collection from a single tree to plot or transect scale. The deployment of such devices over large-scale networks needs a standardized approach for data curation. For this reason, we developed a unified processing workflow according to the user manual. In this paper, we first introduce the concept of a unified TreeTalker data curation process. The idea was formalized into an R-package, and it is freely available as open software. Secondly, we present the different functions available in “ttalkR”, and, lastly, we illustrate the application with a demonstration dataset. With such a unified processing approach, we propose a necessary data curation step to establish a new environmental cyberinfrastructure and allow for synthesis activities across environmental monitoring networks. Our data curation concept is the first step for supporting the TreeTalker data life cycle by improving accessibility and thus creating unprecedented opportunities for TreeTalker-based macroecological analyses.

The ECOTOXicology Knowledgebase: A Curated Database of Ecologically Relevant Toxicity Tests to Support Environmental Research and Risk Assessment.

The need for assembled existing and new toxicity data has accelerated as the amount of chemicals introduced into commerce continues to grow and regulatory mandates require safety assessments for a greater number of chemicals. To address this evolving need, the ECOTOXicology Knowledgebase (ECOTOX) was developed starting in the 1980s and is currently the world's largest compilation of curated ecotoxicity data, providing support for assessments of chemical safety and ecological research through systematic and transparent literature review procedures. The recently released version of ECOTOX (Ver 5, www.epa.gov/ecotox) provides single-chemical ecotoxicity data for over 12,000 chemicals and ecological species with over one million test results from over 50,000 references. Presented is an overview of ECOTOX, detailing the literature review and data curation processes within the context of current systematic review practices and discussing how recent updates improve the accessibility and reusability of data to support the assessment, management, and research of environmental chemicals. Relevant and acceptable toxicity results are identified from studies in the scientific literature, with pertinent methodological details and results extracted following well-established controlled vocabularies and newly extracted toxicity data added quarterly to the public website. Release of ECOTOX, Ver 5, included an entirely redesigned user interface with enhanced data queries and retrieval options, visualizations to aid in data exploration, customizable outputs for export and use in external applications, and interoperability with chemical and toxicity databases and tools. This is a reliable source of curated ecological toxicity data for chemical assessments and research and continues to evolve with accessible and transparent state-of-the-art practices in literature data curation and increased interoperability to other relevant resources. Environ Toxicol Chem 2022;41:1520-1539. © 2022 SETAC. This article has been contributed to by US Government employees and their work is in the public domain in the USA.

Automatic Curation of Court Documents: Anonymizing Personal Data

In order to provide open access to data of public interest, it is often necessary to perform several data curation processes. In some cases, such as biological databases, curation involves quality control to ensure reliable experimental support for biological sequence data. In others, such as medical records or judicial files, publication must not interfere with the right to privacy of the persons involved. There are also interventions in the published data with the aim of generating metadata that enable a better experience of querying and navigation. In all cases, the curation process constitutes a bottleneck that slows down general access to the data, so it is of great interest to have automatic or semi-automatic curation processes. In this paper, we present a solution aimed at the automatic curation of our National Jurisprudence Database, with special focus on the process of the anonymization of personal information. The anonymization process aims to hide the names of the participants involved in a lawsuit without losing the meaning of the narrative of facts. In order to achieve this goal, we need, not only to recognize person names but also resolve co-references in order to assign the same label to all mentions of the same person. Our corpus has significant differences in the spelling of person names, so it was clear from the beginning that pre-existing tools would not be able to reach a good performance. The challenge was to find a good way of injecting specialized knowledge about person names syntax while taking profit of previous capabilities of pre-trained tools. We fine-tuned an NER analyzer and we built a clusterization algorithm to solve co-references between named entities. We present our first results, which, for both tasks, are promising: We obtained a 90.21% of F1-micro in the NER task—from a 39.99% score before retraining the same analyzer in our corpus—and a 95.95% ARI score in clustering for co-reference resolution.

DataOps-4G: On Supporting Generalists in Data Quality Discovery

Data preparation has become a necessary but labor and resource intensive step to perform data analytics. To date, such activities still require considerable manual effort from experts. In this paper, we focus on a specific data preparation activity, namely data quality discovery. We explore different settings in which data workers undertake data quality discovery tasks and the implications of those settings for the efficiency and effectiveness of data workers. To this end, we propose DataOps-4G, a data curation platform for generalists, that allows users to interact with data without the need to write code. We wrap up pre-defined code snippets that implement useful functionalities to explore data quality and bundle the code into so-called DataOps. Then, we conduct a lab-based user study to evaluate our DataOps-4G platform from two perspectives: (i) effectiveness, the accuracy of the outcomes achieved by participants; and (ii) efficiency, their effort and strategies in task completion. Our experimental results uncover how effectiveness and efficiency can be affected by their task completion patterns and strategies. This opens up the possibility of popularizing data curation processes by employing non-experts (e.g., from crowdsourcing platforms) and consequently allowing experts to focus on more complex activities (e.g., building machine learning models).

Prediction of Neutropenic Events in Chemotherapy Patients: A Machine Learning Approach.

Severe and febrile neutropenia present serious hazards to patients with cancer undergoing chemotherapy. We seek to develop a machine learning-based neutropenia prediction model that can be used to assess risk at the initiation of a chemotherapy cycle. We leverage rich electronic medical records (EMRs) data from a large health care system and apply machine learning methods to predict severe and febrile neutropenic events. We outline the data curation process and challenges posed by EMRs data. We explore a range of algorithms with an emphasis on model interpretability and ease of use in a clinical setting. Our final proposed model demonstrates an out-of-sample area under the receiver operating characteristic curve of 0.865 (95% CI, 0.830 to 0.891) in the prediction of neutropenic events on the basis of only 20 clinical features. The model validates known risk factors and offers insight into potential novel clinical indicators and treatment characteristics that elevate risk. It relies on factors that are directly extractable from EMRs, provided a tool can be easily integrated into existing workflows. A cost-based analysis provides insight into optimal risk thresholds and offers a framework for tailoring algorithms to individual hospital needs. A better understanding of neutropenic risk on an individual level enables a more informed approach to patient monitoring and treatment decisions.

Detecting visual design principles in art and architecture through deep convolutional neural networks

Visual design is associated with the use of some basic design elements and principles. Those are applied by the designers in the various disciplines for aesthetic purposes, relying on an intuitive and subjective process. Thus, numerical analysis of design visuals and disclosure of the aesthetic value embedded in them are considered as hard. However, it has become possible with emerging artificial intelligence technologies. This research aims at a neural network model, which recognizes and classifies the design principles over different domains. The domains include artwork produced since the late 20th century; professional photos; and facade pictures of contemporary buildings. The data collection and curation processes, including the production of computationally-based synthetic dataset, is genuine. The proposed model learns from the knowledge of myriads of original designs, by capturing the underlying shared patterns. It is expected to consolidate design processes by providing an aesthetic evaluation of the visual compositions with objectivity.

ERpred: a web server for the prediction of subtype-specific estrogen receptor antagonists.

Estrogen receptors alpha and beta (ERα and ERβ) are responsible for breast cancer metastasis through their involvement of clinical outcomes. Estradiol and hormone replacement therapy targets both ERs, but this often leads to an increased risk of breast and endometrial cancers as well as thromboembolism. A major challenge is posed for the development of compounds possessing ER subtype specificity. Herein, we present a large-scale classification structure-activity relationship (CSAR) study of inhibitors from the ChEMBL database which consisted of an initial set of 11,618 compounds for ERα and 7,810 compounds for ERβ. The IC50 was selected as the bioactivity unit for further investigation and after the data curation process, this led to a final data set of 1,593 and 1,281 compounds for ERα and ERβ, respectively. We employed the random forest (RF) algorithm for model building and of the 12 fingerprint types, models built using the PubChem fingerprint was the most robust (Ac of 94.65% and 92.25% and Matthews correlation coefficient (MCC) of 89% and 76% for ERα and ERβ, respectively) and therefore selected for feature interpretation. Results indicated the importance of features pertaining to aromatic rings, nitrogen-containing functional groups and aliphatic hydrocarbons. Finally, the model was deployed as the publicly available web server called ERpred at http://codes.bio/erpred where users can submit SMILES notation as the input query for prediction of the bioactivity against ERα and ERβ.

Data-driven curation process for describing the blood glucose management in the intensive care unit

Analysis of real-world glucose and insulin clinical data recorded in electronic medical records can provide insights into tailored approaches to clinical care, yet presents many analytic challenges. This work makes publicly available a dataset that contains the curated entries of blood glucose readings and administered insulin on a per-patient basis during ICU admissions in the Medical Information Mart for Intensive Care (MIMIC-III) database version 1.4. Also, the present study details the data curation process used to extract and match glucose values to insulin therapy. The curation process includes the creation of glucose-insulin pairing rules according to clinical expert-defined physiologic and pharmacologic parameters. Through this approach, it was possible to align nearly 76% of insulin events to a preceding blood glucose reading for nearly 9,600 critically ill patients. This work has the potential to reveal trends in real-world practice for the management of blood glucose. This data extraction and processing serve as a framework for future studies of glucose and insulin in the intensive care unit.

DATA CURATION DAN RESEARCH DATA MANAGEMENT UNTUK TERWUJUDNYA INTEGRASI DATA RISET DI INDONESIA

Indonesia is actively supporting the research process that is conducted in their research institutions, by providing research funds for them. Many research institutions in Indonesia work in the similar field and their funding could also come from the same source. Based on that reason, there is a need to evaluate the data curation and research data management processes at Indonesian research institutions. The purposes of this research are to evaluate the existing research data conditions at several research institutions, issues associated with research data management, standards and tools to support the implementation of research data management, also finding a way to achieve research data integration for research institutions in Indonesia. Interviews were used as the method to gain data from several research institutions in Indonesia then it was analysed with the qualitative data analysis. Ultimately, the result shows that data access problem, data collection and awareness of researcher are the main complexities of the research data management process. While policies and regulations on research data management are considered important by research institutions to guide them along the process. The result also shows that integration between research institutions could be achieved by using supporting tools and protocols such as OAI-PMH (The Open Archives Initiative Protocol for Metadata Harvesting) and API (Application Programming Interface), integrated system and standard data format.

MATCOR, a program for the cross-validation of material properties between databases

Data analytics approaches are increasingly often used to facilitate property-specific materials discovery. The uncertainties in these approaches can be greatly affected by the fidelity of the data sets that are used to train the data models. Therefore, data curation is an essential step for obtaining well-constrained model predictions. This can be a challenging task, especially for data sets that are too large for human quality control. We developed MATCOR, an open source, user-friendly, easily adaptable software to facilitate the data curation process. MATCOR processes lists of material identifiers in either AFLOW or Materials Project format and searches for the best matching materials entry in the other database. This is a non-trivial task due to differences in labeling and/or non-unique usage of material labels. MATCOR uses a combination of characteristics such space group, compound formula, crystal structure and use of Hubbard-U to provide the best possible comparison between databases. The capabilities of MATCOR are demonstrated for density, elastic properties, magnetic properties, and band gap correlations between AFLOW and Materials Project. We find that density shows the highest correlation among the tested properties, 93% of verified densities agree to within ±2%. Bulk- and shear-moduli showed deviations of less than ±10% for 80.6% and 65.1% of the materials, respectively. The classification of materials as non-magnetic/paramagnetic and metallic/gapped are consistent among the two databases for 91% and 69% of the materials, respectively. These examples show that MATCOR can be used to automate and thereby accelerate the data curation process prior to materials discovery through data analytical models.

SPARC: SODA, an interactive software for curating SPARC datasets

In this era of data‐intensive research, data sharing and reuse are becoming increasingly important for rapid discovery and innovation. While sharing can be as easy as a few clicks thanks to numerous data sharing platforms, data reuse is still an issue as standards for data curation and annotation are not rigorously or consistently followed by all researchers. As a result, it is often difficult, if not impossible, for researchers to understand and work with data they were not involved with during collection or analysis. To address this concern, the NIH SPARC program has imposed strict guidelines for curating and sharing data resulting from SPARC‐funded research. Complying with the guidelines requires additional time investment from the researchers (as with any data curation and submission standards), and the data curation process could thus become overwhelming as more and more data is generated. The objective of this project was to develop computer software to improve the curation workflow for SPARC researchers.To achieve that, we are developing a software named SODA (Software for Organizing Data Automatically) to assist SPARC investigators in curating and annotating their datasets. Distributed as an open‐source desktop application (Windows, MAC, Linux), the goal of SODA is to bridge a long‐standing and overlooked gap between comprehensive data standards and their convenient application by researchers. SODA provides an interactive interface that, without requiring any coding knowledge, walks SPARC investigators step‐by‐step through the data organization and sharing requirements, while automating repetitive, complex and/or time‐consuming tasks (Fig 1). An introduction to the software and the algorithms implemented will be given. Quantitative results from ongoing performance evaluation (time to curate a SPARC dataset with and without SODA) by beta testers, all of whom receive funding from the SPARC program, will be presented as well.Overall, the beta testers for early versions have acknowledged that SODA has the potential to improve their curation workflow. Data curation is becoming a critical issue, even beyond the SPARC consortium, and SODA could impact the broader research community by providing an exemplar and foundational tool for convenient and time‐efficient curation of data, which could be adopted by other projects. Potential applications, beyond the SPARC program, will be thus discussed as well.Support or Funding InformationSPARC OT2OD025308‐01S2Overview of the major features included in SODA during the on‐going development phaseFigure 1

Collaborative data familiarisation and quality assessment: Reflections from use of a national dataset to investigate palliative care for Indigenous Australians.

Data quality is fundamental to the integrity of quantitative research. The role of external researchers in data quality assessment (DQA) remains ill-defined in the context of secondary use for research of large, centrally curated health datasets. In order to investigate equity of palliative care provided to Indigenous Australian patients, researchers accessed a now-historical version of a national palliative care dataset developed primarily for the purpose of continuous quality improvement. (i) To apply a generic DQA framework to the dataset and (ii) to report the process and results of this assessment and examine the consequences for conducting the research. The data were systematically examined for completeness, consistency and credibility. Data quality issues relevant to the Indigenous identifier and framing of research questions were of particular interest. The dataset comprised 477,518 records of 144,951 patients (Indigenous N = 1515; missing Indigenous identifier N = 4998) collected from participating specialist palliative care services during a period (1 January 2010-30 June 2015) in which data-checking systems underwent substantial upgrades. Progressive improvement in completeness of data over the study period was evident. The data were error-free with respect to many credibility and consistency checks, with anomalies detected reported to data managers. As the proportion of missing values remained substantial for some clinical care variables, multiple imputation procedures were used in subsequent analyses. In secondary use of large curated datasets, DQA by external researchers may both influence proposed analytical methods and contribute to improvement of data curation processes through feedback to data managers.

An India Towards Global Ranking Standards: A Move Towards Data Curation

Ministry of Human Resource Development (MHRD) is not only a source of inspiration for sectors such as Research and Development, Services, Agriculture and Entrepreneurship. They are also enhancing our competitiveness and relevance by proposing and executing different projects across the nation. The approach is in line with the vision 2030. As per 2018 Shanghai report of Times Higher Education (THE), only 40% academic and 10% of employer reputation is built as per the Global Ranking Standards. Indian Higher Educational warehouses with the help of National Institutional Ranking Framework (NIRF), Rashtriya Uchchatar Shiksha Abhiyan (RUSA) and University Grant Commission (UGC) are generating data which is of least relevance to end users. We propose peer to peer data curation model which shall be extended as per business need using a cloud technology. We have harnessed data related to the Higher Education Sector (HES) from Indian Institute of Sciences (IISc), RUSA, and MHRD, to understand Indian contribution towards global ranking standards in Higher Education Sector. The approach is in line with the Escuela Nueva Model and Open Architecture Information System (OAIS) as existing standards in traditional data curation process. It is supported by the patent in large-scale data curation.

An Iterative, Self-Assessing Entity Resolution System: First Steps toward a Data Washing Machine

Data curation is the process of acquiring multiple sources of data, assessing and improving data quality, standardizing, and integrating the data into a usable information product, and eventually disposing of the data. The research describes the building of a proof-of-concept for an unsupervised data curation process addressing a basic form of data cleansing in the form of identifying redundant records through entity resolution and spelling corrections. The novelty of the approach is to use ER as the first step using an unsupervised blocking and stop word scheme based on token frequency. A scoring matrix is used for linking unstandardized references, and an unsupervised process for evaluating linking results based on cluster entropy. The ER process is iterative, and in each iteration, the match threshold is increased. The prototype was tested on 18 fully-annotated test samples of primarily synthetic person data varied in two different ways, good data quality versus poor data quality, and a single record layout versus two different record layouts. In samples with good data quality and using both single and mixed layouts, the final clusters had an average F-measure of 0.91, precision of 0.96, and recall of 0.87 outcomes comparable to results from a supervised ER process. In samples with poor data quality whether mixed or single layout, the average F-measure was 0.78, precision 0.74, and recall 0.83 showing that data quality assessment and improvement is still a critical component of successful data curation. The results demonstrate the feasibility of building an unsupervised ER engine to support data integration for good quality references while avoiding the time and effort to standardize reference sources to a common layout, design, and test matching rules, design blocking keys, or test blocking alignment. Also, the paper proposes how unsupervised data quality improvement processes could also be incorporated into the design allowing the model to address an even broader range of data curation applications.