Data Format Research Articles

Automated data collection tool for real-world cohort studies of chronic hepatitis B: Leveraging OCR and NLP technologies for improved efficiency

BackgroundCollecting and standardizing clinical research data is a very tedious task. This study is to develop an intelligent data collection tool, named CHB-EDC, for real-world cohort studies of chronic hepatitis B (CHB), which can assist in standardized and efficient data collection. MethodsCHB_EDC is capable of automatically processing various formats of data, including raw data in image format, using internationally recognized data standards, OCR, and NLP models. It can automatically populate the data into eCRFs designed in the REDCap system, supporting the integration of patient data from electronic medical record systems through commonly used web application interfaces. This tool enables intelligent extraction and aggregation of data, as well as secure and anonymous data sharing. ResultsFor non-electronic data collection, the average accuracy of manual collection was 98.65 %, with an average time of 63.64 min to collect information for one patient. The average accuracy CHB-EDC was 98.66 %, with an average time of 3.57 min to collect information for one patient. In the same data collection task, CHB-EDC achieved a comparable average accuracy to manual collection. However, in terms of time, CHB-EDC significantly outperformed manual collection (p < 0.05). Our research has significantly reduced the required collection time and lowered the cost of data collection while ensuring accuracy. ConclusionThe tool has significantly improved the efficiency of data collection while ensuring accuracy, enabling standardized collection of real-world data.

Matching and Rewriting Rules in Object-Oriented Databases

Graph query languages such as Cypher are widely adopted to match and retrieve data in a graph representation, due to their ability to retrieve and transform information. Even though the most natural way to match and transform information is through rewriting rules, those are scarcely or partially adopted in graph query languages. Their inability to do so has a major impact on the subsequent way the information is structured, as it might then appear more natural to provide major constraints over the data representation to fix the way the information should be represented. On the other hand, recent works are starting to move towards the opposite direction, as the provision of a truly general semistructured model (GSM) allows to both represent all the available data formats (Network-Based, Relational, and Semistructured) as well as support a holistic query language expressing all major queries in such languages. In this paper, we show that the usage of GSM enables the definition of a general rewriting mechanism which can be expressed in current graph query languages only at the cost of adhering the query to the specificity of the underlying data representation. We formalise the proposed query language in terms declarative graph rewriting mechanisms described as a set of production rules L→R while both providing restriction to the characterisation of L, and extending it to support structural graph nesting operations, useful to aggregate similar information around an entry-point of interest. We further achieve our declarative requirements by determining the order in which the data should be rewritten and multiple rules should be applied while ensuring the application of such updates on the GSM database is persisted in subsequent rewriting calls. We discuss how GSM, by fully supporting index-based data representation, allows for a better physical model implementation leveraging the benefits of columnar database storage. Preliminary benchmarks show the scalability of this proposed implementation in comparison with state-of-the-art implementations.

A novel method for the classification of 3D point clouds based on the improved PointNet++

Abstract In deep learning, point clouds are used as the primary input format for 3D data, which can provide detailed geometric information about objects in the original 3D space. PointNet++ is a deep learning network that uses point cloud data as an input format, which avoids the losses associated with the previous conversion of point cloud into 3D voxelization and a collection of 2D images. Although PointNet++ can directly process point cloud data in various ways, due to the disordered, irregular, and unevenly distributed nature of point cloud data, the effect of extracting point cloud features could be better. The large amount of point cloud data also leads to the training model falling into the local optimal solution, which affects the training results. In recent years, some effective methods and strategies have emerged to address these problems. In this thesis, three methods are proposed based on the PointNet++ network: feature similarity-based attention pooling, small kernel convolution, and diverse branch block method to improve the performance of the PointNet++ network. Experiments show that the improvement methods proposed in this paper effectively improve the feature extraction accuracy, which improves the accuracy of the PointNet++ network for classification on the ModelNet40_Normal_Resampled dataset, with an overall improvement of 1% compared with PointNet++.&amp;#xD;

An Overview of Public Retinal Optical Coherence Tomography Datasets: Access, Annotations, and Beyond.

In ophthalmology, Optical Coherence Tomography (OCT) has become a daily used tool in the diagnostics and therapeutic planning of various diseases. Publicly available datasets play a crucial role in advancing research by providing access to diverse imaging data for algorithm development. The accessibility, data format, annotations, and metadata are not consistent across OCT datasets, making it challenging to efficiently use the available resources. This article provides a comprehensive analysis of different OCT datasets, with particular attention to dataset properties, disease representation, accessibility, and aims to create a catalog of all publicly available OCT datasets. The goal is to improve accessibility to OCT data, increase openness about the availability, and give important new perspectives on the state of OCT imaging resources. Our findings reveal the need for improved data-sharing practices and standardized documentation.

MeDaX: A Knowledge Graph on FHIR.

In Germany, the standard format for exchange of clinical care data for research is HL7 FHIR. Graph databases (GDBs), well suited for integrating complex and heterogeneous data from diverse sources, are currently gaining traction in the medical field. They provide a versatile framework for data analysis which is generally challenging for raw FHIR-formatted data. For generation of a knowledge graph (KG) for clinical research data, we tested different extract-transform-load (ETL) approaches to convert FHIR into graph format. We designed a generalised ETL process and implemented a prototypic pipeline for automated KG creation and ontological structuring. The MeDaX-KG prototype is built from synthetic patient data and currently serves internal testing purposes. The presented approach is easy to customise to expand to other data types and formats.

A Review of Artificial Intelligence and Machine Learning in Product Life Cycle Management.

The pursuit of harnessing data for knowledge creation has been an enduring quest, with the advent of machine learning and artificial intelligence (AI) marking significant milestones in this journey. Machine Learning (ML), a subset of AI, emerged as the practice of employing mathematical models to enable computers to learn and improve autonomously based on their experiences. In the pharmaceutical and biopharmaceutical sectors, a significant portion of manufacturing data remains untapped or insufficient for practical use. Recognizing the potential advantages of leveraging available data for process design and optimization, manufacturers face the daunting challenge of data utilization. Diverse proprietary data formats and parallel data generation systems compound the complexity. The transition to Pharma 4.0 necessitates a paradigm shift in data capture for manufacturing and process operations. This paper highlights the pivotal role of artificial intelligence in converting process data into actionable knowledge to support critical functions throughout the whole process life cycle. Furthermore, it underscores the importance of maintaining compliance with data integrity guidelines, as mandated by regulatory bodies globally. Embracing AI-driven transformations is a crucial step toward shaping the future of the pharmaceutical industry, ensuring its competitiveness and resilience in an evolving landscape.

Graphical User Interface for Biodiversity Digital Twins: Data Challenges

Digital Twins are a new concept in the field of biodiversity (de Koning et al. 2023), one aspect is the user interface for interacting with digital twins. In the BioDT (Biodiversity Digital Twin) (BioDT 2022) project we are creating a graphical web interface (Martinovič et al. 2024) using the R Shiny framework, which allows small-scale data analysis to be done directly on the server running the web interface, while making it possible to offload a large-scale analysis to a supercomputer such as LUMI (Large Unified Modern Infrastructure). Additionally, we foresee that multiple prototype Digital Twins (pDTs) will be available as part of one web application. This brings multiple challenges in optimization of data flows for the computation and interaction with users, especially since data used by the pDTs, developed in the BioDT, are usually stored across multiple data sources. In the BioDT web application, we want to enable users to access different pDTs concerned with questions such as ecosystem services, biodiversity dynamics, DNA-related biodiversity tasks, pollinators, invasive species, disease outbreaks, and more. While some of these pDTs compute results at a remote server and provide only the newest results to the application, others aim to allow users to execute their own pDT runs with their own data and settings. This leads to many different user scenarios and impacts authentication and authorization flows, as well as data flows. In addition to this, we want to make the whole system comply with the FAIR (Findable, Accessible, Interoperable, and Reusable) principles as much as possible. For this, we need to support traceability of data, models, and pDT executions. This means that we advocate for data to be hosted at data providers with APIs for machine-to-machine interaction and extensive metadata support; models to be as open as possible and versioned; and to have a workflow execution orchestrator that can keep track of the models' execution and their related inputs and outputs. In terms of data, we are using research infrastructures (RIs) such as GBIF (Global Biodiversity Information Facility) and eLTER (Integrated European Long-Term Ecosystem, critical zone and socio-ecological Research) for the input data streams and when it is not possible to use established RIs, we are using self-hosted services. This work is still in progress and some of the self-hosted datasets are still being used, but in the future, we are looking into the possibility of having a dynamic system that will support data formats that cannot be currently stored at the established RIs. Our colleagues at BioDT consortium are preparing a special data server (El-Gabbas et al. 2023) that can handle complex data processing and serve data through an API without the need to download data for multiple commonly used formats. Some of the BioDT pDTs leverage the computational strength of High-Performance Computing (HPC) clusters and in such cases, classical cloud workflow orchestrators are not an option due to the specific security policies of such centers. To solve this, we turned to the LEXIS Platform, which can execute predefined workflows on a combination of cloud and HPC resources, and track the executions and related execution metadata. We are looking into exporting the descriptions of the workflow executions to Research Object Crates (RO-Crates) and then uploading this information to a remote server, where users could check their execution settings later. A current main development focus is the question of how to tackle the challenge of multiple authentication systems. This is specifically of concern in the case of sensitive data, which need to remain secure and available only to selected people. Due to interactions with several systems, we are encountering authentication issues of multiple different identities that should be recognized as one. The simplest solution here is to use one platform for data storage, workflow execution and web application. However, in the future we hope to find a more general solution that would not require data transfer to a single platform, since this could lower the usage of the BioDT platform due to legal restriction on some data. Another challenge in optimization of the dataflows is to avoid downloading the same data repeatedly and to be able to provide users with relevant data in the web application as fast as possible, since long waiting time would result in people not using the web application. For these issues, we are considering a smart-caching mechanism, however such functionality is not yet defined.

Basal knowledge in the field of pediatric nephrology and its enhancement following specific training of ChatGPT-4 "omni" and Gemini 1.5 Flash.

We aimed to evaluate the baseline performance and improvement of ChatGPT-4 "omni" (ChatGPT-4o) and Gemini 1.5 Flash (Gemini 1.5) in answering multiple-choice questions related to pediatric nephrology after specific training. Using questions from the "Educational Review" articles published by Pediatric Nephrology between January 2014 and April 2024, the models were tested both before and after specific training with Portable Data Format (PDF) and text (TXT) file formats of the Educational Review articles removing the last page containing the correct answers using a Python script. The number of correct answers was recorded. Before training, ChatGPT-4o correctly answered 75.2% of the 1395 questions, outperforming Gemini 1.5, which answered 64.9% correctly (p < 0.001). After training with PDF files, ChatGPT-4o's accuracy increased to 77.8%, while Gemini 1.5 improved significantly to 84.7% (p < 0.001). Training with TXT files showed similar results, with ChatGPT-4o maintaining 77.8% accuracy and Gemini 1.5 further improving to 87.6% (p < 0.001). The study highlights that while ChatGPT-4o has strong baseline performance, specific training does not significantly enhance its accuracy. Conversely, Gemini 1.5, despite its lower initial performance, shows substantial improvement with training, particularly with TXT files. These findings suggest Gemini 1.5's superior ability to store and retrieve information, making it potentially more effective in clinical applications, albeit with a dependency on additional data for optimal performance.

Human BioMolecular Atlas Program (HuBMAP): 3D Human Reference Atlas Construction and Usage.

The Human BioMolecular Atlas Program (HuBMAP) aims to construct a reference 3D structural, cellular, and molecular atlas of the healthy adult human body. The HuBMAP Data Portal (https://portal.hubmapconsortium.org) serves experimental datasets and supports data processing, search, filtering, and visualization. The Human Reference Atlas (HRA) Portal (https://humanatlas.io) provides open access to atlas data, code, procedures, and instructional materials. Experts from more than 20 consortia are collaborating to construct the HRA's Common Coordinate Framework (CCF), knowledge graphs, and tools that describe the multiscale structure of the human body (from organs and tissues down to cells, genes, and biomarkers) and to use the HRA to understand changes that occur at each of these levels with aging, disease, and other perturbations. The 6th release of the HRA v2.0 covers 36 organs with 4,499 unique anatomical structures, 1,195 cell types, and 2,089 biomarkers (e.g., genes, proteins, lipids) linked to ontologies and 2D/3D reference objects. New experimental data can be mapped into the HRA using (1) three cell type annotation tools (e.g., Azimuth) or (2) validated antibody panels (OMAPs), or (3) by registering tissue data spatially. This paper describes the HRA user stories, terminology, data formats, ontology validation, unified analysis workflows, user interfaces, instructional materials, application programming interface (APIs), flexible hybrid cloud infrastructure, and previews atlas usage applications.

Observational study examining the feasibility of generating real-world evidence (RWE) for new drug applications from the clinical trials database (DB; RELIASE study).

TPS100 Background: The extraction of the data that contributes to regulatory approval from real-world data (RWD) are difficult due to the lack of standardized data format as well as the extraction methodology. In addition, when RWE is used as an external control group, the similarity between internal and external control data has not been evaluated. Methods: In our ‘REALISE’ study, we will investigate the data extraction methodology as external control data of rare molecular subtype. We also elucidate the 'quality' and 'reliability' of RWD/RWE necessary for regulatory submissions. In addition, as most databases are not designed for regulatory use in the creation phase, we will investigate the retrospective methodologies to ensure reliability of RWD/RWE. We compare ‘data quality’ and ‘data reliability’ of ARCAD global DB (N=45,224), SCRUM-Japan registry (N=546), SCRUM-Japan observational study (N=14,325), and the Flatiron health RWD in Japan (N=650) and statistically analyze the differences and similarities among 4 databases. All analyses will be summarized descriptively. Similarity of patient characteristics and efficacy endpoints such as response rate, progression-free survival, overall survival, and all available endpoints will be evaluated by cancer type, treatment line, and standard treatment. For colorectal cancer, data extracted from the SCRUM-Japan registry, SCRUM-Japan observational study, and the Flatiron health RWD will be compared with data extracted from randomized controlled trials in the ARCAD global DB to examine the availability as an external control of RWD. The Kaplan-Meier method will be applied to estimate the survival distribution of time-to-event data. No imputation method for missing data will be used, and the frequency of missing data will also be summarized. We will also examine the methodology for extracting data with sufficient quality from the SCRUM-Japan observational study. In addition, if the reliability of RWD/RWE does not reach the required level, we will also examine the methodologies to retrospectively assure the reliability of the SCRUM-Japan observational study for regulatory submissions. [Table: see text]

LongReadSum: A fast and flexible quality control and signal summarization tool for long-read sequencing data.

While several well-established quality control (QC) tools are available for short reads sequencing data, there is a general paucity of computational tools that provide long read metrics in a fast and comprehensive manner across all major sequencing platforms (such as PacBio, Oxford Nanopore, Illumina Complete Long Read) and data formats (such as ONT POD5, FAST5, basecall summary files and PacBio unaligned BAM). Additionally, none of the current tools provide support for summarizing Oxford Nanopore basecall signal or comprehensive base modification (methylation) information from genomic data. Furthermore, nowadays a single PromethION flowcell on the Oxford Nanopore platform can generate terabytes of signal data, which cannot be handled by existing tools designed for small-scale flowcells. To address these challenges, here we present LongReadSum, a multi-threaded C++ tool which provides fast and comprehensive QC reports on all major aspects of sequencing data (such as read, base, base quality, alignment, and base modification metrics) and produce basecalling signal intensity information from the Oxford Nanopore platform. We demonstrate use cases to analyze cDNA sequencing, direct mRNA sequencing, reduced representation methylation sequencing (RRMS) through adaptive sequencing, as well as whole genome sequencing (WGS) data using diverse long-read platforms.

TOMOMAN: a software package for large scale cryo-electron tomography data preprocessing, community data sharing, and collaborative computing.

Cryo-electron tomography (cryo-ET) and subtomogram averaging (STA) are becoming the preferred methodologies for investigating subcellular and macromolecular structures in native or near-native environments. While cryo-ET is amenable to a wide range of biological problems, these problems often have data processing requirements that need to be individually optimized, precluding the notion of a one-size-fits-all processing pipeline. Cryo-ET data processing is also becoming progressively more complex due to an increasing number of packages for each processing step. Though each package has its own strengths and weaknesses, independent development and different data formats makes them difficult to interface with one another. TOMOMAN (TOMOgram MANager) is an extensible package for streamlining the interoperability of packages, enabling users to develop project-specific processing workflows. TOMOMAN does this by maintaining an internal metadata format and wrapping external packages to manage and perform preprocessing, from raw tilt-series data to reconstructed tomograms. TOMOMAN can also export this metadata between various STA packages. TOMOMAN also includes tools for archiving projects to data repositories; allowing subsequent users to download TOMOMAN projects and directly resume processing where it was previously left off. By tracking essential metadata, TOMOMAN streamlines data sharing, which improves reproducibility of published results, reduces computational costs by minimizing reprocessing, and enables distributed cryo-ET projects between multiple groups and institutions. TOMOMAN provides a way for users to test different software packages to develop processing workflows that meet the specific needs of their biological questions and to distribute their results with the broader scientific community.

OmicScope unravels systems-level insights from quantitative proteomics data

Shotgun proteomics analysis presents multifaceted challenges, demanding diverse tool integration for insights. Addressing this complexity, OmicScope emerges as an innovative solution for quantitative proteomics data analysis. Engineered to handle various data formats, it performs data pre-processing – including joining replicates, normalization, data imputation – and conducts differential proteomics analysis for both static and longitudinal experimental designs. Empowered by Enrichr with over 224 databases, OmicScope performs Over Representation Analysis (ORA) and Gene Set Enrichment Analysis (GSEA). Additionally, its Nebula module facilitates meta-analysis from independent datasets, providing a systems biology approach for enriched insights. Complete with a data visualization toolkit and accessible as Python package and a web application, OmicScope democratizes proteomics analysis, offering an efficient and high-quality pipeline for researchers.

An astronomical analysis of the data in the pseudo-Hipparchus palimpsest in the Codex Climaci Rescriptus

In 2022, Gysembergh et al. published a paper in JHA claiming ‘New evidence for Hipparchus’ Star Catalogue’. In this paper we challenge this hypothesis by stating that (a) we disagree with their astronomical dating and find inconsistencies by using the given numbers, and (b) the terminology and the data format used in the palimpsest do not match Hipparchus or anybody else. Therefore, the palimpsest does not prove anything about Hipparchus’s star catalogue nor did Hipparchus use rectangular constellation borders. Specifically, the constellation of Corona Borealis, typically depicted as a circle since Babylonian times, is not considered a rectangle by Hipparchus. Furthermore, a palimpsest that cannot be dated properly does not confirm the long-known relationship between Hipparchus and Ptolemy as the authors claim.

Building the NHPI Data Policy Platform to Identify the Most Pressing SDOH Policy Domains for the NHPI Community

Abstract: Access to accurate Native Hawaiian and Pacific Islander (NHPI) social drivers of health (SDOH) data is crucial for understanding health needs and shaping effective public health strategies. However, this data often gets obscured within broader racial and ethnic categories making NHPI issues invisible. Moreover, NHPI communities face barriers when published data formats are inaccessible to community-based organization staff. To tackle these challenges, we initiated the Assessing Social Determinants of Health Data Through Local Data Intermediaries Initiative (AHEAD), supported by the federal Office of Minority Health. We developed two community-centered resources: 1. The NH PI Data Policy Platform, shaped by 138 NHPI community leaders from 13 states and Washington, D.C., representing nine NHPI subgroups. This platform highlights NHPI priority issues and provides equitable data policy recommendations for advocates and policymakers. 2. An NHP I SDOH Dashboard for California Counties, tailored to offer accessible data for community-based organization staff, ensuring better-informed interventions and support.

Advanced computer architecture optimization for machine learning/deep learning

Abstract The recent progress in Machine Learning (Géron, 2022) and particularly Deep Learning (Goodfellow, 2016) models exposed the limitations of traditional computer architectures. Modern algorithms demonstrate highly increased computational demands and data requirements that most existing architectures cannot handle efficiently. These demands result in training speed, inference latency, and power consumption bottlenecks, which is why advanced methods of computer architecture optimization are required to enable the development of ML/DL-dedicated efficient hardware platforms (Engineers, 2019). The optimization of computer architecture for applications of ML/DL becomes critical, due to the tremendous demand for efficient execution of complex computations by Neural Networks (Goodfellow, 2016). This paper reviewed the numerous approaches and methods utilized to optimize computer architecture for ML/DL workloads. The following sections contain substantial discussion concerning the hardware-level optimizations, enhancements of traditional software frameworks and their unique versions, and innovative explorations of architectures. In particular, we discussed hardware including specialized accelerators, which can improve the performance and efficiency of a computation system using various techniques, specifically describing accelerators like CPUs (multicore) (Hennessy, 2017), GPUs (Hwu, 2015) and TPUs (Contributors, 2017), parallelism in multicore architectures, data movement in hardware systems, especially techniques such as caching and sparsity, compression, and quantization, other special techniques and configurations, such as using specialized data formats, and measurement sparsity. Moreover, this paper provided a comprehensive analysis of current trends in software frameworks, Data Movement optimization strategies (A.Bienz, 2021), sparsity, quantization and compression methods, using ML for architecture exploration, and, DVFS (Hennessy, 2017),, which provides strategies for maximizing hardware utilization and power consumption during training, machine learning, dynamic voltage, and frequency scaling, runtime systems. Finally, the paper discussed research opportunity directions and the possibilities of computer architecture optimization influence in various industrial and academic areas of ML/DL technologies. The objective of implementing these optimization techniques is to largely minimize the current gap between the computational needs of ML/DL algorithms and the current hardware’s capability. This will lead to significant improvements in training times, enable real-time inference for various applications, and ultimately unlock the full potential of cutting-edge machine learning algorithms.

A comparison of likelihood ratios calculated from surface DNA mixtures using MPS and CE Technologies

This study evaluates the performance of analysing surface DNA samples using massively parallel sequencing (MPS) compared to traditional capillary electrophoresis (CE). A total of 30 samples were collected from various surfaces in an office environment and were analysed with CE and MPS. These were compared against 60 reference samples (office inhabitants). To identify contributors, likelihood ratios (LRs) were calculated for MPS and CE data using the probabilistic genotyping software MPSproto and EuroForMix respectively. Although a higher number of sequences/peaks were observed per DNA profile in MPS compared to CE, LR values were found to be lower for MPS data formats. This might be the result of the increased complexity of MPS data, along with a possible elevation of unknown alleles and/or artefacts. The study highlights avenues for improving MPS data quality and analysis to facilitate more robust interpretation of challenging casework-like samples.

A United Architecture For AES/Present Ciphers and Its Usage in a Soc Environment

Abstract: In order to monitor the operating status of vehicles, it is necessary to collect vehicle operating data in real time through IoT devices and analyze these data. However, the collected data has the characteristics of multi-source heterogeneous, network resources are limited and server performance is poor. It is difficult to truly realize data processing in real time. In addition, data needs to be transmitted over the network, it is particularly important to ensure the safety of data transmission. Considering the above problems, it is necessary to structure the data and define a unified data format to facilitate data transmission and analysis. At the same time, improve the server communication program and improve the server's concurrent processing capabilities. In addition, considering that data needs to be transmitted over the network, in order to ensure that the data is not stolen or tampered with, the PRESENT lightweight encryption algorithm is adopted to ensure the safety of data transmission. Compared with encryption algorithms such as AES, this algorithm has much lower hardware requirements. This article combines the characteristics of the project and uses the number of communications between the device and the server to achieve the dynamic key update which is approximately one-time pad, and greatly improves the security of the data.

A Cloud-Based System for Automated AI Image Analysis and Reporting.

Although numerous AI algorithms have been published, the relatively small number of algorithms used clinically is partly due to the difficulty of implementing AI seamlessly into the clinical workflow for radiologists and for their healthcare enterprise. The authors developed an AI orchestrator to facilitate the deployment and use of AI tools in a large multi-site university healthcare system and used it to conduct opportunistic screening for hepatic steatosis. During the 60-day study period, 991 abdominal CTs were processed at multiple different physical locations with an average turnaround time of 2.8min. Quality control images and AI results were fully integrated into the existing clinical workflow. All input into and output from the server was in standardized data formats. The authors describe the methodology in detail; this framework can be adapted to integrate any clinical AI algorithm.

A scoping review of rule-based clinical decision support malfunctions.

Conduct a scoping review of research studies that describe rule-based clinical decision support (CDS) malfunctions. In April 2022, we searched three bibliographic databases (MEDLINE, CINAHL, and Embase) for literature referencing CDS malfunctions. We coded the identified malfunctions according to an existing CDS malfunction taxonomy and added new categories for factors not already captured. We also extracted and summarized information related to the CDS system, such as architecture, data source, and data format. Twenty-eight articles met inclusion criteria, capturing 130 malfunctions. Architectures used included stand-alone systems (eg, web-based calculator), integrated systems (eg, best practices alerts), and service-oriented architectures (eg, distributed systems like SMART or CDS Hooks). No standards-based CDS malfunctions were identified. The "Cause" category of the original taxonomy includes three new types (organizational policy, hardware error, and data source) and two existing causes were expanded to include additional layers. Only 29 malfunctions (22%) described the potential impact of the malfunction on patient care. While a substantial amount of research on CDS exists, our review indicates there is a limited focus on CDS malfunctions, with even less attention on malfunctions associated with modern delivery architectures such as SMART and CDS Hooks. CDS malfunctions can and do occur across several different care delivery architectures. To account for advances in health information technology, existing taxonomies of CDS malfunctions must be continually updated. This will be especially important for service-oriented architectures, which connect several disparate systems, and are increasing in use.