Audit Logs Research Articles

Health Care Worker Usage of Large-Scale Health Information Exchanges in Japan: User-Level Audit Log Analysis Study.

Over 200 health information exchanges (HIEs) are currently operational in Japan. The most common feature of HIEs is remote on-demand viewing or searching of aggregated patient health data from multiple institutions. However, the usage of this feature by individual users and institutions remains unknown. This study aims to understand usage of the on-demand patient data viewing feature of large-scale HIEs by individual health care workers and institutions in Japan. We conducted audit log analyses of large-scale HIEs. The research subjects were HIEs connected to over 100 institutions and with over 10,000 patients. Each health care worker's profile and audit log data for HIEs were collected. We conducted four types of analyses on the extracted audit log. First, we calculated the ratio of the number of days of active HIE use for each hospital-affiliated doctor account. Second, we calculated cumulative monthly usage days of HIEs by each institution in financial year (FY) 2021/22. Third, we calculated each facility type's monthly active institution ratio in FY2021/22. Fourth, we compared the monthly active institution ratio by medical institution for each HIE and the proportion of cumulative usage days by user type for each HIE. We identified 24 HIEs as candidates for data collection and we analyzed data from 7 HIEs. Among hospital doctors, 93.5% (7326/7833) had never used HIEs during the available period in FY2021/22, while 19 doctors used them at least 30% of days. The median (IQR) monthly active institution ratios were 0.482 (0.470-0.487) for hospitals, 0.243 (0.230-0.247) for medical clinics, and 0.030 (0.024-0.048) for dental clinics. In 51.9% (1781/3434) of hospitals, the cumulative monthly usage days of HIEs was 0, while in 26.8% (921/3434) of hospitals, it was between 1 and 10, and in 3% (103/3434) of hospitals, it was 100 or more. The median (IQR) monthly active institution ratio in medical institutions was 0.511 (0.487-0.529) for the most used HIE and 0.109 (0.0927-0.117) for the least used. The proportion of cumulative usage days of HIE by user type was complex for each HIE, and no consistent trends could be discerned. In the large-scale HIEs surveyed in this study, the overall usage of the on-demand patient data viewing feature was low, consistent with past official reports. User-level analyses of audit logs revealed large disparities in the number of days of HIE use among health care workers and institutions. There were also large disparities in HIE use by facility type or HIE; the percentage of cumulative HIE usage days by user type also differed by HIE. This study indicates the need for further research into why there are large disparities in demand for HIEs in Japan as well as the need to design comprehensive audit logs that can be matched with other official datasets.

Provenance-based APT campaigns detection via masked graph representation learning

Advanced Persistent Threats (APTs) are well-planned, persistent, and highly stealthy cyberattacks designed to steal confidential information or disrupt specific target systems. Recent studies have used system audit logs to construct provenance graphs that describe system interactions to detect potentially malicious activities. Although they are effective, they still suffer from problems such as the need for a priori knowledge, lack of attack data, and high computational overhead that limit their application. In this paper, we propose a self-supervised learning-based APT detection model, APT-MGL, which learns the embedded representations of nodes through a graph mask self-encoder and transforms the detection problem into an outlier detection problem for malicious nodes. APT-MGL characterizes the behavior of nodes based on node type, action, and interaction frequency, and fuses the features through a multi-head self-attention mechanism. Then the node embedding is obtained by combining graph features and structural information using masked graph representation learning. Finally, the unsupervised outlier detection method is used to analyze the computed embeddings and obtain the final detection results. The experimental results show that APT-MGL outperforms existing monitoring models and achieves a small overhead.

The Impact of Diagnosis-Specific Plan Templates on Admission Note Writing Time: A Quality Improvement Initiative.

Background There are limited objective studies regarding the effectiveness of strategies to alleviate the documentation burden on resident physicians. Objective To develop and implement diagnosis-specific templates for the plan of care section of inpatient admission notes, aiming to reduce documentation time. Methods Twelve templates for the plan of care section of admission notes were written by the study authors, reviewed by attending physicians, and shared with the residents through the electronic health record (EHR) on September 23, 2022. EHR audit log data were collected to examine admission note writing times, supplemented by resident feedback on acceptability via an anonymous survey. Feasibility measures included time investment, experience with the EHR, and resident training. Results Between July 1, 2021 and June 30, 2023, 62 pediatric residents contributed 9840 admission notes. The templates were used in 557 admission notes. The mean total time spent on an admission note decreased from 97.9 minutes pre-intervention to 71.0 minutes post-intervention with the use of a template; an adjusted reduction of 23% (95% CI 16%-30%; P<.001). The mean attending time spent editing an admission note was unchanged. The survey results underscored wide acceptability of the templates among the residents. Feasibility data showed that the project required minimal time investment from the health care informatics team and minimal resident training. Conclusions Using templates in the care plan section of admission notes reduces the time residents spend writing admission notes.

Exploring user privacy awareness on GitHub: an empirical study

GitHub provides developers with a practical way to distribute source code and collaboratively work on common projects. To enhance account security and privacy, GitHub allows its users to manage access permissions, review audit logs, and enable two-factor authentication. However, despite the endless effort, the platform still faces various issues related to the privacy of its users. This paper presents an empirical study delving into the GitHub ecosystem. Our focus is on investigating the utilization of privacy settings on the platform and identifying various types of sensitive information disclosed by users. Leveraging a dataset comprising 6,132 developers, we report and analyze their activities by means of comments on pull requests. Our findings indicate an active engagement by users with the available privacy settings on GitHub. Notably, we observe the disclosure of different forms of private information within pull request comments. This observation has prompted our exploration into sensitivity detection using a large language model and BERT, to pave the way for a personalized privacy assistant. Our work provides insights into the utilization of existing privacy protection tools, such as privacy settings, along with their inherent limitations. Essentially, we aim to advance research in this field by providing both the motivation for creating such privacy protection tools and a proposed methodology for personalizing them.

Secure messaging telehealth billing in the digital age: moving beyond time-based metrics.

We proposed adopting billing models for secure messaging (SM) telehealth services that move beyond time-based metrics, focusing on the complexity and clinical expertise involved in patient care. We trained 8 classification machine learning (ML) models using providers' electronic health record (EHR) audit log data for patient-initiated non-urgent messages. Mixed effect modeling (MEM) analyzed significance. Accuracy and area under the receiver operating characteristics curve scores generally exceeded 0.85, demonstrating robust performance. MEM showed that knowledge domains significantly influenced SM billing, explaining nearly 40% of the variance. This study demonstrates that ML models using EHR audit log data can improve and predict billing in SM telehealth services, supporting billing models that reflect clinical complexity and expertise rather than time-based metrics. Our research highlights the need for SM billing models beyond time-based metrics, using EHR audit log data to capture the true value of clinical work.

Blockchains in health information systems: A literature review on use cases and status of implementation of blockchains for electronic health records

BACKGROUND: The careful management of health data is crucial due to its immense value and high sensitivity. Blockchain technologies can manage data in a trustworthy way. OBJECTIVE: The central aim is to identify the current state of blockchain technologies applied to Electronic Health Records (EHR) to identify common structures and functionalities. This common ground could be the starting point for defining clear global standards. METHODS: A systematic literature review is used. RESULTS: The benefits of enhancing the digitalization and cross-institutional accessibility of health data are undoubted. Four main application areas of blockchain for the EHR can be identified: storing, sharing, audit logging, and managing the identity of data accessors. Since on-chain transactions are slow and inefficient, most research promotes a hybrid approach for handling transactions as a combination of off-chain and on-chain approaches. CONCLUSIONS: Several approaches, frameworks, and models exist for applying blockchain technologies in the context of EHR. The research revealed that a) only a few concepts are already implemented, b) the existing system implementations are based on different backgrounds and technology stacks, and c) a lack of comprehensive and global standards and norms. All these factors are barriers to a broader usage of blockchain-based EHRs.

ProcSAGE: an efficient host threat detection method based on graph representation learning

Advanced Persistent Threats (APTs) achieves internal networks penetration through multiple methods, making it difficult to detect attack clues solely through boundary defense measures. To address this challenge, some research has proposed threat detection methods based on provenance graphs, which leverage entity relationships such as processes, files, and sockets found in host audit logs. However, these methods are generally inefficient, especially when faced with massive audit logs and the computational resource-intensive nature of graph algorithms. Effectively and economically extracting APT attack clues from massive system audit logs remains a significant challenge. To tackle this problem, this paper introduces the ProcSAGE method, which detects threats based on abnormal behavior patterns, offering high accuracy, low cost, and independence from expert knowledge. ProcSAGE focuses on processes or threads in host audit logs during the graph construction phase to effectively control the scale of provenance graphs and reduce performance overhead. Additionally, in the feature extraction phase, ProcSAGE considers information about the processes or threads themselves and their neighboring nodes to accurately characterize them and enhance model accuracy. In order to verify the effectiveness of the ProcSAGE method, this study conducted a comprehensive evaluation on the StreamSpot dataset. The experimental results show that the ProcSAGE method can significantly reduce the time and memory consumption in the threat detection process while improving the accuracy, and the optimization effect becomes more significant as the data size expands.

Blockchain-enabled EHR access auditing: Enhancing healthcare data security

In the realm of modern healthcare, Electronic Health Records EHR serve as invaluable assets, yet they also pose significant security challenges. The absence of EHR access auditing mechanisms, which includes the EHR audit trails, results in accountability gaps and magnifies security vulnerabilities. This situation effectively paves the way for unauthorized data alterations to occur without detection or consequences. Inadequate EHR compliance auditing procedures, particularly in verifying and validating access control policies, expose healthcare organizations to risks such as data breaches, and unauthorized data usage. These vulnerabilities result from unchecked unauthorized access activities. Additionally, the absence of EHR audit logs complicates investigations, weakens proactive security measures, and raises concerns to put healthcare institutions at risk. This study addresses the pressing need for robust EHR auditing systems designed to scrutinize access to EHR data, encompassing who accesses it, when, and for what purpose. Our research delves into the complex field of EHR auditing, which includes establishing an immutable audit trail to enhance data security through blockchain technology. We also integrate Purpose-Based Access Control (PBAC) alongside smart contracts to strengthen compliance auditing by validating access legitimacy and reducing unauthorized entries. Our contributions encompass the creation of audit trail of EHR access, compliance auditing via PBAC policy verification, the generation of audit logs, and the derivation of data-driven insights, fortifying EHR access security.

National rollout of a medication safety dashboard to improve testing for latent infections among biologic and targeted synthetic disease-modifying agent users within the Veterans Health Administration.

To develop, deploy, and evaluate a national, electronic health record (EHR)-based dashboard to support safe prescribing of biologic and targeted synthetic disease-modifying agents (b/tsDMARDs) in the United States Veterans Affairs Healthcare System (VA). We extracted and displayed hepatitis B (HBV), hepatitis C (HCV), and tuberculosis (TB) screening data from the EHR for users of b/tsDMARDs using PowerBI (Microsoft) and deployed the dashboard to VA facilities across the United States in 2022; we observed facilities for 44 weeks post-deployment. We examined the association between dashboard engagement by healthcare personnel and the percentage of patients with all screenings complete (HBV, HCV, and TB) at the facility level using an interrupted time series. Based on frequency of sessions, facilities were grouped into high- and low/none-engagement categories. We modeled changes in complete screening pre- and post-deployment of the dashboard. All VA facilities were eligible for inclusion; excluded facilities participated in design of the dashboard or had <20 patients receiving b/tsDMARDs. Session counts from facility personnel were captured using PowerBI audit log data. Outcomes were assessed weekly based on EHR data extracted via the dashboard itself. Totally 117 facilities (serving a total of 41,224 Veterans prescribed b/tsDMARDs) were included. Before dashboard deployment, across all facilities, 61.5% of patients had all screenings complete, which improved to 66.3% over the course of the study period. The largest improvement (15 percentage points, 60.3%-75.3%) occurred among facilities with high engagement (post-intervention difference in outcome between high and low/none-engagement groups was 0.17 percentage points (pp) per week, 95% confidence interval (0.04 pp, 0.30 pp); p = 0.01). We observed significant improvements in screening for latent infections among facilities with high engagement with the dashboard, compared with those with fewer sessions.

Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences.

To develop and validate a novel measure, action entropy, for assessing the cognitive effort associated with electronic health record (EHR)-based work activities. EHR-based audit logs of attending physicians and advanced practice providers (APPs) from four surgical intensive care units in 2019 were included. Neural language models (LMs) were trained and validated separately for attendings' and APPs' action sequences. Action entropy was calculated as the cross-entropy associated with the predicted probability of the next action, based on prior actions. To validate the measure, a matched pairs study was conducted to assess the difference in action entropy during known high cognitive effort scenarios, namely, attention switching between patients and to or from the EHR inbox. Sixty-five clinicians performing 5904429 EHR-based audit log actions on 8956 unique patients were included. All attention switching scenarios were associated with a higher action entropy compared to non-switching scenarios (P < .001), except for the from-inbox switching scenario among APPs. The highest difference among attendings was for the from-inbox attention switching: Action entropy was 1.288 (95% CI, 1.256-1.320) standard deviations (SDs) higher for switching compared to non-switching scenarios. For APPs, the highest difference was for the to-inbox switching, where action entropy was 2.354 (95% CI, 2.311-2.397) SDs higher for switching compared to non-switching scenarios. We developed a LM-based metric, action entropy, for assessing cognitive burden associated with EHR-based actions. The metric showed discriminant validity and statistical significance when evaluated against known situations of high cognitive effort (ie, attention switching). With additional validation, this metric can potentially be used as a screening tool for assessing behavioral action phenotypes that are associated with higher cognitive burden. An LM-based action entropy metric-relying on sequences of EHR actions-offers opportunities for assessing cognitive effort in EHR-based workflows.

Blockchain based secret key management for trusted platform module standard in reconfigurable platform

SummaryThe growing sophistication of cyber attacks, vulnerabilities in high computing systems and increasing dependency on cryptography to protect our digital data, make it more important to keep secret keys safe and secure. A few major issues of secret keys, like incorrect use of keys, inappropriate storage of keys, inadequate protection of keys, insecure movement of keys, lack of audit logging, insider threats and nondestruction of keys can compromise the whole security system severely. In this work, we propose a field programmable gate array (FPGA)‐based trusted platform module (TPM) framework for operating system companies and OS users, utilizing blockchain to address NIST‐recommended secret key management issues. The security processor used in OS user machines is partitioned into three areas such that processor area, confidential area, and crypto area. The isolated secret key memory in confidential area, along with a private blockchain (BC) can log the life cycle of secret keys of TPM standard. We have also implemented a special custom bus interconnect, which receives custom crypto instructions from Processing Element (PE). During the execution of crypto instructions, the architecture ensures that secret keys are present in confidential area and crypto area but never in the processor area. The movements of secret keys between confidential area, and crypto area are recorded cryptographically after the proper authentication process controlled by the proposed hardware‐based private BC framework. To the best of our knowledge, this work is the first attempt to implement a blockchain‐based framework between OS company and OS users to address NIST recommended secret key management issues of TPM standard hardware environment. The additional cost of resource usage and timing complexity we spent to implement the proposed idea is nominal. The proposed architecture is implemented with Xilinx EDA tool using FPGA board.

Nurse-led virtual clinic for liver cirrhosis: impact on quality of life and health outcomes – a mixed methods research proposal

Liver cirrhosis causes high mortality and morbidity, affecting patients' quality of life (QoL) and health outcomes. Given the significant burden of liver cirrhosis, restructuring healthcare services is crucial. Nurses' involvement in secondary prevention for other chronic diseases has shown benefits, including improved symptom control, reduced hospitalisations and enhanced HRQoL. However, the nursing role in liver disease management remains understudied. This mixed-methods study proposal with an embedded sequential design will assess the effectiveness of a nurse-led virtual liver cirrhosis clinic (NLVLCC) on QoL, self-efficacy, and health outcomes in 90 patients with alcohol-related cirrhosis (60 intervention, 30 control). Health outcomes include medication adherence, performance status, dietary adherence, alcohol intake, depression, anxiety, stress and 30-day readmission. Phase one will involve a randomised controlled trial, followed by narrative analysis in phase two. Data collection will utilize standardized and structured tools, including the Chronic Liver Disease Questionnaire, Self-Efficacy for Managing Chronic Disease 6-item Scale, MARS, Karnofsky Performance Scale, 24-hour dietary recall, AUDIT, DASS, and adherence logs. NLVLCC components include patient education, dietary counseling, brief alcohol de-addiction intervention, psychological support, medication adherence, physical activity guidance and regular virtual follow-ups.

Improving the quality of newborn feeding documentation in an EHR using a mixed methods approach

IntroductionNewborn feeding is key to infant growth and survival. Accurate feeding documentation can inform care decisions and planning of care. A nutritional dashboard is available within the Electronic Health Record (EHR) which accumulates feeding data in a graphical display. PurposeTo improve the quality of newborn feeding documentation for post-natal ward babies and babies in the Neonatal Intensive Care Unit (NICU). Designand Methods: A multidisciplinary end user expert group (n = 38) was established. Qualitative thematic analyses from this group were used to design new feeding data entry and review elements. Quantitative pre-post design was used to assess feeding documentation for both post-natal ward baby charts (n = 134) and NICU baby charts (n = 188). Descriptive statistics and Pearson's chi-square were used to assess pre-post differences and statistical significance. The use of a nutritional dashboard was assessed using system audit logs and analyzed using Poisson regression testing. ResultsPost-natal ward babies had improvements in structured feeding documentation by 91.6% (from 17.9% to 34.3%) (p = 0.031). NICU data feeding documentation improved by 25% (from 72.3% to 90.4%) (p = 0.001). Use of the nutritional dashboard however reduced in the post period. ConclusionThis study has positively demonstrated improvements in the quality of newborn feeding documentation within the patient's EHR can be achieved through a collaborative multidisciplinary approach optimising EHR design. Practice implicationsThe benefit of a multidisciplinary approach to EHR design is paramount to promoting superior quality data entry compliant with individual workflows.

The FDA Enhancing Quality Using the Inspection Program (EQUIP) Breast Imaging Quality Initiative. 5-Year Clinical Experience.

To evaluate the effects of the Enhancing Quality Using the Inspection Program (EQUIP) on quality control (QC) and quality assurance (QA) at an academic medical center. EQUIP audit logs for technologist image quality review as well as mammography unit QA and QC formed the basis for study data. One randomly selected screening mammogram was evaluated by the lead interpreting physician (LIP) using EQUIP criteria for each technologist for each imaging site worked, initially semiannually and then monthly. One randomly selected screening mammogram interpreted by each interpreting physician (IP) for each imaging site was evaluated on a semiannual basis. Quarterly, the LIP reviewed QA and QC logs for each mammography unit with deficiencies further investigated. Of 214 965 eligible screening mammograms performed, 5955 (2.8%) underwent EQUIP image quality review. Five were found to be technically inadequate (0.08%, 5955/214 965). The LIP identified 20 significant interpretive differences compared with the clinical interpretation resulting in 10 biopsies and 7 previously undetected malignancies, with supplemental cancer detection rate of 1.2/1000 cases reviewed. Two hundred ninety mammography unit QA/QC reviews identified 31 potential deficiencies, 29 of which were due to human documentation error (93.4%). EQUIP review of both IP and technologists' quality and mammography unit QA/QC logs as performed identified few deficiencies. EQUIP policies should be evaluated at each institution and modified to best utilize resources and provide opportunities for meaningful quality improvement. Although not an EQUIP focus, supplemental cancer detection was observed as might be expected with double reading.

A Scalable and Extensible Logical Data Model of Electronic Health Record Audit Logs for Temporal Data Mining (RNteract): Model Conceptualization and Formulation.

Increased workload, including workload related to electronic health record (EHR) documentation, is reported as a main contributor to nurse burnout and adversely affects patient safety and nurse satisfaction. Traditional methods for workload analysis are either administrative measures (such as the nurse-patient ratio) that do not represent actual nursing care or are subjective and limited to snapshots of care (eg, time-motion studies). Observing care and testing workflow changes in real time can be obstructive to clinical care. An examination of EHR interactions using EHR audit logs could provide a scalable, unobtrusive way to quantify the nursing workload, at least to the extent that nursing work is represented in EHR documentation. EHR audit logs are extremely complex; however, simple analytical methods cannot discover complex temporal patterns, requiring use of state-of-the-art temporal data-mining approaches. To effectively use these approaches, it is necessary to structure the raw audit logs into a consistent and scalable logical data model that can be consumed by machine learning (ML) algorithms. We aimed to conceptualize a logical data model for nurse-EHR interactions that would support the future development of temporal ML models based on EHR audit log data. We conducted a preliminary review of EHR audit logs to understand the types of nursing-specific data captured. Using concepts derived from the literature and our previous experience studying temporal patterns in biomedical data, we formulated a logical data model that can describe nurse-EHR interactions, the nurse-intrinsic and situational characteristics that may influence those interactions, and outcomes of relevance to the nursing workload in a scalable and extensible manner. We describe the data structure and concepts from EHR audit log data associated with nursing workload as a logical data model named RNteract. We conceptually demonstrate how using this logical data model could support temporal unsupervised ML and state-of-the-art artificial intelligence (AI) methods for predictive modeling. The RNteract logical data model appears capable of supporting a variety of AI-based systems and should be generalizable to any type of EHR system or health care setting. Quantitatively identifying and analyzing temporal patterns of nurse-EHR interactions is foundational for developing interventions that support the nursing documentation workload and address nurse burnout.

Methods of digital signature truncation for El-Gamal-type signatures and for DSTU 4145-2002.

Digital signature truncation is important for implementing cryptographic systems for low-resource devices, particularly in systems where signatures are stored for a long term but are checked relatively infrequently. Among such systems we can mention hardware audit logs or secure document repositories. Signature truncation can also be used in cases where memory is limited to represent or store the signature itself (for example, in QR codes). Signature truncation methods for ECDSA and EdDSA standards were proposed in public sources. This paper proposes a method of truncation of digital signatures for the classical El-Gamal scheme and some of its generalizations, which are based on modular arithmetic. The proposed methods do not require any alternations in the signing procedure, and are therefore applicable to arbitrary existing implementations of the El-Gamal scheme; the restoration of a digital signature occurs at the expense of an increase in computational costs on the part of the party verifying the signature. A digital signature truncation method for the national standard DSTU 4145-2002 is also proposed; this method takes into account the properties of the arithmetic of elliptic curves, on which the standard is based. The proposed methods make it possible to effectively implement the truncation of signatures of the El-Gamal type (especially noticeable for DSTU 4145-2002 signatures), discarding up to 32 bits of signature, but the complexity of the verification increases rapidly with the increase in the discarded part of the signature. However, for selected limits of the length of the discarded part, the proposed algorithms have a relatively efficient implementation and therefore can be used to reduce the length of signatures in protocols for low-resource devices.

Machine learning to predict notes for chart review in the oncology setting: a proof of concept strategy for improving clinician note-writing.

Leverage electronic health record (EHR) audit logs to develop a machine learning (ML) model that predicts which notes a clinician wants to review when seeing oncology patients. We trained logistic regression models using note metadata and a Term Frequency Inverse Document Frequency (TF-IDF) text representation. We evaluated performance with precision, recall, F1, AUC, and a clinical qualitative assessment. The metadata only model achieved an AUC 0.930 and the metadata and TF-IDF model an AUC 0.937. Qualitative assessment revealed a need for better text representation and to further customize predictions for the user. Our model effectively surfaces the top 10 notes a clinician wants to review when seeing an oncology patient. Further studies can characterize different types of clinician users and better tailor the task for different care settings. EHR audit logs can provide important relevance data for training ML models that assist with note-writing in the oncology setting.

130 Developing a Conceptual Data Model for Nursing Workload

OBJECTIVES/GOALS: Nurses are leaving the profession at an alarming rate due to increased workload and burnout.#_msocom_1 Computational models that are reliable and reproducible are needed to quantitatively examine nursing workload and estimate potential effect of interventions. This project developed a logical data model to represent nursing EHR interactions. METHODS/STUDY POPULATION: With nursing EHR interactions as a starting point, we expand upon literature that examined the EHR workload of physicians. We conducted an exploratory analysis of nursing EHR audit log data at a large academic medical center, and explored components of nursing workload that can be extracted from other health system data. Using concepts derived from the studying temporal biomedical data patterns, we formulated a data structure that describes nurse EHR interactions, nurse intrinsic and situational characteristics, and nurse outcomes of interest in a scalable and extensible manner. RESULTS/ANTICIPATED RESULTS: Temporal machine learning models are grounded in the concept of vectors. We developed a logical data model that describes tasks performed by nurses (NTask), nurse types (NType), and nursing outcomes (NOutcome). For each nurse (k), we define a function &lt;NTask (k, i)&gt;, i=1 to N as a vector of dimension N, where N is the number of time periods in the study. The i component corresponds to the activity that the nurse is doing. The model will allow the quantitative classification of activity patterns for any finite number of nurses for an arbitrary set of tasks and for time at any specified resolution. The expected outcome is a set of vectors that can then be utilized to quantitatively model nurse activity trajectories and other patterns of nurse EHR interactions. DISCUSSION/SIGNIFICANCE: By instantiating the logical data model, we will demonstrate how nurse EHR interactions can be studied using temporal unsupervised learning and state-of-the-art artificial intelligence methods. We plan to simulate the potential impact of workload interventions and predict risk for nurse burnout.

Variable Impact of Medical Scribes on Physician Electronic Health Record Documentation Practices: A Quantitative Analysis Across a Large, Integrated Health-System.

Medical scribes have been utilized to reduce electronic health record (EHR) associated documentation burden. Although evidence suggests benefits to scribes, no large-scale studies have quantitatively evaluated scribe impact on physician documentation across clinical settings. This study aimed to evaluate the effect of scribes on physician EHR documentation behaviors and performance. This retrospective cohort study used EHR audit log data from a large academic health system to evaluate clinical documentation for all ambulatory encounters between January 2014 and December 2019 to evaluate the effect of scribes on physician documentation behaviors. Scribe services were provided on a first-come, first-served basis on physician request. Based on a physician's scribe use, encounters were grouped into 3 categories: never using a scribe, prescribe (before scribe use), or using a scribe. Outcomes included chart closure time, the proportion of delinquent charts, and charts closed after-hours. Three hundred ninety-five physicians (23% scribe users) across 29 medical subspecialties, encompassing 1,132,487 encounters, were included in the analysis. At baseline, scribe users had higher chart closure time, delinquent charts, and after-hours documentation than physicians who never used scribes. Among scribe users, the difference in outcome measures postscribe compared with baseline varied, and using a scribe rarely resulted in outcome measures approaching a range similar to the performance levels of nonusing physicians. In addition, there was variability in outcome measures across medical specialties and within similar subspecialties. Although scribes may improve documentation efficiency among some physicians, not all will improve EHR-related documentation practices. Different strategies may help to optimize documentation behaviors of physician-scribe dyads and maximize outcomes of scribe implementation.

Longitudinal Medical Student Collaborative Care in Primary Care Patients With Type 2 Diabetes.

The Education-Centered Medical Home (ECMH) is a longitudinal clerkship that emphasizes continuity and quality improvement in primary care. We aimed to evaluate our ECMH's ability to improve type 2 diabetes mellitus (T2DM) care through a systematic chart audit and care planning process. The effect of this intervention was measured by adherence to process and outcome measures. From November 2015 to March 2017, medical students were educated on and performed monthly chart audits of guideline-based quality metrics: hemoglobin A1c systolic blood pressure; lipid and microalbuminuria evaluation; annual ophthalmic and foot examinations; flu, hepatitis, and pneumonia vaccination; and statin therapy. Patients were included if they had a diagnosis of T2DM and were seen by the ECMH clinic before and after the audits started. Students shared audit logs, using them to plan patient appointments. We assessed changes in proportion of patients meeting each guideline with Fisher's exact test. The project included 11 patients with T2DM. ECMH adherence to the annual eye exam increased significantly 1 year postintervention, compared to preintervention (73% vs 55%; P=.03) and 6 months (73% vs 46%, P=.01). The metric with significant improvement during the chart audit, annual eye exam, is a process measure requiring advance planning. This small study suggests that a formal, regular audit process can improve student adherence to evidence-based care guidelines, particularly for tasks that require advance planning or action by the care team outside the day of a patient visit.