Audit Logs Research Articles

DELIA: Distributed Efficient Log Integrity Audit Based on Hierarchal Multi-Party State Channel

Audit log contains the trace of different activities in computing systems, which makes it critical for security management, censorship, and forensics. However, experienced attackers may delete or modify the audit log after their attacks, which makes the audit log unavailable in attack investigation. In this article, we focus on the log integrity audit in the same domain, in which a number of servers update audit logs for a single or several organizations as an alliance. We propose a distributed efficient log integrity audit framework, called DELIA, which employs the distributed ledger technique to protect audit information, and utilizes the idea of state channel to improve the throughput of distributed ledger. To generate stable state from the rapidly-updated logs in the domain, we propose a log state generation scheme, which not only generates state suitable for audit logs, but also enables mutual supervision within the domain. To overcome the high latency in existing state channel schemes, we propose a hierarchal multi-party state channel scheme, which makes the latency in our framework independent of the number of servers in the domain. We implement DELIA on Ethereum and evaluate its performance. The results show that our framework is efficient and secure in practice.

Improving Physician Electronic Health Record Usability and Perception Through Personalized Coaching.

The relationship between poor EHR training and subsequent poor usability is increasingly being recognized. We utilized objective EHR audit log data to personalize EHR training with the goal of improving EHR usability and to identify changes in physician perceptions pre- and post-intervention. We found that time in the system and Pajama time decreased post-coaching intervention. Different physician perceptions were reported pre- and post-coaching. Overall, personalized EHR coaching improved the usability and perceptions of physicians.

Health information systems security: Risks, prospects and frameworks

Information is the most precious asset of any organization and assessing risk to information is a core mandate of any institutional management to ensure availability of effective controls to protect information assets. The increasing digitization of health information and the ever-changing cyber security threat environment, has led to some public health data breaches and as information security become increasingly important to the continued success for businesses, majority of organizations are searching for an appropriate security framework. Security risk assessment framework enables identification of threats and vulnerabilities. Although numerous frameworks available in the market, selection of the right framework to meet the organization’s need is a challenge due to lack of prescriptiveness, standard, inconsistencies, complexity, compliance, cost, and certifications. To address the gap, this study assessed the security of health information system and privacy risks in addition to existing frameworks and developed an enhanced framework. The study adopted a descriptive cross – sectional design and was conducted in Siaya County, in Kenya. A questionnaire was used to collect data which was analyzed and presented inform of tables, and charts. The results indicated that confidentiality of information is good (use of identifiers and passwords at 96.8% approval rate), availability of physical controls to protect authorized access at 95.2%, availability of policies stating staff responsible for protection of information confidentiality at 91.9%, availability of written policy on patient confidentiality and privacy at 74.2% and use of access privileges at 68.8%. The findings on integrity of information was poor with availability of systems to review data accuracy having 71.9% approval rate, frequency of data review at 81.2%, availability of written description of information security manager’s responsibility at 39.5%, monitoring of electronic systems to detect potential breaches at 40%, creation of audit logs to track system transactions at 54% and frequency of reviewing audit logs at 51.5%. The findings on availability of information was good (availability of inventory of computers at 69.9%), regular updates of inventory at 61.3%, updates of patient data on laptops and desktops at 68.2%, sharing of data confidentiality and security policy at 36%, and regular backups of audited logs at 51% approval rates. Regarding the assessment of existing security frameworks, it was noted that HIPAA has the following shortcomings: lacks complete valid risk analysis, not certifiable, security rule is safeguarding electronic protected health information only, the security does not regulate emails and does not require encryption, and commitment on security is verbal. On the other hand, ISO/IEC 27001 is expensive, requires specific IT budget, special expertise, and more time to apply in public hospitals. Finally, NIPP framework is expensive, and uses consequence’s assessment which is outside the scope of this study.

Data station

Pooling and sharing data increases and distributes its value. But since data cannot be revoked once shared, scenarios that require controlled release of data for regulatory, privacy, and legal reasons default to not sharing. Because selectively controlling what data to release is difficult, the few data-sharing consortia that exist are often built around data-sharing agreements resulting from long and tedious one-off negotiations. We introduce Data Station, a data escrow designed to enable the formation of data-sharing consortia. Data owners share data with the escrow knowing it will not be released without their consent. Data users delegate their computation to the escrow. The data escrow relies on delegated computation to execute queries without releasing the data first. Data Station leverages hardware enclaves to generate trust among participants, and exploits the centralization of data and computation to generate an audit log. We evaluate Data Station on machine learning and data-sharing applications while running on an untrusted intermediary. In addition to important qualitative advantages, we show that Data Station: i) outperforms federated learning baselines in accuracy and runtime for the machine learning application; ii) is orders of magnitude faster than alternative secure data-sharing frameworks; and iii) introduces small overhead on the critical path.

Baseline eHealth Behaviors of Service Members: A Retrospective, Cross-Sectional Analysis of Patient Portal Use Before the Pandemic.

ABSTRACTIntroductionThe use of electronic health (eHealth) tools has the potential to support the overall health, wellness, fitness status, and ability to deploy worldwide of active duty service members (SMs). Additionally, the Coronavirus Disease 2019 pandemic forced healthcare organizations to quickly convert to virtual care settings to decrease face-to-face interactions and increase access to healthcare using technology. The shift to virtual care and the push to increase use of eHealth tools heightened the need to understand how military members interact with eHealth tools. Little is known about the factors that influence SMs use of eHealth tools and if having a health condition increases or decreases use. To evaluate these factors, we completed a cross-sectional, retrospective analysis on a sample of 198,388 active duty SMs aged 18 to 68 years.Materials and MethodsWe used two Military Health System (MHS) data sources—Tricare Online (TOL) Patient Portal 2018 audit logs and outpatient electronic health record data. Using eHealth behaviors identified in the audit logs, we evaluated and compared individual characteristics (i.e., “gender”, “age”, “race”, and “marital status”), environmental factors (i.e., “rank”, “military branch”, and “geographic location”), and six available health conditions (i.e., congenital health defects, amputation, anxiety, sleep, traumatic brain injury, and depression). Since moderate usage of eHealth tools is linked to improved health outcomes, adherence, communication, and increased consumer satisfaction, a logistic regression model was developed to find the factors most associated with moderate (3–11 logins per year) use of the portal.ResultsElectronic health use increased by SMs with underlying health conditions or if they were managing family member health. Most SMs who used the TOL Patient Portal were of ages 25-34 years, White, and married. The mean age is 32.53 for males and 29.98 for females. Over half of the TOL Patient Portal SM users utilized the portal one to two times. Most SMs used the TOL Patient Portal in Virginia, Texas, California, Florida, North Carolina, Georgia, and Maryland. The highest use was during the months of March to May. Frequent patient portal actions include searching for appointments, viewing health information, viewing medical encounters, and refilling medications. Although SMs with congenital health defects, anxiety, sleep issues, and depression have higher patient portal use rates, SMs with depression have a negative association with using the patient portal at a “moderate” rate. Viewing family member health information and searching for appointments were strongly associated with patient portal moderate use.ConclusionsOur findings support top military initiatives to improve the overall health, wellness, and readiness of SMs while decreasing the MHS’s overall cost of care while providing a foundation to compare “pre” and “post” pandemic eHealth behaviors. It is essential to note that SMs are more likely to use a patient portal to seek information or manage family member health. This key factor identifies the significance of family health promotion and readiness in the active duty SM’s life. The long-term goal of our study is to build the foundation for delivering tailored health information and eHealth tools to promote health and readiness-centric patient engagement.

Temporal Associations Between EHR-Derived Workload, Burnout, and Errors: a Prospective Cohort Study.

The temporal progression and workload-related causal contributors to physician burnout are not well-understood. To characterize burnout's time course and evaluate the effect of time-varying workload on burnout and medical errors. Six-month longitudinal cohort study with measurements of burnout, workload, and wrong-patient orders every 4 weeks. Seventy-five intern physicians in internal medicine, pediatrics, and anesthesiology at a large academic medical center. Burnout was measured using the Professional Fulfillment Index survey. Workload was collected from electronic health record (EHR) audit logs and summarized as follows: total time spent on the EHR, after-hours EHR time, patient load, inbox time, chart review time, note-writing time, and number of orders. Wrong-patient orders were assessed using retract-and-reorder events. Seventy-five of 104 interns enrolled (72.1%) in the study. A total of 337 surveys and 8,863,318 EHR-based actions were analyzed. Median burnout score across the cohort across all time points was 1.2 (IQR 0.7-1.7). Individual-level burnout was variable (median monthly change 0.3, IQR 0.1-0.6). In multivariable analysis, increased total EHR time (β=0.121 for an increase from 54.5 h per month (25th percentile) to 123.0 h per month (75th percentile), 95%CI=0.016-0.226), increased patient load (β=0.130 for an increase from 4.9 (25th percentile) to 7.1 (75th percentile) patients per day, 95%CI=0.053-0.207), and increased chart review time (β=0.096 for an increase from 0.39 (25th percentile) to 0.59 (75th percentile) hours per patient per day, 95%CI=0.015-0.177) were associated with an increased burnout score. After adjusting for the total number of ordering sessions, burnout was not statistically associated with an increased rate of wrong-patient orders (rate ratio=1.20, 95%CI=0.76-1.89). Burnout and recovery were associated with recent clinical workload for a cohort of physician trainees, highlighting the elastic nature of burnout. Wellness interventions should focus on strategies to mitigate sustained elevations of work responsibilities.

Medical Scribes Have a Variable Impact on Documentation Workflows.

Physicians can reduce their documentation time by working with a scribe. However, what scribes document and how their actions affect existing documentation workflows is unclear. This study leverages electronic health record (EHR) audit logs to observe how scribes affected the documentation workflows of seven physicians and their staff across 13,000 outpatient ophthalmology visits. In addition to editing progress notes, scribes routinely edited exam findings and diagnoses. Scribes with clinical training also edited items such as vital signs that a scribe without clinical training did not. Every physician edited patient records later in the day when working with a scribe and those who deferred their editing the most had some of the largest reductions in EHR time. These results suggest that what scribes document, how physicians work with scribes, and scribe impact on documentation time are all highly variable, highlighting the need for evidence-based best practices.

A flexible approach for cyber threat hunting based on kernel audit records

Hunting the advanced threats hidden in the enterprise networks has always been a complex and difficult task. Due to the variety of attacking means, it is difficult for traditional security systems to detect threats. Most existing methods analyze log records, but the amount of log records generated every day is very large. How to find the information related to the attack events quickly and effectively from massive data streams is an important problem. Considering that the knowledge graph can be used for automatic relation calculation and complex relation analysis, and can get relatively fast feedback, our work proposes to construct the knowledge graph based on kernel audit records, which fully considers the global correlation among entities observed in audit logs. We design the construction and application process of knowledge graph, which can be applied to actual threat hunting activities. Then we explore different ways to use the constructed knowledge graph for hunting actual threats in detail. Finally, we implement a LAN-wide hunting system which is convenient and flexible for security analysts. Evaluations based on the adversarial engagement designed by DARPA prove that our platform can effectively hunt sophisticated threats, quickly restore the attack path or assess the impact of attack.

Identification of Prescribing Errors in an Electronic Health Record Using a Retract-and-Reorder Tool: A Pilot Study.

The aims of this study were to develop and to validate an adapted Retract-and-Reorder (RAR) tool to identify and quantify near-miss/intercepted prescribing errors in an electronic health record. This is a cross-sectional study between February and March 2021 in an Irish maternity hospital. We used the RAR tool to detect near-miss prescribing errors in audit log data. Potential errors flagged by the tool were validated using prescriber interviews. Chart reviews were performed if the prescriber was unavailable for interview. Errors were judged to be clinical decisions in chart reviews through review of narrative notes, order components, and patient's clinical history. Interviews were analyzed with reference to the London Protocol, a process of incident analysis that categorizes causes of errors into various contributory factors including patient factors, task and technology factors, and work environment. Logistic regression with robust clustered standard errors was used to determine predictors for near-miss prescribing errors. We calculated the positive predictive value of the RAR tool by dividing the number of confirmed near-miss prescribing errors by the total number of RAR events identified. Eighty-four RAR events were identified in 27,407 medication orders. Seventy-one events were confirmed near-miss prescribing errors, resulting in a positive predictive value of 85.0% (95% confidence interval, 75%-91%) and an estimated near-miss prescribing error rate of 259/100,000 medication orders. Duplicate prescribing errors were most common (54/71, 76.1%). No errors were reported by prescribers. Consultants were less likely to make an error than nonconsultant hospital doctors (adjusted odds ratio, 0.10; 95% confidence interval, 0.01-0.84). Factors associated with errors included workload, staffing levels, and task structure. Our adapted RAR tool identified a variety of near-miss prescribing errors not otherwise reported. The tool has been implemented in the study hospital as a patient safety resource. Further implementations are planned across Irish hospitals.

Opportunities to use electronic health record audit logs to improve cancer care.

The rapid adoption of electronic health records (EHRs) has created extensive repositories of digitized data that can be used to inform improvements in care delivery, processes, and patient outcomes. While the clinical data captured in EHRs are widely used for such efforts, EHRs also capture audit log data that reflect how users interact with the EHR to deliver care. Automatically collected audit log data provide a unique opportunity for new insights into EHR user behavior and decision‐making processes. Here, we provide an overview of audit log data and examples that could be used to improve oncology care and outcomes in four domains: diagnostic reasoning and consumption, care team collaboration and communication, patient outcomes and experience, and provider burnout/fatigue. This data source could identify gaps in performance and care, physician uptake of EHR features that enhance decision‐making, and integration of data trends for oncology. Ensuring researchers and oncologists are familiar with the data's potential and developing the data engineering capacity to utilize this rich data source, will expand the breadth of research to improve cancer care.

Unusual Insider Behavior Detection Framework on Enterprise Resource Planning Systems Using Adversarial Recurrent Autoencoder

Detecting unusual behaviors of insiders on enterprise resource planning (ERP) systems is one of the essential parts to reduce the risks of threatening and abusing enterprise resources by insiders. Many approaches to detect the behaviors based on rule-based systems and stochastic processes are currently limited to empirical monitoring using manually established algorithms or probabilistic boundaries. Those approaches need prior knowledge such as user permission guideline and process data characteristics. Unfortunately, obtaining prior knowledge is hard in practice, and these are not appropriate to detect atypical unusual behavior which can not be clearly defined using heuristic rules. Therefore, in this article, we propose a novel framework for unusual insider behavior detection (UIBD) for ERP systems. The proposed framework initially derives a discriminative model for normal behavior samples, and UIBD is conducted by computing an error using the model. Since the model is compiled using normal samples only, the error of unusual samples would be larger than normal ones. To derive a robust normal behavior model, we present adversarial recurrent autoencoder (ARAE). To demonstrate the efficiency of the proposed framework based on ARAE, we conduct experiments using a dataset composed of insider behaviors defined by sequences of security audit logs of ERP systems operating in real-world enterprises. The experimental results show that the proposed framework with ARAE can successfully detect unusual insider behaviors and outperform other methods to detect unusual insider behavior or threatening.

Predicting physician burnout using clinical activity logs: Model performance and lessons learned

BackgroundBurnout is a significant public health concern affecting more than half of the healthcare workforce; however, passive screening tools to detect burnout are lacking. We investigated the ability of machine learning (ML) techniques to identify burnout using passively collected electronic health record (EHR)-based audit log data. MethodPhysician trainees participated in a longitudinal study where they completed monthly burnout surveys and provided access to their EHR-based audit logs. Using the monthly burnout scores as the target outcome, we trained ML models using combinations of features derived from audit log data—aggregate measures of clinical workload, time series-based temporal measures of EHR use, and the baseline burnout score. Five ML models were constructed to predict burnout as a continuous score: penalized linear regression, support vector machine, neural network, random forest, and gradient boosting machine. Results88 trainee physicians participated and completed 416 surveys; greater than10 million audit log actions were collected (Mean [Standard Deviation] = 25,691 [14,331] actions per month, per physician). The workload feature set predicted burnout score with a mean absolute error (MAE) of 0.602 (95% Confidence Interval (CI), 0.412–0.826), and was able to predict burnout status with an average AUROC of 0.595 (95% CI 0.355–0.808) and average accuracy 0.567 (95% CI 0.393–0.742). The temporal feature set had a similar performance, with MAE 0.596 (95% CI 0.391–0.826), and AUROC 0.581 (95% CI 0.343–0.790). The addition of the baseline burnout score to the workload features improved the model performance to a mean AUROC of 0.829 (95% CI 0.607–0.996) and mean accuracy of 0.781 (95% CI 0.587–0.936); however, this performance was not meaningfully different than using the baseline burnout score alone. ConclusionsCurrent findings illustrate the complexities of predicting burnout exclusively based on clinical work activities as captured in the EHR, highlighting its multi-factorial and individualized nature. Future prediction studies of burnout should account for individual factors (e.g., resilience, physiological measurements such as sleep) and associated system-level factors (e.g., leadership).

Extensive Preoperative Work Is Required for Revision Hip and Knee Arthroplasty

BackgroundDetermining the clinical effort associated with preparing for revision total hip and knee arthroplasty is necessary to maintain the appropriate work relative value unit rating. We have investigated the work done by the orthopedic surgical team in the days and weeks prior to revision hip and knee arthroplasty using a count of time by team members in the electronic medical record (EMR). MethodsEMR audit logs were generated, and preoperative work (POW) was calculated for members of the surgical team for 200 sequential revision cases. Independent samples t-tests were conducted to compare total POW for procedure, age, gender, insurance, and health literacy; significance threshold was set at P = .05. ResultsPOW was 97.7 minutes (standard deviation [SD] 53.1). Surgeon POW accounted for 10.5 minutes (SD 9.3), nurses for 29.9 minutes (SD 34.2), mid-level providers for 22.1 minutes (SD 17.0), and office technicians for 34.1 minutes (SD 35.2). There was no difference in total POW based on procedure (hip vs knee), age, gender, insurance type, or health literacy. ConclusionRevision arthroplasty requires substantial preoperative preparation from the surgical team. Most of this is by nurses, mid-level providers, and office staff. This does not seem to be different for hip or knee revisions or by age and gender. EMR audit logs capture the bare minimum POW required to prepare a patient for revision arthroplasty.

FBASHI: Fuzzy and Blockchain-Based Adaptive Security for Healthcare IoTs

Internet of Things (IoT) is a system of interconnected devices that have the ability to monitor and transfer data to peers without human intervention. Authentication, Authorization and Audit Logs (AAA) are prime features of Network Security and easily attained in legacy systems, however, remains unachieved in IoT. The IoTs require due security considerations as the conventional security mechanisms are not optimized for such devices due to various aspects such as heterogeneity, resource constrained processing, storage and multiple factors. Additionally, the legacy systems are mostly centralized and thus introduce a single point of failure. In this research, a novel framework, FBASHI is presented that is based on fuzzy logic and blockchain technology to achieve AAA services. The proposed system is developed using Hyperledger that is a blockchain platform providing privacy and fast response capability, therefore, it is best suited for the healthcare IoT environments. This work proposes behavior driven adaptive security mechanism for healthcare IoTs and networks based on blockchain by utilizing fuzzy logic and presents a heuristic approach towards behavior driven adaptive security providing AAA services. FBASHI is implemented to analyze its security and practicality. Furthermore, a comparison is drawn with other blockchain-based solutions.

Authentication model for mobile access subjects

The paper proposes a model for authenticating mobile access subjects in a critical system. The incident detection mechanism is based on training and classification of the user activity audit log dataset embedded in the underlying configuration of the content management system. The proposed mechanisms can be implemented in systems whose subjects have some terrain movement patterns (within a city, district). The proposed model eliminates the disadvantages of existing traditional authentication methods based on the use of explicit verification methods. KNN methods, SVM and random forest algorithm are used for classification. These classification methods are formalized mathematically in the context of the set authentication problem, and to confirm the effectiveness of the algorithm, the robustness of the model by Euclidean metric and Chebyshev metric is evaluated, as well as the information content of the input features that affect the classification by each of the considered methods.

A Blockchain-Based Audit Trail Mechanism: Design and Implementation

Audit logs are a critical component in today’s enterprise business systems as they provide several benefits such as records transparency and integrity and security of sensitive information by creating a layer of evidential support. However, current implementations are vulnerable to attacks on data integrity or availability. This paper presents a Blockchain-based audit trail mechanism that leverages the security features of Blockchain to enable secure and reliable audit trails and to address the aforementioned vulnerabilities. The architecture design and specific implementation are described in detail, resulting in a real prototype of a reliable, secure, and user-friendly audit trail mechanism.

Characterizing styles of clinical note production and relationship to clinical work hours among first-year residents.

To characterize variation in clinical documentation production patterns, how this variation relates to individual resident behavior preferences, and how these choices relate to work hours. We used unsupervised machine learning with clinical note metadata for 1265 progress notes written for 279 patient encounters by 50 first-year residents on the Hospital Medicine service in 2018 to uncover distinct note-level and user-level production patterns. We examined average and 95% confidence intervals of median user daily work hours measured from audit log data for each user-level production pattern. Our analysis revealed 10 distinct note-level and 5 distinct user-level production patterns (user styles). Note production patterns varied in when writing occurred and in how dispersed writing was through the day. User styles varied in which note production pattern(s) dominated. We observed suggestive trends in work hours for different user styles: residents who preferred producing notes in dispersed sessions had higher median daily hours worked while residents who preferred producing notes in the morning or in a single uninterrupted session had lower median daily hours worked. These relationships suggest that note writing behaviors should be further investigated to understand what practices could be targeted to reduce documentation burden and derivative outcomes such as resident work hour violations. Clinical note documentation is a time-consuming activity for physicians; we identify substantial variation in how first-year residents choose to do this work and suggestive trends between user preferences and work hours.

Customized Learning Model Using Learner Activity Analysis

Reshaping education with knowledge and skills is the major objective of many of the developing organization to build competitive knowledge economy. Electronic learning, also popularly known as e-learning is one of the most effective applications of Information and Communication Technology (ICT), which refurbishes the dimension in higher education. COVID-19 pandemic situation justifies the need and real utilization of E-learning. Existing E-learning system provides static content to all learners irrespective of their learning needs. In this paper we propose to create Dynamic Role based access control model based on information from learners needs, audit logs and observations to support adaptive E-learning systems.

Predicting next-day discharge via electronic health record access logs.

Hospital capacity management depends on accurate real-time estimates of hospital-wide discharges. Estimation by a clinician requires an excessively large amount of effort and, even when attempted, accuracy in forecasting next-day patient-level discharge is poor. This study aims to support next-day discharge predictions with machine learning by incorporating electronic health record (EHR) audit log data, a resource that captures EHR users' granular interactions with patients' records by communicating various semantics and has been neglected in outcome predictions. This study focused on the EHR data for all adults admitted to Vanderbilt University Medical Center in 2019. We learned multiple advanced models to assess the value that EHR audit log data adds to the daily prediction of discharge likelihood within 24 h and to compare different representation strategies. We applied Shapley additive explanations to identify the most influential types of user-EHR interactions for discharge prediction. The data include 26 283 inpatient stays, 133 398 patient-day observations, and 819 types of user-EHR interactions. The model using the count of each type of interaction in the recent 24 h and other commonly used features, including demographics and admission diagnoses, achieved the highest area under the receiver operating characteristics (AUROC) curve of 0.921 (95% CI: 0.919-0.923). By contrast, the model lacking user-EHR interactions achieved a worse AUROC of 0.862 (0.860-0.865). In addition, 10 of the 20 (50%) most influential factors were user-EHR interaction features. EHR audit log data contain rich information such that it can improve hospital-wide discharge predictions.

Image-Based Insider Threat Detection via Geometric Transformation

Insider threat detection has been a challenging task over decades; existing approaches generally employ the traditional generative unsupervised learning methods to produce normal user behavior model and detect significant deviations as anomalies. However, such approaches are insufficient in precision and computational complexity. In this paper, we propose a novel insider threat detection method, Image-based Insider Threat Detector via Geometric Transformation (IGT), which converts the unsupervised anomaly detection into supervised image classification task, and therefore the performance can be boosted via computer vision techniques. To illustrate, our IGT uses a novel image-based feature representation of user behavior by transforming audit logs into grayscale images. By applying multiple geometric transformations on these behavior grayscale images, IGT constructs a self-labelled dataset and then trains a behavior classifier to detect anomaly in a self-supervised manner. The motivation behind our proposed method is that images converted from normal behavior data may contain unique latent features which remain unchanged after geometric transformation, while malicious ones cannot. Experimental results on CERT dataset show that IGT outperforms the classical autoencoder-based unsupervised insider threat detection approaches, and improves the instance and user based Area under the Receiver Operating Characteristic Curve (AUROC) by 4% and 2%, respectively.