False Positive Alerts Research Articles

Author commentary on Erik A. Holzwanger et al.

Erik A. Holzwanger et al. Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy

Novel ways to use sensor data to improve mastitis management

Current sensor systems are used to detect cows with clinical mastitis. Although, the systems perform well enough to not negatively affect the adoption of automatic milking systems, the performance is far from perfect. An important advantage of sensor systems is the availability of multiple measurements per day. By clearly defining the need for detection of subclinical mastitis (SCM) and clinical mastitis (CM) from the farmers' management perspective, detection and management of SCM and CM may be improved. Sensor systems may also be used for other aspects of mastitis management. In this paper we have defined 4 mastitis situations that could be managed with the support of sensor systems. Because of differences in the associated management and the epidemiology of these specific mastitis situations, the required demands for performance of the sensor systems do differ. The 4 defined mastitis situations with the requirements of performance are the following: (1) Cows with severe CM needing immediate attention. Sensor systems should have a very high sensitivity (>95% and preferably close to 100%) and specificity (>99%) within a narrow time window (maximum 12 h) to ensure that close to all cows with true cases of severe CM are detected quickly. Although never studied, it is expected that because of the effects of severe CM, such a high detection performance is feasible. (2) Cows with mastitis that do not need immediate attention. Although these cows have a risk of progressing into severe CM or chronic mastitis, they should get the chance to cure spontaneously under close monitoring. Sensor alerts should have a reasonable sensitivity (>80%) and a high specificity (>99.5%). The time window may be around 7 d. (3) Cows needing attention at drying off. For selective dry cow treatment, the absence or presence of an intramammary infection at dry-off needs to be known. To avoid both false-positive and false-negative alerts, sensitivity and specificity can be equally high (>95%). (4) Herd-level udder health. By combining sensor readings from all cows in the herd, novel herd-level key performance indicators can be developed to monitor udder health status and development over time and raise alerts at significant deviances from predefined thresholds; sensitivity should be reasonably high, >80%, and because of the costs for further analysis of false-positive alerts, the specificity should be >99%. The development and validation of sensor-based algorithms specifically for these 4 mastitis situations will encourage situation-specific farmer interventions and operational udder health management.

A practical method for real-time detection of pedicle wall breaching during funneling.

A reliable, real-time method for the detection of pedicle wall breaching during funnelling in spine deformity surgery could be accessible to any surgeon assisted with neuromonitoring. Fifty-six consecutive patients (1066 pedicles), who were submitted to spinal deformity surgery from December 2013 to July 2015 were included in the study group. A control group of 13 consecutive patients (226 pedicles) with spinal deformity surgery were operated on from January to December 2013 and were excluded from finder stimulation. In the study cohort, continuous stimulation during funnelling was delivered via a finder and subsequently a compound muscle action potential (CMAP) threshold was determined. Following funnelling, manual inspection of the pedicular internal walls was performed. The CMAP thresholds were compared with the results of palpation to determine the sensitivity and specificity of the technique for detecting pedicular breaching. To cover common ranges of damage, the medial and lateral breaches were compared and the concave-apical breaches compared to the non-apical or convex-apical breaches. In addition, a pedicle screw test was estimated for all patients. ROC analysis showed 9mA cut-off to have a sensitivity of 88.0% and a specificity of 89.5% for predicting pedicular breaching, with an area under the curve of 0.92 (95% confidence interval 0.90-0.94; P < 0.001). Using 9mA threshold as an alert criterion, funnelling at the concave-apical pedicles showed significantly more true and false positive alerts and fewer true negative alerts when compared with the non-apical and convex-apical pedicles (P < 0.001). Medial breaches had significantly lower stimulation thresholds than lateral breaches (P < 0.001). Thresholds of screw-testing were significantly higher for study than for control-patients (P = 0.002). Finder stimulation has a considerably higher sensitivity and specificity for prediction of pedicular breaching, most prominent for medial breaches. Screw-testing displayed significantly better results in patients undergoing the finder stimulation technique, as compared with the control group. The main advantages of our method are its high safety level and low cost, which may be critical in less affluent countries. III.

Variable selection for monitoring sickness behavior in lactating dairy cattle with the application of control charts

The present observational study investigated the application of multivariate cumulative sum (MCUSUM) control charts by including variables selected by principal component analysis and partial least squares (PLS) regression to identify sickness behavior in dairy cattle. Therefore, sensor information (24 variables) was collected from 480 milking cows on a German dairy farm between September 2018 and December 2019. These variables were gathered in potentially different scenarios on farm. In total, data from 749 animals were available for evaluation. Variables were chosen based on the information of 499 cows (62 healthy; 437 sick) with 93,598 observations. The available diagnoses were collected together to form 1,025 sickness events. Hence, the different numbers of selected variables were included into the MCUSUM control charts. The performance of the MCUSUM control charts was evaluated by a 10-fold cross validation; hence, 90% of the original data set (749 cows) represented the training data, and the remaining 10% was used to test the training results. On average, the 10 training data sets included 124,871 observations with 1,392 sickness events, and the 10 testing data sets included, on average, 13,704 observations with 153 sickness events. The MCUSUM generated from the variables selected by principal component analysis showed comparable results in training and testing in all scenarios; therefore, 70.0 to 97.4% of the sickness events were detected. The false-positive rates ranged from 8.5 to 29.6%, and thus they created at least 2.6 false-positive alerts per day in testing. The variables selected by the PLS regression approach showed comparable sickness detection rates (70.0-99.9%) as well as false-positive rates (8.2-62.8%) in most scenarios. The best performing scenario produced 2.5 false-positive alerts in testing. Summarizing, both approaches showed potential for practical implementation; however, the PLS variable selection approach showed fewer false positives. Therefore, the PLS regression approach could generate a more reliable sickness detection algorithm, if combined with MCUSUM control charts, and considered for practical implementation.

Wind Turbine Fault Detection Using Highly Imbalanced Real SCADA Data

Wind power is cleaner and less expensive compared to other alternative sources, and it has therefore become one of the most important energy sources worldwide. However, challenges related to the operation and maintenance of wind farms significantly contribute to the increase in their overall costs, and, therefore, it is necessary to monitor the condition of each wind turbine on the farm and identify the different states of alarm. Common alarms are raised based on data acquired by a supervisory control and data acquisition (SCADA) system; however, this system generates a large number of false positive alerts, which must be handled to minimize inspection costs and perform preventive maintenance before actual critical or catastrophic failures occur. To this end, a fault detection methodology is proposed in this paper; in the proposed method, different data analysis and data processing techniques are applied to real SCADA data (imbalanced data) for improving the detection of alarms related to the temperature of the main gearbox of a wind turbine. An imbalanced dataset is a classification data set that contains skewed class proportions (more observations from one class than the other) which can cause a potential bias if it is not handled with caution. Furthermore, the dataset is time dependent introducing an additional variable to deal with when processing and splitting the data. These methods are aimed to reduce false positives and false negatives, and to demonstrate the effectiveness of well-applied preprocessing techniques for improving the performance of different machine learning algorithms.

Evaluation and characterization of estrus alerts and behavioral parameters generated by an ear-attached accelerometer-based system for automated detection of estrus

Our objectives were to evaluate the performance of an ear-attached automated estrus detection (AED) system (Smartbow; Zoetis) that monitored physical activity and rumination time, and to characterize AED system estrus alert features (i.e., timing and duration). Lactating Holstein cows (n = 216) commenced a protocol for the synchronization of estrus at 50 ± 3 DIM or 18 ± 3 d after artificial insemination. For 7 d after induction of luteolysis with PGF2α (d 0), we used visual observation of estrous behavior (30 min, 2 times per day) and data from an automated mounting behavior monitoring system based on a pressure-activated tail-head sensor (HeatWatch; Cowchips LLC) as a reference test (RTE) to detect behavioral estrus. Concomitantly, estrus alerts and their features were collected from the AED system. Progesterone levels confirmed luteal regression, and transrectal ultrasonography confirmed the occurrence and timing of ovulation. Performance metrics for the AED system were estimated with PROC FREQ in SAS, using the RTE or ovulation only as a reference. Performance was also estimated after the removal of cows with a discrepancy between the RTE and ovulation. Continuous outcomes with or without repeated measurements were evaluated by ANOVA using PROC MIXED in SAS. Based on the RTE, 86.6% (n = 187) of the cows presented estrus and ovulated; 1.4% (n = 3) presented estrus and did not ovulate; 6.4% (n = 14) did not present estrus but ovulated; and 5.6% (n = 12) did not present estrus or ovulation. We found no difference in the proportion of cows detected in estrus and with ovulation for the AED system (83.4%) and the RTE (86.6%). Compared with estrus events as detected by the RTE, sensitivity for the AED was 91.6% (95% CI: 87.6-95.5) and specificity was 69.2% (95% CI: 51.5-87.0). Using ovulation as reference, sensitivity was 89.6% (95% CI: 85.3-93.8) and specificity was 86.7% (95% CI 69.5-100). For all cows with agreement between the RTE and ovulation, sensitivity was 92.5% (95% CI: 88.7-96.3) and specificity was 91.7% (95% CI: 76.0-100). The mean (±SD) interval from induction of luteolysis to estrus alerts, estrus alert duration, and the onset of estrus alerts to ovulation interval were 72.2 ± 18.1, 13.5 ± 3.8, and 23.8 ± 7.1 h, respectively. We concluded that an ear-attached AED system that monitored physical activity and rumination time was effective at detecting cows in estrus and generated few false positive alerts when accounting for ovulation, cow physiological limitations, and the limitations of the RTE.

One size does not fit all: Factors associated with increased frequency of radiation overexposure alerts based on fixed-alert thresholds.

Quantify the expected rate of CT radiation dose alerts for three body regions using accepted radiation dose benchmarks and assess key determinants of alert frequency. This IRB-approved retrospective cohort study evaluated consecutive CT examinations performed between July and December 2013 within an academic medical system. CTDIvol x-ray tube output metrics were compared to the body-region-specific benchmark levels, Achievable Doses (AD), Diagnostic Reference Levels (DRL), and Dose Notification Values (DNV). A logistic regression model for the simulated alerts was fit as a function of the independent predictors: scanner, body region, gender, weight, and age. For 17,000 exams, the proportion of events triggering alerts increased with patient weight. Significant covariates were scanner, body region, patient weight and patient age (all p<0.0001). Odds of alert generation for the AD, DRL, and DNV benchmarks increased by 7.6%, 6.6% and 2.9% per kilogram, respectively, and by 0.8%, 1.1% and -2.7% per year of age (all p<0.0001). Compared to the most highly optimized scanner, odds of alert generation varied by a factor of 595 for AD, 1126 for DRL, and 13 for DNV. Alert frequency was significantly correlated with weight, age, body region and scanner. Controllable factors include scanner functionality and associated protocol optimization. Patient factors driving alert frequency are predominantly weight, and to a lesser degree, age. Size-agnostic fixed dose thresholds can frequently produce false positive alerts in appropriately performed exams of large patients, while missing opportunities to identify outlier scans of higher-than-expected dose in small patients.

A Combination Techniques of Intrusion Prevention and Detection for Cloud Computing

Cloud computing provides scalable, on-demand, and highly available computing services over the Internet to both the public and organizations on a pay-per-use basis. It provides a variety of services such as networking, storage space, and applications. The key issue for cloud computing is ensuring the confidentiality and privacy of cloud resources and data. Enticing the user to purchase cloud services requires their trust which cannot be guaranteed unless the infrastructure is effectively protected because attacks at this level will threaten the whole system. To this end, we propose the Integrated Intrusion Prevention and Detection System (IIPDS) to prevent and detect different types of attacks to the infrastructure level of the cloud system. The proposed design uses Trusted Third Party (TTP) services and SSL/TLS protocols as intrusion prevention methods to secure the communication between the cloud provider and the user. It also uses multiple Intrusion Detection Systems (IDS) distributed over multiple cloud regions. The IDS system is capable of detecting known and unknown attacks using anomaly and rule-based (signature) intrusion detection techniques. The simulation results proved the efficiency of the system in detecting a wide range of attacks with low false positive alerts and low computational overhead.

Entropy Based Features Distribution for Anti-DDoS Model in SDN

In modern network infrastructure, Distributed Denial of Service (DDoS) attacks are considered as severe network security threats. For conventional network security tools it is extremely difficult to distinguish between the higher traffic volume of a DDoS attack and large number of legitimate users accessing a targeted network service or a resource. Although these attacks have been widely studied, there are few works which collect and analyse truly representative characteristics of DDoS traffic. The current research mostly focuses on DDoS detection and mitigation with predefined DDoS data-sets which are often hard to generalise for various network services and legitimate users’ traffic patterns. In order to deal with considerably large DDoS traffic flow in a Software Defined Networking (SDN), in this work we proposed a fast and an effective entropy-based DDoS detection. We deployed generalised entropy calculation by combining Shannon and Renyi entropy to identify distributed features of DDoS traffic—it also helped SDN controller to effectively deal with heavy malicious traffic. To lower down the network traffic overhead, we collected data-plane traffic with signature-based Snort detection. We then analysed the collected traffic for entropy-based features to improve the detection accuracy of deep learning models: Stacked Auto Encoder (SAE) and Convolutional Neural Network (CNN). This work also investigated the trade-off between SAE and CNN classifiers by using accuracy and false-positive results. Quantitative results demonstrated SAE achieved relatively higher detection accuracy of 94% with only 6% of false-positive alerts, whereas the CNN classifier achieved an average accuracy of 93%.

Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy.

The occurrence of false-positive alerts is an important outcome measure in computer-aided colon polyp detection (CADe) studies. However, there is no consensus definition of a false positive in clinical trials evaluating CADe in colonoscopy. We aimed to study the diagnostic performance of CADe based on different threshold definitions for false-positive alerts. A previously validated CADe system was applied to screening/surveillance colonoscopy videos. Different thresholds for false-positive alerts were defined based on the time an alert box was continuously traced by the system. Primary outcomes were false-positive results and specificity using different threshold definitions of false positive. 62 colonoscopies were analyzed. CADe specificity and accuracy were 93.2 % and 97.8 %, respectively, for a threshold definition of ≥ 0.5 seconds, 98.6 % and 99.5 % for a threshold definition of ≥ 1 second, and 99.8 % and 99.9 % for a threshold definition of ≥ 2 seconds. Our analysis demonstrated how different threshold definitions of false positive can impact the reported diagnostic performance of CADe for colon polyp detection.

POTENTIAL DISGUISING ATTACK VECTORS ON SECURITY OPERATION CENTERS AND SIEM SYSTEMS

In this article we highlight several potential vectors of attacks that can be carried out on a monitoring capacities powered by SOC SIEM using its common features and misconfigurations. Widely spread problems like excessive amounts of false positive alerts or not absolutely accurate configuration of the correlation rules may lead to situation where an attacker is able to trigger an undesired state of the monitoring system. We’ve find three potential vectors for evasion the SIEM powered SOCs monitoring. The first vector grounds on mechanisms used to collect event data – log collectors: the malfunctioning SIEM state can be achieved with generating and submitting the bogus event data to the processing party like SIEM. Fake data flow may cause generation of mistaken alerts which can confuse the analytics stuff. The second vector employs some of the attacker’s knowledge about actual SIEM configuration – exploitation of correlation rule flaws. Taking into account the fact that correlation rules are mostly hand-written, they are prone to some logic flaws – certain detection rules may not be triggered by all of the malicious attack indicators. An attacker with knowledge about that feature may fulfill the unrecorded conditions and trick the SIEM to treat the attack flow as benign activity. The last researched vector is based on redundantly sensitive detection rules which produce a lot of false positive alarms but are not removed. An attacker may trigger the malfunctioning alarm continuously to distract the analytics stuff and perform its actions under the cover of noise. Those discussed vectors are derived from analysis of the actual SIEM installations and SOC processes used as best practices. We have no actual indicators that those attacks are carried out “in wild” at the moment of issuing of this article, but it is highly probable that those tactics may be used in the future. The purpose of this research is to highlight the possible risks for the security operation centers connected with actual processes and practices used in industry and to develop the remediation strategy in perspective.

Methods to Accurately Exclude Renal Dialysis Patients from The Welsh Acute Kidney Injury E-Alert System

IntroductionAcute Kidney Injury (AKI) effects up to 20% of emergency hospital admissions. It can be identified from serum creatinine (SCr) levels and as such an e-alert system has been introduced to hospitals in England and Wales to identify AKI. This system relies on manual review by pathology laboratory staff to prevent false alerts in dialysis patients.&#x0D; Objectives and ApproachOur aim is to use routinely collected dialysis timeline and dialysis treatment data in order to create dialysis treatment spells using Structured Query Language (SQL), which can then be compared with dialysis prevalence results in the UK renal registry. Furthermore, we will link this data with biochemical results in Secure Anonymised Information Linkage (SAIL) Databank to create an AKI cohort.&#x0D; Through identifying when patients are on renal replacement therapy (RRT) at a specific time we are able to exclude false positive AKI Alerts. This is done in two ways – one from a timeline within the data which is used in the UK renal registry. The other using individual dialysis sessions to get accurate coverage, as short periods of acute dialysis are often not been recorded in UK renal registry data.&#x0D; ResultsUsing the renal dataset in SAIL we were able identify more renal dialysis patients than the UK renal registry between (2009-17). By applying this to Swansea and Bridgend pathology data in SAIL we were able to identify 60,072 SCr tests in dialysis patients between 2011 and 2013 out of 1,776,101 SCr tests. In this cohort, without ruling out dialysis patients there would be 91,140 AKI alerts, however 19.6% (17,855) of the alerts are during dialysis treatments spells.&#x0D; Conclusion / ImplicationsWe were able to recreate the ‘e-Alert’ algorithm using SQL with SAIL and get an accurate picture of RRT in Wales, enabling more accurate AKI analysis.

Neuromonitoring in Cervical Spine Surgery: When Is a Signal Drop Clinically Significant?

Study DesignRetrospective cohort study.PurposeTo identify the clinical significance of different patterns of intraoperative neuromonitoring (IONM) signal alerts.Overview of LiteratureIONM is a long-established valuable adjunct to complex spine surgeries. IONM for cervical spine surgery is in the form of somatosensory evoked potential (SSEP) and motor evoked potential (MEP). The efficacy of both modalities (individually or in combination) to detect clinically significant neurological compromise is constantly being debated and requires conclusive suggestions.MethodsClinical and neuromonitoring data of 207 consecutive adult patients who underwent cervical spine surgeries at multiple surgical centers using bimodal IONM were analyzed. Signal changes were divided into three groups. Group 0 had transient signal changes in either MEPs or SSEPs, group 1 had sustained unimodal changes, and group 2 had sustained changes in both MEPs and SSEPs. The incidences of true neurological deficits in each group were recorded.ResultsA total of 25% (52/207) had IONM signal alerts. Out of these signal drops, 96% (50/52) were considered to be false positives. Groups 0 and 1 had no incidence of neurological deficits, while group 2 had a 29% (2/7) rate of true neurological deficits. The sensitivities of both MEP and SSEP were 100%. SSEP had a specificity of 96.6%, while MEP had a lower specificity at 76.6%. C5 palsy rate was 6%, and there was no correlation with IONM signal alerts (p=0.73).ConclusionsThis study shows that we can better predict its clinical significance by dividing IONM signal drops into three groups. A sustained, bimodal (MEP and SSEP) signal drop had the highest risk of true neurological deficits and warrants a high level of caution. There were no clear risk factors for false-positive alerts but there was a trend toward patients with cervical myelopathy.

Evaluation of Attackers’ Skill Levels in Multi-Stage Attacks

The rapid move to digitalization and usage of online information systems brings new and evolving threats that organizations must protect themselves from and respond to. Monitoring an organization’s network for malicious activity has become a standard practice together with event and log collection from network hosts. Security operation centers deal with a growing number of alerts raised by intrusion detection systems that process the collected data and monitor networks. The alerts must be processed so that the relevant stakeholders can make informed decisions when responding to situations. Correlation of alerts into more expressive intrusion scenarios is an important tool in reducing false-positive and noisy alerts. In this paper, we propose correlation rules for identifying multi-stage attacks. Another contribution of this paper is a methodology for inferring from an alert the values needed to evaluate the attack in terms of the attacker’s skill level. We present our results on the CSE-CIC-IDS2018 data set.

The optimal threshold for prompt clinical review: An external validation study of the national early warning score.

The aim of this study was to determine the optimal threshold for national early warning score in clinical practice. The national early warning score is an aggregate early warning score aiming to predict patient mortality. Studies validating national early warning score did not use standardised patient outcomes or did not always include clinical workload in their results. Since all patients with a positive national early warning score require a clinical workup, it is crucial to determine the optimal threshold to limit false-positive alerts. An external validation study using retrospectively collected data of patient admissions in six Belgian hospitals. We adhered to the STARD guideline for reporting. Two sample groups were selected: the cross-sectional sample (admitted patients, 1day every 4months) and the serious adverse event sample (all patients with unexpected death, cardiac arrest and unplanned admission to the intensive care unit). The maximum registered national early warning score value was collected in both groups. Predictive values were used as estimates for clinical workload. We collected 1,523 in the cross-sectional sample and 390 patients in the serious adverse event sample. A national early warning score ≥5 had a predictive value of 6.8% and a negative predictive value of 99.5% to predict unexpected death, cardiac arrest with cardiopulmonary resuscitation or unplanned admission to intensive care (AUROC 0.841). The performance of national early warning score differed between outcome measures. Considering the predictive value, the optimal threshold for national early warning score is ≥5. We validated national early warning score to be applied in general hospital wards and confirmed the optimal threshold (≥5). When a patient has a national early warning score <5, we may assume that in the next 24hr this patient is less likely to die unexpectedly, receive cardiopulmonary resuscitation or be transferred to the ICU. Because of the significant number of false positives when national early warning score is ≥5, hospitals should create workable guidelines for clinical practice.

An Individualized, Alert-based Heartlogic Management Strategy Can Identify Patients At Risk For Impending Heart Failure Events

Background HeartLogic is a diagnostic tool that uses multiple sensors (heart sounds, thoracic impedance, respiration, heart rate, patient activity) to track physiologic trends and combine them into one composite index designed to alert clinicians of patients at risk for impending heart failure events (HFE). While prior studies have shown an alert threshold of 16 to be predictive of HFE, there is currently no standard treatment protocol to manage these alerts. Objective To design a management algorithm and assess the utility of HeartLogic alerts in predicting HFE among patients with chronic systolic heart failure. Methods All patients with activated HeartLogic devices at a large academic center were followed prospectively for three months and monitored for elevated HeartLogic index defined as 16 or greater. All alerts were managed by the heart failure clinical team who assessed each patient via telephone call. Diuretic changes were made at the discretion of the clinical team. HFE, defined as unscheduled visits or heart failure hospitalizations occurring within 6 weeks after the initial alert were recorded. Results We identified a total of 80 patients with activated HeartLogic devices. A total of 38 patients (48%) had at least one HeartLogic alert during the study period. Of these, 26 patients (68%) appeared to have false positive alerts and did not require diuretic change nor had HFE, while 12 (32%) had heart failure symptoms requiring diuretic adjustment or urgent follow-up. Three of these 12 patients failed outpatient diuretic augmentation and were admitted for decompensated heart failure. Among the 42 patients (52%) who did not have an alert, only one patient had a HFE. Overall, HeartLogic demonstrated a sensitivity of 92%, specificity of 61%, positive predictive value of 32% and negative predictive value of 98%. Increasing HeartLogic alert threshold to >20 in this cohort would improve specificity to 90% and positive predictive value to 56%, thereby reducing staff burden, with an acceptable sensitivity (69%) and negative predictive value (94%). Conclusion This is the first study to evaluate the effects of an individualized HeartLogic threshold in an alert-based follow-up management strategy. This strategy identified actionable alerts leading to change in diuretic management. We do not recommend a standardized diuretic change algorithm for alerts given the heterogeneity of the index value and high false positive rate of alerts. Future studies are needed to determine if this management strategy leads to decrease in heart failure admissions. HeartLogic is a diagnostic tool that uses multiple sensors (heart sounds, thoracic impedance, respiration, heart rate, patient activity) to track physiologic trends and combine them into one composite index designed to alert clinicians of patients at risk for impending heart failure events (HFE). While prior studies have shown an alert threshold of 16 to be predictive of HFE, there is currently no standard treatment protocol to manage these alerts. To design a management algorithm and assess the utility of HeartLogic alerts in predicting HFE among patients with chronic systolic heart failure. All patients with activated HeartLogic devices at a large academic center were followed prospectively for three months and monitored for elevated HeartLogic index defined as 16 or greater. All alerts were managed by the heart failure clinical team who assessed each patient via telephone call. Diuretic changes were made at the discretion of the clinical team. HFE, defined as unscheduled visits or heart failure hospitalizations occurring within 6 weeks after the initial alert were recorded. We identified a total of 80 patients with activated HeartLogic devices. A total of 38 patients (48%) had at least one HeartLogic alert during the study period. Of these, 26 patients (68%) appeared to have false positive alerts and did not require diuretic change nor had HFE, while 12 (32%) had heart failure symptoms requiring diuretic adjustment or urgent follow-up. Three of these 12 patients failed outpatient diuretic augmentation and were admitted for decompensated heart failure. Among the 42 patients (52%) who did not have an alert, only one patient had a HFE. Overall, HeartLogic demonstrated a sensitivity of 92%, specificity of 61%, positive predictive value of 32% and negative predictive value of 98%. Increasing HeartLogic alert threshold to >20 in this cohort would improve specificity to 90% and positive predictive value to 56%, thereby reducing staff burden, with an acceptable sensitivity (69%) and negative predictive value (94%). This is the first study to evaluate the effects of an individualized HeartLogic threshold in an alert-based follow-up management strategy. This strategy identified actionable alerts leading to change in diuretic management. We do not recommend a standardized diuretic change algorithm for alerts given the heterogeneity of the index value and high false positive rate of alerts. Future studies are needed to determine if this management strategy leads to decrease in heart failure admissions.

Improving blood glucose level predictability using machine learning.

This study was designed to improve blood glucose level predictability and future hypoglycemic and hyperglycemic event alerts through a novel patient-specific supervised-machine-learning (SML) analysis of glucose level based on a continuous-glucose-monitoring system (CGM) that needs no human intervention, and minimises false-positive alerts. The CGM data over 7 to 50 non-consecutive days from 11 type-1 diabetic patients aged 18 to 39 with a mean HbA1C of 7.5% ± 1.2% were analysed using four SML models. The algorithm was constructed to choose the best-fit model for each patient. Several statistical parameters were calculated to aggregate the magnitudes of the prediction errors. The personalised solutions provided by the algorithm were effective in predicting glucose levels 30 minutes after the last measurement. The average root-mean-square-error was 20.48 mg/dL and the average absolute-mean-error was 15.36 mg/dL when the best-fit model was selected for each patient. Using the best-fit-model, the true-positive-hypoglycemia-prediction-rate was 64%, whereas the false-positive- rate was 4.0%, and the false-negative-rate was 0.015%. Similar results were found even when only CGM samples below 70 were considered. The true-positive-hyperglycemia-prediction-rate was 61%. State-of-the-art SML tools are effective in predicting the glucose level values of patients with type-1diabetes and notifying these patients of future hypoglycemic and hyperglycemic events, thus improving glycemic control. The algorithm can be used to improve the calculation of the basal insulin rate and bolus insulin, and suitable for a closed loop "artificial pancreas" system. The algorithm provides a personalised medical solution that can successfully identify the best-fit method for each patient.

Preferential Hyperacuity Home Testing for Monitoring Intermediate Age-related Macular Degeneration in Routine Clinical Practice

Objectives: A home monitoring device (ForeseeHome; Notal Vision, Tel Aviv, Israel) detected choroidal neovascularization in age-related macular degeneration (AMD) earlier than home and office monitoring. This study describes device usage in routine clinical practice. Materials and Methods: An Institutional Review Board-approved retrospective chart review of intermediate AMD patients who used the device at a single tertiary care center. Primary outcomes were duration and frequency of usage and outcomes of alerts generated. Results: Sixty-two patients (106 eyes) completed monitoring 4.2 ± 2.1 times per week (avg ± std dev) from February 2015 to February 2019. Forty-five eyes discontinued monitoring after an average of 65 days (range 13–237 days), due to poor quality tests (62%), and false-positive alerts (16%). Eyes that discontinued device monitoring were older and had poorer visual acuity than eyes that continued monitoring (P &lt; 0.01). Fourteen alerts were generated, of which one represented conversion to exudative AMD. Two additional converted eyes were identified by routine office visit and Amsler grid monitoring. Conclusions: Home monitoring has the potential to improve the detection of exudative AMD. Identifying barriers to device utilization are necessary in the AMD population.

MP41: Crowdsourcing to save lives: A scoping review of bystander alert technologies for out-of-hospital cardiac arrest

Introduction: Out-of-hospital cardiac arrest (OHCA) constitutes a significant global health burden, with a survival rate of only 10-12%. Early intervention is vital but limited by ambulance response times, low rates of bystander assistance, and access to AEDs. Smartphone technologies have been developed that crowdsource willing volunteers to nearby OHCAs in order to initiate resuscitation prior to ambulance arrival. We performed a scoping review to map the available literature on these crowdsourcing technologies and compared their key operational features. Methods: A search strategy was developed for five online databases: Medline, Cochrane, Embase, and Web of Science, as well as Google Scholar. We searched for primary studies and grey literature describing mobile phone technologies that alerted users of nearby cardiac arrests in the community. Two reviewers independently screened all articles and extracted relevant study information. Subsequently, we performed a search of the Google and Apple app stores, a general internet search, and consulted experts to identify all available technologies that might not be described in literature. We contacted developers for information on technology use and specifications to create a detailed features table. Results: We included 72 articles examining bystander alerting technologies from 15 countries worldwide, owing to the increasing importance of this topic. We identified 25 unique technologies, of which 18 were described in the included literature. Technologies were either text message-based systems (n = 4) or mobile phone applications (n = 21). Most (23/25) used global positioning systems to direct bystanders to victims and nearby AEDs. Response radii for alerts varied widely from 200m to 10km. Some technologies had advanced features such as video-conferencing with ambulance dispatch and detailed alert settings. Not all systems required volunteers to have first aid training. There were 18 studies examining effects on bystander intervention, all of which showed significant improvements using the technologies. However, only six studies assessed impact on survival outcomes. Key barriers discussed included false positive alerts, legal liability, and potential psychological impact on volunteers. Conclusion: Our review provides a comprehensive overview of crowdsourcing technologies for bystander intervention in out-of-hospital cardiac arrest. Future work in this growing field should focus on survival outcomes and methods of addressing barriers to implementation.

Improving SIEM alert metadata aggregation with a novel kill-chain based classification model

Today’s information networks face increasingly sophisticated and persistent threats, where new threat tools and vulnerability exploits often outpace advancements in intrusion detection systems. Current detection systems often create too many alerts, which contain insufficient data for analysts. As a result, the vast majority of alerts are ignored, contributing to security breaches that might otherwise have been prevented. Security Information and Event Management (SIEM) software is a recent development designed to improve alert volume and content by correlating data from multiple sensors. However, insufficient SIEM configuration has thus far limited the promise of SIEM software for improving intrusion detection. The focus of our research is the implementation of a hybrid kill-chain framework as a novel configuration of SIEM software. Our research resulted in a new log ontology capable of normalizing security sensor data in accordance with modern threat research. New SIEM correlation rules were developed using the new log ontology, and the effectiveness of the new configuration was tested against a baseline configuration. The novel configuration was shown to improve detection rates, give more descriptive alerts, and lower the number of false positive alerts.