Frequency Data Streams Research Articles

A minimal set of physiomarkers in continuous high frequency data streams predict adult sepsis onset earlier

PurposeSepsis is a life-threatening condition with high mortality rates and expensive treatment costs. To improve short- and long-term outcomes, it is critical to detect at-risk sepsis patients at an early stage. MethodsA data-set consisting of high-frequency physiological data from 1161 critically ill patients was analyzed. 377 patients had developed sepsis, and had data at least 3 h prior to the onset of sepsis. A random forest classifier was trained to discriminate between sepsis and non-sepsis patients in real-time using a total of 132 features extracted from a moving time-window. The model was trained on 80% of the patients and was tested on the remaining 20% of the patients, for two observational periods of lengths 3 and 6 h prior to onset. ResultsThe model that used continuous physiological data alone resulted in sensitivity and F1 score of up to 80% and 67% one hour before sepsis onset. On average, these models were able to predict sepsis 294.19 ± 6.50 min (5 h) before the onset. ConclusionsThe use of machine learning algorithms on continuous streams of physiological data can allow for early identification of at-risk patients in real-time with high accuracy.

Streaming MASSIF: Cascading Reasoning for Efficient Processing of IoT Data Streams.

In the Internet of Things (IoT), multiple sensors and devices are generating heterogeneous streams of data. To perform meaningful analysis over multiple of these streams, stream processing needs to support expressive reasoning capabilities to infer implicit facts and temporal reasoning to capture temporal dependencies. However, current approaches cannot perform the required reasoning expressivity while detecting time dependencies over high frequency data streams. There is still a mismatch between the complexity of processing and the rate data is produced in volatile domains. Therefore, we introduce Streaming MASSIF, a Cascading Reasoning approach performing expressive reasoning and complex event processing over high velocity streams. Cascading Reasoning is a vision that solves the problem of expressive reasoning over high frequency streams by introducing a hierarchical approach consisting of multiple layers. Each layer minimizes the processed data and increases the complexity of the data processing. Cascading Reasoning is a vision that has not been fully realized. Streaming MASSIF is a layered approach allowing IoT service to subscribe to high-level and temporal dependent concepts in volatile data streams. We show that Streaming MASSIF is able to handle high velocity streams up to hundreds of events per second, in combination with expressive reasoning and complex event processing. Streaming MASSIF realizes the Cascading Reasoning vision and is able to combine high expressive reasoning with high throughput of processing. Furthermore, we formalize semantically how the different layers in our Cascading Reasoning Approach collaborate.

Text Classification Using the N-Gram Graph Representation Model Over High Frequency Data Streams

A prominent challenge in our information age is the classification over high frequency data streams. In this research, we propose an innovative and high-accurate text stream classification model that is designed in an elastic distributed way and is capable to service text load with fluctuated frequency. In this classification model, text is represented as N-Gram Graphs and the classification process takes place using text preprocessing, graph similarity and feature classification techniques following the supervised machine learning approach. The work involves the analysis of many variations of the proposed model and its parameters, such as various representations of text as N-Gram Graphs, graph comparisons metrics and classification methods in order to conclude to the most accurate setup. To deal with the scalability, the availability and the timely response in case of high frequency text we employ the Beam programming model. Using the Beam programming model the classification process occurs as a sequence of distinct tasks and facilitates the distributed implementation of the most computational demanding tasks of the inference stage. The proposed model and the various parameters that constitute it are evaluated experimentally and the high frequency stream emulated using two public datasets (20NewsGroup and Reuters-21578) that are commonly used in the literature for text classification

Early and Accurate Recognition of Highway Traffic Maneuvers Considering Real World Application: A Novel Framework Using Bayesian Networks

This paper presents a novel application of artificial cognitive systems to traffic scene understanding and early recognition of highway maneuvers. This is achieved by use of Bayesian networks for knowledge representation, to mimic the human reasoning on situation analysis and to manage inherited uncertainties in the automotive domain, that requires efficient and effective analysis of high volume and frequency data streams. The maneuver recognition uses features, analyzing the observed vehicles behavior and available free space on the target lane.

A Data Imputation Model in Phasor Measurement Units Based on Bagged Averaging of Multiple Linear Regression

Synchrophasor data from the phasor measurement unit (PMU) indicate the health of the electrical system. However, the PMU performance relies entirely on a supported communication network. The existing communication approaches do not guarantee the error-free channel for the PMU network. Meanwhile, the mean or median approaches are commonly used to impute the missing value. These methods, however, fail to recover some valuable frequency events. In this paper, we first proved the multiple linear regression features in many synchronized frequency data streams in the short-time window. Then, we proposed the Bagged Averaging of Multiple Linear Regression model, which handles and fulfills the missing values in synchronized frequency data measurement fast and efficiently. This technique was based on the ensemble learning by bootstrapping and averaging many multiple linear regressions to predict the missing values. Various experiments on the synchronized frequency measurement data from the Texas synchrophasor network have demonstrated the effectiveness of our proposed approach in recovering the missing frequency events. Our proposed approach guarantees the performance of real-time wide area monitoring system applications, such as frequency analysis or stability monitoring and trending.

Efficient parallel reasoning on fuzzy goal models for run time requirements verification

As software applications become highly interconnected in dynamically provisioned platforms, they form the so-called systems-of-systems. Therefore, a key issue that arises in such environments is whether specific requirements are violated, when these applications interact in new unforeseen ways as new resources or system components are dynamically provisioned. Such environments require the continuous use of frameworks for assessing compliance against specific mission critical system requirements. Such frameworks should be able to (a) handle large requirements models, (b) assess system compliance repeatedly and frequently using events from possibly high velocity and high frequency data streams, and (c) use models that can reflect the vagueness that inherently exists in big data event collection and in modeling dependencies between components of complex and dynamically re-configured systems. In this paper, we introduce a framework for run time reasoning over medium and large-scale fuzzy goal models, and we propose a process which allows for the parallel evaluation of such models. The approach has been evaluated for time and space performance on large goal models, exhibiting that in a simulation environment, the parallel reasoning process offers significant performance improvement over a sequential one.

Information management at the North Temperate Lakes Long-term Ecological Research site — Successful support of research in a large, diverse, and long running project

Information management has been an integral part of the research process at the North Temperate Lakes Long-term Ecological Research (NTL LTER) program for over 30years. A combination of factors has made the information management system (IMS) at NTL very successful. Significant resources have been invested in the IMS from the beginning, the Information Manager has been part of the leadership team at NTL and later in various roles at the LTER network level; the NTL IMS was a very early adopter of database systems, standardized metadata, and a data delivery system based on those metadata. This approach has made data easily accessible to NTL researchers and the broader scientific community. Data management workflows have become increasingly more automated with adoption of modern technologies as they became available, making the system efficient enough to handle core data as well as all one-time research data generated within NTL and several related projects. More than three decades of core data from eleven lakes are reused extensively as critical background information and as the limnological go-to site for many synthesis projects within and beyond LTER.The NTL IMS continues to implement new technologies for improving data management efficiency, discovery, access, integration, and synthesis. Accordingly, the functionality of the original online data access system programmed in Java and JavaServer Pages (JSP) was ported to the modern content management system, Drupal and integrated into LTER's Drupal Ecological Information Management System (DEIMS). NTL has invested in sensor technology for studying lake conditions over the long term, which necessitated a sophisticated management system tailored to high frequency data streams. Several technologies have been used at different times for automation of management, quality control and archiving of these high volume data. Near real time lake conditions can be accessed on the NTL website and smart phone Apps.Easy access to long-term and sensor data in the NTL IMS has led NTL researchers to develop new analytical methods and the publication of several R statistical packages. Recent graduate students are now employed as data scientists helping define a new career path inspired by the availability of data.The NTL project has amassed one of the world's most comprehensive long-term datasets on lakes and their surrounding landscapes. The NTL IMS facilitates the use of these data by multiple groups for research, education, and communication of science to the public.

A gathered EnKF for continuous reservoir model updating

Abstract The increasing use of in situ permanent downhole sensors gives new incentives for continuous model updating. The Ensemble Kalman filter is a widely used recursive data assimilation method that is well suited to continuous updating. However, effective integration of high frequency data streams received from permanent sensors remains a challenge. On one hand, too-frequent model updating imposes a computational burden and may degrade the ensemble fidelity because of sampling errors introduced to the Kalman gain. On the other hand, the full value of high-frequency data can only be realized if we use these data to update the reservoir model. We propose a gathered EnKF method in which some data are stored instead of being assimilated immediately. Model updating is done only if the gathered data volume exceeds a specified threshold, or if the observations diverge significantly from prior ensemble predictions. A theoretical prediction of the effects of gather size is given and tested with a scalar case. It is shown that, for a linear dynamic model, sampling errors are reduced by gathering the observations. The gathered EnKF is further tested with 2D synthetic reservoir models and compared with a standard EnKF, indicating that the gathered EnKF may be more stable than the traditional EnKF and may save computational effort in simulation restarts, file transfers, and input–output at EnKF assimilation steps — even for nonlinear reservoir dynamics models.

Expert System for the LHC CMS Cathode Strip Chambers (CSC) detector

Modern High Energy Physics experiments are of high demand for a generic and consolidated solution to integrate and process high frequency data streams by applying experts' knowledge and inventory configurations. In this paper we present the Expert System application that was built for the Compact Muon Solenoid (CMS) Cathode Strip Chambers (CSC) detector at the Large Hadron Collider (LHC) aiming to support the detector operations and to provide integrated monitoring. The main building blocks are the integration platform, rule-based complex event processing engine, ontology-based knowledge base, persistent storage and user interfaces for results and control.