Analysis Of Data Streams Research Articles

Machine learning models predicting multidrug resistant urinary tract infections using \u201cDsaaS\u201d

BackgroundThe scope of this work is to build a Machine Learning model able to predict patients risk to contract a multidrug resistant urinary tract infection (MDR UTI) after hospitalization. To achieve this goal, we used different popular Machine Learning tools. Moreover, we integrated an easy-to-use cloud platform, called DSaaS (Data Science as a Service), well suited for hospital structures, where healthcare operators might not have specific competences in using programming languages but still, they do need to analyze data as a continuous process. Moreover, DSaaS allows the validation of data analysis models based on supervised Machine Learning regression and classification algorithms.ResultsWe used DSaaS on a real antibiotic stewardship dataset to make predictions about antibiotic resistance in the Clinical Pathology Operative Unit of the Principe di Piemonte Hospital in Senigallia, Marche, Italy. Data related to a total of 1486 hospitalized patients with nosocomial urinary tract infection (UTI). Sex, age, age class, ward and time period, were used to predict the onset of a MDR UTI. Machine Learning methods such as Catboost, Support Vector Machine and Neural Networks were utilized to build predictive models. Among the performance evaluators, already implemented in DSaaS, we used accuracy (ACC), area under receiver operating characteristic curve (AUC-ROC), area under Precision-Recall curve (AUC-PRC), F1 score, sensitivity (SEN), specificity and Matthews correlation coefficient (MCC). Catboost exhibited the best predictive results (MCC 0.909; SEN 0.904; F1 score 0.809; AUC-PRC 0.853, AUC-ROC 0.739; ACC 0.717) with the highest value in every metric.Conclusionsthe predictive model built with DSaaS may serve as a useful support tool for physicians treating hospitalized patients with a high risk to acquire MDR UTIs. We obtained these results using only five easy and fast predictors accessible for each patient hospitalization. In future, DSaaS will be enriched with more features like unsupervised Machine Learning techniques, streaming data analysis, distributed calculation and big data storage and management to allow researchers to perform a complete data analysis pipeline. The DSaaS prototype is available as a demo at the following address: https://dsaas-demo.shinyapps.io/Server/

Temporal analysis of multimodal data to predict collaborative learning outcomes

AbstractThe analysis of multiple data streams is a long‐standing practice within educational research. Both multimodal data analysis and temporal analysis have been applied successfully, but in the area of collaborative learning, very few studies have investigated specific advantages of multiple modalities versus a single modality, especially combined with temporal analysis. In this paper, we investigate how both the use of multimodal data and moving from averages and counts to temporal aspects in a collaborative setting provides a better prediction of learning gains. To address these questions, we analyze multimodal data collected from 25 9–11‐year‐old dyads using a fractions intelligent tutoring system. Assessing the relation of dual gaze, tutor log, audio and dialog data to students' learning gains, we find that a combination of modalities, especially those at a smaller time scale, such as gaze and audio, provides a more accurate prediction of learning gains than models with a single modality. Our work contributes to the understanding of how analyzing multimodal data in temporal manner provides additional information around the collaborative learning process.

An analysis of technological frameworks for data streams

Real-time data analysis is becoming increasingly important in Big Data environments for addressing data stream issues. To this end, several technological frameworks have been developed, both open-source and proprietary, for the analysis of streaming data. This paper analyzes some open-source technological frameworks available for data streams, detailing their main characteristics. The objective is to facilitate decisions on which framework to use, meeting the needs of data mining methods for data streams. In this sense, there are important factors affecting the choice about which framework is most suitable for this purpose. Some of these factors are the existence of data mining libraries, the available documentation, the maturity of the platform, fault tolerance and processing guarantees, among others. Another decisive factor when choosing a data stream framework is its performance. For this reason, two comparisons have been made: a performance and latency comparison between Spark Streaming, Spark Structured Streaming, Storm, Flink and Samza following the Yahoo Streaming Benchmark methodology, and a comparison between Spark Streaming and Flink with a clustering algorithm for data streaming called streaming K-means.

RES: Real-Time Video Stream Analytics Using Edge Enhanced Clouds

With increasing availability and use of Internet of Things (IoT) devices such as sensors and video cameras, large amounts of streaming data is now being produced at high velocity. Applications which require low latency response such as video surveillance, augmented reality and autonomous vehicles demand a swift and efficient analysis of this data. Existing approaches employ cloud infrastructure to store and perform machine learning-based analytics on this data. This centralized approach has limited ability to support real-time analysis of large-scale streaming data due to network bandwidth and latency constraints between data source and cloud. We propose RealEdgeStream (RES) an edge enhanced stream analytics system for large-scale, high performance data analytics. The proposed approach investigates the problem of video stream analytics by proposing (i) filtration and (ii) identification phases. The filtration phase reduces the amount of data by filtering low-value stream objects using configurable rules. The identification phase uses deep learning inference to perform analytics on the streams of interest. The phases consist of stages which are mapped onto available in-transit and cloud resources using a placement algorithm to satisfy the Quality of Service (QoS) constraints identified by a user. We demonstrate that for a 10K element data streams, with a frame rate of 15–100 per second, the job completion in the proposed system takes 49 percent less time and saves 99 percent bandwidth compared to a centralized cloud-only based approach.

2019 John M. Eisenberg Patient Safety and Quality Awards: SPOTting Sepsis to Save Lives: A Nationwide Computer Algorithm for Early Detection of Sepsis: Innovation in Patient Safety and Quality at the National Level (Eisenberg Award)

In recognition of the potential of data to drive care and the need for early identification of patients with sepsis, HCA Healthcare developed an automated sepsis detection algorithm-SPOT (Sepsis Prediction and Optimization of Therapy). The algorithm was deployed at scale and served as a mechanism to reduce the time to detection and improve sepsis mortality in 173 hospitals across the United States. The SPOT algorithm was designed as a rules-based detection of defined criteria that would interpret available electronic and basic laboratory data in near real time. Working from an organizational recognition of the need to construct a national clinical data warehouse to allow for the aggregation and analysis of data streams, HCA Healthcare designed and deployed SPOT and delivered the alert from the algorithm to the bedside to initiate existing clinical workflows for patients with sepsis. SPOT improved the timeliness of sepsis detection by providing alerts when signals of sepsis become available, triggering initiation of sepsis screens. This gave an advantage of about six hours over the legacy practice of sepsis screening at shift change. When deployed alongside existing sepsis improvement initiatives, SPOT was associated with an acceleration of improvement in mortality-particularly in the not-present-on-admission (NPOA) septic shock population, the patients at greatest risk for mortality. This population had seen little improvement with prior initiatives, but mortality improved 3.9 percentage points from 2018 to 2019. When accounting for seasonal variation, there was a decline in mortality rate following the deployment of SPOT, as compared to the year prior, of 9.9% for NPOA severe sepsis and 5.1% for NPOA septic shock. Development of the SPOT algorithm for the detection of sepsis from data available in the electronic health record resulted in more timely recognition, faster initiation of treatment, and improved survival for patients.

Whence to Learn? Transferring Knowledge in Configurable Systems Using BEETLE

As software systems grow in complexity and the space of possible configurations increases exponentially, finding the near-optimal configuration of a software system becomes challenging. Recent approaches address this challenge by learning performance models based on a sample set of configurations. However, collecting enough sample configurations can be very expensive since each such sample requires configuring, compiling, and executing the entire system using a complex test suite. When learning on new data is too expensive, it is possible to use <i>Transfer Learning</i> to “transfer” old lessons to the new context. Traditional transfer learning has a number of challenges, specifically, (a) learning from excessive data takes excessive time, and (b) the performance of the models built via transfer can deteriorate as a result of learning from a poor source. To resolve these problems, we propose a novel transfer learning framework called BEETLE, which is a “bellwether”-based transfer learner that focuses on identifying and learning from the most relevant source from amongst the old data. This paper evaluates BEETLE with 57 different software configuration problems based on five software systems (a video encoder, an SAT solver, a SQL database, a high-performance C-compiler, and a streaming data analytics tool). In each of these cases, BEETLE found configurations that are as good as or better than those found by other state-of-the-art transfer learners while requiring only a fraction ( <inline-formula><tex-math notation="LaTeX">$\frac{1}{7}$</tex-math></inline-formula> th) of the measurements needed by those other methods. Based on these results, we say that BEETLE is a new high-water mark in optimally configuring software.

On the design of a framework integrating an optimization engine with streaming technologies

A number of streaming technologies have appeared in the last years as a result of the rising of Big Data applications. Nowadays, deciding which technology to adopt is not an easy task due not only to the number of available data streaming processing projects, but also because they are continuously evolving. In this paper, we focus on how these issues have affected jMetalSP, a framework for dynamic multi-objective optimization that incorporates streaming features. jMetalSP allows the development of three tier optimization workflows where the central component is an optimizer that is continuously solving a dynamic multi-objective optimization problem. This problem can change as a consequence of the analysis of data streams carried out by components that use the Apache Spark streaming engine. A third kind of components receive and process the Pareto front approximations being yielded by the optimization algorithm. However, all jMetalSP elements are tightly coupled and linked to Spark, making it difficult to use a different streaming system. To overcome this issue, we have redesigned the jMetalSP architecture to make it flexible enough to avoid the dependence of any particular streaming system. This way, popular Apache projects such as Spark Structured Streaming, Kafka Streams, or Flink can be used without requiring to change the rest of components of the application. Furthermore, Kafka can be used for inter-process communication, what enables the execution of components in different nodes of a cluster, independently of their implementation languages thanks to the serialization of data streams with Apache Avro. We show how the embraced solution provides a high degree of flexibility that enhances the usability of jMetalSP. To this end, a representative case study based on a transport problem is conducted that focuses on data representation and performance evaluation of the Spark, Flink, and Kafka systems.

A universal middleware streaming framework and data analytics: Analysing their economic feasibility in road transport planning

The individual paths followed by road transport operators in adopting and using information and communication technologies (ICT) result in heterogeneous planning practices. This includes working with multiple IT systems, data sources and applications. There is no coordination or standardisation that ensures a straightforward approach to develop and implement cross-sector IT solutions. Different integrative state-of-the-art solutions are sub-optimal, not flexible and relatively expensive to maintain.The proposed methodology starts by investigating the current working practices of a road transport operator. It reports on an own exploration initiative conducted by researchers in logistics and computer science, and brings together their complementary expertise. It identifies firstly the ICT solutions used at an operational level and points out the necessities for further integration, functionalities or application development. Secondly, it develops a theoretical model that assists road transport operators in checking whether using a uniform integration layer and data analytics tools is economically feasible. The model shows the fleet size from which significant costs reductions are made. These costs reductions are enabled by implementing a middleware framework and data analytics algorithm that assists human operators in operational planning.

Designing translucent learning analytics with teachers: an elicitation process

ABSTRACTLearning Analytics (LA) systems can offer new insights into learners’ behaviours through analysis of multiple data streams. There remains however a dearth of research about how LA interfaces can enable effective communication of educationally meaningful insights to teachers and learners. This highlights the need for a participatory, horizontal co-design process for LA systems. Inspired by the notion of translucence, this paper presents LAT-EP (Learning Analytics Translucence Elicitation Process), a five-step process to design for the effective use of translucent LA systems. LAT-EP was operationalised in an authentic multimodal learning analytics (MMLA) study in the context of teamwork in clinical simulation. Results of this process are illustrated through a series of visual proxies co-designed with teachers, each presenting traces of social, physical, affective and epistemic evidence captured while teams of student nurses practised clinical skills in a simulated hospital setting.

A multimodal approach for human activity recognition based on skeleton and RGB data

Human action recognition plays a fundamental role in the design of smart solution for home environments, particularly in relation to ambient assisted living applications, where the support of an automated system could improve the quality of life of humans trying to interpret and anticipate user needs, recognizing unusual behaviors or preventing dangerous situations (e.g. falls). In this work the potentialities of the Kinect sensor are fully exploited to design a robust approach for activity recognition combining the analysis of skeleton and RGB data streams. The skeleton representation is designed to capture the most representative body postures, while the temporal evolution of actions is better highlighted by the representation obtained from RGB images. The experimental results confirm that the combination of these two data sources allow to capture highly discriminative features resulting in an approach able to achieve state-of-the-art performance on public benchmarks.

Novelty Detection and Online Learning for Chunk Data Streams.

Datastream analysis aims at extracting discriminative information for classification from continuously incoming samples. It is extremely challenging to detect novel data while incrementally updating the model efficiently and stably, especially for high-dimensional and/or large-scale data streams. This paper proposes an efficient framework for novelty detection and incremental learning for unlabeled chunk data streams. First, an accurate factorization-free kernel discriminative analysis (FKDA-X) is put forward through solving a linear system in the kernel space. FKDA-X produces a Reproducing Kernel Hilbert Space (RKHS), in which unlabeled chunk data can be detected and classified by multiple known-classes in a single decision model with a deterministic classification boundary. Moreover, based on FKDA-X, two optimal methods FKDA-CX and FKDA-C are proposed. FKDA-CX uses the micro-cluster centers of original data as the input to achieve excellent performance in novelty detection. FKDA-C and incremental FKDA-C (IFKDA-C) using the class centers of original data as their input have extremely fast speed in online learning. Theoretical analysis and experimental validation on under-sampled and large-scale real-world datasets demonstrate that the proposed algorithms make it possible to learn unlabeled chunk data streams with significantly lower computational costs and comparable accuracies than the state-of-the-art approaches.

The hybrid ant colony optimization and ensemble method for solving the data stream e-mail foldering problem

The e-mail foldering problem is a special classification problem. It concerns a situation where e-mail users create new folders and, at the same time, stop using some of the folders created in the past. Additionally, messages arrive in the system at different time stamps. This article proposes a novel approach to ant colony optimization adapted to data stream analysis. The article is related to the revision of the ant colony optimization algorithm in the e-mail foldering problem and the proposition of a new solution adapted to the data stream. The goal of this work is to allow the classification of messages arriving at the system as data packages; however, due to the large number of decision classes (folders in the inbox), successive packages lead to a large concept drift. To assure the stability of the algorithm, an approach based on the memory being represented as a pheromone trail is introduced. This concept is known from the ant colony optimization methods. At the same time, multiple numbers of classifiers (similar to an ensemble method) are included. The proposed approach was tested on real-world data from the Enron e-mail dataset. An analysis of the two proposed methods related to the data stream was proposed. Both methods were compared with the methods used in the literature. The results achieved, in terms of the accuracy as well as the stability, confirm that (according to a statistical analysis) the proposed solutions are capable of better classifying e-mail messages derived from the system as data packages.

Change detection in streaming data analytics: A comparison of Bayesian online and martingale approaches

On line change detection is a key activity in streaming analytics, which aims to determine whether the current observation in a time series marks a change point in some important characteristic of the data, given the sequence of data observed so far. It can be a challenging task when monitoring complex systems, which are generating streaming data of significant volume and velocity. While applicable to diverse problem domains, it is highly relevant to monitoring high value and critical engineering assets. This paper presents an empirical evaluation of two algorithmic approaches for streaming data change detection. These are a modified martingale and a Bayesian online detection algorithm. Results obtained with both synthetic and real world data sets are presented and relevant advantages and limitations are discussed.

FENCE: Fast, ExteNsible, and ConsolidatEd Framework for Intelligent Big Data Processing

The proliferation of smart devices and the advancement of data-intensive services has led to explosion of data, which uncovers massive opportunities as well as challenges related to real-time analysis of big data streams. The edge computing frameworks implemented over manycore systems can be considered as a promising solution to address these challenges. However, in spite of the availability of modern computing systems with a large number of processing cores and high memory capacity, the performance and scalability of manycore systems can be limited by the software and operating system (OS) level bottlenecks. In this work, we focus on these challenges, and discuss how accelerated communication, efficient caching, and high performance computation can be provisioned over manycore systems. The proposed Fast, ExteNsible, and ConsolidatEd (FENCE) framework leverages the availability of a large number of computing cores and overcomes the OS level bottlenecks to provide high performance and scalability for intelligent big data processing. We implemented a prototype of FENCE and the experiment results demonstrate that FENCE provides improved data reception throughput, read/write throughput, and application processing performance as compared to the baseline Linux system.

A Framework for Ensemble Classification and Sensitivity Analysis in Privacy Preserving Data Mining

The perturbation mechanism for data streams is a challenging task. In the emerging world, data is erupting from various sources. The core applications need care on the data streams for further analysis and experimentation. As the micro data available with the core applications shall not be revealed to the public without taking any chance of breach, the perturbation challenges the analysis to get through the like results as of on the original datasets. In this paper, we have applied a concept of Perlin noise to distract the original data from the eyes of the analysts, however allowing them to perform their activities well. Applying security dynamically on such data is a challenging task. This paper deals about the concepts of generation of smooth noise and syntactic perturbation mechanism on the selective tuples as selective perturbation.

МЕТОД СИНТЕЗА ИНТЕЛЛЕКТУАЛЬНЫХ СИСТЕМ КОНТРОЛЯ ТЕХНИЧЕСКОГО СОСТОЯНИЯ ТЕРРИТОРИАЛЬНО-РАСПРЕДЕЛЕННЫХ КОМПЛЕКСОВ СВЯЗИ СПЕЦИАЛЬНОГО НАЗНАЧЕНИЯ

The work is devoted to the development of an intelligent control system for geographically distributed communications systems of special purpose. The relevance of the study is caused by the fact that the evolutionary development of controlled telecommunications systems, on the example of special-purpose communication complexes characterized by a changing nature, application conditions in the course of their operation, large coverage and territorial distribution, structural, topological, as well as space-time heterogeneity should be taken into account and correlated with systems for monitoring their technical condition, taking into account the development of basic artificial intelligence technologies, which is conceptually important in the short term. Method of synthesis of intelligent control system characterized by multilevel architecture of construction and functioning, depending on the degree of emergency situation, in mode of express and complete control is presented, which involves use of databases (knowledge) and adaptation of control processes to the effects of destabilizing factors. The work shows the approach to processes of proactive inspection of parameters of objects, consists in adaptive adjustment of preventive tolerances for checked parameters and use of adaptive mechanisms for polling intelligent sensors based on polling models; taking into account the correlation of actuation of a group of sensors within the control object in emergency situations and contributing to an increase in the speed of recording and transmitting the alarm signal, and reducing redundancy of measurement information in the measurement path of the monitoring system. As a result of the synthesis of a multilevel intelligent control system, the system architecture is proposed, which allows determining the most rational ratio of measuring channels (intelligent sensors, analog-to-digital converters, sampling and storage devices), taking into account the noise immunity and throughput of measuring paths, as well as the required number of communication channels. The multi-level monitoring system architecture allows analysis of measuring information (alarms) at the sensor level of the system and online analysis of streaming data (telemetry messages) to quickly detect cases of possible failures, predict the residual resource of equipment of special-purpose communication systems.

Smart Surveillance System Using Computer Vision and Internet of Things

With the evolution of technology ensuring people for their safety and security all around the time constantly is a big challenge. We propose an advanced technique based on deep learning and artificial intelligence platform that can monitor the people, their homes and their surroundings providing them a quantifiable increase in security. We have surveillance cameras in our homes for video capture as well as security purposes. Our proposed technique is to detect and classify as well as inform the user if there is any breach in security of the classified object using the cameras by implementing deep learning techniques and the technology of internet of things. It can serve as a perimeter monitoring and intruder alert system in smart surveillance environment. This paper provides a well-defined structure for live stream data analysis. It overcomes the challenge of static closed circuit cameras television as it serves as a motion based tracking system and monitors events in real time to ensure activities are limited to specific persons within authorized areas. It has the advantage of creating multiple bounding boxes to track down the objects which could be any living or non-living thing based on the trained modules. The trespasser or intruder can be efficiently detected using the CCTV camera surveillance which is being supported by the real-time object classifier algorithm at the intermediate module. The proposed method is mainly supported by the real time object detection and classification which is implemented using Mobile Net and Single shot detector.

Stochastic AUC optimization with general loss

Recently, there is considerable work on developing efficient stochastic optimization algorithms for AUC maximization. However, most of them focus on the least square loss which may be not the best option in practice. The main difficulty for dealing with the general convex loss is the pairwise nonlinearity w.r.t. the sampling distribution generating the data. In this paper, we use Bernstein polynomials to uniformly approximate the general losses which are able to decouple the pairwise nonlinearity. In particular, we show that this reduction for AUC maximization with a general loss is equivalent to a weakly convex (nonconvex) min-max formulation. Then, we develop a novel SGD algorithm for AUC maximization with per-iteration cost linearly w.r.t. the data dimension, making it amenable for streaming data analysis. Despite its non-convexity, we prove its global convergence by exploring the appealing convexity-preserving property of Bernstein polynomials and the intrinsic structure of the min-max formulation. Experiments are performed to validate the effectiveness of the proposed approach.

Scalable Detection of Concept Drift: A Learning Technique Based on Support Vector Machines

The issue of concept drift describes how static machine-learning models build on historical data can become unreliable over time and pose a significant challenge to many applications. Although, there is a growing body of literature investigating concept drift existing solutions are often limited to a small number of samples or features and do not work well in Industry 4.0 scenarios. We are proposing a novel algorithm that extends the existing concept drift algorithm FLORA3 by utilizing support vector machines for the classification process. Through this combination of dynamic and static approaches the algorithm is capable of effectively analyzing data streams of high volume. For evaluation, we tested our algorithm on the publicly available data set ‘elec2’, which is based on the energy market in Australia. Our results show that the proposed algorithm needs less computational resources compared to other algorithms while maintaining a high level of accuracy.

A TTL-based Approach for Data Aggregation in Geo-distributed Streaming Analytics

Streaming data analytics has been an important topic of research in recent years. Large quantities of data are generated continuously over time across a variety of application domains such as web and social analytics, scientific computing and energy analytics. One of the key requirements in modern data analytics services is the real-time analysis of these data streams to extract useful and timely information for the analyst. Several distributed data analytics platforms have been developed in recent times to meet this growing requirement of real-time streaming analytics. Nowadays, a large amount of data is generated continuously by geographically distributed sources (e.g., agents, sensors, mobile devices, edge nodes, etc.) in many streaming applications. For instance, services like Facebook, Twitter and Netflix continuously gather data from the end users for a variety of analytical purposes such as finding the popular web content amongst their users or monitoring the QoS metrics. Large content delivery networks (CDNs) like Akamai that serve a significant fraction of content on the Internet continuously collect data from their edge servers and clients from around the globe to understand what, where and how content is accessed for the purpose of providing content analytics insights to businesses.