Events In Data Streams Research Articles

ANOMALY DETECTION USING MODIFIED LINEARITY BASED GREY SWARM OPTIMIZATION ALGORITHM AND EFFICIENT HYPER PARAMETER OPTIMIZATION TECHNIQUES

Attacks and anomalies are quiet common in today’s technological world. However it is important to identify these anomalies as soon as possible since these anomalies can cause various problem. Therefore anomaly detection plays a vital role in detecting these outliers. Anomaly detection detects suspicious activity which fundamentally falls outside of the respected pattern of behaviour. Anomaly detection aims at identifying the unpredicted and unexpected events in data streams which are commonly referred to as anomalous events. Therefore proposed modified linearity based grey swarm algorithm and hyperparameter optimization techniques for improving the performance of the proposed model. Hyperparameter optimization techniques implemented in the proposed model are genetic algorithm (GA), Grid Search and Random Search. Feature selection is implemented in the proposed model is modified linearity based grey swarm algorithm since it has the ability to reduce the convergence. In addition to modified linearity based grey swarm algorithm, modified objective function is employed in the proposed model as it has the capability to find solutions to problems which are difficult to solve. Finally the performance of the proposed model is evaluated and assessed using various performance metrics such as accuracy, recall, precision and F1 score. At last, proposed model is compared with the existing models and found that proposed model outperformed other existing models.

Multimodal Shot Prediction Based on Spatial-Temporal Interaction between Players in Soccer Videos

Sports data analysis has significantly advanced and become an indispensable technology for planning strategy and enhancing competitiveness. In soccer, shot prediction has been realized on the basis of historical match situations, and its results contribute to the evaluation of plays and team tactics. However, traditional event prediction methods required tracking data acquired with expensive instrumentation and event stream data annotated by experts, and the benefits were limited to only some professional athletes. To tackle this problem, we propose a novel shot prediction method using soccer videos. Our method constructs a graph considering player relationships with audio and visual features as graph nodes. Specifically, by introducing players’ importance into the graph edge based on their field positions and team information, our method enables the utilization of knowledge that reflects the detailed match situation. Next, we extract latent features considering spatial–temporal interactions from the graph and predict event occurrences with uncertainty based on the probabilistic deep learning method. In comparison with several baseline methods and ablation studies using professional soccer match data, our method was confirmed to be effective as it demonstrated the highest average precision of 0.948, surpassing other methods.

Spiking Transfer Learning from RGB Image to Neuromorphic Event Stream.

Recent advances in bio-inspired vision with event cameras and associated spiking neural networks (SNNs) have provided promising solutions for low-power consumption neuromorphic tasks. However, as the research of event cameras is still in its infancy, the amount of labeled event stream data is much less than that of the RGB database. The traditional method of converting static images into event streams by simulation to increase the sample size cannot simulate the characteristics of event cameras such as high temporal resolution. To take advantage of both the rich knowledge in labeled RGB images and the features of the event camera, we propose a transfer learning method from the RGB to the event domain in this paper. Specifically, we first introduce a transfer learning framework named R2ETL (RGB to Event Transfer Learning), including a novel encoding alignment module and a feature alignment module. Then, we introduce the temporal centered kernel alignment (TCKA) loss function to improve the efficiency of transfer learning. It aligns the distribution of temporal neuron states by adding a temporal learning constraint. Finally, we theoretically analyze the amount of data required by the deep neuromorphic model to prove the necessity of our method. Numerous experiments demonstrate that our proposed framework outperforms the state-of-the-art SNN and artificial neural network (ANN) models trained on event streams, including N-MNIST, CIFAR10-DVS and N-Caltech101. This indicates that the R2ETL framework is able to leverage the knowledge of labeled RGB images to help the training of SNN on event streams.

A bio-inspired hierarchical spiking neural network with biological synaptic plasticity for event camera object recognition

With inherent sparse spike-based coding and asynchronous event-driven computation, spiking neural network (SNN) is naturally suitable for processing event stream data of event cameras. In order to improve the feature extraction and classification performance of bio-inspired hierarchical SNNs, in this paper an event camera object recognition system based on biological synaptic plasticity is proposed. In our system input event streams were firstly segmented adaptively using spiking neuron potential to improve computational efficiency of the system. Multi-layer feature learning and classification are implemented by our bio-inspired hierarchical SNN with synaptic plasticity. After Gabor filter-based event-driven convolution layer which extracted primary visual features of event streams, we used a feature learning layer with unsupervised spiking timing dependent plasticity (STDP) rule to help the network extract frequent salient features, and a feature learning layer with reward-modulated STDP rule to help the network learn diagnostic features. The classification accuracies of the network proposed in this paper on the four benchmark event stream datasets were better than the existing bio-inspired hierarchical SNNs. Moreover, our method showed good classification ability for short event stream input data, and was robust to input event stream noise. The results show that our method can improve the feature extraction and classification performance of this kind of SNNs for event camera object recognition.

Score-Based Learning of Graphical Event Models with Background Knowledge Augmentation

Graphical event models (GEMs) are representations of temporal point process dynamics between different event types. Many real-world applications however involve limited event stream data, making it challenging to learn GEMs from data alone. In this paper, we introduce approaches that can work together in a score-based learning paradigm, to augment data with potentially different types of background knowledge. We propose novel scores for learning an important parametric class of GEMs; in particular, we propose a Bayesian score for leveraging prior information as well as a more practical simplification that involves fewer parameters, analogous to Bayesian networks. We also introduce a framework for incorporating easily assessed qualitative background knowledge from domain experts, in the form of statements such as `event X depends on event Y' or `event Y makes event X more likely'. The proposed framework has Bayesian interpretations and can be deployed by any score-based learner. Through an extensive empirical investigation, we demonstrate the practical benefits of background knowledge augmentation while learning GEMs for applications in the low-data regime.

New hybrid ensemble method for anomaly detection in data science

Anomaly detection is a significant research area in data science. Anomaly detection is used to find unusual points or uncommon events in data streams. It is gaining popularity not only in the business world but also in different of other fields, such as cyber security, fraud detection for financial systems, and healthcare. Detecting anomalies could be useful to find new knowledge in the data. This study aims to build an effective model to protect the data from these anomalies. We propose a new hyper ensemble machine learning method that combines the predictions from two methodologies the outcomes of isolation forest-k-means and random forest using a voting majority. Several available datasets, including KDD Cup-99, Credit Card, Wisconsin Prognosis Breast Cancer (WPBC), Forest Cover, and Pima, were used to evaluate the proposed method. The experimental results exhibit that our proposed model gives the highest realization in terms of receiver operating characteristic performance, accuracy, precision, and recall. Our approach is more efficient in detecting anomalies than other approaches. The highest accuracy rate achieved is 99.9%, compared to accuracy without a voting method, which achieves 97%.

EventMix: An efficient data augmentation strategy for event-based learning

High-quality and challenging event stream datasets play an important role in the design of an efficient event-driven mechanism that mimics the brain. Although event cameras can provide high dynamic range and low-energy event stream data, the scale is smaller and more difficult to obtain than traditional frame-based data, which restricts the development of neuromorphic computing. Data augmentation can improve the quantity and quality of the original data by processing more representations from the original data. This paper proposes an efficient data augmentation strategy for event stream data: EventMix. We carefully design the mixing of different event streams by Gaussian Mixture Model (GMM) to generate random 3D masks and achieve arbitrary shape mixing of event streams in the spatio-temporal dimension. By computing the relative distances of event streams, we propose a more reasonable way to assign labels to the mixed samples. The experimental results on multiple neuromorphic datasets have shown that our strategy can improve performance on neuromorphic classification tasks as well as neuromorphic human action recognition tasks both for ANNs and SNNs, and we have achieved state-of-the-art performance on DVS-CIFAR10, N-Caltech101, and DVS-Gesture datasets.

Event-Image Fusion Stereo Using Cross-Modality Feature Propagation

Event cameras asynchronously output the polarity values of pixel-level log intensity alterations. They are robust against motion blur and can be adopted in challenging light conditions. Owing to these advantages, event cameras have been employed in various vision tasks such as depth estimation, visual odometry, and object detection. In particular, event cameras are effective in stereo depth estimation to find correspondence points between two cameras under challenging illumination conditions and/or fast motion. However, because event cameras provide spatially sparse event stream data, it is difficult to obtain a dense disparity map. Although it is possible to estimate disparity from event data at the edge of a structure where intensity changes are likely to occur, estimating the disparity in a region where event occurs rarely is challenging. In this study, we propose a deep network that combines the features of an image with the features of an event to generate a dense disparity map. The proposed network uses images to obtain spatially dense features that are lacking in events. In addition, we propose a spatial multi-scale correlation between two fused feature maps for an accurate disparity map. To validate our method, we conducted experiments using synthetic and real-world datasets.

Event-based imaging velocimetry: an assessment of event-based cameras for the measurement of fluid flows

Contrary to conventional frame-based imaging, event-based vision or dynamic vision sensing asynchronously records binary signals of intensity changes for given pixels with microsecond resolution. The present work explores the possibilities of harnessing the potential of event-based vision for fluid flow measurement. The described implementations of event-based imaging velocimetry rely on imaging small particles that are illuminated by a laser light sheet which is similar to classical two-dimensional, two-component (2d–2c) PIV with the difference that a continuously operating laser light sheet is used without modulation of the laser or camera. The moving particles generate continuous time-stamped events on the detector that are later used to infer their velocity using patch-wise processing schemes. Two flow estimation algorithms are proposed; one uses a “motion compensation” that maximizes the local contrast and the other is based on a sum-of-correlations approach. The underlying motion detection schemes in combination with the complete absence of background signal allow straightforward retrieval of the events associated with individual particles thereby allowing the reconstruction of individual particle tracks. Alternatively, the event data can be processed with conventional PIV algorithms using images reconstructed from the event data stream. The concepts are demonstrated on simple flows in water and air.Graphic abstract

Exploring jump back behavior patterns and reasons in e-book system

With the increasing use of digital learning materials in higher education, the accumulated operational log data provide a unique opportunity to analyzing student learning behaviors and their effects on student learning performance to understand how students learn with e-books. Among the students’ reading behaviors interacting with e-book systems, we find that jump-back is a frequent and informative behavior type. In this paper, we aim to understand the student’s intention for a jump-back using user learning log data on the e-book materials of a course in our university. We at first formally define the “jump-back” behaviors that can be detected from the click event stream of slide reading and then systematically study the behaviors from different perspectives on the e-book event stream data. Finally, by sampling 22 learning materials, we identify six reading activity patterns that can explain jump backs. Our analysis provides an approach to enriching the understanding of e-book learning behaviors and informs design implications for e-book systems.

Event-VPR: End-to-End Weakly Supervised Deep Network Architecture for Visual Place Recognition Using Event-Based Vision Sensor

Traditional visual place recognition (VPR) methods generally use frame-based cameras, which will easily fail due to rapid illumination changes or fast motion. To overcome this, we propose an end-to-end VPR network using event cameras, which can achieve good recognition performance in challenging environments (e.g., large-scale driving scenes). The key idea of the proposed algorithm is first to characterize the event streams with the EST voxel grid representation, then extract features using a deep residual network, and, finally, aggregate features using an improved VLAD network to realize end-to-end VPR using event streams. To verify the effectiveness of the proposed algorithm, on the event-based driving datasets (MVSEC, DDD17, and Brisbane-Event-VPR) and the synthetic event datasets (Oxford RobotCar and CARLA), we analyze the performance of our proposed method on large-scale driving sequences, including cross-weather, cross-season, and illumination changing scenes, and then, we compare the proposed method with the state-of-the-art event-based VPR method (Ensemble-Event-VPR) to prove its advantages. Experimental results show that the performance of the proposed method is better than that of the event-based ensemble scheme in challenging scenarios. To the best of our knowledge, for the VPR task, this is the first end-to-end weakly supervised deep network architecture that directly processes event stream data.

Multioutput Gaussian Process Modulated Poisson Processes for Event Prediction

Prediction of events such as part replacement and failure events plays a critical role in reliability engineering. Event stream data are commonly observed in manufacturing and teleservice systems. Designing predictive models for individual units based on such event streams is challenging and an underexplored problem. In this work, we propose a nonparametric prognostic framework for individualized event prediction based on the inhomogeneous Poisson processes with a multivariate Gaussian convolution process (MGCP) prior on the intensity functions. The MGCP prior on the intensity functions of the inhomogeneous Poisson processes maps data from similar historical units to the current unit under study which facilitates sharing of information and allows for analysis of flexible event patterns. To facilitate inference, we derive a variational inference scheme for learning and estimation of parameters in the resulting MGCP modulated Poisson process model. Experimental results are shown on both synthetic data as well as real-world data for fleet-based event prediction.

Ordinal Historical Dependence in Graphical Event Models with Tree Representations

Graphical event models are representations that capture process independence between different types of events in multivariate temporal point processes. The literature consists of various parametric models and approaches to learn them from multivariate event stream data. Since these models are interpretable, they are often able to provide beneficial insights about event dynamics. In this paper, we show how to compactly model the situation where the order of occurrences of an event’s causes in some recent historical time interval impacts its occurrence rate; this sort of historical dependence is common in several real-world applications. To overcome the practical challenge of parameter explosion due to the number of potential orders that is super-exponential in the number of parents, we introduce a novel graphical event model based on a parametric tree representation for capturing ordinal historical dependence. We present an approach to learn such a model from data, demonstrating that the proposed model fits several real-world datasets better than relevant baselines. We also showcase the potential advantages of such a model to an analyst during the process of knowledge discovery.

Data stream event prediction based on timing knowledge and state transitions

We study a practical problem of predicting the upcoming events in data streams using a novel approach. Treating event time orders as relationship types between event entities, we build a dynamic knowledge graph and use it to predict future event timing. A unique aspect of this knowledge graph embedding approach for prediction is that we enhance conventional knowledge graphs with the notion of "states"---in what we call the ephemeral state nodes---to characterize the state of a data stream over time. We devise a complete set of methods for learning relevant events, for building the event-order graph stream from the original data stream, for embedding and prediction, and for theoretically bounding the complexity. We evaluate our approach with four real world stream datasets and find that our method results in high precision and recall values for event timing prediction, ranging between 0.7 and nearly 1, significantly outperforming baseline approaches. Moreover, due to our choice of efficient translation-based embedding, the overall throughput that the stream system can handle, including continuous graph building, training, and event predictions, is over one thousand to sixty thousand tuples per second even on a personal computer---which is especially important in resource constrained environments, including edge computing.

Modeling Opinion of IPTV Viewers Based on Implicit Feedback and Content Metadata

In today's hyper-connected world, rich social data feeds can be obtained from various sources, including the data exhaust of many commonly used systems. In this paper, we analyze the social pulse, obtained from viewer activity in an IPTV network-we attempt to validate a framework for determining public opinion and public interest through implicit feedback of IPTV viewers. First, we address the hypothesis that implicit viewer feedback in the form of channel change events, paired with the content metadata, can be used to model viewers' opinion and interest. For this, we design a controlled experiment to collect explicit feedback by rating a set of general-interest news clips. In addition to collecting demographic information, we also survey viewers' opinion, interest, and the probability of channel change during each clip. Furthermore, we extract weighted feature vectors from the closed captions of the video; this data, combined with the reported probability of channel change, is used to build a model that classifies opinion in five categories based on the probability of channel change and content. Next, we build a simplified model that classifies opinion in five categories based on the interest, which shows a linear relationship, but further consideration of content, in this case, provides better accuracy and possibility to analyze anomalous cases. Finally, we discuss and analyze the applications of such models in large systems and the necessary modifications to scale the system and to ensure the adequate performance on massive IPTV event data streams.

Fueling Prediction of Player Decisions: Foundations of Feature Engineering for Optimized Behavior Modeling in Serious Games

As a digital learning medium, serious games can be powerful, immersive educational vehicles and provide large data streams for understanding player behavior. Educational data mining and learning analytics can effectively leverage big data in this context to heighten insight into student trajectories and behavior profiles. In application of these methods, distilling event-stream data down to a set of salient features for analysis (i.e. feature engineering) is a vital element of robust modeling. This paper presents a process for systematic game-based feature engineering to optimize insight into player behavior: the IDEFA framework (Integrated Design of Event-stream Features for Analysis). IDEFA aligns game design and data collection for high-resolution feature engineering, honed through critical, iterative interplay with analysis. Building on recent research in game-based data mining, we empirically investigate IDEFA application in serious games. Results show that behavioral models which used a full feature set produced more meaningful results than those with no feature engineering, with greater insight into impactful learning interactions, and play trajectories characterizing groups of players. This discovery of emergent player behavior is fueled by the data framework, resultant base data stream, and rigorous feature creation process put forward in IDEFA—integrating iterative design, feature engineering, and analysis for optimal insight into serious play.

Deep representation via convolutional neural network for classification of spatiotemporal event streams

Abstract Different from traditional frame-based cameras, event-based dynamic vision sensor (DVS) converts the visual information into spatiotemporal event streams. Convolutional neural networks (CNNs) have recently achieved outstanding classification performance while require a very large number of annotated samples. However, a lack of available large-scale event-stream datasets prevents application of CNNs to classification of such event streams. In this work, we show how the deep representation learned with an originally optimized CNN is efficiently transferred to the event-stream classification tasks. In our classification method, a spike-event temporal coding is used to encoding the spike-event information of each pixel. This temporal coding mechanism is implemented based on the subthreshold dynamic of the leaky integrate-and-fire (LIF) model. Three popular event-stream datasets were used to evaluate the performance of the proposed method. Results show that the proposed method leads to significantly improved classification accuracy, outperforming the current state of the art methods on the three event-stream datasets. Besides, the robustness of our method was verified in the MNIST-DVS dataset when Gaussian temporal noises were added to the timestamps of the events. Finally, we find that fine tuning with a small amount of event-stream data would improve the classification performance. This work can be easily extended to more complex scenarios and more fascinating and potential visual applications.

Sentiment-based and hashtag-based Chinese online bursty event detection

How to detect bursty events in data streams on social media is a hot research topic in natural language processing. However, current methods for extracting bursty events suffer from poor accuracy and low efficiency. Fortunately, sentiment analysis has been applied to event detection, which has improved the performance greatly. Inspired by this, this paper proposes a new model which utilizes sentiment analysis for Chinese bursty event detection. First, we build a sentiment co-occurrence graph offline and apply it to analyze microblog sentiment. Plutchik’s emotion wheel is the base for the sentiment classification of the graph. Second, sentiment is used as features to detect bursts in microblog streams online. At last, we exploit regular expressions to extract hashtags in bursty periods and segment hashtags into keywords. By using mutual information and frequent patterns, we fetch words relevant to hashtags as keywords to form events. This approach can detect bursty events online while analyzing the sentiment of microblogs. The experimental results on a large real dataset show that our method can detect bursty events with higher accuracy in a shorter time than traditional methods.

Imprecise temporal associations and decision support systems

The quick and pervasive infiltration of decision support systems, artificial intelligence, and data mining in consumer electronics and everyday life in general has been significant in recent years. Fields such as UX have been facilitating the integration of such technologies into software and hardware, but the back-end processing is still based on binary foundations. This article describes an approach to mining for imprecise temporal associations among events in data streams, taking into account the very natural concept of approximation. This type of association analysis is likely to lead to more meaningful and actionable decision support systems.

Complex event recognition in the big data era

The concept of event processing is established as a generic computational paradigm in various application fields, ranging from data processing in Web environments, over maritime and transport, to finance and medicine. Events report on state changes of a system and its environment. Complex Event Recognition (CER) in turn, refers to the identification of complex/composite events of interest, which are collections of simple events that satisfy some pattern, thereby providing the opportunity for reactive and proactive measures. Examples include the recognition of attacks in computer network nodes, human activities on video content, emerging stories and trends on the Social Web, traffic and transport incidents in smart cities, fraud in electronic marketplaces, cardiac arrhythmias, and epidemic spread. In each scenario, CER allows to make sense of Big event Data streams and react accordingly. The goal of this tutorial is to provide a step-by-step guide for realizing CER in the Big Data era. To do so, it elaborates on major challenges and describes algorithmic toolkits for optimized manipulation of event streams characterized by high volume, velocity and/or lack of veracity, placing emphasis on distributed CER over potentially heterogeneous (data variety) event sources. Finally, we highlight future research directions in the field.