Problem Of Event Detection Research Articles

Sound Event Detection and Time–Frequency Segmentation from Weakly Labelled Data

Sound event detection (SED) aims to detect when and recognize what sound events happen in an audio clip. Many supervised SED algorithms rely on strongly labelled data which contains the onset and offset annotations of sound events. However, many audio tagging datasets are weakly labelled, that is, only the presence of the sound events is known, without knowing their onset and offset annotations. In this paper, we propose a time-frequency (T-F) segmentation framework trained on weakly labelled data to tackle the sound event detection and separation problem. In training, a segmentation mapping is applied on a T-F representation, such as log mel spectrogram of an audio clip to obtain T-F segmentation masks of sound events. The T-F segmentation masks can be used for separating the sound events from the background scenes in the time-frequency domain. Then a classification mapping is applied on the T-F segmentation masks to estimate the presence probabilities of the sound events. We model the segmentation mapping using a convolutional neural network and the classification mapping using a global weighted rank pooling (GWRP). In SED, predicted onset and offset times can be obtained from the T-F segmentation masks. As a byproduct, separated waveforms of sound events can be obtained from the T-F segmentation masks. We remixed the DCASE 2018 Task 1 acoustic scene data with the DCASE 2018 Task 2 sound events data. When mixing under 0 dB, the proposed method achieved F1 scores of 0.534, 0.398 and 0.167 in audio tagging, frame-wise SED and event-wise SED, outperforming the fully connected deep neural network baseline of 0.331, 0.237 and 0.120, respectively. In T-F segmentation, we achieved an F1 score of 0.218, where previous methods were not able to do T-F segmentation.

Accurate Detection of Important Events in WSNs

Security applications of wireless sensor networks (such as surveillance systems) require highly accurate detection systems for distinguishing the events of vital importance from simultaneously occurring random events. In order to improve the accuracy of the detection of such events of critical importance, a distributed in-network inference scheme is proposed in this paper. The problem of detection of such events is formulated as a sparse event detection problem and the awake cycle of every node is partitioned into smaller time blocks. Such a division allows the distributed inference algorithm to precisely identify the events of importance and their exact location in the physical sensor field. In order to further minimize the false alarm rate (FAR), spatially correlated data, from targeted regions, are obtained via a compressed sensing based data gathering scheme. The proposed scheme is able to achieve a significantly high accuracy rate of 0.9875 and exceptionally low FAR of 0.01. The prodigious performance of the proposed scheme continues in the experiments performed in outdoor environment and simulations.

An On-Line and Adaptive Method for Detecting Abnormal Events in Videos Using Spatio-Temporal ConvNet

We address in this paper the problem of abnormal event detection in video-surveillance. In this context, we use only normal events as training samples. We propose to use a modified version of pretrained 3D residual convolutional network to extract spatio-temporal features, and we develop a robust classifier based on the selection of vectors of interest. It is able to learn the normal behavior model and detect potentially dangerous abnormal events. This unsupervised method prevents the marginalization of normal events that occur rarely during the training phase since it minimizes redundancy information, and adapt to the appearance of new normal events that occur during the testing phase. Experimental results on challenging datasets show the superiority of the proposed method compared to the state of the art in both frame-level and pixel-level in anomaly detection task.

User-driven geolocated event detection in social media

Event detection is one of the most important research topics in social media analysis. Despite this interest, few researchers have addressed the problem of identifying geolocated events in an unsupervised way, and none includes user interests during the process. In this paper, we tackle the problem of local event detection from social media data. We present a method to automatically identify events by evaluating the burstiness of hashtags in a geographical area and a time interval, and at the same time integrating user feedback. We devise two algorithms to discover user-driven events. The first one relies on an exact enumeration process, while the other directly samples the space of events. In our empirical study, we provide evidence that geolocated events cannot be detected by non location-aware methods. We also show that our methods (i) outperform by a factor of two to several orders of magnitude state-of-the-art methods designed to discover geolocated events, (ii) are more robust to noise, and (iii) produce high quality events with respect to user interests.

Multi-label Prediction in Time Series Data using Deep Neural Networks

This paper addresses a multi-label predictive fault classification problem for multidimensional time-series data. While fault (event) detection problems have been thoroughly studied in literature, most of the state-of-the-art techniques can’t reliably predict faults (events) over a desired future horizon. In the most general setting of these types of problems, one or more samples of data across multiple time series can be assigned several concurrent fault labels from a finite, known set and the task is to predict the possibility of fault occurrence over a desired time horizon. This type of problem is usually accompanied by strong class imbalances where some classes are represented by only a few samples. Importantly, in many applications of the problem such as fault prediction and predictive maintenance, it is exactly these rare classes that are of most interest. To address the problem, this paper proposes a general approach that utilizes a multi-label recurrent neural network with a new cost function that accentuates learning in the imbalanced classes. The proposed algorithm is tested on two public benchmark datasets: an industrial plant datasetfrom the PHM Society Data Challenge, and a human activity recognition dataset. The results are compared with state-ofthe-art techniques for time-series classification and evaluation is performed using the F1-score, precision and recall.

Sparse Coding Guided Spatiotemporal Feature Learning for Abnormal Event Detection in Large Videos

Abnormal event detection in large videos is an important task in research and industrial applications, which has attracted considerable attention in recent years. Existing methods usually solve this problem by extracting local features and then learning an outlier detection model on training videos. However, most previous approaches merely employ hand-crafted visual features, which is a clear disadvantage due to their limited representation capacity. In this paper, we present a novel unsupervised deep feature learning algorithm for the abnormal event detection problem. To exploit the spatiotemporal information of the inputs, we utilize the deep three-dimensional convolutional network (C3D) to perform feature extraction. Then, the key problem is how to train the C3D network without any category labels. Here, we employ the sparse coding results of the hand-crafted features generated from the inputs to guide the unsupervised feature learning. Specifically, we define a multilevel similarity relationship between these inputs according to the statistical information of the shared atoms. In the following, we introduce the quadruplet concept to model the multilevel similarity structure, which could be used to construct a generalized triplet loss for training the C3D network. Furthermore, the C3D network could be utilized to generate the features for sparse coding again, and this pipeline could be iterated for several times. By jointly optimizing between the sparse coding and the unsupervised feature learning, we can obtain robust and rich feature representations. Based on the learned representations, the sparse reconstruction error is applied to predicting the anomaly score of each testing input. Experiments on several publicly available video surveillance datasets in comparison with a number of existing works demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods.

Challenges of event detection from social media streams

The area of Event Detection (ED) has attracted researchers' attention over the last few years because of the wide use of social media. Many studies have examined the problem of ED in various social media platforms, like Twitter, Facebook, YouTube, etc. The ED task for social networks involves many issues, including the processing of huge volumes of data with a high level of noise, data collection and privacy issues, etc. Hence, this article discusses and presents the wide range of challenges encountered in the ED process from unstructured text data for the most popular Social Networks (SNs), such as Facebook and Twitter. The main goal is to aid the researchers to understand the main challenges and to discuss the future directions in the ED area.

Propagation-Based Temporal Network Summarization

Modern networks are very large in size and also evolve with time. As their sizes grow, the complexity of performing network analysis grows as well. Getting a smaller representation of a temporal network with similar properties will help in various data mining tasks. In this paper, we study the novel problem of getting a smaller diffusion-equivalent representation of a set of time-evolving networks. We first formulate a well-founded and general temporal-network condensation problem based on the so-called system-matrix of the network. We then propose NetCondense , a scalable and effective algorithm which solves this problem using careful transformations in sub-quadratic running time, and linear space complexities. Our extensive experiments show that we can reduce the size of large real temporal networks (from multiple domains such as social, co-authorship, and email) significantly without much loss of information. We also show the wide-applicability of NetCondense by leveraging it for several tasks: for example, we use it to understand, explore, and visualize the original datasets and to also speed-up algorithms for the influence-maximization and event detection problems on temporal networks.

Learning audio and image representations with bio-inspired trainable feature extractors

Since when very young, we can quickly learn new concepts, and distinguish between different kinds of object or sound. If we see a single object or hear a particular sound, we are then able to recognize such sample or even different versions of it in other scenarios. As an example, if one sees a iron chair and associates the object to the general concept of “chairs”, he will be able to detect and recognize also wooden or wicker chairs. Similarly, when we hear the sound of a particular event, such as a scream, we are then able to recognize other kinds of scream that occur in different environments. We continuously learn representations of the real world, which we then use in order to understand new and changing environments.In the field of pattern recognition, traditional methods typically require a careful design of data representations (i.e. features), which involves considerable domain knowledge and effort by experts. Recently, approaches for automated learning of representations from training data were introduced and based on popular deep learning techniques and convolutional neural networks (CNN). Representation learning aims at avoiding engineering of hand-crafted features and providing automatically learned features suitable for the recognition tasks. In this work, we proposed novel trainable filters for representation learning in audio and image processing. The structure of these filters is not fixed in the implementation but rather configured directly from single prototype patterns of interest [4].In the context of audio processing, we focused on the problem of audio event detection and classification in noisy environments, also in cases where the signal to noise ratio (SNR) is null or negative. We released two data sets, namely the MIVIA audio events and the MIVIA road events data sets, and obtained baseline results (recognition rate of about 85%) with a real-time method for event detection based on the bag of features classification scheme [3, 2].We designed novel trainable feature extractors, which we call COPE (Combination of Peaks of Energy), that are able to detect specific constellations of energy peak points in time-frequency representations of input audio signals [8]. The particular constellation of energy peaks to be detected by a COPE feature extractor is determined in an automatic configuration process performed on a given prototype sound. The design of COPE feature extractors was inspired by some functions of the cochlea membrane and the inner hair cells in the inner auditory system, which convert the sound pressure waves into neural stimuli on the auditory nerve.We proposed a method that uses COPE feature extractors together with a classification system to perform real-time audio event detection and classification, also in cases where sounds have null and negative SNR. The performance results (recognition rate over 90%) that we obtained on several benchmarking data sets for audio events detection in different contexts are higher than state-of-the-art approaches.In the second part of the work, we introduced B-COSFIRE filters for detection of elongated and curvilinear patterns in images and apply them to the delineation of blood vessels in retinal images [1, 6]. The B-COSFIRE filters are trainable, that is their structure is automatically configured from prototype elongated patterns. The design of the B-COSFIRE filters is inspired by the functions of some neurons, called simple cells, in area V1 of the visual system, which fire when presented with line or contour stimuli. A B-COSFIRE filter achieves orientation selectivity by computing the weighted geometric mean of the output of a pool of Difference-of-Gaussians (DoG) filters, whose supports are aligned in a collinear manner. Rotation invariance is efficiently obtained by appropriate shiftings of the DoG filter responses.After configuring a large bank of B-COSFIRE filters selective for vessels (i.e. lines) and vessel-endings (i.e. line-endings) of various thickness (i.e. scale), we employed different techniques based on information theory and machine learning to select an optimal subset of B-COSFIRE filters for the vessel delineation task [5, 7]. We considered the selected filters as feature extractors to construct a pixel-wise feature vector which we used in combination with a classifier to classify the pixels in the testing image as vessel and non-vessel pixels. We carried out experiments on public benchmarking data sets (DRIVE, STARE, CHASE DB1 and HRF data sets) and the results that we achieved are higher than many existing methods.We studied the computational requirements of the proposed algorithms in order to evaluate their applicabilityin real-world applications and the fulfillment of real-time constraints given by the considered problems. The MATLAB implementation of the proposed algorithms are publicly released for research purposes.This work contributes to the development of algorithms for representation learning in audio and imageprocessing and promotes their use in higher-level pattern recognition systems.

A Sequential Algorithm for Signal Segmentation.

The problem of event detection in general noisy signals arises in many applications; usually, either a functional form of the event is available, or a previous annotated sample with instances of the event that can be used to train a classification algorithm. There are situations, however, where neither functional forms nor annotated samples are available; then, it is necessary to apply other strategies to separate and characterize events. In this work, we analyze 15-min samples of an acoustic signal, and are interested in separating sections, or segments, of the signal which are likely to contain significant events. For that, we apply a sequential algorithm with the only assumption that an event alters the energy of the signal. The algorithm is entirely based on Bayesian methods.

Video feature descriptor combining motion and appearance cues with length-invariant characteristics

Feature descriptor is one of the important subjects in video analysis problem. In this paper, we propose one video feature descriptor combining motion and appearance cues. The length-invariant characteristics of this proposed feature descriptor are clarified. Further, this feature descriptor is adopted to represent the video sequence for abnormal event detection problem, which is one challenging research field in the video surveillance. We proposed one abnormal event detection algorithm which consists of the feature descriptor and the nonlinear one-class classification method. Experiments on the benchmark dataset and comparisons with the state-of-the-art methods validate the advantages of our proposed feature descriptor.

Compressive detection and localization of multiple heterogeneous events in sensor networks

This paper focuses on the comprehensive event detection and localization problem which efficiently detects not only the number and the position, but also the event signal strength of events in sensor networks. We consider the practical situation where multiple events may simultaneously occur, their signal with heterogeneous strength attenuates over distance and their signal propagation region may overlap. The problem becomes even more challenging when we get rid of the commonly made impractical assumptions, such as the oversimplified binary detection model, the awareness of the number and potential positions of future events, and the existing of super sensor nodes with unlimited sensing range. Inspired by spatially sparse event occurrences, we propose the efficient compressive sensing based approach called CED. Instead of collecting complete sensor readings, our self-driven and fully distributed measurement construction process makes only a small number of qualified measurements, enabling compressive sensing based data recovery. The distinguishing feature of our approach is that it requires no knowledge of, and is adaptive to, the number of occurred events which is changing over time. We have validated signal attenuation model of real-world events and implemented the proposed approach on a testbed of 36 TelosB motes. Testbed experiments and simulation results jointly demonstrate that our approach can achieve high detection rate with event occurred grids while incurring modest transmission overhead.

Multi-event Detection with Rechargeable Sensors

In this paper, we use a rechargeable sensor network to detect important events. Different from most existing works which mainly pay attention to the event detection problems of single event, we focus on multi-sensor and multi-event problems. Our goal is to maximize capture probabilities of all the events in a special area. First, we analyze the single-event case and find a multi-sensor cooperation schedule. It is shown that when each sensor’s battery capacity is large enough, the detecttion rate gained from cooperation schedule tends to be the optimal value. Then, we extend results to multi-sensor multi-event case. We use a weighted sum method to transform the multi-objective problem into a general linear programming. By considering the characteristics of this programming, a Pareto optimal solution can be found directly. We can also use weights and constraints to reflect the importance of each event, and get different Pareto optimal solutions via changing these parameters.

TagBook: A Semantic Video Representation Without Supervision for Event Detection

We consider the problem of event detection in video for scenarios where only a few, or even zero, examples are available for training. For this challenging setting, the prevailing solutions in the literature rely on a semantic video representation obtained from thousands of pretrained concept detectors. Different from existing work, we propose a new semantic video representation that is based on freely available social tagged videos only, without the need for training any intermediate concept detectors. We introduce a simple algorithm that propagates tags from a video's nearest neighbors, similar in spirit to the ones used for image retrieval, but redesign it for video event detection by including video source set refinement and varying the video tag assignment. We call our approach TagBook and study its construction, descriptiveness, and detection performance on the TRECVID 2013 and 2014 multimedia event detection datasets and the Columbia Consumer Video dataset. Despite its simple nature, the proposed TagBook video representation is remarkably effective for few-example and zero-example event detection, even outperforming very recent state-of-the-art alternatives building on supervised representations.

Topic Model Based New Event Detection Within Topics

Traditional new event detection is first proposed by Topic Detection and Tracking and it is actually first event detection. However, one topic usually consists of many events. The automatic instant detection of each event in one topic, not only the first event but also the second, the third and so on, is very useful for users to correctly understand the main development trend of the topic. In this paper, we address the problem of new event detection in one single topic and propose a novel topic model to detect new events along with the topic evolution. Our topic model treats new event detection as novel semantic aspect identification in one topic, rather than measuring the analog degrees between content items by lexical congruence. Besides, it can automatically determine the appropriate number of aspects needed and can naturally adapt dynamic change in the vocabulary along with the topic evolution. We use a sequential Gibbs sampling algorithm for posterior inference, which well realizes the online new event detection. Experiments are presented to show the performance of our proposed technique. It is found that our proposed technique outperforms the comparable techniques in previous work.

Joint event detection and environment perception in decentralised wireless sensor networks

In an event detection application, sensor nodes measure signals that are emitted from multiple events and attenuated by the environment, and then collaborate to estimate the locations and magnitudes of the events. Existing event detection algorithms often assume that the number of the events is known in advance and/or the attenuation coefficient of the environment is given. This paper considers the case that both the number of the events and the attenuation coefficient of the environment are unknown. Through exploiting the sparse nature of the events, we propose an l-norm regularised least squares formulation that automatically estimates the number of the events as well as their locations and magnitudes; the attenuation coefficient of the environment also appears as an optimisation variable. We develop a decentralised algorithm and its accelerated variant to solve the joint event detection and environment perception problem using the alternating direction method of multipliers ADMM.

Learning to detect video events from zero or very few video examples

In this work we deal with the problem of high-level event detection in video. Specifically, we study the challenging problems of i) learning to detect video events from solely a textual description of the event, without using any positive video examples, and ii) additionally exploiting very few positive training samples together with a small number of “related” videos. For learning only from an event's textual description, we first identify a general learning framework and then study the impact of different design choices for various stages of this framework. For additionally learning from example videos, when true positive training samples are scarce, we employ an extension of the Support Vector Machine that allows us to exploit “related” event videos by automatically introducing different weights for subsets of the videos in the overall training set. Experimental evaluations performed on the large-scale TRECVID MED 2014 video dataset provide insight on the effectiveness of the proposed methods.

Categorizing events using spatio-temporal and user features from Flickr

Even though the problem of event detection from social media has been well studied in recent years, few authors have looked at deriving structured representations for their detected events. We envision the use of social media for extracting large-scale structured event databases, which could in turn be used for answering complex (historical) queries. As a key stepping-stone towards this goal, we introduce a method for discovering the semantic type of extracted events, focusing in particular on how this type is influenced by the spatio-temporal grounding of the event, the profile of its attendees, and the semantic type of the venue and other entities which are associated with the event. We estimate the aforementioned characteristics from metadata associated with Flickr photos of the event and then use an ensemble learner to identify its most likely semantic type. Experimental results based on an event dataset from Upcoming.org and Last.fm show a marked improvement over bag-of-words based methods.

Language agnostic meme-filtering for hashtag-based social network analysis

Users in social networks utilize hashtags for a variety of reasons. In many cases, hashtags serve retrieval purposes by labeling the content they accompany. More often than not, hashtags are used to promote content, ideas, or conversations producing viral memes. This paper addresses a specific case of hashtag classification: meme-filtering. We argue that hashtags that are correlated with memes may hinder many valuable social media algorithms like trend detection and event identification. We propose and evaluate a set of language-agnostic features that aid the separation of these two classes: meme-hashtags and event-hashtags. The proposed approach is evaluated on two large datasets of Twitter messages written in English and German. A proof-of-concept application of the meme-filtering approach to the problem of event detection is presented.

Approximate event detection over multi-modal sensing data

Composite event detection is one of the fundamental tasks for heterogeneous wireless sensor networks (WSNs). Multi-modal data generated by heterogeneous sensors bring new challenges for composite event monitoring in heterogeneous WSNs. By exploiting the correlations between different types of data, the approximate composite event detection problem is investigated in this paper. The optimal transmitting scheme problem is proposed to calculate the optimal transmitting scheme with minimum cost on the constraint that the confidence of the composite event must exceed the threshold. The optimal transmitting scheme problem is proved to belong to NP-complete. A dynamic programming based algorithm is presented for simple linear confidence combination operators, which runs in pseudo-polynomial time. The greedy based approximate algorithm is also designed for general confidence combination operators and the approximate ratio is proved to be 2 for "+" as the confidence combination operator. The simulation results show that our algorithms can reduce energy consumption significantly.