Normal Data Points Research Articles

Triplet-set feature proximity learning for video anomaly detection

The identification of anomalies in videos is a particularly complex visual challenge, given the wide variety of potential real-world events. To address this issue, our paper introduces a unique approach for detecting divergent behavior in surveillance videos, utilizing triplet-loss for video anomaly detection. Our method involves selecting a triplet set of video segments from normal (n) and abnormal (a) data points for deep feature learning. We begin by creating a database of triplet sets of two types: a-a-n and n-n-a. By computing a triplet loss, we model the proximity between n-n chunks and the distance between ‘a’ chunks from the n-n ones. Additionally, we train the deep network to model the closeness of a-a chunks and the divergent behavior of ‘n’ from the a-a chunks.The model acquired in the initial stage can be viewed as a prior, which is subsequently employed for modeling normality. As a result, our method can leverage the advantages of both straightforward classification and normality modeling-based techniques. We also present a data selection mechanism for the efficient generation of triplet sets. Furthermore, we introduce a novel video anomaly dataset, AnoVIL, designed for human-centric anomaly detection. Our proposed method is assessed using the UCF-Crime dataset encompassing all 13 categories, the IIT-H accident dataset, and AnoVIL. The experimental findings demonstrate that our method surpasses the current state-of-the-art approaches. We conduct further evaluations of the performance, considering various configurations such as cross-dataset evaluation, loss functions, siamese structure, and embedding size. Additionally, an ablation study is carried out across different settings to provide insights into our proposed method.

Deep clustering hierarchical latent representation for anomaly-based cyber-attack detection

In the field of anomaly detection, well-known techniques and state-of-the-art models often face challenges when interpreting the latent space, which hinders their behavioral classification accuracy. Firstly, the sub-optimal distribution of data points within the latent space makes normal behavioral regions verbose and indistinguishable from abnormal regions. Secondly, within the latent space, it can be difficult to identify meaningful, separable, and indicative features. Finally, the processing time at the inference stage is still relatively slow.This paper aims to improve the accuracy of network anomaly detection mechanisms by proposing two novel deep hierarchical representation learning models: Deep Nested Clustering Auto-Encoder (DNCAE) and Deep Clustering Hierarchical Auto-Encoder (DCHAE). Both models adopt a nested branch structure, utilizing dual deep auto-encoders to establish hierarchical latent spaces; in each, clustering algorithms are used to spatially optimize and refine the data points. This approach results in improved separation between normal and abnormal data points, and easier identification of notable and/or indicative features.To ascertain the effectiveness of the approach and the quality of resulting features, both models were used in conjunction with ten different one-class anomaly detectors. Each of these ten anomaly detectors was evaluated on popular network intrusion datasets, notably: NSL-KDD, UNSW-NB15, CIC-IDS-2017, CSE-CIC-IDS-2018, and CTU13. Experimental results have confirmed that both of the proposed models produced higher levels of accuracy than existing baselines and current state-of-the-art models. Additionally, the processing time at the inference stage shows a significant reduction.

Self-Supervised Random Forest on Transformed Distribution for Anomaly Detection.

Anomaly detection, the task of differentiating abnormal data points from normal ones, presents a significant challenge in the realm of machine learning. Numerous strategies have been proposed to tackle this task, with classification-based methods, specifically those utilizing a self-supervised approach via random affine transformations (RATs), demonstrating remarkable performance on both image and non-image data. However, these methods encounter a notable bottleneck, the overlap of constructed labeled datasets across categories, which hampers the subsequent classifiers' ability to detect anomalies. Consequently, the creation of an effective data distribution becomes the pivotal factor for success. In this article, we introduce a model called "self-supervised forest (sForest)", which leverages the random Fourier transform (RFT) and random orthogonal rotations to craft a controlled data distribution. Our model utilizes the RFT to map input data into a new feature space. With this transformed data, we create a self-labeled training dataset using random orthogonal rotations. We theoretically prove that the data distribution formulated by our methodology is more stable compared to one derived from RATs. We then use the self-labeled dataset in a random forest (RF) classifier to distinguish between normal and anomalous data points. Comprehensive experiments conducted on both real and artificial datasets illustrate that sForest outperforms other anomaly detection methods, including distance-based, kernel-based, forest-based, and network-based benchmarks.

Prediction of Gasoline Orders at Gas Stations in South Korea Using VAE-Based Machine Learning Model to Address Data Asymmetry

South Korea has developed road-based transportation and uses a lot of gasoline. South Korea imports gasoline since it is not produced domestically. So, fluctuations in gasoline prices have a significant impact on the national economy. Currently, gasoline orders, which are based on gasoline consumption, are analyzed in relation to fluctuations in gasoline prices. However, gasoline orders can also change due to various non-price factors. Therefore, to understand the trend of gasoline orders, it is important to identify additional factors that gas stations consider when determining orders. We collected 180 monthly samples of data on 167 variables. Sudden international issues lead to rapid fluctuations in gasoline orders, which can lead to outliers. A class imbalance occurs because outliers are generally fewer in number than the normal data points. Therefore, to address the class imbalance, we proposed a method that grouped the data samples into 11 clusters using the K-means clustering algorithm and then augmented the data into 85 datasets in each cluster through the Variational Auto-Encoder. We evaluated the augmented datasets through the R-Squared, Root Mean Squared Errors, and accuracy of various regression models. Based on the experimental results, when predicting gasoline orders at gas stations in South Korea using augmented datasets, linear regression showed the best performance.

Semi-supervised anomaly detection methods for leakage identification in water distribution networks: A comparative study

This study presents a comprehensive evaluation of 10 state of the art semi-supervised anomaly detection (AD) methods for leakage identification in water distribution networks (WDNs). The performances of the semi-supervised AD methods is evaluated on LeakDB, a benchmark consisting of independent leakage scenarios that also account for the various sources of uncertainties arising in WDNs. Three performance metrics (Fβ Measure, PR AUC Score, and Identification Lag Time) that collectively capture the different facets of leakage identification in WDNs is utilised to measure the efficacy of semi-supervised AD methods. Additionally, the TOPSIS MCDM tool supported with two weighting approaches is implemented to simultaneously consider all performance metrics in ranking the performance of semi-supervised AD methods. The results of this extensive comparative study shows that Local Outlier factor (LOF) is the overall best performing semi-supervised AD method on LeakDB. It is also evident that proximity based semi-supervised AD methods are superior to linear and probabilistic AD methods due to their ability to unearth leak events in the neighbourhood of normal operational data points. Finally, the impact of uncertainties on the performance of the semi-supervised AD models is discussed in addition to general recommendations on the usage of semi-supervised AD methods in leakage identification.

Detecting outliers in GNSS position time series using machine learning techniques

The Global Navigation Satellite System (GNSS) position time series is applied in studies that require high-precision positioning, such as monitoring tectonic movements and Earth deformation. Outliers in GNSS position time series can significantly impact the accuracy of station positioning and movement parameters, leading to distorted data analysis outcomes. This study investigates the effectiveness of three machine learning techniques, including-Isolation Forest, One-Class Support Vector Machines (O-C SVM), and Local Outlier Factor (LOF) for outlier detection in GNSS position time series, with a specific focus on the SYNT model where outliers account for a substantial proportion (15%). Through comprehensive analysis, our results highlight the exceptional performance of the Isolation Forest method. It demonstrates remarkable accuracy in identifying outliers, effectively detecting the majority of them, and achieving an area under the ROC curve close to 1. In contrast, the LOF method performs less effectively in outlier detection, while the O-C SVM method displays relatively higher accuracy in identifying normal data points. These findings emphasize the significant advantages of leveraging machine learning approaches in processing continuous GNSS measurement data. By effectively identifying and handling outliers, these techniques enhance the accuracy and reliability of data analysis in GNSS position time series, ultimately establishing their superiority in the field of data analysis.

A Probabilistic Transformation of Distance-Based Outliers

The scores of distance-based outlier detection methods are difficult to interpret, and it is challenging to determine a suitable cut-off threshold between normal and outlier data points without additional context. We describe a generic transformation of distance-based outlier scores into interpretable, probabilistic estimates. The transformation is ranking-stable and increases the contrast between normal and outlier data points. Determining distance relationships between data points is necessary to identify the nearest-neighbor relationships in the data, yet most of the computed distances are typically discarded. We show that the distances to other data points can be used to model distance probability distributions and, subsequently, use the distributions to turn distance-based outlier scores into outlier probabilities. Over a variety of tabular and image benchmark datasets, we show that the probabilistic transformation does not impact outlier ranking (ROC AUC) or detection performance (AP, F1), and increases the contrast between normal and outlier score distributions (statistical distance). The experimental findings indicate that it is possible to transform distance-based outlier scores into interpretable probabilities with increased contrast between normal and outlier samples. Our work generalizes to a wide range of distance-based outlier detection methods, and, because existing distance computations are used, it adds no significant computational overhead.

Anomaly detection-based condition monitoring

The impact of an anomaly is domain-dependent. In a dataset of network activities, an anomaly can imply an intrusion attack. Other objectives of anomaly detection are industrial damage detection, data leak prevention, identifying security vulnerabilities or military surveillance. Anomalies are observations or a sequence of observations in which the distribution deviates remarkably from the general distribution of the whole dataset. A large majority of the dataset consists of normal (healthy) data points. The anomalies form only a very small part of the dataset. Anomaly detection is the technique used to find these observations and its methods are specific to the type of data. While there is a wide spectrum of anomaly detection approaches available today, it becomes increasingly difficult to keep track of all the techniques. In fact, it is not clear which of the three categories of detection methods, ie statistical approaches, machine learning approaches or deep learning approaches, is more appropriate to detect anomalies in time-series data, which are mainly used in industry. A typical industrial device has multi-dimensional characteristics. It is possible to measure voltage, current, active power, vibrations, rotational speed, temperature, pressure difference, etc, on such a device. Early detection of the anomalous behaviour of industrial devices can help reduce or prevent serious damage, which could lead to significant financial loss. This paper presents a summary of the methods used to detect anomalies in condition monitoring applications.

FL-MGVN: Federated learning for anomaly detection using mixed gaussian variational self-encoding network

Anomalous data are such data that deviate from a large number of normal data points, which often have negative impacts on various systems. Current anomaly detection technology suffers from low detection accuracy, high false alarm rate and lack of labeled data. Anomaly detection is of great practical importance as an effective means to detect anomalies in the data and provide important support for the normal operation of various systems. In this paper, we propose an anomaly detection classification model that incorporates federated learning and mixed Gaussian variational self-encoding networks, namely MGVN. The proposed MGVN network model first constructs a variational self-encoder using a mixed Gaussian prior to extracting features from the input data, and then constructs a deep support vector network with the mixed Gaussian variational self-encoder to compress the feature space. The MGVN finds the minimum hypersphere to separate the normal and abnormal data and measures the abnormal fraction by calculating the Euclidean distance between the data features and the hypersphere center. Federated learning is finally incorporated with MGVN (FL-MGVN) to effectively address the problems that multiple participants collaboratively train a global model without sharing private data. The experiments are conducted on the benchmark datasets such as NSL-KDD, MNIST and Fashion-MNIST, which demonstrate that the proposed FL-MGVN has higher recognition performance and classification accuracy than other methods. The average AUC on MNIST and Fashion-MNIST reached 0.954 and 0.937, respectively.

Unsupervised learning for fault detection and diagnosis of air handling units

Supervised learning techniques have witnessed significant successes in fault detection and diagnosis (FDD) for heating ventilation and air-conditioning (HVAC) systems. Despite the good performance, these techniques heavily rely on balanced datasets that contain a large amount of both faulty and normal data points. In real-world scenarios, however, it is often very challenging to collect a sufficient amount of faulty training samples that are necessary for building a balanced training dataset. In this paper, we introduce a framework that utilizes the generative adversarial network (GAN) to address the imbalanced data problem in FDD for air handling units (AHUs). To this end, we first show the necessary procedures of applying GAN to increase the number of faulty training samples in the training pool and re-balance the training dataset. The proposed framework then uses supervised classifiers to train the re-balanced datasets. Finally, we present a comparative study that illustrates the advantages of the proposed method for FDD of AHU with various evaluation metrics. Our work demonstrates the promising prospects of performing robust FDD of AHU with a limited number of faulty training samples.

A Modified Approach of OPTICS Algorithm for Data Streams

Data are continuously evolving from a huge variety of applications in huge volume and size. They are fast changing, temporally ordered and thus data mining has become a field of major interest. A mining technique such as clustering is implemented in order to process data streams and generate a set of similar objects as an individual group. Outliers generated in this process are the noisy data points that shows abnormal behavior compared to the normal data points. In order to obtain the clusters of pure quality outliers should be efficiently discovered and discarded. In this paper, a concept of pruning is applied on the stream optics algorithm along with the identification of real outliers, which reduces memory consumption and increases the speed for identifying potential clusters.

Clustering Data and Incorporating Topographical Variables for Improving Spatial Interpolation of Rainfall in Mountainous Region

This study was an attempt to quantify the improvement in the accuracy of rainfall interpolation in the mountainous terrain by clustering of rainfall data at the data preparation stage and incorporating topographical variables at interpolation stage. The univariate kriging techniques, ordinary kriging (OK), simple kriging and universal kriging (UK) were compared with multivariate kriging method of ordinary cokriging (OCK). The elevation, slope and terrain ruggedness index (TRI) computed from digital elevation model were incorporated as explanatory variable in OCK. These algorithms were applied to the normal annual and seasonal rainfall data points located in Central Himalayas of Uttarakhand region. The study area was divided into two different zones of lowland and upland based on the elevation variability. This zonation was then used as a basis for clustering the rainfall data. The performance of the interpolation techniques was assessed for subdivided regions and compared with the results obtained for complete region. The evaluation was based on absolute error metrics of root mean square error (RMSE) in combination with RMSE-observations standard deviation ratio. The absolute percentage error (APE) statistics was calculated for every observation points and then the percentage of data for with APE ≤ 30% (APE30) was analyzed. It was observed that performance efficiency of the interpolation methods improves by 5-20% if the rainfall data is clustered based on homogeneity of terrain elevation as against considering the complete set of data. In complex terrain, the inclusion of topographical variables improves the cokriging based rainfall prediction if it is correlated with rainfall.

Can psychosine and galactocerebrosidase activity predict early-infantile Krabbe's disease presymptomatically?

Krabbe's disease (KD) is a fatal neurodegenerative disorder, with the early-infantile form (EIKD) defined by onset of symptoms before age 6 months. Early and highly accurate identification of EIKD is required to maximize benefits of hematopoietic stem cell transplantation treatment. This study investigates the potential for accurate prediction of EIKD based on a novel newborn screening (NBS) tool developed from two biomarkers, galactocerebrosidase (GALC) enzyme activity and galactosylsphingosine concentration (psychosine [PSY]). Normative information about PSY and GALC, derived from distinct samples of normal newborns, was used to develop the novel diagnostic tool. Bivariate normal limits (BVNL) were constructed, assuming a multivariate normal distribution of natural logarithms of GALC and PSY of normal newborns. The (lnGALC, lnPSY) points for newborns in various "abnormal groups," including one group of infants who subsequently suffered EIKD, were plotted on a graph of BVNL. The points for all EIKD patients fell outside of BVNL (100% sensitivity). In a simulation study to compare the false-positive rate of existing univariate methods of diagnosis with our new BVNL-based method, we generated 100 million normal newborn data points. All fell within BVNL (i.e., zero false positives), whereas 5,682 false positives were observed when applying a two-tiered univariate method of the type suggested in the literature. These results suggest that (lnGALC, lnPSY) BVNLs will allow highly accurate prediction of EIKD, whereas two-tiered univariate approaches will not. Redevelopment of the BVNL based on GALCs and PSYs measured on a common large sample of normal newborns is required for NBS use. © 2016 Wiley Periodicals, Inc.

An Improved Algorithm of Precise Point Cloud Registration

Basing on the application of digital protection of cultural sites, this paper presented a precise algorithm for the multi-resolution point cloud based on sequence iterative. Firstly, based on the synchronous scanning image, a rough registration of the adjacent point cloud can be obtained in an interactive way. Secondly, based on a normal-based algorithm of registration sphere center, the points of sphere surface are filtered from all scanned point cloud data according to the normal data points and the characteristic of viewpoint, and then the center of registration sphere can be calculated accurately and the initial registration of the adjacent point cloud can be obtained by setting Matching Registration Labels mode as the constraint condition. Finally, based on the 3D edge points of the adjacent point cloud from mahalanobis distance calculations, some overlapping images can be eliminated by the way of constantly optimizing the transition matrix in the iteration process. The engineering practice of the Small Wild Goose Pagoda in Tang Dynasty and the Ancient Tomb in Han Dynasty proves that the method is reliable and easy to design and implement and can effectively restrain the accumulative errors of sequence registrations.

Evaluation of Outlier Specific Correction Procedure for Areal Surface Texture

Surface texture data measured by optical type profilometer such as confocal microscope often contains outliers which may disturb the characterization of the surface texture. This paper evaluates an outlier specific correction procedure (OSCP) for areal surface texture data which can removes the outliers without affecting normal data points. The outliers are identified based on the median of its relative height to neighbouring data points within the detection window. The application of OSCP to areal topography data measured by confocal laser scanning microscope is compared to Gaussian filter and median filter. The result shows that, OSPC is better in outlier correction without affecting normal data points but there is a room of improvement.

Harnessing the Power of GPUs to Speed Up Feature Selection for Outlier Detection

Acquiring a set of features that emphasize the differences between normal data points and outliers can drastically facilitate the task of identifying outliers. In our work, we present a novel non-parametric evaluation criterion for filter-based feature selection which has an eye towards the final goal of outlier detection. The proposed method seeks the subset of features that represent the inherent characteristics of the normal dataset while forcing outliers to stand out, making them more easily distinguished by outlier detection algorithms. Experimental results on real datasets show the advantage of our feature selection algorithm compared with popular and state-of-the-art methods. We also show that the proposed algorithm is able to overcome the small sample space problem and perform well on highly imbalanced datasets. Furthermore, due to the highly parallelizable nature of the feature selection, we implement the algorithm on a graphics processing unit (GPU) to gain significant speedup over the serial version. The benefits of the GPU implementation are two-fold, as its performance scales very well in terms of the number of features, as well as the number of data points.

Normal type-2 fuzzy rational B-spline curve

In this paper, we proposed a new form of type-2 fuzzy data points(T2FDPs) that is normal type-2 data points(NT2FDPs). These brand-new forms of data were defined by using the definition of normal type-2 triangular fuzzy number(NT2TFN). Then, we applied fuzzification(alpha-cut) and type-reduction processes towards NT2FDPs after they had been redefined based on the situation of NT2FDPs. Furthermore, we redefine the defuzzification definition along with the new definitions of fuzzification process and type-reduction method to obtain crisp type-2 fuzzy solution data points. For all these processes from the defining the NT2FDPs to defuzzification of NT2FDPs, we demonstrate through curve representation by using the rational B-spline curve function as the example form modeling these NT2FDPs.

Perfectly normal type-2 fuzzy interpolation B-spline curve

In this paper, we proposed another new form of type-2 fuzzy data points(T2FDPs) that is perfectly normal type-2 data points(PNT2FDPs). These kinds of brand-new data were defined by using the existing type-2 fuzzy set theory(T2FST) and type-2 fuzzy number(T2FN) concept since we dealt with the problem of defining complex uncertainty data. Along with this restructuring, we included the fuzzification(alpha-cut operation), type-reduction and defuzzification processes against PNT2FDPs. In addition, we used interpolation B-spline curve function to demonstrate the PNT2FDPs.

Robust Regression Approach to Estimating Fish Mortality Rates with a Cohort-Based Model

Natural mortality rate (M) and fishing catchability coefficient (q) were estimated by using the Paloheimo pairwise linear regression model with simulated and real catch–effort data. The four estimation methods used were least squares (LS), least absolute value (LAV), least median of squares (LMS), and LMS-based reweighted least squares (RLS). When catch or effort data or both contained homogeneous error variances, mean squared errors (MSE) of LS and RLS were significantly smaller than those of LAV and LMS. However, there was no significant difference in MSE between LS and RLS. When catch–effort data contained variances inconsistent among years, MSEs of the robust methods were much smaller than those of LS. Reweighted least squares substantially reduced the variances in the LS-estimated q and M, and it had the smallest MSE among the four estimation methods. We suggest using the following procedures in estimating q and M based on the Paloheimo pairwise regression model: (1) applying LMS to the model to identify outliers in the data; and (2) weighting the defined outliers by a factor of 0 and normal data points by a factor of 1 and applying LS to the model to estimate q and M and their associated variances.