Abnormal Detection Algorithm Research Articles

Short-term Load Forecasting by Using Improved GEP and Abnormal Load Recognition

Load forecasting in short term is very important to economic dispatch and safety assessment of power system. Although existing load forecasting in short-term algorithms have reached required forecast accuracy, most of the forecasting models are black boxes and cannot be constructed to display mathematical models. At the same time, because of the abnormal load caused by the failure of the load data collection device, time synchronization, and malicious tampering, the accuracy of the existing load forecasting models is greatly reduced. To address these problems, this article proposes a Short-Term Load Forecasting algorithm by using Improved Gene Expression Programming and Abnormal Load Recognition (STLF-IGEP_ALR). First, the Recognition algorithm of Abnormal Load based on Probability Distribution and Cross Validation is proposed. By analyzing the probability distribution of rows and columns in load data, and using the probability distribution of rows and columns for cross-validation, misjudgment of normal load in abnormal load data can be better solved. Second, by designing strategies for adaptive generation of population parameters, individual evolution of populations and dynamic adjustment of genetic operation probability, an Improved Gene Expression Programming based on Evolutionary Parameter Optimization is proposed. Finally, the experimental results on two real load datasets and one open load dataset show that compared with the existing abnormal data detection algorithms, the algorithm proposed in this article have higher advantages in missing detection rate, false detection rate and precision rate, and STLF-IGEP_ALR is superior to other short-term load forecasting algorithms in terms of the convergence speed, MAE, MAPE, RSME, and R 2 .

IQVision: An Image-Based Evaluation Tool for Quantitative Lateral Flow Immunoassay Kits.

The development of quantitative lateral flow immunoassay test strips involves a lot of research from kit manufacturers’ standpoint. Kit providers need to evaluate multiple parameters, including the location of test regions, sample flow speed, required sample volumes, reaction stability time, etc. A practical visualization tool assisting manufacturers in this process is very much required for the design of more sensitive and reliable quantitative LFIA test strips. In this paper, we present an image-based quantitative evaluation tool determining the practical functionality of fluorescence-labelled LFIA test cartridges. Image processing-based algorithms developed and presented in this paper provide a practical analysis of sample flow rates, reaction stability times of samples under test, and detect any abnormalities in test strips. Evaluation of the algorithm is done with Glycated Hemoglobin (HbA1C) and Vitamin D test cartridges. Practical sample flow progress for HbA1C test cartridges is demonstrated. The reaction stability time of HbA1C test samples is measured to be 12 min, while that of Vitamin D test samples is 24 min. Experimental evaluation of the abnormality detection algorithm is carried out, and sample flow abnormalities are detected with 100% accuracy while membrane irregularities are detected with 96% accuracy.

Group Abnormal Behaviour Detection Algorithm Based on Global Optical Flow

Abnormal behaviour detection algorithm needs to conduct behaviour analysis on the basis of continuous video inclination tracking, and the robustness of the algorithm is reduced for the occlusion of moving targets, the occlusion of the environment, and the movement of targets with the same colour. For this reason, the optical flow information between RGB (red, green, and blue) images and video frames is used as the input of the network in view of group behaviour. Then, the direction, velocity, acceleration, and energy of the crowd were weighted and fused into a global optical flow descriptor. At the same time, the crowd trajectory map is extracted from the original image of a single frame. Following, in order to realize the detection of large displacement moving target and solve the problem that the traditional optical flow algorithm is only suitable for the detection of displacement moving target, a video abnormal behaviour detection algorithm based on the double-flow convolutional neural network is proposed. The network uses two network branches to learn spatial dimension information and temporal dimension information, respectively, and uses short- and long-time neural network to model the dependency relationship between long-time video frames, so as to obtain the final behaviour classification results. Simulation test results show that the proposed method can achieve good recognition effect on multiple datasets, and the performance of abnormal behaviour detection can be significantly improved by using interframe motion information.

Abnormal event detection algorithm based on dual attention future frame prediction and gap fusion discrimination

Rapid and accurate detection of crowd abnormal events, such as stampedes and violent attacks in public places, has great research significance and application value. Due to the diversity and uncertainty of abnormal events, almost all existing methods tackle the problem by minimizing the reconstruction errors of training data, which cannot guarantee a larger reconstruction error for all abnormal events. According to the idea that “normal events can be predicted, abnormal events cannot be predicted,” we proposed a future frame prediction-based anomaly detection algorithm. First, the generative adversarial network (GAN) is trained by the normal videos to predict normal future frames. Then, it can determine the existence of abnormal events by identifying the difference between the ground truth and predicted video frame. In the design of the GAN, the attention module is introduced to improve the prediction level of the network. At the same time, the optical flow information is added for motion constraint to improve the constraint ability on the appearance characteristics. In the testing stage, the appearance gap and optical flow gap between the ground truth and the predicted video frame are fused to determine whether the frame is abnormal. The experimental results on the datasets of CUHK Avenue, UCSD, and ShanghaiTech show that the proposed algorithm is superior to that of the current mainstream anomaly detection algorithms.

Optical fiber network abnormal data detection algorithm based on deep learning

Optical fiber network abnormal data detection algorithm based on deep learning

Morphological Similarity Clustering and Its Applications in Anomaly Detection of Time Series

Time series data clustering is an important branch and difficult topic in the field of data clustering. In this paper, the definition of temporal data morphological similarity is proposed, a set of affine invariant morphological similarity measurement methods of time series data is established, and a morphological clustering algorithm based on morphological similarity measurement is developed. Using morphological similarity measurement of time series data, two groups of abnormal change detection algorithms for time series data are established, which can be used to detect the morphological consistency of different periodical sampling series in the same time series and the morphological consistency among several time series in the same period. Based on these algorithms stated above, the multiple monitoring algorithms are proposed, which can be used to monitor states of many kinds of industry process. The effectiveness of the methods and algorithms is verified with theoretical deduction and simulation results. Simulation results show that these algorithms are very valuable for mining, clustering, modeling, statistical learning of multi-source time series data, as well as the detection and diagnosis of abnormal process changes.

Toward Anomaly Behavior Detection as an Edge Network Service Using a Dual-Task Interactive Guided Neural Network

How to use artificial intelligence technology to mine human abnormal behavior from considerable video data generated by the Internet-of-Things system has been intensively studied for a long time. Existing deep learning anomaly detection algorithms deployed in the cloud typically perform supervised learning based on constant kinds of abnormal behavior data. However, this supervised learning model with preset abnormal behavior categories ignores the diversity and unpredictability of abnormal occurrences in open scenarios. Thus, we propose an abnormal behavior detection algorithm as an edge network service by combining the advantages of cloud computing and the efficiency of edge networks. This method combines the double verification of global behavior detection and local fine-grained action cycle alignment to detect whether a behavior is abnormal. Moreover, to enable abnormal behavior detection models to predict test samples whose categories do not appear during the training stage, we propose an active label learning algorithm based on cycle clustering, which not only improves the efficiency of data transmission between the edge and the cloud but also makes model updates in the cloud more efficient. Extensive and quantitative experimental results show that our method can not only accurately detect abnormal human behavior at the edge of limited resources but also has strong robustness under the interference of test samples of unknown categories.

Incremental learning of an abnormal behavior detection algorithm based on principal components

In this paper, we propose an incremental learning scheme for the abnormal behavior detection algorithm based on principal component. The results obtained on a UCSD dataset and our experimental videos at a different number of training samples show that error rates are similar to conventional learning. Moreover, the proposed scheme allows the incremental learning time to be significantly reduced in comparison with a method based on matrix eigendecomposition.

A novel adaptive federated filter for GNSS/INS/VO integrated navigation system

In order to solve the problem of decreased navigation performance of the Global Navigation Satellite System (GNSS)/inertial navigation system (INS) integrated navigation systems in GNSS-denied environments, and to improve the navigation accuracy and robustness of the navigation system, a novel adaptive federated filter with a feedback scheme for a GNSS/INS/visual odometry (VO) integrated navigation system is proposed in this paper. A visual-inertial odometry system model with a multi-state constraint Kalman filter structure based on a polar geometry and trifocal tensor geometry between different images is established, which can provide better navigation accuracy in GNSS-denied environments. Moreover, a new method to obtain the information allocation factor according to the different navigation performances of local filters is deduced in this paper, which has low computational complexity and a simple structure. Meanwhile, an abnormal measurement detection algorithm based on fuzzy logic is proposed to detect the abnormal measurements of local filters. The results of the vehicle experiment with the publicly available real-world KITTI dataset show that the proposed algorithm can obtain reliable navigation results in GNSS-denied environments and improve the navigation accuracy and robustness of the GNSS/INS/VO integrated navigation system.

Deep neural network combined with MapReduce for abnormal data mining and detection in cloud storage

Aiming at the abnormal data behavior such as huge amount of data and easy to be stolen or lost in the process of distributed cloud computing in cloud storage environment, an abnormal data mining and detection algorithm of MapReduce based on Hadoop distributed file system (HDFS) and deep neural network is proposed. Firstly, the algorithm analyzes the MAC timestamp characteristics generated by HDFS folder replication, establishes the detection and measurement methods of replication behavior, and ensures that all the patterns that lead to data anomalies, including theft, packet loss and malicious attack, can be detected. Secondly, the algorithm combines deep neural network to design a task partition strategy suitable for arbitrary MapReduce data, and records the input dataset of HDFS hierarchical relationship. Finally, combined with the parallel processing ability of MapReduce, the efficient analysis of massive timestamp data is realized by designing the dataset and algorithm execution scheme suitable for MapReduce task partition. The experimental results show that the algorithm can control the missed detection rate and the number of false detection folders through the segmentation detection strategy. Compared with the existing big data anomaly detection method, the algorithm has higher execution efficiency and good scalability.

Bayesian Cooperative Localization With NLOS and Malicious Vehicle Detection in GNSS-Challenged Environments

Owing to the inter-vehicle non-line-of-sight (NLOS) measurement and malicious attack in global navigation satellite system (GNSS) challenged environment, the vehicle position precision is seriously damaged. In order to improve the vehicle position accuracy, we propose a new Bayesian cooperative localization scheme which tackles this problem by combining the vehicle position measurements and inter-vehicle distance measurements. In the proposed scheme, an abnormal vehicle detection algorithm (AVDA) is presented to eliminate the impacts of NLOS and malicious attack. Simulation results demonstrate that the proposed scheme can achieve excellent localization performance in the presence of NLOS and malicious attacks. Based on these results, the abnormal and normal detection rates of AVDA are approximate and the root mean square error (RMSE) is reduced to the sub-meter level. The performances of the proposed scheme are also verified in real environmental conditions by using the simulation of urban mobility (SUMO).

Research on abnormal data detection of optical fiber communication network based on data mining

为了提高光纤通信网络中异常数据的识别能力，提出了基于熵目标函数最优化的异常数据检测算法。首先，对数据样本进行属性分类，依据异常数据特征密度指标完成邻域区间半径的选取；其次，通过对高阶统计量的大数据聚类度循环迭代，完成特征提取参数的优化；最后，由样本属性概率计算熵目标函数的最优值，并利用最优值完成异常数据检测。实验对1 000组通信数据进行测试，结果显示，该算法的检测精度均值约为95.7%，其数据融合率、检测耗时与平均误检率均优于2种传统方法。该算法具有精度高、收敛快、误检率低的优势，具有一定的应用价值。

Abnormal Trajectory Detection Based on a Sparse Subgraph

Traditional abnormal trajectory detection algorithms mainly involve the measurement of a single feature; however, the influence of other features on abnormal trajectory is ignored, resulting in the inability to fully discover the abnormal trajectory in the trajectory database. To overcome this limitation, we propose an abnormal trajectory detection method - called TADSS - to find the hidden abnormal trajectory by using a comprehensive measurement. Firstly, we employ three kernel functions to measure the time, velocity and position feature values of trajectory data, where the kernel functions extract semantic feature of the position, time feature of trajectory, and velocity feature of object motion from each trajectory data. Secondly, we propose a feature fusion strategy to measure the similarity of trajectory data, where we assign weights to the above kernel functions and then use a linear combination approach to fuse the weighted kernel functions. Thirdly, we build a trajectory feature graph by using the above fused kernel functions, and then divide the trajectory feature graph into a plurality of subgraphs by using a conventional graph clustering technique. Last, we propose a sparse subgraph method to detect abnormal trajectory, where a novel weight coefficient concept is used to distinguish sparse subgraph. Experimental results driven by both the vehicle trajectory data of Shanghai city and the Atlantic hurricane data demonstrate the performance of our TADSS.

A hybrid OpenFlow with intelligent detection and prediction models for preventing BGP path hijack on SDN

The Border Gateway Protocol (BGP) is a path vector protocol whose fundamental aim is to exchange the information across the Internet, which directs data between autonomous systems. The significant drawback of the BGP is that it does not address security; path hijacking is one of the top-rated cyber hijacks. Existing methods such as sBGP, soBGP and PGBGP have focused more on detecting path hijacking rather than preventing. Hence, we propose an intelligent model to detect abnormal behavior of a network and to predict and prevent BGP path hijacking (DPPBGP) in software-defined networks. The main objective of our proposed model is to reduce detection time and the controller workload with SFlow-integrated OpenFlow. Three modules of our model are as follows: (1) Based on the abnormal behavior of the network, we evaluated the statistics. We use the statistic features in the cumulative sum abnormal detection algorithm to detect abnormal behavior and flows proficiently and perfectly with less detection time. (2) An intelligent machine learning approach knows as a Pattern Sequence Forecasting algorithm is used to forecast the behavior of the network. (3) After the detection or the forecast of abnormality, path hijack is prevented by killing the appropriate PID based on SFlow analyzer. Simulation results show how large the network of this model can perform accurately and effectively.

Anomaly Detection Algorithm Based On Electric Equipment

Traditional methods for detecting data anomalies of power equipment fail to fully mine the data characteristics. And it has shortcomings such as complex calculation, poor flexibility and low accuracy. To solve the problem of abnormal fault detection of electric equipment, the abnormal detection algorithm based on statistical analysis and machine learning is used in the paper. The reconstruction method based on Long Short Term Memory (LSTM) time sequence is proposed for time series data abnormal detection. Experiments show that the new method is effective in detecting and correcting abnormal data, which reduces the detection time and improves the accuracy of state estimation results. There are huge differences within the positive samples in anomaly detection, so several methods are studied in the paper. One-class SVM algorithm is often used in novelty detection and isolation forest algorithm is used in outlier detection. Machine learning methods which is based on statistical analysis will play an increasingly important role in the fields of equipment monitoring and predictive maintenance, safety of electric equipment.

Analyzing multiple types of behaviors from traffic videos via nonparametric topic model

Intelligent video surveillance systems have garnered substantial research attention in recent years within the transportation surveillance field. The systems can assist in identifying activities, interactions, and abnormal behaviors of individuals in traffic. We propose a novel unsupervised learning framework based on a two-layer BNBP-PFA topic model to simultaneously model multiple types of behaviors in crowded and complicated traffic videos. We provide the model’s structure, its inference algorithm, and design a corresponding likelihood function based on an abnormality detection algorithm. Compared to similar existing algorithms, ours readily reveals both the local topic-motion pattern and the global topic-traffic pattern. Comparative experiments on two public traffic video datasets show that our model outperforms the state-of-art algorithms in regards to effective topic discovery and abnormality detection.

Anomaly Detection Based on PMF Encoding and Adversarially Learned Inference

In order to solve the problem of increasing the dimension and sparse feature space caused by the categorization coding method in the existing abnormal traffic detection problem, a coding method based on Probability Mass Function (PMF) is proposed. Secondly, in order to improve the ability of abnormal traffic detection algorithms to identify unknown attack type data and improve detection efficiency, we use Adversarially Learned Inference as the basic detection algorithm. The comparison experiments on the standard dataset show that the proposed method has improved the accuracy and detection efficiency greatly compared with the existing anomaly detection methods.

Abnormal behavior detection based on dense trajectories

In this paper, we propose abnormal behavior detection algorithms based on dense trajectories and principal components for video surveillance applications. The result shows that the proposed algorithms are faster than an algorithm based on lengths of displacement vectors but the accuracy is only retained if the bag-of-features model is trained on a balanced sample of behavior features.

Abnormal Detection of Electric Management System based on Spatial-temporal User Profile

Abnormal detection of user behavior is essential for Electric Management Information System to ensure its stable operations. In this paper, we propose a user profile based abnormal behavior detection algorithm based on spatial-temporal data mining. First, the user log recording the operation and corresponding timestamps are generated and aggregated according to the user catalog. Then the semantic tags to describe the users are defined and detected from the user log files. Next user profile is generated based on the detected semantic tags. Finally, we detect the abnormal behavior by comparing the user profile with the profile of the group that user belongs to. The algorithm proposed in this paper can detect the regular abnormal behavior such as working late or working overtime, which is proofed by experiment results.

ECG Abnormality Detection Algorithm

The monitoring and early detection of abnormalities in the cardiac cycle morphology have significant impact on the prevention of heart diseases and their associated complications. Electrocardiogram (ECG) is very effective in detecting irregularities of the heart muscle functionality. In this work, we investigate the detection of possible abnormalities in ECG signal and the identification of the corresponding heart disease in real-time using an efficient algorithm. The algorithm relies on cross-correlation theory to detect abnormalities in ECG signal. The algorithm incorporates two cross-correlations steps. The first step detects abnormality in a real-time ECG signal trace while the second step identifies the corresponding disease. The optimization of search-time is the main advantage of this algorithm.