Detection Of Anomalous Behavior Research Articles

Machine Learning in NextG Networks via Generative Adversarial Networks

Generative Adversarial Networks (GANs) are Machine Learning (ML) algorithms that have the ability to address competitive resource allocation problems together with detection and mitigation of anomalous behavior. In this paper, we investigate their use in next-generation (NextG) communications within the context of cognitive networks to address <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i)</i> spectrum sharing, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ii)</i> detecting anomalies, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">iii)</i> mitigating security attacks. GANs have the following advantages. First, they can learn and synthesize field data, which can be costly, time consuming, and nonrepeatable. Second, they enable pre-training classifiers by using semi-supervised data. Third, they facilitate increased resolution. Fourth, they enable the recovery of corrupted bits in the spectrum. The paper provides the basics of GANs, a comparative discussion on different kinds of GANs, performance measures for GANs in computer vision and image processing as well as wireless applications, a number of datasets for wireless applications, performance measures for general classifiers, a survey of the literature on GANs for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i)–iii)</i> above, and future research directions. As a use case of GAN for NextG communications, we show that a GAN can be effectively applied for anomaly detection in signal classification (e.g., user authentication) outperforming another state-of-the-art ML technique, an autoencoder.

A deep learning model based anomalous behavior detection for supporting verifiable access control scheme in cloud servers

Due to the growth of Big Data (BD) storage and access in cloud computing infrastructure, the detection of anomalies for Cloud Servers (CSs) is essential to ensure data confidentiality. Over the past decades, different security systems have been designed based on various methods like encryption, Access Policy (AP) control schemes, signcryption and so on. Among many security systems, a new Improved NTRU (INTRU) decryption based on the AP strategy has been suggested to secure the BD processed by the CSs. Also, the shared secret data was authenticated to defend the clients from anomalies in the cloud. But, the AP upgrade must not degrade the confidentiality of storing information, reveal trust in the CS or cause any different security challenges. It is not considered that such security challenges occur when the data owner shares its data with many CSs. Hence in this article, an INTRU with Detecting Anomalous in CS (INTRU-DACS) system is proposed that employs a deep learning-based Anomaly Detection System (ADS) to handle and secure the BD stored in the CSs. The main goal of this method is to effectively identify the abnormalities in the real world by the conduct utilization, i.e., the System Call Identifier Sequences (SCISs) created from CSs in which these conducts are associated with BD. Initially, effective data summarization is constructed via different feature states to analyze the SCISs of specific durations. After that, an anomaly identification algorithm is proposed to train and test the streaming of raw SC sequences. This observable SCs execution task of CSs is gathered from log files. The variations of such SCISs having a specified duration are random for usual and unusual sequences. So, the fact of current normal and abnormal services is recognized regarding their SCISs. Such normal and abnormal behavioral states are learned from Convolutional Neural Network-Hidden Markov Model (CNNHMM) classifier to identify the anomalies in CSs. But, it is still a challenging process because of the patterns of usual and unusual events. The performance is not effective since it models only the conduct of a huge number of SCISs created from a single CS. As a result, a Secure Access Control Scheme with DACS (SACS-DACS) system is proposed in which a Multidimensional Feature Misbehavior Server Detection method (MFMSD) is introduced for detecting anomalies in multiple CSs. In this method, large-scale SCISs of multiple CSs are extracted, including different features such as network traffic sequence features, CPU energy usage and memory usage from host logs. These extracted multidimensional features are fed to the CNNHMM that identifies the anomalies and maximizes the detection accuracy. At last, the simulation results demonstrate the effectiveness of the SACS-DACS and INTRU-DACS as compared to the INTRU.

Detection of Anomalous Behavior of Smartphone Devices using Changepoint Analysis and Machine Learning Techniques

Detecting anomalous behavior on smartphones is challenging since malware evolution. Other methodologies detect malicious behavior by analyzing static features of the application code or dynamic data samples obtained from hardware or software. Static analysis is prone to code’s obfuscation while dynamic needs that malicious activities to cease to be dormant in the shortest possible time while data samples are collected. Triggering and capturing malicious behavior in data samples in dynamic analysis is challenging since we need to generate an efficient combination of user’s inputs to trigger these malicious activities. We propose a general model which uses a data collector and analyzer to unveil malicious behavior by analyzing the device’s power consumption since this summarizes the changes in software. The data collector uses an automated tool to generate user inputs. The data analyzer uses changepoint analysis to extract features from power consumption and machine learning techniques to train these features. The data analyzer stage contains two methodologies that extract features using parametric and non-parametric changepoint. Our methodologies are efficient in data collection time than a manual method and the data analyzer provides higher accuracy compared to other techniques, reaching over 94% F1-measure for emulated and real malware.

Decision Support System to Classify and Optimize the Energy Efficiency in Smart Buildings: A Data Analytics Approach.

In this paper, an intelligent data analysis method for modeling and optimizing energy efficiency in smart buildings through Data Analytics (DA) is proposed. The objective of this proposal is to provide a Decision Support System (DSS) able to support experts in quantifying and optimizing energy efficiency in smart buildings, as well as reveal insights that support the detection of anomalous behaviors in early stages. Firstly, historical data and Energy Efficiency Indicators (EEIs) of the building are analyzed to extract the knowledge from behavioral patterns of historical data of the building. Then, using this knowledge, a classification method to compare days with different features, seasons and other characteristics is proposed. The resulting clusters are further analyzed, inferring key features to predict and quantify energy efficiency on days with similar features but with potentially different behaviors. Finally, the results reveal some insights able to highlight inefficiencies and correlate anomalous behaviors with EE in the smart building. The approach proposed in this work was tested on the BlueNet building and also integrated with Eugene, a commercial EE tool for optimizing energy consumption in smart buildings.

Anomalous Vehicle Recognition in Smart Urban Traffic Monitoring as an Edge Service

The past decades witnessed an unprecedented urbanization and the proliferation of modern information and communication technologies (ICT), which makes the concept of Smart City feasible. Among various intelligent components, smart urban transportation monitoring is an essential part of smoothly operational smart cities. Although there is fast development of Smart Cities and the growth of Internet of Things (IoT), real-time anomalous behavior detection in Intelligent Transportation Systems (ITS) is still challenging. Because of multiple advanced features including flexibility, safety, and ease of manipulation, quadcopter drones have been widely adopted in many areas, from service improvement to urban surveillance, and data collection for scientific research. In this paper, a Smart Urban traffic Monitoring (SurMon) scheme is proposed employing drones following an edge computing paradigm. A dynamic video stream processing scheme is proposed to meet the requirements of real-time information processing and decision-making at the edge. Specifically, we propose to identify anomalous vehicle behaviors in real time by creatively applying the multidimensional Singular Spectrum Analysis (mSSA) technique in space to detect the different vehicle behaviors on roads. Multiple features of vehicle behaviors are fed into channels of the mSSA procedure. Instead of trying to create and define a database of normal activity patterns of vehicles on the road, the anomaly detection is reformatted as an outlier identifying problem. Then, a cascaded Capsules Network is designed to predict whether the behavior is a violation. An extensive experimental study has been conducted and the results have validated the feasibility and effectiveness of the SurMon scheme.

5G Network Management System With Machine Learning Based Analytics

Application of intelligent data analytics using machine learning in management of 5G networks can enable autonomous networking capabilities in 5G networks. This paper describes the design and implementation of CygNet MaSoN, a management system supporting advanced aggregation and analytics features combined with machine learning. The system supports detection of anomalous network behaviour, detection of degradation in network performance and service quality and also supports resource optimization. The main objective is to achieve self-organizing and closed loop automation functionalities expected as part of autonomous functioning of 5G networks. Details of the system architecture and components are presented. Three real-life use cases implemented on this system are then described. The features considered, machine learning models built and synthetic data generation methods adopted are presented. The results obtained using the MaSoN system are also presented to demonstrate the effectiveness of the system in 5G network operations.

Illegal fishing, anomalous vessel behavior detection through automatic identification system

Border cross fishing and illegal fishing are big problems around the world. Observations show, there are many records for intensive and often illegal fishing of West Africa’s waters by Asian and European hasty that reduce the regular catch for the native people. They are in poverty levels and then illegal fishing impacts rare spices and life in the oceans and threatens the food supply of coastal communities. This paper aims to reduce and decline illegal fishing and cross fishing. Automatic Identification System (AIS) which is collected by satellites and terrestrial receivers, to identify distinct fishing behavior based on the movement of vessels over time. Using the Automatic tracking System(ATS) or AIS each vessels periodically reports information including the Vessel’s identity, type, positions, speed Navigational status and other safety-related information By tracking ship moments, the data is receivers by the Global Fishing watch(GFW) the result will be GFW Detector the vessel’s location. Frequently analyst and terminate the illegal fishing and monitor vessel moments in the oceans.

Anomaly detection method applied to vehicle monitoring

Anomaly detection is a topic studied in the literature for a large number of applications. As more and more data is collected continuously, numerous algorithms have been developed for anomaly detection in time series. In this paper, we first propose a method for time series anomaly detection that can be used in different fields. This method is then applied in a real case study for the detection of anomalous behavior of hybrid trucks. Finally, the results obtained are analyzed to propose lines of improvement.

Intrusion detection in internet of things networks based on machine learning methods

Introduction: The growing amount of digital data generated, among others, by smart devices of the Internet of Things makes it important to study the application of machine learning methods to the detection of network traffic anomalies, namely the presence of network attacks. Purpose: To propose a unified approach to detecting attacks at different levels of IoT network architecture, based on machine learning methods. Results: It was shown that at the wireless sensor network level, attack detection is associated with the detection of anomalous behavior of IoT devices, when the deviation of an IoT device behavior from its profile exceeds a predetermined level. Smart IoT devices are profiled on the basis of statistical characteristics, such as the intensity and duration of packet transmission, the proportion of retransmitted packets, etc. At the level of a local or global wired IoT network, data is aggregated and then analyzed using machine learning methods. Trained classifiers can become a part of a network attack detection system, making decisions about compromising a node on the fly. Models of classifiers of network attacks were experimentally selected both at the level of a wireless sensor network and at the level of a local or global wired network. The best results in terms of completeness and accuracy estimates are demonstrated by the random forest method for a wired local and/or global network and by all the considered methods for a wireless sensor network. Practical relevance: The proposed models of classifiers can be used for developing intrusion detection systems in IoT networks.

Online network monitoring

An important problem in network analysis is the online detection of anomalous behaviour. In this paper, we introduce a network surveillance method bringing together network modelling and statistical process control. Our approach is to apply multivariate control charts based on exponential smoothing and cumulative sums in order to monitor networks generated by temporal exponential random graph models (TERGM). The latter allows us to account for temporal dependence while simultaneously reducing the number of parameters to be monitored. The performance of the considered charts is evaluated by calculating the average run length and the conditional expected delay for both simulated and real data. To justify the decision of using the TERGM to describe network data, some measures of goodness of fit are inspected. We demonstrate the effectiveness of the proposed approach by an empirical application, monitoring daily flights in the United States to detect anomalous patterns.

Research into vessel behaviour pattern recognition in the maritime domain: Past, present and future

A comprehensive literature review on vessel behaviour pattern recognition in maritime and littoral environments, spanning a period of three decades, is presented. A vast majority of research focuses on anomalous vessel behaviour detection from Automated Identification System (AIS) data, using artificial intelligence (AI), Bayesian networks and machine learning based methods. AIS is a cooperative system that employs VHF transceivers to share vessel information through terrestrial and satellite communication networks. To track small boats not fitted with AIS, uncooperative surveillance systems are employed. These systems have been studied in a relatively small number of publications. In vessel behaviour pattern recognition, target classification is critically important, particularly, in asymmetric warfare situations. The paper concludes with a discussion of future outlook in vessel behaviour pattern recognition research.

Anomalous behaviour detection using one-class support vector machine and remote sensing images: a case study of algal bloom occurrence in inland waters

ABSTRACT Algal blooms are a frequent subject in scientific discussions and are the focus of many recent studies, mainly due to their adverse effect on society. Given the lack of ground truth data and the need to develop tools for their detection and monitoring, this research proposes a novel method to automate detection. Concepts derived from multi-temporal image series processing, spectral indices and classification with One-class Support Vector Machine (OC-SVM) are used in this proposal. Imagery from multi-spectral sensors on Landsat-8 and MODIS were acquired through the Google Earth Engine API (GEE API). In order to evaluate our method, two bloom detection case studies (Lake Erie (USA) and Lake Taihu (China)) were performed. Comparisons were made with methods based on spectral index thresholds. Also, to demonstrate the performance of the OC-SVM classifier compared to other machine learning methods, the proposal was adapted to be used with a Random Forest (RF) classifier, having its results added to the analysis. In situ measurements show that the proposed method delivers highly accurate results compared to spectral index thresholding approaches. However, a drawback of the proposal refers to its higher computational cost. The application of the new method to a real-world bloom case is demonstrated.

Anomalous Behavior Detection Framework Using HTM-Based Semantic Folding Technique

Upon the working principles of the human neocortex, the Hierarchical Temporal Memory model has been developed which is a proposed theoretical framework for sequence learning. Both categorical and numerical types of data are handled by HTM. Semantic Folding Theory (SFT) is based on HTM to represent a data stream for processing in the form of sparse distributed representation (SDR). For natural language perception and production, SFT delivers a solid structural background for semantic evidence description to the fundamentals of the semantic foundation during the phase of language learning. Anomalies are the patterns from data streams that do not follow the expected behavior. Any stream of data patterns could have a number of anomaly types. In a data stream, a single pattern or combination of closely related patterns that diverges and deviates from standard, normal, or expected is called a static (spatial) anomaly. A temporal anomaly is a set of unexpected changes between patterns. When a change first appears, this is recorded as an anomaly. If this change looks a number of times, then it is set to a “new normal” and terminated as an anomaly. An HTM system detects the anomaly, and due to continuous learning nature, it quickly learns when they become the new normal. A robust anomalous behavior detection framework using HTM-based SFT for improving decision-making (SDR-ABDF/P2) is a proposed framework or model in this research. The researcher claims that the proposed model would be able to learn the order of several variables continuously in temporal sequences by using an unsupervised learning rule.

A Low-Cost Jamming Detection Approach Using Performance Metrics in Cluster-Based Wireless Sensor Networks.

Wireless Sensor Networks constitute an important part of the Internet of Things, and in a similar way to other wireless technologies, seek competitiveness concerning savings in energy consumption and information availability. These devices (sensors) are typically battery operated and distributed throughout a scenario of particular interest. However, they are prone to interference attacks which we know as jamming. The detection of anomalous behavior in the network is a subject of study where the routing protocol and the nodes increase power consumption, which is detrimental to the network’s performance. In this work, a simple jamming detection algorithm is proposed based on an exhaustive study of performance metrics related to the routing protocol and a significant impact on node energy. With this approach, the proposed algorithm detects areas of affected nodes with minimal energy expenditure. Detection is evaluated for four known cluster-based protocols: PEGASIS, TEEN, LEACH, and HPAR. The experiments analyze the protocols’ performance through the metrics chosen for a jamming detection algorithm. Finally, we conducted real experimentation with the best performing wireless protocols currently used, such as Zigbee and LoRa.

The 51% Attack on Blockchains: A Mining Behavior Study

The applications that use blockchain are cryptocurrencies, decentralized finance applications, video games, and many others. Most of these applications trust that the blockchain will prevent issues like fraud, thanks to the built-in cryptographic mechanisms provided by the data structure and the consensus protocol. However, blockchains suffer from what is called a 51% attack or majority attack, which is considered a high risk for the integrity of these blockchains, where if a miner, or a group of them, has more than half the computing capability of the network, it can rewrite the blockchain. Even though this attack is possible in theory, it is regarded as hard-achievable in practice, due to the assumption that, with enough active members, it is very complicated to have that much computing power; however, this assumption has not been studied with enough detail. In this work, a detailed characterization of the miners in the Bitcoin and Crypto Ethereum blockchains is presented, with the aim of proving the computing distribution assumption and creating profiles that may allow the detection of anomalous behaviors and prevent 51% attacks. The results of the analysis show that, in the last years, there has been an increasing concentration of hash rate power in a very small set of miners, which generates a real risk for current blockchains. Also, that there is a pattern in mining among the main miners, which makes it possible to identify out-of-normal behavior.

A Video Prediction Method Based on Optical Flow Estimation and Pixel Generation

Video prediction has developed rapidly after the booming of deep learning. As an important part of unsupervised representation learning, it plays an important role in anomalous behavior detection, autonomous driving, video games, and other fields. However, the prediction method based on optical flow estimation is susceptible to brightness change and camera shake, and it is difficult to predict occluded objects. While the prediction method based on pixel generation is difficult to fit ambiguous and complex scenes, which leads to a blurry prediction. In this work, we proposed an end-to-end video prediction framework that combines the optical flow estimation module with the pixel generation module by a learnable mask weight to predict high-fidelity videos. To further improve the prediction effect, we introduce adversarial training to the framework. We introduced a frame discriminator and a sequence discriminator to ensure the consistency of the spatio-temporal distribution of predicted video frames and real video frames. The results of experiments on challenging datasets demonstrate the practicability and effectiveness of our proposed video prediction framework. On the one hand, our proposed framework has achieved an equal quality compared with the current latest model, which requires fewer parameters and has a faster prediction speed. On the other hand, the results of ablation experiments demonstrate the effect of fusing different modules and the effectiveness of adversarial training.

Review on anomalous gait behavior detection using machine learning algorithms

A review on anomalous behavior in crime by other researchers is discussed in this study that focused specifically on the linkage between anomalous behaviors. Next, comprehensive reviews related to gait recognition in utilizing machine learning algorithms for detection and recognition of anomalous behavior is elaborated too. The review begins with the conventional approach of gait recognition that includes feature extraction and classification using PCA, OLS, ANN, and SVM. Further, the review focused on utilization of deep learning namely CNN for anomalous gait behavior detection and transfer learning using pre-trained CNNs such as AlexNet, VGG, and a few more. To the extent of our knowledge, very few studies investigated and explored crime related anomalous behavior based on their gaits, hence this will be the next study that we will explore.

Adaptive Performance Anomaly Detection in Distributed Systems Using Online SVMs

Performance anomaly detection is crucial for long running, large scale distributed systems. However, existing works focus on the detection of specific types of anomalies, rely on historical failure data, and cannot adapt to changes in system behavior at run time. In this work, we propose an adaptive framework for the detection and identification of complex anomalous behaviors, such as deadlocks and livelocks, in distributed systems without historical failure data. Our framework employs a two-step process involving two online SVM classifiers on periodically collected system metrics to identify at run time normal and anomalous behaviors such as deadlock, livelock, unwanted synchronization, and memory leaks. Our approach achieves over 0.70 F-score in detecting previously unseen anomalies and 0.78 F-score in identifying the type of known anomalies with a short delay after the anomalies appear, and with minimal expert intervention. Our experimental analysis uses system execution traces from our in-house distributed system with varied behaviors and a dataset by Yahoo!, and shows the benefits of our approach as well as future research challenges.

Early detection of anomalous degradation behavior in lithium-ion batteries

• Identifying ongoing reliability testing samples with anomalous degradation behavior at the earliest stage. • User-dependent confidence levels to determine the anomaly thresholds. • Evaluation of five data-driven methods on two datasets with 35 samples. • Development of an anomaly detection approach that considers an ensemble of methods. Before lithium-ion batteries are purchased in volume, they are typically tested (qualified) to determine if they meet the life-cycle reliability requirements for the targeted applications. To ensure that subsequent production lots of batteries continue to meet the reliability requirements, ongoing reliability testing is often conducted on production lot samples. However, a key challenge is how to quickly determine if the samples have substantially similar reliability as those batteries that were initially qualified, and, in particular, how to detect early signs of unacceptable degradation. This paper uses five data-driven methods (regression model with prediction bound, one-class support vector machine, local outlier factor, Mahalanobis distance, and sequential probability ratio test) to detect anomalous degradation behavior of samples from actual production lots subjected to ongoing reliability tests. An ensemble approach was then developed because it was observed that no single method always gave the earliest warning. The approach can be used by device companies for warranty, recall, and technical decisions.

Detection of Anomalous Behavior in Modern Smartphones Using Software Sensor-Based Data.

This paper describes the steps involved in obtaining a set of relevant data sources and the accompanying method using software-based sensors to detect anomalous behavior in modern smartphones based on machine-learning classifiers. Three classes of models are investigated for classification: logistic regressions, shallow neural nets, and support vector machines. The paper details the design, implementation, and comparative evaluation of all three classes. If necessary, the approach could be extended to other computing devices, if appropriate changes were made to the software infrastructure, based upon mandatory capabilities of the underlying hardware.