OPTIMIZED DEEP AUTOENCODER WITH L1 REGULARIZATION AND DROPOUT FOR ANOMALY DETECTION IN 6G NETWORK SLICING

The l1 regularization technique has been developed for damage detection by utilizing the sparsity feature of structural damage. However, the sensitivity matrix in the damage identification exhibits a strong correlation structure, which does not suffice the independency criteria of the l1 regularization technique. This study employs the elastic net method to solve the problem by combining the l1 and l2 regularization techniques. Moreover, the proposed method enables the grouped structural damage being identified simultaneously, whereas the l1 regularization cannot. A numerical cantilever beam and an experimental three-story frame are utilized to demonstrate the effectiveness of the proposed method. The results showed that the proposed method is able to accurately locate and quantify the single and multiple damages, even when the number of measurement data is much less than the number of elements. In particular, the present elastic net technique can detect the grouped damaged elements accurately, whilst the l1 regularization method cannot.

Video anomaly detection by fusing self-attention and autoencoder

Sensitivity and specificity are foundational metrics for cancer detection tools. However, most machine learning algorithms prioritize overall accuracy during optimization, which fails to align with clinical priorities of early detection. We aim to develop a feature selection machine learning algorithm while maximizing sensitivity at a given specificity. We developed SMAGS-LASSO, a machine learning algorithm that combines our developed Sensitivity Maximization at a Given Specificity (SMAGS) framework with L1 regularization for feature selection. This approach simultaneously optimizes sensitivity at user-defined specificity thresholds while performing feature selection. SMAGS-LASSO utilizes a custom loss function with L1 regularization and multiple parallel optimization techniques. We used train-test splits and cross-validation, comparing against LASSO and Random Forest using sensitivity and AUC metrics. We evaluated our method on synthetic datasets and real-world protein colorectal cancer biomarker data. In synthetic datasets designed to contain strong signals for both sensitivity and specificity, SMAGS-LASSO significantly outperformed standard LASSO, achieving sensitivity of 1.00 (95% CI: 0.98-1.00) compared to 0.19 (95% CI: 0.13-0.23) for LASSO at 99.9% specificity. In colorectal cancer data, SMAGS-LASSO demonstrated 21.8% improvement over LASSO (p-value = 2.24E-04) and 38.5% over Random Forest (p-value = 4.62E-08) at 98.5% specificity while selecting the same number of biomarkers. SMAGS-LASSO enables development of minimal biomarker panels that maintain high sensitivity at predefined specificity thresholds, offering superior performance for early cancer detection. This method provides a promising approach for early cancer detection and other medical diagnostics requiring sensitivity-specificity optimization.

Anomaly and outlier detection is a long-standing problem in machine learning. In some cases, anomaly detection is easy, such as when data are drawn from well-characterized distributions such as the Gaussian. However, when data occupy high-dimensional spaces, anomaly detection becomes more difficult. We present CLAM (Clustered Learning of Approximate Manifolds), a manifold mapping technique in any metric space. CLAM begins with a fast hierarchical clustering technique and then induces a graph from the cluster tree, based on overlapping clusters as selected using several geometric and topological features. Using these graphs, we implement CHAODA (Clustered Hierarchical Anomaly and Outlier Detection Algorithms), exploring various properties of the graphs and their constituent clusters to find outliers. CHAODA employs a form of transfer learning based on a training set of datasets, and applies this knowledge to a separate test set of datasets of different cardinalities, dimensionalities, and domains. On 24 publicly available datasets, we compare CHAODA (by measure of ROC AUC) to a variety of state-of-the-art unsupervised anomaly-detection algorithms. Six of the datasets are used for training. CHAODA outperforms other approaches on 16 of the remaining 18 datasets. CLAM and CHAODA scale to large, high-dimensional "big data" anomaly-detection problems, and generalize across datasets and distance functions. Source code to CLAM and CHAODA are freely available on GitHub <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

Anomaly detection is one of the most important methods to process and find abnormal data, as this method can distinguish between normal and abnormal behaviour. Anomaly detection has been applied in many areas such as the medical sector, fraud detection in finance, fault detection in machines, intrusion detection in networks, surveillance systems for security, as well as forensic investigations. Abnormal behaviour can give information or answer questions when an investigator is performing an investigation. Anomaly detection is one way to simplify big data by focusing on data that have been grouped or clustered by the anomaly detection method. Forensic data usually consists of heterogeneous data which have several data forms or types such as qualitative or quantitative, structured or unstructured, and primary or secondary. For example, when a crime takes place, the evidence can be in the form of various types of data. The combination of all the data types can produce rich information insights. Nowadays, data has become ‘big’ because it is generated every second of every day and processing has become time-consuming and tedious. Therefore, in this study, a new method to detect abnormal behaviour is proposed using heterogeneous data and combining the data using data fusion technique. Vast challenge data and image data are applied to demonstrate the heterogeneous data. The first contribution in this study is applying the heterogeneous data to detect an anomaly. The recently introduced anomaly detection technique which is known as Empirical Data Analytics (EDA) is applied to detect the abnormal behaviour based on the data sets. Standardised eccentricity (a newly introduced within EDA measure offering a new simplified form of the well-known Chebyshev Inequality) can be applied to any data distribution. Then, the second contribution is applying image data. The image data is processed using pre-trained deep learning network, and classification is done using a support vector machine (SVM). After that, the last contribution is combining anomaly result from heterogeneous data and image recognition using new data fusion technique. There are five types of data with three different modalities and different dimensionalities. The data cannot be simply combined and integrated. Therefore, the new data fusion technique first analyses the abnormality in each data type separately and determines the degree of suspicious between 0 and 1 and sums up all the degrees of suspicion data afterwards. This method is not intended to be a fully automatic system that resolves investigations, which would likely be unacceptable in any case. The aim is rather to simplify the role of the humans so that they can focus on a small number of cases to be looked in more detail. The proposed approach does simplify the processing of such huge amounts of data. Later, this method can assist human experts in their investigations and making final decisions.

Abnormal ship behavior detection is essential for maritime navigation safety. Most existing abnormal ship behavior detection methods only build A ship trajectory position outlier detection model; however, the construction of a ship speed outlier detection model is also significant for maritime navigation safety. In addition, in most existing methods for detecting a ship's abnormal behavior based on abnormal thresholds, one unsuitable threshold leads to the risk of the ship not being minimized as much as possible. In this paper, we proposed an abnormal ship behavior detection method based on distance measurement and an isolation mechanism. First, to address the problem of traditional trajectory compression methods and density clustering methods only using ship position information, the minimum description length principle based on acceleration (AMDL) algorithm and Multi-Dimensional Density Clustering (MDDBSCAN) algorithm is used in this study. These algorithms not only considered the position information of the ship, but also the speed information. Second, regarding the issue of the difficulty in determining the anomaly threshold, one method for determining the anomaly threshold based on the relationship between the velocity weights and noise points of the MDDBSCAN algorithm has been introduced. Finally, due to the randomness issue of the selected segmentation value in iForest, a strategy of selectively constructing isolated trees was proposed, thus further improving the efficiency of abnormal ship behavior detection. The experimental results on the historical automatic identification system data set of Xiamen port prove the practicality and effectiveness of our proposed method. Our experiment results show that the proposed method achieves an improvement of about 10% over the trajectory outlier detection based on the local outlier fraction method, about 14% over the isolation-based online anomalous trajectory method in terms of the accuracy of ship position information anomaly detection, and about 3% over the feature fusion method in terms of the accuracy of ship speed anomaly detection. This method improves algorithm efficiency by about 5% compared to the traditional isolation forest anomaly detection algorithm.

Background: In the last decade, numerous methods have been proposed to define and detect outliers, particularly in complex environments like networks, where anomalies significantly deviate from normal patterns. Although defining a clear standard is challenging, anomaly detection systems have become essential for network administrators to efficiently identify and resolve irregularities. Methods: This study develops and evaluates a machine learning-based system for network anomaly detection, focusing on point anomalies within network traffic. It employs both unsupervised and supervised learning techniques, including change point detection, clustering, and classification models, to identify anomalies. SHAP values are utilized to enhance model interpretability. Results: Unsupervised models effectively captured temporal patterns, while supervised models, particularly Random Forest (94.3%), demonstrated high accuracy in classifying anomalies, closely approximating the actual anomaly rate. Conclusions: Experimental results indicate that the system can accurately predict network anomalies in advance. Congestion and packet loss were identified as key factors in anomaly detection. This study demonstrates the potential for real-world deployment of the anomaly detection system to validate its scalability.

The autonomous and adaptable identification of anomalies in industrial contexts, particularly in the physical processes of Cyber-Physical Production Systems (CPPS), requires using critical technologies to identify failures correctly. Most of the existing solutions in the anomaly detection research area do not consider such systems’ dynamics. Due to the complexity and multidimensionality of CPPS, a scalable, adaptable, and rapid anomaly detection system is needed, considering the new design specifications of Industry 4.0 solutions. Immune-based models, such as the Dendritic Cell Algorithm (DCA), may provide a rich source of inspiration for detecting anomalies, since the anomaly detection problem in CPPS greatly resembles the functionality of the biological dendritic cells in defending the human body from hazardous pathogens. This paper tackles DCA limitations that may compromise its usage in anomaly detection applications, such as the manual characterization of safe and danger signals, data analysis not suitable for online classification, and the lack of an object-oriented implementation of the algorithm. The proposed approach, the Cursory Dendritic Cell Algorithm (CDCA), is a novel variation of the DCA, developed to be flexible and monitor physical industrial processes continually while detecting anomalies in an online fashion. This work’s contribution is threefold. First, it provides a comprehensive review of Artificial Immune Systems (AIS), focusing on AIS applied to the anomaly detection problem. Then, a new object-oriented architecture for the DCA implementation is described, enabling the modularity and abstraction of the algorithm stages into different classes (modules). Finally, the CDCA for the anomaly detection problem is proposed. The CDCA was successfully validated in two industrial-oriented dataset benchmarks for physical anomaly and network intrusion detection, the Skoltech Anomaly Benchmark (SKAB) and M2M using OPC UA. When compared to other algorithms, the proposed approach exhibits promising classification results. It was placed fourth on the SKAB scoreboard and presented a competitive performance with the incremental Dendritic Cell Algorithm (iDCA).

Optimisation of artefact detection in photoplethysmography heart rate data: influence of different classifiers in machine learning models

Ensuring pedestrian safety in walkways gains more significance in urban surroundings. Anomaly detection (AD) approaches had a crucial role in assuring pedestrian safety in pathways, where abnormal events or behaviours can give rise to potential risks. To attain this, potential AD and object detection approaches are essential to identify abnormal behaviours and potential hazards in real-time. This survey presents a complete overview of AD and object detection algorithms particularly devised for pedestrian walkways. The survey starts by discussing the importance of pedestrian safety and the difficulties related to identifying anomalies and objects in walkway settings. Then it delivers a comprehensive analysis of different object detection algorithms that includes existing deep learning methods and classical computer vision (CV) based methods. The applications, strengths, and limitations of all methods are inspected, emphasizing their potentiality in pedestrian tracking and detection. Moreover, the survey explored AD approach that intends to determine abnormal events or behaviours in walkways. Such methods have machine learning-based methods, statistical modeling, and AD methods designed for pedestrian-centric situations. The survey inspects their respective pros, cons, and real-time applications. The survey concluded by discussing future directions and emerging trends in object detection and AD for pedestrian safety in walkways.

Detection of anomalies in video surveillance plays a key role in ensuring the safety and security of public spaces. The number of surveillance cameras is growing, making it harder to monitor them manually. So, automated systems are needed. This change increases the demand for automated systems that detect abnormal events or anomalies, such as road accidents, fighting, snatching, car fires, and explosions in real-time. These systems improve detection accuracy, minimize human error, and make security operations more efficient. In this study, we proposed the Composite Recurrent Bi-Attention (CRBA) model for detecting anomalies in surveillance videos. The CRBA model combines DenseNet201 for robust spatial feature extraction with BiLSTM networks that capture temporal dependencies across video frames. A multi-attention mechanism was also incorporated to direct the model’s focus to critical spatiotemporal regions. This improves the system’s ability to distinguish between normal and abnormal behaviors. By integrating these methodologies, the CRBA model improves the detection and classification of anomalies in surveillance videos, effectively addressing both spatial and temporal challenges. Experimental assessments demonstrate that the CRBA model achieves high accuracy on both the University of Central Florida (UCF) and the newly developed Road Anomaly Dataset (RAD). This model enhances detection accuracy while also improving resource efficiency and minimizing response times in critical situations. These advantages make it an invaluable tool for public safety and security operations, where rapid and accurate responses are needed for maintaining safety.

Context: Many software systems are highly configurable. Different configuration options could lead to varying performances of the system. It is difficult to measure system performance in the presence of an exponential number of possible combinations of these options.Goal: Predicting software performance by using a small configuration sample.Method: This paper proposes Perf-AL to address this problem via adversarial learning. Specifically, we use a generative network combined with several different regularization techniques (L1 regularization, L2 regularization and a dropout technique) to output predicted values as close to the ground truth labels as possible. With the use of adversarial learning, our network identifies and distinguishes the predicted values of the generator network from the ground truth value distribution. The generator and the discriminator compete with each other by refining the prediction model iteratively until its predicted values converge towards the ground truth distribution.Results: We argue that (i) the proposed method can achieve the same level of prediction accuracy, but with a smaller number of training samples. (ii) Our proposed model using seven real-world datasets show that our approach outperforms the state-of-the-art methods. This help to further promote software configurable performance.Conclusion: Experimental results on seven public real-world datasets demonstrate that PERF-AL outperforms state-of-the-art software performance prediction methods.

Abnormal behaviors in industrial systems may be early warnings on critical events that may cause severe damages to facilities and security. Thus, it is important to detect abnormal behaviors accurately and timely. However, the anomaly detection problem is hard to solve in practice, mainly due to the rareness and the expensive cost to get the labels of the anomalies. Deep generative models parameterized by neural networks have achieved state-of-the-art performance in practice for many unsupervised and semisupervised learning tasks. We present a new deep generative model, Latent Enhanced regression/classification Deep Generative Model (LEDGM), for the anomaly detection problem with multidimensional data. Instead of using two-stage decoupled models, we adopt an end-to-end learning paradigm. Instead of conditioning the latent on the class label, LEDGM conditions the label prediction on the learned latent so that the optimization goal is more in favor of better anomaly detection than better reconstruction that the previously proposed deep generative models have been trained for. Experimental results on several synthetic and real-world small- and large-scale datasets demonstrate that LEDGM can achieve improved anomaly detection performance on multidimensional data with very sparse labels. The results also suggest that both labeled anomalies and labeled normal are valuable for semisupervised learning. Generally, our results show that better performance can be achieved with more labeled data. The ablation experiments show that both the original input and the learned latent provide meaningful information for LEDGM to achieve high performance.

Abnormal ship navigation behaviors in cross-sea bridge waters pose significant threats to maritime safety, creating a critical need for accurate anomaly detection methods. Ship AIS trajectory data contain complex temporal features but often lack explicit labels. Most existing anomaly detection methods heavily rely on labeled or semi-supervised data, thus limiting their applicability in scenarios involving completely unlabeled ship trajectory data. Furthermore, these methods struggle to capture long-term temporal dependencies inherent in trajectory data. To address these limitations, this study proposes an unsupervised trajectory anomaly detection model combining a transformer architecture with a variational autoencoder (transformer–VAE). By training on large volumes of unlabeled normal trajectory data, the transformer–VAE employs a multi-head self-attention mechanism to model both local and global temporal relationships within the latent feature space. This approach significantly enhances the model’s ability to learn and reconstruct normal trajectory patterns, with reconstruction errors serving as the criterion for anomaly detection. Experimental results show that the transformer–VAE outperforms conventional VAE and LSTM–VAE in reconstruction accuracy and achieves better detection balance and robustness compared to LSTM–-VAE and transformer–GAN in anomaly detection. The model effectively identifies abnormal behaviors such as sudden changes in speed, heading, and trajectory deviation under fully unsupervised conditions. Preliminary experiments using the POT method validate the feasibility of dynamic thresholding, enhancing the model’s adaptability in complex maritime environments. Overall, the proposed approach enables early identification and proactive warning of potential risks, contributing to improved maritime traffic safety.

The surge of constantly evolving network attacks can be addressed by designing an effective and efficient Intrusion Detection System (IDS). Various Deep Learning (DL) techniques have been used for designing intelligent IDS. However, DL techniques face an issue of overfitting because of complex network structure and high-dimensional data sets. Dropout and regularization are two competently perceived concepts of DL used for handling overfitting issue to enhance the performance of DL techniques. In this paper, we aim to apply fusion of various regularization techniques, namely, L1, L2, and elastic net regularization, with dropout regularization technique, for analyzing and enhancing the performance of Deep Neural Network (DNN)-based IDS. Experiments are performed using NSL-KDD, UNSW_NB-15, and CIC-IDS-2017 data sets. The value of dropout probability is derived using GridSearchCV-based hyperparameter optimization technique. Moreover, the paper also implements state-of-the-art Machine Learning techniques for the performance comparison. Apart from DNN, we have also presented performance analysis of various DL techniques, namely, Recurrent Neural Network, Long Short-Term Memory, Gated Recurrent Unit, and Convolutional Neural Network using a fusion of regularization techniques for intrusion detection and classification. The empirical study shows that among the techniques implemented, dropout has proved to be more effective compared with L1, L2, and elastic net regularization. Moreover, fusion of dropout with other regularization techniques achieved better results compared with L1 regularization, L2 regularization, and elastic net regularization, individually. The techniques implemented for DNN-based IDS are also statistically tested using the Wilcoxon signed-rank test.