Feature Drift Research Articles

Dynamic feature selection model for adaptive cross site scripting attack detection using developed multi-agent deep Q learning model

Web applications’ popularity has raised attention in various service domains, which increased the concern about cyber-attacks. One of these most serious and frequent web application attacks is a Cross-site scripting attack (XSS). It causes grievous harm to victims. Existing security methods against XSS fail due to the evolving nature of XSS attacks. One evolving aspect of XSS attacks is feature drift which changes the feature relevancy and causes degradation in the performance. Unfortunately, dynamic awareness of drift occurrence is missing. Thus, this study attempts to fill the gap by proposing a feature drift-aware algorithm for detecting the evolved XSS attacks. The proposed approach is a dynamic feature selection based on a deep Q-network multi-agent feature selection (DQN-MAFS) framework. Each agent is associated with one feature and is responsible for selecting or deselecting its feature. DQN-MAFS provides a sub-model for reward distribution over agents, which is named as fair agent reward distribution based dynamic feature selection FARD-DFS. This framework is capable of supporting real-time, dynamic updates and adjustment of embedded knowledge as long as new labelled data arrives. DQN-MAFS has been evaluated using four real XSS attack datasets with various feature length sizes. The evaluation process was conducted and compared with state-of-the-art works. The obtained results show the superiority of our FARD-DFS over the benchmarks in terms of the majority of metrics. The improvement percentages of the mean accuracy and F1-measure ranged from 1.01% to 12.1% and from 0.55% to 6.88%, respectively, in comparison with the benchmarks. This approach can be deployed as an autonomous detection system without the need for any offline retraining process of the model to detect the evolved XSS attack.

Feature Drift in Fake News Detection: An Interpretable Analysis

In recent years, fake news detection and its characteristics have attracted a number of researchers. However, most detection algorithms are driven by data rather than theories, which causes the existing approaches to only perform well on specific datasets. To the extreme, several features only perform well on specific datasets. In this study, we first define the feature drift in fake news detection methods, and then demonstrate the existence of feature drift and use interpretable models (i.e., Shapley Additive Explanations and Partial Dependency Plots) to verify the feature drift. Furthermore, by controlling the distribution of tweets’ creation times, a novel sampling method is proposed to explain the reason for feature drift. Finally, the Anchors method is used in this paper as a supplementary interpretation to exhibit the potential characteristics of feature drift further. Our work provides deep insights into the temporal patterns of fake news detection, proving that the model’s performance is also highly related to the distribution of datasets.

Robustness benchmark for unsupervised anomaly detection models

Due to the complexity and diversity of production environments, it is essential to understand the robustness of unsupervised anomaly detection models to common corruptions. To explore this issue systematically, we propose a dataset named MVTec-C to evaluate the robustness of unsupervised anomaly detection models. Based on this dataset, we explore the robustness of approaches in five paradigms, namely, reconstruction-based, representation similarity-based, normalizing flow-based, self-supervised representation learning-based, and knowledge distillation-based paradigms. Furthermore, we explore the impact of different modules within two optimal methods on robustness and accuracy. This includes the multi-scale features, the neighborhood size, and the sampling ratio in the PatchCore method, as well as the multi-scale features, the MMF module, the OCE module, and the multi-scale distillation in the Reverse Distillation method. Finally, we propose a feature alignment module (FAM) to reduce the feature drift caused by corruptions and combine PatchCore and the FAM to obtain a model with both high performance and high accuracy. We hope this work will serve as an evaluation method and provide experience in building robust anomaly detection models in the future.

Feature Drift Aware for Intrusion Detection System Using Developed Variable Length Particle Swarm Optimization in Data Stream

Feature Drift Aware for Intrusion Detection System Using Developed Variable Length Particle Swarm Optimization in Data Stream

Visual Odometry With Point and Line Features Based on Underground Tunnel Environment

Compared with the interior environment, there exists a lot of noise and dust in an underground tunnel, and the light is unstable, which is difficult to track by the direct method. Moreover, errors in line segment projection and line feature drift under the influence of light can lead to significant deviations in the odometry. Consequently, to improve the accuracy and robustness of visual odometry, a point-line feature stereo visual odometry system is proposed in this paper. The system combines ORB features and LSD line features; using the angle relationship of line projection, we propose a new method for calculating the reprojection error of line features, reconstruct the reprojection model based on line features, and construct a new reprojection error model based on point-line features, which adds an angle constraint to the reprojection of line features and solves the instability caused by line projection error. It is shown in our experiments on the KITTI, New College dataset, that the translation error of our system is reduced by about 40% on average compared to PLVO, with a reduction in relative positioning error. Experiments in the hallway and underground tunnel environments have shown that the maximum positioning error of our system has been reduced by 75% in hallways and by 56.7% in an underground tunnel. Therefore, our algorithm effectively improves the localization accuracy and is more advantageous in low-texture environments.

Health Assessment of Rotating Equipment With Unseen Conditions Using Adversarial Domain Generalization Toward Self-Supervised Regularization Learning

Traditional health assessment models work well under the assumption that the test and training samples obey a similar distribution. However, it is practically impossible to eliminate domain shifts between different tasks. Thus, most work tries to establish a data-driven approach via domain adaptation to accomplish transfer learning between different operating conditions. Sufficient target data are needed to participate in the training, which may not normally be available due to most working scenarios being unseen. An adversarial domain generalization framework with regularization learning (ADGR) is proposed for the health assessment to mine latent domains. Also, the latent domain is expanded to the unseen domain as possible. More specifically, the diversity of the sample distribution is augmented by adversarial training and the maximization of the domain discrepancy between the latent and source domains. Meanwhile, self-supervised interdomain regularization and semantical consistent regularization are proposed to mitigate the feature drift of the domain classifier and semantic divergence between source and latent domains. The case study shows that the ADGR-based health assessment approach achieves competitive prediction accuracy under unseen conditions, demonstrating its potential as a diagnostic solution.

AUV Drift Track Prediction Method Based on a Modified Neural Network

Since AUV works in the complex marine environment without cable and unmanned, there will be a loss of contact when an accident occurs. It is necessary to carry out research on the drift track prediction of AUV for the sake of salvage and recovery of the AUV in time. It is worth noting that the volume of AUV is small, and the drift track changes significantly when it is affected by the marine environment. Consequently, when the AUV drifts to different ocean layers, there will be a feature drift problem which will lead to a significant drop in the prediction accuracy. In this paper, a new method of AUV drift track prediction is proposed. Inspired by the human emotion modulation mechanism in psychology, a modified neural network (ECRNet) is proposed to correct the prediction error in different ocean layers. Through experimental verification, the network reduces the prediction error and achieves a better prediction performance.

RffAe-S: Autoencoder Based on Random Fourier Feature With Separable Loss for Unsupervised Signal Modulation Clustering

Unsupervised signal modulation clustering is becoming increasingly important due to its application in the dynamic spectrum access process of 5G wireless communication and threat detection at the physical layer of Internet of Things. The need for better clustering results makes it a challenge to avoid feature drift and improve feature separability. This article proposes a novel separable loss function to address the issue. Besides, the high-level semantic properties of modulation types make it difficult for networks to extract their features. An autoencoder structure based on the random Fourier feature (RffAe) is proposed to simulate the demodulation process of unknown signals. Combined with the separable loss of RffAe (RffAe-S), it has excellent feature extraction ability. Great experiments were carried out on RADIOML 2016.10 A and RADIOML 2016.10 B. Experimental evaluations on these datasets show that our approach RffAe-S achieves state-of-the-art results compared to classical and the most relevant deep clustering methods.

Core Surface Flow Changes Associated With the 2017 Pacific Geomagnetic Jerk.

A geomagnetic jerk was seen in Swarm satellite data in 2017 over the Pacific region. We invert time series of spatial gradient secular variation data between 2014 and 2020, reduced to a grid of points at satellite altitude, for spatially‐ and temporally‐regularized core surface flow. Pacific region flow acceleration was almost constant before and after the jerk, with a sharp change, especially in the azimuthal component, at the jerk epoch, despite the temporal regularization. Azimuthal acceleration is oppositely signed either side of 160°W, where it effectively vanishes, and also reverses sign at the jerk epoch. Acceleration features drift westward at about 900 km year−1. Unlike previous studies, the evidence presented here for low latitude waves does not depend on imposing flow equatorial symmetry, quasi‐ or tangential geostrophy, or band‐pass filtering, and has no reliance on stochastic models or numerical simulations.

Adaptive PCA-based feature drift detection using statistical measure

Adaptive PCA-based feature drift detection using statistical measure

DeTarNet: Decoupling Translation and Rotation by Siamese Network for Point Cloud Registration

Point cloud registration is a fundamental step for many tasks. In this paper, we propose a neural network named DetarNet to decouple the translation t and rotation R, so as to overcome the performance degradation due to their mutual interference in point cloud registration. First, a Siamese Network based Progressive and Coherent Feature Drift (PCFD) module is proposed to align the source and target points in high-dimensional feature space, and accurately recover translation from the alignment process. Then we propose a Consensus Encoding Unit (CEU) to construct more distinguishable features for a set of putative correspondences. After that, a Spatial and Channel Attention (SCA) block is adopted to build a classification network for finding good correspondences. Finally, the rotation is obtained by Singular Value Decomposition (SVD). In this way, the proposed network decouples the estimation of translation and rotation, resulting in better performance for both of them. Experimental results demonstrate that the proposed DetarNet improves registration performance on both indoor and outdoor scenes. Our code will be available in https://github.com/ZhiChen902/DetarNet.

Contrastive Quantization with Code Memory for Unsupervised Image Retrieval

The high efficiency in computation and storage makes hashing (including binary hashing and quantization) a common strategy in large-scale retrieval systems. To alleviate the reliance on expensive annotations, unsupervised deep hashing becomes an important research problem. This paper provides a novel solution to unsupervised deep quantization, namely Contrastive Quantization with Code Memory (MeCoQ). Different from existing reconstruction-based strategies, we learn unsupervised binary descriptors by contrastive learning, which can better capture discriminative visual semantics. Besides, we uncover that codeword diversity regularization is critical to prevent contrastive learning-based quantization from model degeneration. Moreover, we introduce a novel quantization code memory module that boosts contrastive learning with lower feature drift than conventional feature memories. Extensive experiments on benchmark datasets show that MeCoQ outperforms state-of-the-art methods. Code and configurations are publicly released.

Dreaming Differently

Dreaming Differently

Identifying gene expression-based biomarkers in online learning environments.

Gene expression-based classifiers are often developed using historical data by training a model on a small set of patients and a large set of features. Models trained in such a way can be afterwards applied for predicting the output for new unseen patient data. However, very often the accuracy of these models starts to decrease as soon as new data is fed into the trained model. This problem, known as concept drift, complicates the task of learning efficient biomarkers from data and requires special approaches, different from commonly used data mining techniques. Here, we propose an online ensemble learning method to continually validate and adjust gene expression-based biomarker panels over increasing volume of data. We also propose a computational solution to the problem of feature drift where gene expression signatures used to train the classifier become less relevant over time. A benchmark study was conducted to classify the breast tumors into known subtypes by using a large-scale transcriptomic dataset (∼3500 patients), which was obtained by combining two datasets: SCAN-B and TCGA-BRCA. Remarkably, the proposed strategy improves the classification performances of gold-standard biomarker panels (e.g. PAM50, OncotypeDX and Endopredict) by adding features that are clinically relevant. Moreover, test results show that newly discovered biomarker models can retain a high classification accuracy rate when changing the source generating the gene expression profiles. github.com/UEFBiomedicalInformaticsLab/OnlineLearningBD. Supplementary data are available at Bioinformatics Advances online.

Dynamic feature weighting for data streams with distribution-based log-likelihood divergence

Data streams are expected to undergo changes in data distribution, a phenomenon called concept drift. Another closely related phenomenon is the feature drift of data streams. Feature drifts occur whenever a subset of features becomes, or ceases to be, relevant to the learning task. Identifying the most relevant feature subset from a high-dimensional feature space is challenging in the stream mining scenario. In this study, we propose an online dynamic feature weighting algorithm. Specifically, a feature drift detection scheme is introduced that monitors the changes in the class relevance of the features through a change-detection algorithm based on the log-likelihood divergence score. The score is computed via the kernel density estimator based on the information-theoretic feature merit values. The algorithm is evaluated on both synthetic and real-world datasets, and it is shown that the proposed distribution-based drift detection framework can boost the Nearest Neighbor and Naive Bayes classifier accuracy rates (an average of 2.7% for Nearest Neighbor and 5.5% for Naive Bayes). It also signals feature drifts much faster than traditional methods based on detecting changes in accuracy rates. Finally, the limitations of the proposed method are assessed, and future research directions are discussed.

A deep fast learning framework towards exploring Imbalanced data and Multi-class Drift in Evolving Data Streams

Data stream classification poses great challenges in the text based data mining community towards handling evolving data stream. Identification of feature evolution and imbalanced data on the class generated is an important research area for data stream classification on employing of traditional machine learning classifiers. Class evolution and drift is the phenomenon of class emergence and disappearance. Due to class evolutions, performance of the learning model degrades drastically over time. Class evolution problem has been handled on analysis of feature drift and multi class drift. Multi-class drift occur according to probability and time, is categorized as sudden, gradual and recurring drift. Multi class drift has been captured by proposing a new framework in this paper which is named as “Deep Fast Learning Framework”. Initially feature has been extracted using ensemble of technique such as Incremental Kernel Principle Component Analysis, Incremental linear Discriminant analysis and Incremental Linear Principle Component Analysis. These techniques for feature extraction have treated as online feature extraction process. Extracted feature has been processed in the deep fast learning classifier framework which is composed of hybrid ensemble classifiers which follows chuck based ensemble and online ensemble classifiers in parallel on basis of gradual class evolution on block of data on the data streams in form of features. Base learner or classifier has been established using deep neural network to generate the fast learning model on deep analysis of the features obtained and its relationship with existing classes on continuously updating the learner by replacing the older model with newly trained model. Further Base learner will remove the emerging classes which is least utilized and detect the recurring classes on basis of the feature obtained easily. This model is effective in determining the novel classes and recurring class to features which has the possibility of multi class drift. Finally class imbalance problem has been handled on employing under sampling method for base learning model. Experimental results has proved the superiority of the proposed framework on benchmark dataset against state of art approaches on the performance measures such as precision , recall and f measure.

Adaptive Quick Reduct for Feature Drift Detection

Data streams are ubiquitous and related to the proliferation of low-cost mobile devices, sensors, wireless networks and the Internet of Things. While it is well known that complex phenomena are not stationary and exhibit a concept drift when observed for a sufficiently long time, relatively few studies have addressed the related problem of feature drift. In this paper, a variation of the QuickReduct algorithm suitable to process data streams is proposed and tested: it builds an evolving reduct that dynamically selects the relevant features in the stream, removing the redundant ones and adding the newly relevant ones as soon as they become such. Tests on five publicly available datasets with an artificially injected drift have confirmed the effectiveness of the proposed method.

A comparative study of feature selection methods for binary text streams classification

Text streams are a continuous flow of high-dimensional text, transmitted at high-volume and high-velocities. They are expected to be classified in real-time, which is challenging due to the high dimensionality of feature space. Applying feature selection algorithms is one solution to reduce text streams feature space and improve the learning process. However, since text streams are potentially unbounded, it is expected a change in their probabilistic distribution over time, the so-called Concept Drift. The concept drift impacts the feature selection process due to the feature drift when the relevance of features is also subject to changes over time. This paper presents a comparative study of six feature selection methods for binary text streams classification, even in the presence of feature drift. We also propose the Online Feature Selection with Evolving Regularization (OFSER) algorithm, a modified version of the Online Feature Selection (OFS) algorithm, which uses evolving regularization to dynamically penalize model complexity, reducing feature drift impacts on the feature selection process. We conducted the experimental analysis on eleven real-world, commonly used datasets for text classification. The OFSER algorithm showed F1-scores up to 12.92% higher than other algorithms in some cases. The results using Iman and Davenport and Bergmann–Hommel’s tests show that OFSER algorithm is statistically superior to Information Gain and Extremal Feature Selection algorithms in terms of improving the base classifier predictive power.

Evolution of the OH and H2O Maser Emission in the Active Star-Forming Region IRAS 05358+3543 (S231)

We present the results of our observations of the star-forming region located in the Perseus arm and associated with the Sharpless nebula S231 in the lines of H2O at a wavelength of 1.35 cm and OH at a wavelength of 18 cm obtained with the Pushchino 22-m radio telescope (Russia) and the Nancay Large Radio Telescope (France), respectively. We provide a catalog of H2O maser spectra in the period from March 2003 to March 2020. The variability of the integrated flux that we have calculated based on the monitoring by Felli et al. (1987–2007) and our monitoring (2003–2020) has a quasi-cyclic pattern with a cycle duration from 4.3 to 7.7 yr. Flares of maser emission of both single and several features occurred in each activity cycle. The mean rise and decay times of the flare emission were $$0.30 \pm 0.02$$ and $$0.35 \pm 0.02$$ yr, respectively. The observed radial velocity drift and jumps of the H2O emission features can be a consequence of the impact of shocks emerging as the protostar activity rises on them. The complex pattern of variability shows that the H2O masing medium is fragmented and small-scale turbulent motions of matter can occur in it. We have observed the OH maser emission in the 1665 and 1667 MHz main lines and the 1720 MHz satellite line. A large number of OH emission features and their variability have been detected. We have found one Zeeman pair at ‒9.2 km/s with a small splitting, with the longitudinal magnetic field in the period 2008–2020 having decreased monotonically from 0.24 to 0.10 mG. For the three most intense features we have calculated the degrees of linear polarization and position angles. A correlation between the H2O and OH maser activities has been found.

Cluster Prediction Model for Market Basket Analysis: Quest for Better Alternatives to Associative Rule Mining Approach

Market basket analysis seeks to apply association rule mining on the massive sales transaction data. It yields an outcome that either aims to suppress product stock-up unnecessarily and/or product being stock-out. Such decision support system seeks to avoid the unnecessary demurrage and help businesses to keep their customers via better decision and improved service. Market data are time-bound on supply-demand value chain. With customer behavior varying in time, we seek to predict purchase of commonly combined itemset for a next period – so that businesses can better support their decisions via adequate provisions of the required inventory. We use 3-KDD dataset and Delta Mall dataset – adapting a time-clustering algorithm that examines buying behavior of customers, their preferences and frequency with which goods are purchased in common as a basket. Model yields average 162-rules for four-dataset from dataset. Result shows that previous basket items by random customers allow the selection purchase of items of similar value as best combined due to its shelf-placement using the concept of feature drift.