Domain Generation Algorithms Detection Research Articles

KDTM: Multi-Stage Knowledge Distillation Transfer Model for Long-Tailed DGA Detection

As the most commonly used attack strategy by Botnets, the Domain Generation Algorithm (DGA) has strong invisibility and variability. Using deep learning models to detect different families of DGA domain names can improve the network defense ability against hackers. However, this task faces an extremely imbalanced sample size among different DGA categories, which leads to low classification accuracy for small sample categories and even classification failure for some categories. To address this issue, we introduce the long-tailed concept and augment the data of small sample categories by transferring pre-trained knowledge. Firstly, we propose the Data Balanced Review Method (DBRM) to reduce the sample size difference between the categories, thus a relatively balanced dataset for transfer learning is generated. Secondly, we propose the Knowledge Transfer Model (KTM) to enhance the knowledge of the small sample categories. KTM uses a multi-stage transfer to transfer weights from the big sample categories to the small sample categories. Furthermore, we propose the Knowledge Distillation Transfer Model (KDTM) to relieve the catastrophic forgetting problem caused by transfer learning, which adds knowledge distillation loss based on the KTM. The experimental results show that KDTM can significantly improve the classification performance of all categories, especially the small sample categories. It can achieve a state-of-the-art macro average F1 score of 84.5%. The robustness of the KDTM model is verified using three DGA datasets that follow the Pareto distributions.

Botnet DGA Domain Name Classification Using Transformer Network with Hybrid Embedding

One of the severest threats to cyber security is botnet, which typically uses domain names generated by Domain Generation Algorithms (DGAs) to communicate with their Command and Control (C&C) infrastructure. DGA detection and classification play an important role of assisting cyber security researchers to detect botnet C&C servers. However, many of the existing DGA detection models only focus on single scale word embedding method, and very few models are specially designed to extract more effective features for DGA detection from multiple scales word embedding. To alleviate above questions, first we propose a hybrid word embedding method, which combines character level embedding and bigram level embedding to make full use of the domain names information, and then, we design a deep neural network with hybrid embedding method to distinguish DGA domains from known legitimate domains. Finally, we evaluate our hybrid embedding method and the proposed model on ONIST dataset and compare our methods with several state-of-the-art DGA classification methods.

Domain generation algorithms detection with feature extraction and Domain Center construction.

Network attacks using Command and Control (C&C) servers have increased significantly. To hide their C&C servers, attackers often use Domain Generation Algorithms (DGA), which automatically generate domain names for C&C servers. Researchers have constructed many unique feature sets and detected DGA domains through machine learning or deep learning models. However, due to the limited features contained in the domain name, the DGA detection results are limited. In order to overcome this problem, the domain name features, the Whois features and the N-gram features are extracted for DGA detection. To obtain the N-gram features, the domain name whitelist and blacklist substring feature sets are constructed. In addition, a deep learning model based on BiLSTM, Attention and CNN is constructed. Additionally, the Domain Center is constructed for fast classification of domain names. Multiple comparative experiment results prove that the proposed model not only gets the best Accuracy, Precision, Recall and F1, but also greatly reduces the detection time.

A Bayesian optimization-based LSTM model for DGA domain name identification approach

In recent years, with the rapid development and rise of mobile Internet, network security issues have also posed a great threat to people. Botnets are an important problem faced by current network security. DNS protocol-based botnets widely use domain generation algorithm (DGA), which can randomly change the domain name to hide itself, and therefore is very likely to threaten people’s network security. In this paper, we use the domain names of the top 1 million websites in the Alexa global ranking as white samples, and for the DGA sample data, we use the open data of 360netlab as black samples. The character sequence model is used for feature extraction, and the LSTM with Bayesian optimization neural network is used to optimize the hyperparameter combination, which finally makes the accuracy of the model above 97%, and the model has superior performance to compare with the conventional model, which can effectively improve the accuracy of DGA detection and recognition.

HAGDetector: Heterogeneous DGA domain name detection model

The botnet relies on the Command and Control (C&C) channels to conduct its malicious activities remotely. The Domain Generation Algorithm (DGA) is often used by botnets to hide their Command and Control (C&C) server and evade take-down attempts, which allows the bot to generate a large number of domain names until it finds its C&C server. The lengths of domain names generated by DGAs are different. Our research finds that the length of the domain name has an impact on the performance of the DGA domain name detection model. In other words, the model is sensitive to the length of the domain name. In this case, attackers can evade detection simply by designing domain names of specific lengths. Moreover, the detection accuracy of DGA domain names still needs to be further improved. To solve these problems, three feature extraction methods adapted to the length of the domain name are proposed in this paper. For extra-short domain names, we use the attention-based method to extract features, which can make use of the character-level semantic feature. For moderate-length domain names, a two-dimensional structure, namely Right Shifted Tensor (RST), is constructed to make the domain name present apparent features similar to images. For the extra-long domain name, the effective classification of domain names can be achieved by manually crafted easy-to-calculate features. Then, different detection structures are designed based on these tree feature extraction methods to form a heterogeneous DGA detection model, namely HAGDetector. In addition, the public suffix is an important part of the domain name. We further analyze the public suffix to evaluate its impact on the detection of DGA domain names. Finally, the experiments are conducted to assess the validity of HAGDetector, as well as compare our approach with the current state-of-the-art and highlight the impact of the domain name length. The experimental results show that our method greatly improves the detection performance.

Robust Botnet DGA Detection: Blending XAI and OSINT for Cyber Threat Intelligence Sharing

We investigated 12 years DNS query logs of our campus network and identified phenomena of malicious botnet domain generation algorithm (DGA) traffic. DGA-based botnets are difficult to detect using cyber threat intelligence (CTI) systems based on blocklists. Artificial intelligence (AI)/machine learning (ML)-based CTI systems are required. This study (1) proposed a model to detect DGA-based traffic based on statistical features with datasets comprising 55 DGA families, (2) discussed how CTI can be expanded with computable CTI paradigm, and (3) described how to improve the explainability of the model outputs by blending explainable AI (XAI) and open-source intelligence (OSINT) for trust problems, an antidote for skepticism to the shared models and preventing automation bias. We define the XAI-OSINT blending as aggregations of OSINT for AI/ML model outcome validation. Experimental results show the effectiveness of our models (96.3% accuracy). Our random forest model provides better robustness against three state-of-the-art DGA adversarial attacks (CharBot, DeepDGA, MaskDGA) compared with character-based deep learning models (Endgame, CMU, NYU, MIT). We demonstrate the sharing mechanism and confirm that the XAI-OSINT blending improves trust for CTI sharing as evidence to validate our proposed computable CTI paradigm to assist security analysts in security operations centers using an automated, explainable OSINT approach (for second opinion). Therefore, the computable CTI reduces manual intervention in critical cybersecurity decision-making.

Evaluating hybrid quantum-classical deep learning for cybersecurity botnet DGA detection

Hybrid quantum machine learning (QML) algorithms have potentials for current quantum computing technologies since only part of the model is computed by a quantum device. This paper proposes a novel hybrid quantum-classical deep learning model for cybersecurity application: domain generation algorithms (DGA)-based botnet detection. We analyzed our hybrid model’s performance compared with the classical model counterpart to investigate the quantum circuit’s effectivity as a layer in a deep learning model. We used four features of the Botnet DGA dataset: MinREBotnets, CharLength, TreeNewFeature, and nGramReputation_Alexa. The hybrid model’s quantum circuit is a combination of Pennylane’s embedding (Angle Embedding and IQP Embedding) and layers circuit (Basic Entangler Layers, Random Layers, and Strongly Entangling Layers). Also, we implemented noise models for assessing the applicability of the model for current Noisy Intermediate Scale Quantum (NISQ) technology. We found that in some cases, the hybrid model reached high performance (maximum accuracy up to 94,7% using n=100; 93,9% using n=1,000). We discovered that the combination of Angle Embedding and Strongly Entangled delivers high accuracy, superior to the classical deep learning model in n=100 experiments). However, the overall performance is still inferior for the rest cases compared to the classical deep learning model counterpart.

DGA-based botnet detection toward imbalanced multiclass learning

Botnets based on the Domain Generation Algorithm (DGA) mechanism pose great challenges to the main current detection methods because of their strong concealment and robustness. However, the complexity of the DGA family and the imbalance of samples continue to impede research on DGA detection. In the existing work, the sample size of each DGA family is regarded as the most important determinant of the resampling proportion; thus, differences in the characteristics of various samples are ignored, and the optimal resampling effect is not achieved. In this paper, a Long Short-Term Memory-based Property and Quantity Dependent Optimization (LSTM.PQDO) method is proposed. This method takes advantage of LSTM to automatically mine the comprehensive features of DGA domain names. It iterates the resampling proportion with the optimal solution based on a comprehensive consideration of the original number and characteristics of the samples to heuristically search for a better solution around the initial solution in the right direction; thus, dynamic optimization of the resampling proportion is realized. The experimental results show that the LSTM.PQDO method can achieve better performance compared with existing models to overcome the difficulties of unbalanced datasets; moreover, it can function as a reference for sample resampling tasks in similar scenarios.

The Ecosystem of Detection and Blocklisting of Domain Generation

Malware authors use domain generation algorithms to establish more reliable communication methods that can avoid reactive defender blocklisting techniques. Network defense has sought to supplement blocklists with methods for detecting machine-generated domains. We present a repeatable evaluation and comparison of the available open source detection methods. We designed our evaluation with multiple interrelated aspects, to improve both interpretability and realism. In addition to evaluating detection methods, we assess the impact of the domain generation ecosystem on prior results about the nature of blocklists and how they are maintained. The results of the evaluation of open source detection methods finds all methods are inadequate for practical use. The results of the blocklist impact study finds that generated domains decrease the overlap among blocklists; however, while the effect is large in relative terms, the baseline is so small that the core conclusions of the prior work are sustained. Namely, that blocklist construction is very targeted, context-specific, and as a result blocklists do no overlap much. We recommend that Domain Generation Algorithm detection should also be similarly narrowly targeted to specific algorithms and specific malware families, rather than attempting to create general-purpose detection for machine-generated domains.

Adversarial Defense: DGA-Based Botnets and DNS Homographs Detection Through Integrated Deep Learning

Cybercriminals use domain generation algorithms (DGAs) to prevent their servers from being potentially blacklisted or shut down. Existing reverse engineering techniques for DGA detection is labor intensive, extremely time-consuming, prone to human errors, and have significant limitations. Hence, an automated real-time technique with a high detection rate is warranted in such applications. In this article, we present a novel technique to detect randomly generated domain names and domain name system (DNS) homograph attacks without the need for any reverse engineering or using nonexistent domain (NXDomain) inspection using deep learning. We provide an extensive evaluation of our model over four large, real-world, publicly available datasets. We further investigate the robustness of our model against three different adversarial attacks: DeepDGA, CharBot, and MaskDGA. Our evaluation demonstrates that our method is effectively able to identify DNS homograph attacks and DGAs and also is resilient to common evading cyberattacks. Promising results show that our approach provides a more effective detection rate with an accuracy of 0.99. Additionally, the performance of our model is compared against the most popular deep learning architectures. Our findings highlight the essential need for more robust detection models to counter adversarial learning.

Real-Time Detection of Dictionary DGA Network Traffic Using Deep Learning

Botnets and malware continue to avoid detection by static rule engines when using domain generation algorithms (DGAs) for callouts to unique, dynamically generated web addresses. Common DGA detection techniques fail to reliably detect DGA variants that combine random dictionary words to create domain names that closely mirror legitimate domains. To combat this, we created a novel hybrid neural network, Bilbo the “bagging” model, that analyses domains and scores the likelihood they are generated by such algorithms and therefore are potentially malicious. Bilbo is the first parallel usage of a convolutional neural network (CNN) and a long short-term memory (LSTM) network for DGA detection. Our unique architecture is found to be the most consistent in performance in terms of AUC, F_1 score, and accuracy when generalising across different dictionary DGA classification tasks compared to current state-of-the-art deep learning architectures. We validate using reverse-engineered dictionary DGA domains and detail our real-time implementation strategy for scoring real-world network logs within a large enterprise. In 4 h of actual network traffic, the model discovered at least five potential command-and-control networks that commercial vendor tools did not flag.

CLETer: A Character-level Evasion Technique Against Deep Learning DGA Classifiers

The detection of pseudo-random domain names generated by Domain Generation Algorithms (DGAs) is one of the effective ways to find botnets. Study on the vulnerability of deep learning models to adversarial attacks can enhance the robustness of DGA detection mechanism. This paper proposes CLETer, an i

Exploiting statistical and structural features for the detection of Domain Generation Algorithms

Nowadays, malware campaigns have reached a high level of sophistication, thanks to the use of cryptography and covert communication channels over traditional protocols and services. In this regard, a typical approach to evade botnet identification and takedown mechanisms is the use of domain fluxing through the use of Domain Generation Algorithms (DGAs). These algorithms produce an overwhelming amount of domain names that the infected device tries to communicate with to find the Command and Control server, yet only a small fragment of them is actually registered. Due to the high number of domain names, the blacklisting approach is rendered useless. Therefore, the botmaster may pivot the control dynamically and hinder botnet detection mechanisms. To counter this problem, many security mechanisms result in solutions that try to identify domains from a DGA based on the randomness of their name.In this work, we explore hard to detect families of DGAs, as they are constructed to bypass these mechanisms. More precisely, they are based on the use of dictionaries or adversarial approaches so the generated domains seem to be user-generated. Therefore, the corresponding generated domains pass many filters that look for, e.g. high entropy strings or n-grams. To address this challenge, we propose an accurate and efficient probabilistic approach to detect them. We test and validate the proposed solution through extensive experiments with a sound dataset containing all the wordlist-based DGA families that exhibit this behaviour, as well as several adversarial DGAs, and compare it with other state-of-the-art methods, practically showing the efficacy and prevalence of our proposal.

A dictionary-based method for detecting machine-generated domains

ABSTRACT Internet robots, also known as bots, have transformed the businesses and society with convenience. However, the dynamics of these interactions could be under adversarial circumstances with detrimental effects on network security. Bots that use domain-generation algorithms (DGAs) can generate many random domains dynamically so that static domain blacklists become ineffective in preventing malicious attacks by botnets. Various families of recent botnets have used DGA to establish communication with the bots. Researchers have introduced various detection methods with moderate success. Methods proposed so far either detect only DGAs that use non-variations forms or focus on the classification accuracy instead of time complexity, which would be critical in real-world production. The goal of this article is to explore how machine learning can help in detecting machine-generated domain names. To that end, we propose a dictionary-based n-gram method that can detect 39 DGA variations. We compared our method with existing research and found that our method can improve the performance of the existing classification algorithms. At last, our method can achieve competitive results as the LSTM model while requiring less time and complexity. Our research helps real-time production for DGA detection and provides insight in protecting DNS server and information security.

DNS anti-attack machine learning model for DGA domain name detection

Domain Name System (DNS) is a vital service for the Internet, Domain names generated by Domain Generation Algorithm (DGA) hidden in DNS distributed database have potential risks because it may be malicious. It is difficult for traditional methods to detect DGA domain names for the reason that DGA domain names are random, dynamic and numerous. In this paper, a machine learning-based DGA domain name detection method is proposed. We analyzed the characteristics of DGA domain name by five feature extraction methods. Then we apply six kinds of Machine Learning algorithms with five types of feature sets to obtain thirty candidate DGA detection models. The optimized DGA detection model is obtained in comparative experiments with the evaluation indexes of Accuracy, Precision, Recall, F1 score and Training Time. The experimental results show that the approach based on Machine Learning can effectively identify DGA domain names.

A DGA domain names detection modeling method based on integrating an attention mechanism and deep neural network

Command and control (C2) servers are used by attackers to operate communications. To perform attacks, attackers usually employee the Domain Generation Algorithm (DGA), with which to confirm rendezvous points to their C2 servers by generating various network locations. The detection of DGA domain names is one of the important technologies for command and control communication detection. Considering the randomness of the DGA domain names, recent research in DGA detection applyed machine learning methods based on features extracting and deep learning architectures to classify domain names. However, these methods are insufficient to handle wordlist-based DGA threats, which generate domain names by randomly concatenating dictionary words according to a special set of rules. In this paper, we proposed a a deep learning framework ATT-CNN-BiLSTM for identifying and detecting DGA domains to alleviate the threat. Firstly, the Convolutional Neural Network (CNN) and bidirectional Long Short-Term Memory (BiLSTM) neural network layer was used to extract the features of the domain sequences information; secondly, the attention layer was used to allocate the corresponding weight of the extracted deep information from the domain names. Finally, the different weights of features in domain names were put into the output layer to complete the tasks of detection and classification. Our extensive experimental results demonstrate the effectiveness of the proposed model, both on regular DGA domains and DGA that hard to detect such as wordlist-based and part-wordlist-based ones. To be precise,we got a F1 score of 98.79% for the detection and macro average precision and recall of 83% for the classification task of DGA domain names.

Inline Detection of DGA Domains Using Side Information

Malware applications typically use a command and control (C&C) server to manage bots to perform malicious activities. Domain Generation Algorithms (DGAs) are popular methods for generating pseudo-random domain names that can be used to establish a communication between an infected bot and the C&C server. In recent years, machine learning based systems have been widely used to detect DGAs. There are several well known state-of-the-art classifiers in the literature that can detect DGA domain names in real-time applications with high predictive performance. However, these DGA classifiers are highly vulnerable to adversarial attacks in which adversaries purposely craft domain names to evade DGA detection classifiers. In our work, we focus on hardening DGA classifiers against adversarial attacks. To this end, we train and evaluate state-of-the-art deep learning and random forest (RF) classifiers for DGA detection using side information that is harder for adversaries to manipulate than the domain name itself. Additionally, the side information features are selected such that they are easily obtainable in practice to perform inline DGA detection. The performance and robustness of these models is assessed by exposing them to one day of real-traffic data as well as domains generated by adversarial attack algorithms. We found that the DGA classifiers that rely on both the domain name and side information have high performance and are more robust against adversaries.

Detecting Stealthy Domain Generation Algorithms Using Heterogeneous Deep Neural Network Framework

Distinguishing malicious domain names generated by various domain generation algorithms (DGA) is critical for defending a network against sophisticated network attacks. In recent years, stealthy domain generation algorithms (SDGA) have been proposed and revealed significantly stronger stealthiness comparing to the traditional character-based DGA. Existing state-of-the-art detection schemes are not effective enough for detecting SDGA. In this paper, we exploit the character-level characteristics of the SDGA domain names and propose a heterogeneous deep neural network framework (HDNN) for detecting SDGA. HDNN employs a proposed improved parallel CNN (IPCNN) architecture with multi-sizes of convolution kernel for extracting multi-scale local features from a domain name. The framework also contains a proposed self-attention based bidirectional long short term memory (SA-Bi-LSTM) architecture which can extract the bidirectional global features with attention mechanism from a domain name. Besides that, the focal loss function is introduced to mitigate the imbalance of the sample quantity in the training phase. The benchmark experiments are carried out based on the database composed of the collected benign domain names, real-world DGA and SDGA ones. Compared to the 6 influential deep-learning-based DGA detection schemes, the proposed scheme has achieved state-of-the-art detection results on SDGAs, and also achieved state-of-the-art results on binary and multiclass classification for traditional DGAs.

Khaos: An Adversarial Neural Network DGA With High Anti-Detection Ability

A botnet is a network of remote-controlled devices that are infected with malware controlled by botmasters in order to launch cyber attacks. To evade detection, the botmaster frequently changes the domain name of his Command and Control (C&C) server. Notice that most of these types of domain names are generated by domain generation algorithms (DGAs). In this paper, we propose Khaos, a novel DGA with high anti-detection ability based on neural language models and the Wasserstein Generative Adversarial Network (WGAN). The key insight of our research is that real domain names are composed of readable syllables and acronyms, and thus we can arrange syllables and acronyms using neural language models to mimic real domain names. In Khaos, we first find the most common ${n}$ -grams in real domain names, then tokenize these domain names into ${n}$ -grams, and finally synthesize new domain names after learning arrangements of ${n}$ -grams from real domain names. We carry out experiments using a variety of state-of-the-art DGA detection approaches: the statistics-based, the distribution-based, the LSTM-based and the graph-based detection approach. Our experimental results show that the average distance for detecting Khaos under the distribution-based detection approach is 0.64, the AUCs of Khaos under the statistics-based and the LSTM-based detection approach are 0.76 and 0.57, respectively, and the precision of Khaos under the graph-based detection approach is 0.68. Our work proves that the existing detection approaches have big troubles in detecting Khaos, and Khaos has better anti-detection ability than state-of-the-art DGAs. In addition, we find that training the existing detection approach on a dataset including the domain names generated by Khaos can improve its detection ability.

DGA CapsNet: 1D Application of Capsule Networks to DGA Detection

Domain generation algorithms (DGAs) represent a class of malware used to generate large numbers of new domain names to achieve command-and-control (C2) communication between the malware program and its C2 server to avoid detection by cybersecurity measures. Deep learning has proven successful in serving as a mechanism to implement real-time DGA detection, specifically through the use of recurrent neural networks (RNNs) and convolutional neural networks (CNNs). This paper compares several state-of-the-art deep-learning implementations of DGA detection found in the literature with two novel models: a deeper CNN model and a one-dimensional (1D) Capsule Networks (CapsNet) model. The comparison shows that the 1D CapsNet model performs as well as the best-performing model from the literature.