Domain Generation Algorithms Detection Research Articles

WordDGA: Hybrid Knowledge-Based Word-Level Domain Names Against DGA Classifiers and Adversarial DGAs

A Domain Generation Algorithm (DGA) employs botnets to generate domain names through a communication link between the C&C server and the bots. A DGA can generate pseudo-random AGDs (algorithmically generated domains) regularly, a handy method for detecting bots on the C&C server. Unlike current DGA detection methods, AGDs can be identified with lightweight, promising technology. DGAs can prolong the life of a viral operation, improving its profitability. Recent research on the sensitivity of deep learning to various adversarial DGAs has sought to enhance DGA detection techniques. They have character- and word-level classifiers; hybrid-level classifiers may detect and classify AGDs generated by DGAs, significantly diminishing the effectiveness of DGA classifiers. This work introduces WordDGA, a hybrid RCNN-BiLSTM-based adversarial DGA with strong anti-detection capabilities based on NLP and cWGAN, which offers word- and hybrid-level evasion techniques. It initially models the semantic relationships between benign and DGA domains by constructing a prediction model with a hybrid RCNN-BiLSTM network. To optimize the similarity between benign and DGA domain names, it modifies phrases from each input domain using the prediction model to detect DGA family categorizations. The experimental results reveal that dodging numerous wordlists and mixed-level DGA classifiers with training and testing sets improves word repetition rate, domain collision rate, attack success rate, and detection rate, indicating the usefulness of cWGAN-based oversampling in the face of adversarial DGAs.

An end-to-end framework for private DGA detection as a service.

Domain Generation Algorithms (DGAs) are used by malware to generate pseudorandom domain names to establish communication between infected bots and command and control servers. While DGAs can be detected by machine learning (ML) models with great accuracy, offering DGA detection as a service raises privacy concerns when requiring network administrators to disclose their DNS traffic to the service provider. The main scientific contribution of this paper is to propose the first end-to-end framework for privacy-preserving classification as a service of domain names into DGA (malicious) or non-DGA (benign) domains. Our framework achieves these goals by carefully designed protocols that combine two privacy-enhancing technologies (PETs), namely secure multi-party computation (MPC) and differential privacy (DP). Through MPC, our framework enables an enterprise network administrator to outsource the problem of classifying a DNS (Domain Name System) domain as DGA or non-DGA to an external organization without revealing any information about the domain name. Moreover, the service provider's ML model used for DGA detection is never revealed to the network administrator. Furthermore, by using DP, we also ensure that the classification result cannot be used to learn information about individual entries of the training data. Finally, we leverage post-training float16 quantization of deep learning models in MPC to achieve efficient, secure DGA detection. We demonstrate that by using quantization achieves a significant speed-up, resulting in a 23% to 42% reduction in inference runtime without reducing accuracy using a three party secure computation protocol tolerating one corruption. Previous solutions are not end-to-end private, do not provide differential privacy guarantees for the model's outputs, and assume that model embeddings are publicly known. Our best protocol in terms of accuracy runs in about 0.22s.

On DGA Detection and Classification Using P4 Programmable Switches

Domain Generation Algorithms (DGAs) are highly effective strategies employed by malware to establish connections with Command and Control (C2) servers. Mitigating DGAs in high-speed networks can be challenging, as it often requires resource-intensive tasks such as extracting high-dimensional features from domain names or collecting extensive network heuristics. In this paper, we propose an innovative framework leveraging the flexibility, per-packet granularity, and Terabits per second (Tbps) processing capabilities of P4 programmable data plane switches for the rapid and accurate detection and classification of DGA families. Specifically, we use P4 switches to extract a combination of unique network heuristics and domain name features through shallow and Deep Packet Inspection (DPI) with minimal impact on throughput. We employ a two-fold approach, comprising a line-rate compact Machine Learning (ML) classifier in the data plane for DGA detection and a more comprehensive classifier in the control plane for DGA detection and classification. To validate our approach, we collected malware samples totaling hundreds of Gigabytes (GBs), representing over 50 DGA families, and utilized campus traffic from normal benign users. Our results demonstrate that our proposed approach can swiftly and accurately detect DGAs with an accuracy of 97% and 99% in the data plane and the control plane, respectively. Furthermore, we present promising findings and preliminary results for detecting DGAs in encrypted Domain Name System (DNS) traffic. Our framework enables the immediate halting of malicious communications, empowering network operators to implement effective mitigation, incident management, and provisioning strategies.

Data Science in Cybersecurity to Detect Malware-Based Domain Generation Algorithm: Improvement, Challenges, and Prospects

Nowadays, the malware communicates with command and control servers using domains generated algorithmically. Domain generation algorithms ( DGAs) are continually evolving, which degrades the accuracy of the existing methods calls for the continuous tracking of how DGAs develop and their detection methods and calls for a good evaluation of the stage to open horizons for new detection methods. Data science plays a key role in cybersecurity by providing methods for detecting and analyzing network traffic data, including DGAs, and helping to improve the overall security of computer systems and networks. It can also be used to analyze large datasets of domain names and to develop and optimize solutions for DGA detection, by applying techniques such as machine learning, deep learning, and genetic algorithms, which have shown their effectiveness in detecting new and unknown DGAs. This paper reviews the role of data science in cybersecurity systems to detect DGAs. Hence, it also brings together publicly available domain name datasets and data science techniques utilized in recent DGA detection systems to highlight current issues and potential directions. This article additionally explains issues related to DGA detection. This will assist researchers in improving the current DGA detection algorithms as well as creating new powerful models.

KDTM: Multi-Stage Knowledge Distillation Transfer Model for Long-Tailed DGA Detection

As the most commonly used attack strategy by Botnets, the Domain Generation Algorithm (DGA) has strong invisibility and variability. Using deep learning models to detect different families of DGA domain names can improve the network defense ability against hackers. However, this task faces an extremely imbalanced sample size among different DGA categories, which leads to low classification accuracy for small sample categories and even classification failure for some categories. To address this issue, we introduce the long-tailed concept and augment the data of small sample categories by transferring pre-trained knowledge. Firstly, we propose the Data Balanced Review Method (DBRM) to reduce the sample size difference between the categories, thus a relatively balanced dataset for transfer learning is generated. Secondly, we propose the Knowledge Transfer Model (KTM) to enhance the knowledge of the small sample categories. KTM uses a multi-stage transfer to transfer weights from the big sample categories to the small sample categories. Furthermore, we propose the Knowledge Distillation Transfer Model (KDTM) to relieve the catastrophic forgetting problem caused by transfer learning, which adds knowledge distillation loss based on the KTM. The experimental results show that KDTM can significantly improve the classification performance of all categories, especially the small sample categories. It can achieve a state-of-the-art macro average F1 score of 84.5%. The robustness of the KDTM model is verified using three DGA datasets that follow the Pareto distributions.

Botnet DGA Domain Name Classification Using Transformer Network with Hybrid Embedding

One of the severest threats to cyber security is botnet, which typically uses domain names generated by Domain Generation Algorithms (DGAs) to communicate with their Command and Control (C&C) infrastructure. DGA detection and classification play an important role of assisting cyber security researchers to detect botnet C&C servers. However, many of the existing DGA detection models only focus on single scale word embedding method, and very few models are specially designed to extract more effective features for DGA detection from multiple scales word embedding. To alleviate above questions, first we propose a hybrid word embedding method, which combines character level embedding and bigram level embedding to make full use of the domain names information, and then, we design a deep neural network with hybrid embedding method to distinguish DGA domains from known legitimate domains. Finally, we evaluate our hybrid embedding method and the proposed model on ONIST dataset and compare our methods with several state-of-the-art DGA classification methods.

Domain generation algorithms detection with feature extraction and Domain Center construction.

Network attacks using Command and Control (C&C) servers have increased significantly. To hide their C&C servers, attackers often use Domain Generation Algorithms (DGA), which automatically generate domain names for C&C servers. Researchers have constructed many unique feature sets and detected DGA domains through machine learning or deep learning models. However, due to the limited features contained in the domain name, the DGA detection results are limited. In order to overcome this problem, the domain name features, the Whois features and the N-gram features are extracted for DGA detection. To obtain the N-gram features, the domain name whitelist and blacklist substring feature sets are constructed. In addition, a deep learning model based on BiLSTM, Attention and CNN is constructed. Additionally, the Domain Center is constructed for fast classification of domain names. Multiple comparative experiment results prove that the proposed model not only gets the best Accuracy, Precision, Recall and F1, but also greatly reduces the detection time.

A Bayesian optimization-based LSTM model for DGA domain name identification approach

In recent years, with the rapid development and rise of mobile Internet, network security issues have also posed a great threat to people. Botnets are an important problem faced by current network security. DNS protocol-based botnets widely use domain generation algorithm (DGA), which can randomly change the domain name to hide itself, and therefore is very likely to threaten people’s network security. In this paper, we use the domain names of the top 1 million websites in the Alexa global ranking as white samples, and for the DGA sample data, we use the open data of 360netlab as black samples. The character sequence model is used for feature extraction, and the LSTM with Bayesian optimization neural network is used to optimize the hyperparameter combination, which finally makes the accuracy of the model above 97%, and the model has superior performance to compare with the conventional model, which can effectively improve the accuracy of DGA detection and recognition.

HAGDetector: Heterogeneous DGA domain name detection model

The botnet relies on the Command and Control (C&C) channels to conduct its malicious activities remotely. The Domain Generation Algorithm (DGA) is often used by botnets to hide their Command and Control (C&C) server and evade take-down attempts, which allows the bot to generate a large number of domain names until it finds its C&C server. The lengths of domain names generated by DGAs are different. Our research finds that the length of the domain name has an impact on the performance of the DGA domain name detection model. In other words, the model is sensitive to the length of the domain name. In this case, attackers can evade detection simply by designing domain names of specific lengths. Moreover, the detection accuracy of DGA domain names still needs to be further improved. To solve these problems, three feature extraction methods adapted to the length of the domain name are proposed in this paper. For extra-short domain names, we use the attention-based method to extract features, which can make use of the character-level semantic feature. For moderate-length domain names, a two-dimensional structure, namely Right Shifted Tensor (RST), is constructed to make the domain name present apparent features similar to images. For the extra-long domain name, the effective classification of domain names can be achieved by manually crafted easy-to-calculate features. Then, different detection structures are designed based on these tree feature extraction methods to form a heterogeneous DGA detection model, namely HAGDetector. In addition, the public suffix is an important part of the domain name. We further analyze the public suffix to evaluate its impact on the detection of DGA domain names. Finally, the experiments are conducted to assess the validity of HAGDetector, as well as compare our approach with the current state-of-the-art and highlight the impact of the domain name length. The experimental results show that our method greatly improves the detection performance.

Robust Botnet DGA Detection: Blending XAI and OSINT for Cyber Threat Intelligence Sharing

We investigated 12 years DNS query logs of our campus network and identified phenomena of malicious botnet domain generation algorithm (DGA) traffic. DGA-based botnets are difficult to detect using cyber threat intelligence (CTI) systems based on blocklists. Artificial intelligence (AI)/machine learning (ML)-based CTI systems are required. This study (1) proposed a model to detect DGA-based traffic based on statistical features with datasets comprising 55 DGA families, (2) discussed how CTI can be expanded with computable CTI paradigm, and (3) described how to improve the explainability of the model outputs by blending explainable AI (XAI) and open-source intelligence (OSINT) for trust problems, an antidote for skepticism to the shared models and preventing automation bias. We define the XAI-OSINT blending as aggregations of OSINT for AI/ML model outcome validation. Experimental results show the effectiveness of our models (96.3% accuracy). Our random forest model provides better robustness against three state-of-the-art DGA adversarial attacks (CharBot, DeepDGA, MaskDGA) compared with character-based deep learning models (Endgame, CMU, NYU, MIT). We demonstrate the sharing mechanism and confirm that the XAI-OSINT blending improves trust for CTI sharing as evidence to validate our proposed computable CTI paradigm to assist security analysts in security operations centers using an automated, explainable OSINT approach (for second opinion). Therefore, the computable CTI reduces manual intervention in critical cybersecurity decision-making.

Evaluating hybrid quantum-classical deep learning for cybersecurity botnet DGA detection

Hybrid quantum machine learning (QML) algorithms have potentials for current quantum computing technologies since only part of the model is computed by a quantum device. This paper proposes a novel hybrid quantum-classical deep learning model for cybersecurity application: domain generation algorithms (DGA)-based botnet detection. We analyzed our hybrid model’s performance compared with the classical model counterpart to investigate the quantum circuit’s effectivity as a layer in a deep learning model. We used four features of the Botnet DGA dataset: MinREBotnets, CharLength, TreeNewFeature, and nGramReputation_Alexa. The hybrid model’s quantum circuit is a combination of Pennylane’s embedding (Angle Embedding and IQP Embedding) and layers circuit (Basic Entangler Layers, Random Layers, and Strongly Entangling Layers). Also, we implemented noise models for assessing the applicability of the model for current Noisy Intermediate Scale Quantum (NISQ) technology. We found that in some cases, the hybrid model reached high performance (maximum accuracy up to 94,7% using n=100; 93,9% using n=1,000). We discovered that the combination of Angle Embedding and Strongly Entangled delivers high accuracy, superior to the classical deep learning model in n=100 experiments). However, the overall performance is still inferior for the rest cases compared to the classical deep learning model counterpart.

DGA-based botnet detection toward imbalanced multiclass learning

Botnets based on the Domain Generation Algorithm (DGA) mechanism pose great challenges to the main current detection methods because of their strong concealment and robustness. However, the complexity of the DGA family and the imbalance of samples continue to impede research on DGA detection. In the existing work, the sample size of each DGA family is regarded as the most important determinant of the resampling proportion; thus, differences in the characteristics of various samples are ignored, and the optimal resampling effect is not achieved. In this paper, a Long Short-Term Memory-based Property and Quantity Dependent Optimization (LSTM.PQDO) method is proposed. This method takes advantage of LSTM to automatically mine the comprehensive features of DGA domain names. It iterates the resampling proportion with the optimal solution based on a comprehensive consideration of the original number and characteristics of the samples to heuristically search for a better solution around the initial solution in the right direction; thus, dynamic optimization of the resampling proportion is realized. The experimental results show that the LSTM.PQDO method can achieve better performance compared with existing models to overcome the difficulties of unbalanced datasets; moreover, it can function as a reference for sample resampling tasks in similar scenarios.

The Ecosystem of Detection and Blocklisting of Domain Generation

Malware authors use domain generation algorithms to establish more reliable communication methods that can avoid reactive defender blocklisting techniques. Network defense has sought to supplement blocklists with methods for detecting machine-generated domains. We present a repeatable evaluation and comparison of the available open source detection methods. We designed our evaluation with multiple interrelated aspects, to improve both interpretability and realism. In addition to evaluating detection methods, we assess the impact of the domain generation ecosystem on prior results about the nature of blocklists and how they are maintained. The results of the evaluation of open source detection methods finds all methods are inadequate for practical use. The results of the blocklist impact study finds that generated domains decrease the overlap among blocklists; however, while the effect is large in relative terms, the baseline is so small that the core conclusions of the prior work are sustained. Namely, that blocklist construction is very targeted, context-specific, and as a result blocklists do no overlap much. We recommend that Domain Generation Algorithm detection should also be similarly narrowly targeted to specific algorithms and specific malware families, rather than attempting to create general-purpose detection for machine-generated domains.

Adversarial Defense: DGA-Based Botnets and DNS Homographs Detection Through Integrated Deep Learning

Cybercriminals use domain generation algorithms (DGAs) to prevent their servers from being potentially blacklisted or shut down. Existing reverse engineering techniques for DGA detection is labor intensive, extremely time-consuming, prone to human errors, and have significant limitations. Hence, an automated real-time technique with a high detection rate is warranted in such applications. In this article, we present a novel technique to detect randomly generated domain names and domain name system (DNS) homograph attacks without the need for any reverse engineering or using nonexistent domain (NXDomain) inspection using deep learning. We provide an extensive evaluation of our model over four large, real-world, publicly available datasets. We further investigate the robustness of our model against three different adversarial attacks: DeepDGA, CharBot, and MaskDGA. Our evaluation demonstrates that our method is effectively able to identify DNS homograph attacks and DGAs and also is resilient to common evading cyberattacks. Promising results show that our approach provides a more effective detection rate with an accuracy of 0.99. Additionally, the performance of our model is compared against the most popular deep learning architectures. Our findings highlight the essential need for more robust detection models to counter adversarial learning.

Real-Time Detection of Dictionary DGA Network Traffic Using Deep Learning

Botnets and malware continue to avoid detection by static rule engines when using domain generation algorithms (DGAs) for callouts to unique, dynamically generated web addresses. Common DGA detection techniques fail to reliably detect DGA variants that combine random dictionary words to create domain names that closely mirror legitimate domains. To combat this, we created a novel hybrid neural network, Bilbo the “bagging” model, that analyses domains and scores the likelihood they are generated by such algorithms and therefore are potentially malicious. Bilbo is the first parallel usage of a convolutional neural network (CNN) and a long short-term memory (LSTM) network for DGA detection. Our unique architecture is found to be the most consistent in performance in terms of AUC, F_1 score, and accuracy when generalising across different dictionary DGA classification tasks compared to current state-of-the-art deep learning architectures. We validate using reverse-engineered dictionary DGA domains and detail our real-time implementation strategy for scoring real-world network logs within a large enterprise. In 4 h of actual network traffic, the model discovered at least five potential command-and-control networks that commercial vendor tools did not flag.

CLETer: A Character-level Evasion Technique Against Deep Learning DGA Classifiers

The detection of pseudo-random domain names generated by Domain Generation Algorithms (DGAs) is one of the effective ways to find botnets. Study on the vulnerability of deep learning models to adversarial attacks can enhance the robustness of DGA detection mechanism. This paper proposes CLETer, an i

Exploiting statistical and structural features for the detection of Domain Generation Algorithms

Nowadays, malware campaigns have reached a high level of sophistication, thanks to the use of cryptography and covert communication channels over traditional protocols and services. In this regard, a typical approach to evade botnet identification and takedown mechanisms is the use of domain fluxing through the use of Domain Generation Algorithms (DGAs). These algorithms produce an overwhelming amount of domain names that the infected device tries to communicate with to find the Command and Control server, yet only a small fragment of them is actually registered. Due to the high number of domain names, the blacklisting approach is rendered useless. Therefore, the botmaster may pivot the control dynamically and hinder botnet detection mechanisms. To counter this problem, many security mechanisms result in solutions that try to identify domains from a DGA based on the randomness of their name.In this work, we explore hard to detect families of DGAs, as they are constructed to bypass these mechanisms. More precisely, they are based on the use of dictionaries or adversarial approaches so the generated domains seem to be user-generated. Therefore, the corresponding generated domains pass many filters that look for, e.g. high entropy strings or n-grams. To address this challenge, we propose an accurate and efficient probabilistic approach to detect them. We test and validate the proposed solution through extensive experiments with a sound dataset containing all the wordlist-based DGA families that exhibit this behaviour, as well as several adversarial DGAs, and compare it with other state-of-the-art methods, practically showing the efficacy and prevalence of our proposal.

A dictionary-based method for detecting machine-generated domains

ABSTRACT Internet robots, also known as bots, have transformed the businesses and society with convenience. However, the dynamics of these interactions could be under adversarial circumstances with detrimental effects on network security. Bots that use domain-generation algorithms (DGAs) can generate many random domains dynamically so that static domain blacklists become ineffective in preventing malicious attacks by botnets. Various families of recent botnets have used DGA to establish communication with the bots. Researchers have introduced various detection methods with moderate success. Methods proposed so far either detect only DGAs that use non-variations forms or focus on the classification accuracy instead of time complexity, which would be critical in real-world production. The goal of this article is to explore how machine learning can help in detecting machine-generated domain names. To that end, we propose a dictionary-based n-gram method that can detect 39 DGA variations. We compared our method with existing research and found that our method can improve the performance of the existing classification algorithms. At last, our method can achieve competitive results as the LSTM model while requiring less time and complexity. Our research helps real-time production for DGA detection and provides insight in protecting DNS server and information security.

DNS anti-attack machine learning model for DGA domain name detection

Domain Name System (DNS) is a vital service for the Internet, Domain names generated by Domain Generation Algorithm (DGA) hidden in DNS distributed database have potential risks because it may be malicious. It is difficult for traditional methods to detect DGA domain names for the reason that DGA domain names are random, dynamic and numerous. In this paper, a machine learning-based DGA domain name detection method is proposed. We analyzed the characteristics of DGA domain name by five feature extraction methods. Then we apply six kinds of Machine Learning algorithms with five types of feature sets to obtain thirty candidate DGA detection models. The optimized DGA detection model is obtained in comparative experiments with the evaluation indexes of Accuracy, Precision, Recall, F1 score and Training Time. The experimental results show that the approach based on Machine Learning can effectively identify DGA domain names.

A DGA domain names detection modeling method based on integrating an attention mechanism and deep neural network

Command and control (C2) servers are used by attackers to operate communications. To perform attacks, attackers usually employee the Domain Generation Algorithm (DGA), with which to confirm rendezvous points to their C2 servers by generating various network locations. The detection of DGA domain names is one of the important technologies for command and control communication detection. Considering the randomness of the DGA domain names, recent research in DGA detection applyed machine learning methods based on features extracting and deep learning architectures to classify domain names. However, these methods are insufficient to handle wordlist-based DGA threats, which generate domain names by randomly concatenating dictionary words according to a special set of rules. In this paper, we proposed a a deep learning framework ATT-CNN-BiLSTM for identifying and detecting DGA domains to alleviate the threat. Firstly, the Convolutional Neural Network (CNN) and bidirectional Long Short-Term Memory (BiLSTM) neural network layer was used to extract the features of the domain sequences information; secondly, the attention layer was used to allocate the corresponding weight of the extracted deep information from the domain names. Finally, the different weights of features in domain names were put into the output layer to complete the tasks of detection and classification. Our extensive experimental results demonstrate the effectiveness of the proposed model, both on regular DGA domains and DGA that hard to detect such as wordlist-based and part-wordlist-based ones. To be precise,we got a F1 score of 98.79% for the detection and macro average precision and recall of 83% for the classification task of DGA domain names.