Domain Generation Algorithm Domains Research Articles

An end-to-end framework for private DGA detection as a service.

Domain Generation Algorithms (DGAs) are used by malware to generate pseudorandom domain names to establish communication between infected bots and command and control servers. While DGAs can be detected by machine learning (ML) models with great accuracy, offering DGA detection as a service raises privacy concerns when requiring network administrators to disclose their DNS traffic to the service provider. The main scientific contribution of this paper is to propose the first end-to-end framework for privacy-preserving classification as a service of domain names into DGA (malicious) or non-DGA (benign) domains. Our framework achieves these goals by carefully designed protocols that combine two privacy-enhancing technologies (PETs), namely secure multi-party computation (MPC) and differential privacy (DP). Through MPC, our framework enables an enterprise network administrator to outsource the problem of classifying a DNS (Domain Name System) domain as DGA or non-DGA to an external organization without revealing any information about the domain name. Moreover, the service provider's ML model used for DGA detection is never revealed to the network administrator. Furthermore, by using DP, we also ensure that the classification result cannot be used to learn information about individual entries of the training data. Finally, we leverage post-training float16 quantization of deep learning models in MPC to achieve efficient, secure DGA detection. We demonstrate that by using quantization achieves a significant speed-up, resulting in a 23% to 42% reduction in inference runtime without reducing accuracy using a three party secure computation protocol tolerating one corruption. Previous solutions are not end-to-end private, do not provide differential privacy guarantees for the model's outputs, and assume that model embeddings are publicly known. Our best protocol in terms of accuracy runs in about 0.22s.

KDTM: Multi-Stage Knowledge Distillation Transfer Model for Long-Tailed DGA Detection

As the most commonly used attack strategy by Botnets, the Domain Generation Algorithm (DGA) has strong invisibility and variability. Using deep learning models to detect different families of DGA domain names can improve the network defense ability against hackers. However, this task faces an extremely imbalanced sample size among different DGA categories, which leads to low classification accuracy for small sample categories and even classification failure for some categories. To address this issue, we introduce the long-tailed concept and augment the data of small sample categories by transferring pre-trained knowledge. Firstly, we propose the Data Balanced Review Method (DBRM) to reduce the sample size difference between the categories, thus a relatively balanced dataset for transfer learning is generated. Secondly, we propose the Knowledge Transfer Model (KTM) to enhance the knowledge of the small sample categories. KTM uses a multi-stage transfer to transfer weights from the big sample categories to the small sample categories. Furthermore, we propose the Knowledge Distillation Transfer Model (KDTM) to relieve the catastrophic forgetting problem caused by transfer learning, which adds knowledge distillation loss based on the KTM. The experimental results show that KDTM can significantly improve the classification performance of all categories, especially the small sample categories. It can achieve a state-of-the-art macro average F1 score of 84.5%. The robustness of the KDTM model is verified using three DGA datasets that follow the Pareto distributions.

CI_GRU: An efficient DGA botnet classification model based on an attention recurrence plot

Malware is often embedded with domain generation algorithms (DGAs) to prevent firewall interception and domain black-and-white list comparison detection while hiding command and control (C&C) servers to tighten the control of botnets. DGA domains are diverse and difficult to obtain, resulting in highly unbalanced datasets. Domain names generated by different DGA families do not differ much at the sequence data level and it is difficult to extract their features. The above characteristics lead to poor accuracy, poor generalization ability, and bloatedness of DGA domain name classification models based on deep learning. To solve the above problems, the visual representation of sequence data and the DGA domain classification model are presented in this paper. First, the DGA domain name is mapped to the attention recurrence plot (Att_RP) proposed in this paper, which can enrich the data phase space features and differentiate the key phase space features. After that, Att_RP is sent to a DGA domain name classification model (CI_GRU) proposed in this paper for data dimension transformation processing, followed by classification. Experiments show that the classification accuracy, F1_score, and recall of the model for a variety of DGA families in the wild are higher than 99%, and can also accurately classify four types of crafted DGA families. Compared with similar models, the model has high classification accuracy, low time consumption, low generalization error, and high efficiency, and the size of the model is less than one-tenth of similar models.

Botnet DGA Domain Name Classification Using Transformer Network with Hybrid Embedding

One of the severest threats to cyber security is botnet, which typically uses domain names generated by Domain Generation Algorithms (DGAs) to communicate with their Command and Control (C&C) infrastructure. DGA detection and classification play an important role of assisting cyber security researchers to detect botnet C&C servers. However, many of the existing DGA detection models only focus on single scale word embedding method, and very few models are specially designed to extract more effective features for DGA detection from multiple scales word embedding. To alleviate above questions, first we propose a hybrid word embedding method, which combines character level embedding and bigram level embedding to make full use of the domain names information, and then, we design a deep neural network with hybrid embedding method to distinguish DGA domains from known legitimate domains. Finally, we evaluate our hybrid embedding method and the proposed model on ONIST dataset and compare our methods with several state-of-the-art DGA classification methods.

Domain generation algorithms detection with feature extraction and Domain Center construction.

Network attacks using Command and Control (C&C) servers have increased significantly. To hide their C&C servers, attackers often use Domain Generation Algorithms (DGA), which automatically generate domain names for C&C servers. Researchers have constructed many unique feature sets and detected DGA domains through machine learning or deep learning models. However, due to the limited features contained in the domain name, the DGA detection results are limited. In order to overcome this problem, the domain name features, the Whois features and the N-gram features are extracted for DGA detection. To obtain the N-gram features, the domain name whitelist and blacklist substring feature sets are constructed. In addition, a deep learning model based on BiLSTM, Attention and CNN is constructed. Additionally, the Domain Center is constructed for fast classification of domain names. Multiple comparative experiment results prove that the proposed model not only gets the best Accuracy, Precision, Recall and F1, but also greatly reduces the detection time.

A Bayesian optimization-based LSTM model for DGA domain name identification approach

In recent years, with the rapid development and rise of mobile Internet, network security issues have also posed a great threat to people. Botnets are an important problem faced by current network security. DNS protocol-based botnets widely use domain generation algorithm (DGA), which can randomly change the domain name to hide itself, and therefore is very likely to threaten people’s network security. In this paper, we use the domain names of the top 1 million websites in the Alexa global ranking as white samples, and for the DGA sample data, we use the open data of 360netlab as black samples. The character sequence model is used for feature extraction, and the LSTM with Bayesian optimization neural network is used to optimize the hyperparameter combination, which finally makes the accuracy of the model above 97%, and the model has superior performance to compare with the conventional model, which can effectively improve the accuracy of DGA detection and recognition.

HAGDetector: Heterogeneous DGA domain name detection model

The botnet relies on the Command and Control (C&C) channels to conduct its malicious activities remotely. The Domain Generation Algorithm (DGA) is often used by botnets to hide their Command and Control (C&C) server and evade take-down attempts, which allows the bot to generate a large number of domain names until it finds its C&C server. The lengths of domain names generated by DGAs are different. Our research finds that the length of the domain name has an impact on the performance of the DGA domain name detection model. In other words, the model is sensitive to the length of the domain name. In this case, attackers can evade detection simply by designing domain names of specific lengths. Moreover, the detection accuracy of DGA domain names still needs to be further improved. To solve these problems, three feature extraction methods adapted to the length of the domain name are proposed in this paper. For extra-short domain names, we use the attention-based method to extract features, which can make use of the character-level semantic feature. For moderate-length domain names, a two-dimensional structure, namely Right Shifted Tensor (RST), is constructed to make the domain name present apparent features similar to images. For the extra-long domain name, the effective classification of domain names can be achieved by manually crafted easy-to-calculate features. Then, different detection structures are designed based on these tree feature extraction methods to form a heterogeneous DGA detection model, namely HAGDetector. In addition, the public suffix is an important part of the domain name. We further analyze the public suffix to evaluate its impact on the detection of DGA domain names. Finally, the experiments are conducted to assess the validity of HAGDetector, as well as compare our approach with the current state-of-the-art and highlight the impact of the domain name length. The experimental results show that our method greatly improves the detection performance.

Siamese Neural Network and Machine Learning for DGA Classification

Domain Generation Algorithms (DGA) are systems used to create immediate multiple and varying domain names. Such “artificial” domains can be then used for siting command and control servers which in turn oversee recruiting/infecting devices, and finally turning them into new resources to be exploited. In this sense, identifying DGA domain names can be crucial, to avoid cyberattacks like Phishing, Spam sending, Bitcoin mining, and many other. Usually, domain names generated by DGAs, are comprised by illegible character strings, but new “intelligent” DGAs tend to generate names using combination of words in dictionaries making its detection a challenging task. For this reason, in this work, we propose to address this problem using a combination of Machine Learning algorithms for improving the classification of DGAs domains. In particular, we propose to combine Siamese Neural Networks and traditional supervised Machine Learning algorithms in order to expand the input domain into separable n-dimensional data points and then achieve the domain classification. The proposed approach can be separated into 3 phases. In a first phase, domain names are encoded, by a one-hot encoder and a variation of this, named probabilistic one-hot encoder, which are implemented separately. Then, in the second phase, Long Short-Term Memory and Convolutional Siamese embedders are tested and compared. In particular, the first one is combined with the one-hot, while the Convolution algorithm is applied with the probabilistic one-hot encoded data. In the final step, five Machine Learning algorithms are tested using the two ways embedded data. Both embedder approaches reach very high results in terms of F1-score and Accuracy (about 91%) depending on the implemented classifier. The promising results obtained by the application of the proposed method shows that it is possible to perform DGA domain classification uniquely over the domain names, without considering external information such as DNS packets features.

DGA-based botnets detection using DNS traffic mining

Botnet is a network of infected workstations that are remotely managed by BotMaster via the command and control (C&C) server. Botnets pose a serious threat to network security since they are the source of a variety of malicious behaviors such as information theft, phishing, and Distributed Denial of Service (DDoS) assaults. Using a Domain Generation Algorithm (DGA) to produce a vast set of domain names is one of the most prevalent ways for hiding the identity of the C&C server.As a result, existing defensive methods have a limited chance of detecting and defeating such infrastructure. In this study, a system is suggested that employs machine learning techniques to categorize domain names intomalicious orlegitimate domain names. The suggested method is based on assessing the linguistic qualities of domain names requested from various hosts. Fifteen associated linguistic features were collected from the domain wordings to determine the degree of randomization, rarity, typing difficulty, and other related factors. The proposed system is tested with DNS requests gathered from various sources and seven distinct DGA botnet families. The findings reveal that the suggested technique can detect DGA domains with a 99.1% and a 0.6% false-positive rate.

D3-SACNN: DGA Domain Detection With Self-Attention Convolutional Network

Botnets are currently one of the main cyber security threats. In order to enhance the concealment, botnets usually use Domain Generation Algorithm (DGA) to establish communication between bots and command and control servers. Character-based deep learning methods are widely researched in the classification of DGA domains to detect botnets and have achieved good results. But the pronounceable DGA domain detection is still a challenge, since the linguistic statistical characteristics of the pronounceable DGA domains and benign domains are very similar. We propose a multi-head self-attention convolutional network method for DGA domain classification task. We use a shallow convolutional neural network to extract hidden features of domain characters. The multi-head self-attention mechanism with different input values is used to effectively obtain the relationship between the characters and the extracted implicit features, which will help us more effectively distinguish between pronounceable DGA domains and benign domains. Experiments on public data show that our model can effectively detect various types of DGA domains. Especially for the pronounceable DGA domains, our method is significantly better than other detection methods.

Applied machine learning in recognition of DGA domain names

Recognition of domain names generated by domain generation algorithms (DGAs) is the essential part of malware detection by inspection of network traffic. Besides basic heuristics (HE) and limited detection based on blacklists, the most promising course seems to be machine learning (ML). There is a lack of studies that extensively compare different ML models in the field of DGA binary classification, including both conventional and deep learning (DL) representatives. Also, those few that exist are either focused on a small set of models, use a poor set of features in ML models or fail to secure unbiased independence between training and evaluation samples. To overcome these limitations, we engineered a robust feature set, and accordingly trained and evaluated 14 ML, 9 DL, and 2 comparative models on two independent datasets. Results show that if ML features are properly engineered, there is a marginal difference in overall score between top ML and DL representatives. This paper represents the first attempt to neutrally compare the performance of many different models for the recognition of DGA domain names, where the best models perform as well as the top representatives from the literature.

A Lightweight Hybrid Detection Method for Botnet

Botnet is a serious threat for the Internet and it has created great damage to the Internet. How to detect botnet has become an ongoing endeavor research. Series of methods have been discussed in recent research. However, one of the remaining challenges is that the high computational overhead. In this paper, a lightweight hybrid botnet detection method is proposed. Considering the features in the botnet data packets and the characteristic of employing DGA (Domain Generation Algorithm) domain names to connect to the botnet, two sensors are designed and deployed individually and parallelly. Signature detection is used on the gateway sensor to dig out known bot software and deep learning based techniques are used on the DNS (Domain Name Server) server sensor to find DGA domain names. With this method, the computational overhead would be shared by the two sensors and experiments are conducted and the results indicate that the method is effective in detecting botnet

DGA-based botnet detection toward imbalanced multiclass learning

Botnets based on the Domain Generation Algorithm (DGA) mechanism pose great challenges to the main current detection methods because of their strong concealment and robustness. However, the complexity of the DGA family and the imbalance of samples continue to impede research on DGA detection. In the existing work, the sample size of each DGA family is regarded as the most important determinant of the resampling proportion; thus, differences in the characteristics of various samples are ignored, and the optimal resampling effect is not achieved. In this paper, a Long Short-Term Memory-based Property and Quantity Dependent Optimization (LSTM.PQDO) method is proposed. This method takes advantage of LSTM to automatically mine the comprehensive features of DGA domain names. It iterates the resampling proportion with the optimal solution based on a comprehensive consideration of the original number and characteristics of the samples to heuristically search for a better solution around the initial solution in the right direction; thus, dynamic optimization of the resampling proportion is realized. The experimental results show that the LSTM.PQDO method can achieve better performance compared with existing models to overcome the difficulties of unbalanced datasets; moreover, it can function as a reference for sample resampling tasks in similar scenarios.

Real-Time Detection of Dictionary DGA Network Traffic Using Deep Learning

Botnets and malware continue to avoid detection by static rule engines when using domain generation algorithms (DGAs) for callouts to unique, dynamically generated web addresses. Common DGA detection techniques fail to reliably detect DGA variants that combine random dictionary words to create domain names that closely mirror legitimate domains. To combat this, we created a novel hybrid neural network, Bilbo the “bagging” model, that analyses domains and scores the likelihood they are generated by such algorithms and therefore are potentially malicious. Bilbo is the first parallel usage of a convolutional neural network (CNN) and a long short-term memory (LSTM) network for DGA detection. Our unique architecture is found to be the most consistent in performance in terms of AUC, F_1 score, and accuracy when generalising across different dictionary DGA classification tasks compared to current state-of-the-art deep learning architectures. We validate using reverse-engineered dictionary DGA domains and detail our real-time implementation strategy for scoring real-world network logs within a large enterprise. In 4 h of actual network traffic, the model discovered at least five potential command-and-control networks that commercial vendor tools did not flag.

Efficient Deep Learning Models for DGA Domain Detection

In recent years, cyberattacks using command and control (C&C) servers have significantly increased. To hide their C&C servers, attackers often use a domain generation algorithm (DGA), which automatically generates domain names for the C&C servers. Accordingly, extensive research on DGA domain detection has been conducted. However, existing methods cannot accurately detect continuously generated DGA domains and can easily be evaded by an attacker. Recently, long short-term memory- (LSTM-) based deep learning models have been introduced to detect DGA domains in real time using only domain names without feature extraction or additional information. In this paper, we propose an efficient DGA domain detection method based on bidirectional LSTM (BiLSTM), which learns bidirectional information as opposed to unidirectional information learned by LSTM. We further maximize the detection performance with a convolutional neural network (CNN) + BiLSTM ensemble model using Attention mechanism, which allows the model to learn both local and global information in a domain sequence. Experimental results show that existing CNN and LSTM models achieved F1-scores of 0.9384 and 0.9597, respectively, while the proposed BiLSTM and ensemble models achieved higher F1-scores of 0.9618 and 0.9666, respectively. In addition, the ensemble model achieved the best performance for most DGA domain classes, enabling more accurate DGA domain detection than existing models.

Detection of Algorithmically Generated Domain Names Using the Recurrent Convolutional Neural Network with Spatial Pyramid Pooling.

Domain generation algorithms (DGAs) use specific parameters as random seeds to generate a large number of random domain names to prevent malicious domain name detection. This greatly increases the difficulty of detecting and defending against botnets and malware. Traditional models for detecting algorithmically generated domain names generally rely on manually extracting statistical characteristics from the domain names or network traffic and then employing classifiers to distinguish the algorithmically generated domain names. These models always require labor intensive manual feature engineering. In contrast, most state-of-the-art models based on deep neural networks are sensitive to imbalance in the sample distribution and cannot fully exploit the discriminative class features in domain names or network traffic, leading to decreased detection accuracy. To address these issues, we employ the borderline synthetic minority over-sampling algorithm (SMOTE) to improve sample balance. We also propose a recurrent convolutional neural network with spatial pyramid pooling (RCNN-SPP) to extract discriminative and distinctive class features. The recurrent convolutional neural network combines a convolutional neural network (CNN) and a bi-directional long short-term memory network (Bi-LSTM) to extract both the semantic and contextual information from domain names. We then employ the spatial pyramid pooling strategy to refine the contextual representation by capturing multi-scale contextual information from domain names. The experimental results from different domain name datasets demonstrate that our model can achieve 92.36% accuracy, an 89.55% recall rate, a 90.46% F1-score, and 95.39% AUC in identifying DGA and legitimate domain names, and it can achieve 92.45% accuracy rate, a 90.12% recall rate, a 90.86% F1-score, and 96.59% AUC in multi-classification problems. It achieves significant improvement over existing models in terms of accuracy and robustness.

Effective DGA-Domain Detection and Classification with TextCNN and Additional Features

Malicious codes, such as advanced persistent threat (APT) attacks, do not operate immediately after infecting the system, but after receiving commands from the attacker’s command and control (C&C) server. The system infected by the malicious code tries to communicate with the C&C server through the IP address or domain address of the C&C server. If the IP address or domain address is hard-coded inside the malicious code, it can analyze the malicious code to obtain the address and block access to the C&C server through security policy. In order to circumvent this address blocking technique, domain generation algorithms are included in the malware to dynamically generate domain addresses. The domain generation algorithm (DGA) generates domains randomly, so it is very difficult to identify and block malicious domains. Therefore, this paper effectively detects and classifies unknown DGA domains. We extract features that are effective for TextCNN-based label prediction, and add additional domain knowledge-based features to improve our model for detecting and classifying DGA-generated malicious domains. The proposed model achieved 99.19% accuracy for DGA classification and 88.77% accuracy for DGA class classification. We expect that the proposed model can be applied to effectively detect and block DGA-generated domains.

DNS anti-attack machine learning model for DGA domain name detection

Domain Name System (DNS) is a vital service for the Internet, Domain names generated by Domain Generation Algorithm (DGA) hidden in DNS distributed database have potential risks because it may be malicious. It is difficult for traditional methods to detect DGA domain names for the reason that DGA domain names are random, dynamic and numerous. In this paper, a machine learning-based DGA domain name detection method is proposed. We analyzed the characteristics of DGA domain name by five feature extraction methods. Then we apply six kinds of Machine Learning algorithms with five types of feature sets to obtain thirty candidate DGA detection models. The optimized DGA detection model is obtained in comparative experiments with the evaluation indexes of Accuracy, Precision, Recall, F1 score and Training Time. The experimental results show that the approach based on Machine Learning can effectively identify DGA domain names.

A DGA domain names detection modeling method based on integrating an attention mechanism and deep neural network

Command and control (C2) servers are used by attackers to operate communications. To perform attacks, attackers usually employee the Domain Generation Algorithm (DGA), with which to confirm rendezvous points to their C2 servers by generating various network locations. The detection of DGA domain names is one of the important technologies for command and control communication detection. Considering the randomness of the DGA domain names, recent research in DGA detection applyed machine learning methods based on features extracting and deep learning architectures to classify domain names. However, these methods are insufficient to handle wordlist-based DGA threats, which generate domain names by randomly concatenating dictionary words according to a special set of rules. In this paper, we proposed a a deep learning framework ATT-CNN-BiLSTM for identifying and detecting DGA domains to alleviate the threat. Firstly, the Convolutional Neural Network (CNN) and bidirectional Long Short-Term Memory (BiLSTM) neural network layer was used to extract the features of the domain sequences information; secondly, the attention layer was used to allocate the corresponding weight of the extracted deep information from the domain names. Finally, the different weights of features in domain names were put into the output layer to complete the tasks of detection and classification. Our extensive experimental results demonstrate the effectiveness of the proposed model, both on regular DGA domains and DGA that hard to detect such as wordlist-based and part-wordlist-based ones. To be precise,we got a F1 score of 98.79% for the detection and macro average precision and recall of 83% for the classification task of DGA domain names.

Inline Detection of DGA Domains Using Side Information

Malware applications typically use a command and control (C&C) server to manage bots to perform malicious activities. Domain Generation Algorithms (DGAs) are popular methods for generating pseudo-random domain names that can be used to establish a communication between an infected bot and the C&C server. In recent years, machine learning based systems have been widely used to detect DGAs. There are several well known state-of-the-art classifiers in the literature that can detect DGA domain names in real-time applications with high predictive performance. However, these DGA classifiers are highly vulnerable to adversarial attacks in which adversaries purposely craft domain names to evade DGA detection classifiers. In our work, we focus on hardening DGA classifiers against adversarial attacks. To this end, we train and evaluate state-of-the-art deep learning and random forest (RF) classifiers for DGA detection using side information that is harder for adversaries to manipulate than the domain name itself. Additionally, the side information features are selected such that they are easily obtainable in practice to perform inline DGA detection. The performance and robustness of these models is assessed by exposing them to one day of real-traffic data as well as domains generated by adversarial attack algorithms. We found that the DGA classifiers that rely on both the domain name and side information have high performance and are more robust against adversaries.