URL Classification Research Articles

Phishing URL detection with neural networks: an empirical study

Cybercriminals create phishing websites that mimic legitimate websites to get sensitive information from companies, individuals, or governments. Therefore, using state-of-the-art artificial intelligence and machine learning technologies to correctly classify phishing and legitimate URLs is imperative. We report the results of applying deterministic and probabilistic neural network models to URL classification. Key achievements of this work are: (1) The development of a unique approach based on probabilistic neural networks that improves classification accuracy. (2) We show for the first time in URL phishing research that a machine learning model trained on a combination of open source and private datasets is successful in production. The dataset is constructed from open sources like Alexa, PhishTank, or OpenPhish and, most importantly, real-world production data from EasyDMARC. The daily validation of the model using daily reported URL data and corresponding labels, both from open-source platforms and private production, reach on average a 97% accuracy on the validation dataset, labeled by PhishTank, OpenPhish and EasdDMARC where possible mislabeled data can not be excluded and was not possible to check due to large number of URLs. Feature engineering was done without third-party dependencies. Lastly, the evaluation of both deterministic and probabilistic models shows high accuracy on short and long URLs, where short URLs are defined as having less than 50 characters.

Recognition of Malicious URLs Using Machine Learning

The surge in online activities has heightened the risk of cyber-attacks, with phishing emerging as a notably widespread threat. Conventional phishing detection methods, such as blacklists and heuristic analysis, fall short when it comes to identifying novel and complex phishing schemes. To tackle this issue, we propose a machine learning-based solution for detecting phishing websites. Our model utilizes a combination of features—such as URL structure, domain attributes, and content characteristics—to categorize websites as either phishing or legitimate. The model's outstanding overall accuracy of 96.9% underscores the need to evaluate both precision and recall when determining the effectiveness of machine learning models. The results of this study are significant for developing robust URL classification models and advancing the field of cybersecurity. Future research could aim to improve the model's performance by integrating more features, developing real-time phishing detection systems, and exploring new attributes that can be derived from URLs. Keywords: Phishing, URL, Machine Learning, Classification, Detection

Malicious URL Detection using Machine Learning and CSV

The proposed program item is at risk to meet the security possibly filtering pernicious URLs. This is Python based code, a client is provoked to enter a URL, which is at that point put away in a CSV record. The code peruses the CSV record, extricates different highlights of the URL such as length, the number of characters, and the number of registries in the URL. It too checks whether the URL employments an IP address or a abbreviated URL benefit. These highlights can be utilized for URL classification and distinguishing proof of possibly malevolent URLs. The expanding predominance of cyber dangers such as phishing, malware dispersion, and other malevolent exercises has made URL discovery basic for online security. This consider presents a Python-based device for identifying possibly pernicious URLs by analyzing basic highlights such as URL length, character check, and catalog profundity. The device leverages machine learning models to classify URLs based on these highlights. Test comes about illustrate that the proposed strategy can successfully recognize between generous and pernicious URLs, giving a strong arrangement for upgrading cybersecurity.

Political Optimization Algorithm with a Hybrid Deep Learning Assisted Malicious URL Detection Model

With the enhancement of the Internet of Things (IoT), smart cities have developed the idea of conventional urbanization. IoT networks permit distributed smart devices to collect and process data in smart city structures utilizing an open channel, the Internet. Accordingly, challenges like security, centralization, privacy (i.e., execution data poisoning and inference attacks), scalability, transparency, and verifiability restrict faster variations of smart cities. Detecting malicious URLs in an IoT environment is crucial to protect devices and the network from potential security threats. Malicious URL detection is an essential element of cybersecurity. It is established that malicious URL attacks mean large risks in smart cities, comprising financial damages, losses of personal identifications, online banking, losing data, and loss of user confidentiality in online businesses, namely e-commerce and employment of social media. Therefore, this paper concentrates on the proposal of a Political Optimization Algorithm by a Hybrid Deep Learning Assisted Malicious URL Detection and Classification for Cybersecurity (POAHDL-MDC) technique. The presented POAHDL-MDC technique identifies whether malicious URLs occur. To accomplish this, the POAHDL-MDC technique performs pre-processing to transform the data to a compatible format, and a Fast Text word embedding process is involved. For malicious URL recognition, a Hybrid Deep Learning (HDL) model integrates the features of stacked autoencoder (SAE) and bi-directional long short-term memory (Bi-LSTM). Finally, POA is exploited for optimum hyperparameter tuning of the HDL technique. The simulation values of the POAHDL-MDC approach are tested on a Malicious URL database, and the outcome exhibits an improvement of the POAHDL-MDC technique with a maximal accuracy of 99.31%.

Detection of malicious websites across multiple classes using n-gram features and VGG based on URL analysis

Due to the ubiquity of the internet, cyber-attacks implemented through websites have become a severe issue with high frequency and appreciable overall financial damage. Detecting malicious URLs has become one of the most common solutions to tackle this threat, which is widely applied in the market and researched. Inspired by relevant work on URL classification using n-gram techniques and convolutional neural networks in other research areas, a method for detecting malicious websites using n-gram statistical features of URLs and a VGG-style neural network has been developed, which aims to provide classification for multiple website classes with arbitrary URL input lengths. Experimental results show that the method proposed in this paper provides an average accuracy of 96.60% on the 5-class ISCX-URL2016 dataset and 96.33% on the 4-class Malicious URLs dataset, which is 1.5 times larger. A further comparison reveals that the accuracies are competitive with similar methods for binary classifications that also use either n-gram features or a VGG-based network.

Classification of Malicious URLs Using Machine Learning.

Amid the rapid proliferation of thousands of new websites daily, distinguishing safe ones from potentially harmful ones has become an increasingly complex task. These websites often collect user data, and, without adequate cybersecurity measures such as the efficient detection and classification of malicious URLs, users' sensitive information could be compromised. This study aims to develop models based on machine learning algorithms for the efficient identification and classification of malicious URLs, contributing to enhanced cybersecurity. Within this context, this study leverages support vector machines (SVMs), random forests (RFs), decision trees (DTs), and k-nearest neighbors (KNNs) in combination with Bayesian optimization to accurately classify URLs. To improve computational efficiency, instance selection methods are employed, including data reduction based on locality-sensitive hashing (DRLSH), border point extraction based on locality-sensitive hashing (BPLSH), and random selection. The results show the effectiveness of RFs in delivering high precision, recall, and F1 scores, with SVMs also providing competitive performance at the expense of increased training time. The results also emphasize the substantial impact of the instance selection method on the performance of these models, indicating its significance in the machine learning pipeline for malicious URL classification.

URLGEN—Toward Automatic URL Generation Using GANs

URLs play an essential role on the Internet, allowing access to Web resources. Automatically generating URLs is helpful in various tasks, such as application debugging, API testing, and blocklist creation for security applications. Current testing suites deeply embed experts’ domain knowledge to generate suitable URLs, resulting in an ad-hoc solution for each given application. These tools thus require heavy manual intervention, with the expensive coding of rules that are hard to maintain. We here introduce URLGEN, a system that uses Generative Adversarial Networks (GANs) to tackle the automatic URL generation problem. URLGEN is designed for web API testing and generates URL samples for an application without any system expertise, complementing the existing tools. It leverages Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) architectures, augmented by an embedding layer that simplifies the URL learning and generation process. We show that URLGEN learns to generate new valid URLs from samples of real URLs without requiring any domain knowledge and following a purely data-driven approach. We compare the GAN architecture of URLGEN against other design options and show that the LSTM architecture can better capture the correlation among URL characters, outperforming previously proposed solutions. Finally, we show that the URLGEN approach can be extended to other scenarios, which we illustrate with two use cases, i.e., cybersquatting domain prediction and URL classification.

An Adversarial Attack Analysis on Malicious Advertisement URL Detection Framework

Malicious advertisement URLs pose a security risk since they are the source of cyber-attacks, and the need to address this issue is growing in both industry and academia. Several attempts have been made in recent years for malicious URL detection using machine learning (ML). The most widely used techniques extract linguistic features of URL string to extract features like bag-of-words (BoW) before applying ML model. Existing malicious URL detection techniques require effective manual feature engineering that can handle unseen features and generalise to test data. In this study, we extract a novel set of lexical and web-scrapped features and employ ML techniques for fraudulent advertisement URL detection. The combination set of six different kinds of features precisely overcomes the obfuscation in fraudulent URL classification. Based on distinct statistical properties, we use twelve differently formatted datasets for detection, prediction and classification task. We extend our prediction analysis for mismatched and unlabelled datasets. For this framework, we analyze the performance of four ML techniques: Random Forest, Gradient Boost, XGBoost and AdaBoost in the detection part. With our proposed method, we achieve a false negative rate up to 0.0037 while maintaining high detection accuracy of 99.63%. Moreover, we employ an unsupervised learning technique for data clustering using the K-Means algorithm for the visual analysis. This paper analyses the vulnerability of decision tree-based models using the limited knowledge attack scenario. We considered the exploratory attack during the test phase and implemented Zeroth Order Optimization adversarial attack on the detection models.

Evaluating the effectiveness of Discriminator network in GAN architecture for phishing URL classification

Phishing attack by illegitimate URLs is of the most common security challenges for both individuals and companies in ensuring the security of their information resources. The user passwords, credit card information, or other sensitive information can be stolen by clicking on the malicious URL links. Recently, machine learning based approach is being popularly applied to detect phishing URLs. The classifiers, such as SVM, Random Forest, LSTM, etc., are built on the standard datasets to make a prediction about a URL sample is malign or benign one. Some recent researches focus on using GAN network for enrichment of malicious URL samples utilized in classifier training based on deep learning models. In this work, we explore the ability of training a standard GAN architecture which consists of two adversarial networks of Discriminator and Generator. The URL samples are generated by the Generator network will be refined and feed backed to the Generator by the Discriminator. This helps the Generator generate URL samples that are more and more similar to the real ones. Accordingly, the Discriminant network also learns the malicious and clean characteristics of the URL patterns. In order to evaluate the effectiveness of this learning, the experiments are conducted on completely new testing datasets beyond the training datasets. The experimental results are promising with the classification accuracy of both malign and benign URLs are about 97%.

Classification of Malicious URLs by CNN Model Based on Genetic Algorithm

Researchers have proposed many models for the identification of malicious URLs in network security, but they have not achieved good results. In order to improve this defect, the current popular machine learning algorithm is combined to train the model, thus improving the accuracy of malicious URL classification. This paper proposes a model of a convolutional neural network based on genetic algorithm optimization. Firstly, the genetic algorithm was used to reduce the data dimension of the grammatical features, structural features, and probabilistic features in the extracted malicious URL text, and then the convolutional neural network was used to establish the model and classify the malicious URL. Through experimental verification, the model has achieved good results. Compared with the traditional machine learning model, it improves the accuracy of malicious URL recognition and provides a reference for malicious URL recognition.

A novel robust convolutional neural network for uniform resource locator classification from the view of cyber security

SummaryUniform resource locator (URL)‐based cyber‐attacks form a major part of security threats in cyberspace. Even though the experience and awareness of the end‐users help them protect themselves from these attacks, a software‐based solution is necessary for comprehensive protection. To this end, a novel robust URL classification model based on convolutional neural network is proposed in this study. The proposed model classifies given URLs into five classes, namely, () , () , () , () , and () . The proposed model was trained and evaluated on a gold standard URL dataset comprising of samples. According to the experimental result, the proposed model obtained an accuracy as high as which outperformed the state‐of‐the‐art. Based on the same architecture, we proposed another classifier, a binary classifier that detects malicious URLs without dealing with their types. This binary classifier obtained an accuracy as high as which outperformed the state‐of‐the‐art as well. The experimental result demonstrates the feasibility of the proposed solution.

Using and Comparing Machine Learning Techniques for Automatic Detection of Spam Website URLs

With the developing technology, the issue of cyber security has become one of the most common and current issues in recent years. Spam URLs are one of the most common and dangerous issues for cybersecurity. Spam URLs are one of the most widely used attacks to defraud users. These attacks cause users to suffer monetary losses, steal private information, and install malicious software on their devices. It is very important to detect such threats promptly and to take precautions against these threats. Detection of malicious URLs is mostly done by using blacklists. However, these lists are insufficient to detect newly created URLs. In recent years, machine learning techniques have been developed to overcome this deficiency. In this study, URL classification was made using different machine learning techniques. In the study, 9 different classifiers were preferred for URL classification. The performances of the classifiers were compared in the URL classification process. In addition, similar studies in the literature have been comprehensively examined and these studies have been discussed. In addition, since the preparation of data sets in the natural language processing process has a great effect on the training of models, these steps are discussed in detail.

Optimized URL Feature Selection Based on Genetic-Algorithm-Embedded Deep Learning for Phishing Website Detection

Deep learning models for phishing URL classification based on character- and word-level URL features achieve the best performance in terms of accuracy. Various improvements have been proposed through deep learning parameters, including the structure and learning strategy. However, the existing deep learning approach shows a degradation in recall according to the nature of a phishing attack that is immediately discarded after being reported. An additional optimization process that can minimize the false negatives by selecting the core features of phishing URLs is a promising avenue of improvement. To search the optimal URL feature set and to fully exploit it, we propose a combined searching and learning strategy that effectively models the URL classifier for recall. By incorporating the deep-learning-based URL classifier with the genetic algorithm to search the optimal feature set that minimizing the false negatives, an optimized classifier that guarantees the best performance was obtained. Extensive experiments on three real-world datasets consisting of 222,541 URLs showed the highest recall among the deep learning models. We demonstrated the superiority of the method by 10-fold cross-validation and confirmed that the recall improved compared to the latest deep learning method. In particular, the accuracy and recall were improved by 4.13%p and 7.07%p, respectively, compared to the convolutional–recurrent neural network in which the feature selection optimization was omitted.

K-Nearest Neighbor Based URL Identification Model for Phishing Attack Detection

Phishing causes many problems in business industry. The electronic commerce and electronic banking such as mobile banking involves a number of online transaction. In such online transactions, we have to discriminate features related to legitimate and phishing websites in order to ensure security of the online transaction. In this study, we have collected data form phish tank public data repository and proposed K-Nearest Neighbors (KNN) based model for phishing attack detection. The proposed model detects phishing attack through URL classification. The performance of the proposed model is tested empirically and result is analyzed. Experimental result on test set reveals that the model is efficient on phishing attack detection. Furthermore, the K value that gives better accuracy is determined to achieve better performance on phishing attack detection. Overall, the average accuracy of the proposed model is 85.08%.

K-Nearest Neighbor Based URL Identification Model for Phishing Attack Detection

Phishing causes many problems in business industry. The electronic commerce and electronic banking such as mobile banking involves a number of online transaction. In such online transactions, we have to discriminate features related to legitimate and phishing websites in order to ensure security of the online transaction. In this study, we have collected data form phish tank public data repository and proposed K-Nearest Neighbors (KNN) based model for phishing attack detection. The proposed model detects phishing attack through URL classification. The performance of the proposed model is tested empirically and result is analyzed. Experimental result on test set reveals that the model is efficient on phishing attack detection. Furthermore, the K value that gives better accuracy is determined to achieve better performance on phishing attack detection. Overall, the average accuracy of the proposed model is 85.08%.

An anti‐phishing enterprise environ model using feed‐forward backpropagation and Levenberg‐Marquardt method

AbstractPhishing in an enterprise is serious issue rising in wide scale and complexity, as phishers use email phishing via obfuscated, malicious or phished URLs and continuously adapt or innovate their strategies to lure victims for identity theft for financial benefits. To gain victim's trust and confidence phishers have started using visceral factors and familiarity cues. Phishing is not always money centric; phisher defame the user's goodwill and character. Defamation in enterprise could be much more traumatic than being embarrassed at a social networking site. It is a challenging task to address this issue. It is evident through literature review that single phishing detection filter approaches are insufficient to detect phishing in enterprise environ. Therefore, a novel anti‐phishing model for enterprise using artificial neural network is proposed. In addition, this model effectively identifies whether the phishing email is known phishing or unknown phishing to reduce the trust and familiarity‐based email phishing enterprise environ. The feed‐forward backpropagation and Levenberg‐Marquart methods of ANN are adopted to enhance the URL classification process and with Fuzzy Inference System to get result with imprecise data of social features. The proposed model can accurately classify the known and unknown email phishing via URLs.

Frequent rule reduction for phishing URL classification using fuzzy deep neural network model

Phishing is an online extensive fraud, which can trick Internet clients into uncovering their mystery data and qualifications, e.g., login id, secret key, charge card number, and so on. Phishing is one of the real computer security dangers looked by the digital world and could prompt monetary losses for both industries and people. In this paper, two datasets (Phishtank and UCI) were considered for malicious URL detection analysis. At first, the optimal features are extracted from the datasets using DBA-based detector module and then seventy-five optimal rules are generated using Association Rule mining based on these features. However, it is having some repetitive features which can lessen the efficiency of the malicious URL detection process. To beat this issue, the Frequent Rule Reduction algorithm along with the classification approach for predicting Phishing Websites should be implemented. The outcomes show that the proposed Fuzzy Deep Neural Network classifier accomplishes the most extreme accuracy in the prediction analysis of phishing, non-phishing, and suspicious URL.

Website Reputation System

Because of the fast development of the web, sites have turned into the interloper’s principle target. As the quantity of web pages expands, the vindictive pages are likewise expanding and the assault is progressively turned out to be modern developing different ways to trick a client into visiting malicious websites extracting credential information. This paper presents a detailed account of ensemble based machine learning approach for URL classification. Models already existing either use outdated techniques or limited set of features in their attack detection model and thus leads to lower detection rate. But ensemble classifiers along with a selection of robust feature list for single and multi attack type detection outperform all the previous deployed techniques. Focus of the study is being able to come up with a system model that yields us better results with a higher accuracy rate.

A Data Mining approach to Deal with Phishing URL Classification Problem

A Data Mining approach to Deal with Phishing URL Classification Problem

Robust URL Classification With Generative Adversarial Networks

Classifying URLs is essential for different applications, such as parental control, URL filtering and Ads/tracking protection. Such systems historically identify URLs by means of regular expressions, even if machine learning alternatives have been proposed to overcome the time-consuming maintenance of classification rules. Classical machine learning algorithms, however, require large samples of URLs to train the models, covering the diverse classes of URLs (i.e., a ground truth), which somehow limits the applicability of the approach. We here give a first step towards the use of Generative Adversarial Neural Networks (GANs) to classify URLs. GANs are attractive for this problem for two reasons. First, GANs can produce samples of URLs belonging to specific classes even if exposed to a limited training set, outputting both synthetic traces and a robust discriminator. Second, a GAN can be trained to discriminate a class of URLs without being exposed to all other URLs classes - i.e., GANs are robust even if not exposed to uninteresting URL classes during training. Experiments on real data show that not only the generated synthetic traces are somehow realistic, but also the URL classification is accurate with GANs.