URL Detection Research Articles

SOLVING THE PROBLEM OF DETECTING PHISHING WEBSITES USING ENSEMBLE LEARNING MODELS

Due to the popularity of the easiest way to obtain personal information among attackers, phishing detection is becoming a popular area for research aimed at countering the implementation of such attacks. Malicious website detection is essential to prevent the spread of malware and protect end users from victims. Unfortunately, malicious URL detection still needs to be better understood due to a lack of features and inaccurate classification. Possible sources were examined in order to investigate the subject. Based on the collected information from previous studies, this study is devoted to solving the problem of detecting phishing websites using Ensemble Learning. The aim of the work is to choose the most optimal algorithm for classifying phishing websites using gradient boosting algorithms. AdaBoost, CatBoost, and Gradient Boosting Classifier were chosen as Ensemble Learning algorithms and were used to improve the efficiency of classifiers. Practical studies of the parameters of each algorithm for finding the optimal classification model are given. Research and experiments were carried out on a dataset containing information extracted from the contents of a URL: main URL, domain, directory, and file. A thorough Exploratory Data Analysis (EDA) was carried out, as a result of which the main dependencies and patterns of determining phishing resources were identified using correlation analysis. ROC AUC Score was chosen as an evaluation metric for the algorithms. The best result for predicting phishing websites was demonstrated by the AdaBoost Classifier algorithm, with an average ROC AUC score of 99%. The results of the experiments were illustrated in the form of graphs and tables.

Malicious URL Detection Model Based on Bidirectional Gated Recurrent Unit and Attention Mechanism

With the rapid development of Internet technology, numerous malicious URLs have appeared, which bring a large number of security risks. Efficient detection of malicious URLs has become one of the keys for defense against cyber attacks. Deep learning methods bring new developments to the identification of malicious web pages. This paper proposes a malicious URL detection method based on a bidirectional gated recurrent unit (BiGRU) and attention mechanism. The method is based on the BiGRU model. A regularization operation called a dropout mechanism is added to the input layer to prevent the model from overfitting, and an attention mechanism is added to the middle layer to strengthen the feature learning of URLs. Finally, the deep learning network DA-BiGRU model is formed. The experimental results demonstrate that the proposed method can achieve better classification results in malicious URL detection, which has high significance for practical applications.

A Heterogeneous Machine Learning Ensemble Framework for Malicious Webpage Detection

The growing dependence on digital systems has heightened the risks posed by cybersecurity threats. This paper proposes a new method for detecting malicious webpages among several adversary activities. As shown in previous studies, malicious URL detection performance is significantly affected by the learning dataset features. The overall performance of different machine learning models varies depending on the data features, and using a particular model alone is not always desirable in any given environment. To address these limitations, we propose an ensemble approach using different machine learning models. Our proposed method outperforms the existing single model by 6%, allowing for the detection of an additional 141 malicious URLs. In this study, repetitive tasks are automated, improving the performance of different machine learning models. In addition, the proposed framework builds an advanced feature set based on URL and web content and includes the most optimized detection model structure. The proposed technology can contribute to define an advanced feature set based on URL and web content and includes the most optimized detection model structure and research on automated technology for the detection of malicious websites, such as phishing websites and malicious code distribution.

Malicious URL Detection Using Decision Tree-based Lexical Features Selection and Multilayer Perceptron Model

Network information security risks multiply and become more dangerous. Hackers today generally target end-to-end technology and take advantage of human weaknesses. Furthermore, hackers take advantage of technology weaknesses by applying various methods to attack. Nowadays, one of the greatest dangers to the modern digital world is malicious URLs, and stopping them is one of the biggest challenges in the field of cyber security. Detecting harmful URLs using machine learning and deep learning algorithms have been the subject of various academic papers. However, time and accuracy are the two biggest challenges of these tools. This paper proposes a multilayer perceptron (MLP) model that utilizes two significant aspects to make it more practical, lightweight, and fast: Using only lexical features and a decision tree (DT) algorithm to select the best relevant subset of features. The effectiveness of the experimental outcomes is evaluated in terms of time, accuracy, and error reduction. The results show that a MLP model using 35 features could achieve an accuracy of 94.51% utilizing only URL lexical features. Furthermore, the model is improved in time after applying the DT as feature selection with a slight improvement in accuracy and loss.

Deep Learning-Based Intrusion Detection Methods in Cyber-Physical Systems: Challenges and Future Trends

A cyber-physical system (CPS) integrates various interconnected physical processes, computing resources, and networking units, as well as monitors the process and applications of the computing systems. Interconnection of the physical and cyber world initiates threatening security challenges, especially with the increasing complexity of communication networks. Despite efforts to combat these challenges, it is difficult to detect and analyze cyber-physical attacks in a complex CPS. Machine learning-based models have been adopted by researchers to analyze cyber-physical security systems. This paper discusses the security threats, vulnerabilities, challenges, and attacks of CPS. Initially, the CPS architecture is presented as a layered approach including the physical layer, network layer, and application layer in terms of functionality. Then, different cyber-physical attacks regarding each layer are elaborated, in addition to challenges and key issues associated with each layer. Afterward, deep learning models are analyzed for malicious URLs and intrusion detection in cyber-physical systems. A multilayer perceptron architecture is utilized for experiments using the malicious URL detection dataset and KDD Cup99 dataset, and its performance is compared with existing works. Lastly, we provide a roadmap of future research directions for cyber-physical security to investigate attacks concerning their source, complexity, and impact.

PhishNot: A Cloud-Based Machine-Learning Approach to Phishing URL Detection

Phishing is constantly growing to be one of the most adopted tools for conducting cyber-attacks. Recent statistics indicated that 97% of users could not recognize a sophisticated phishing email. With over 1.5 million new phishing websites being created every month, legacy black lists and rule-based filters can no longer mitigate the increasing risks and sophistication level of phishing. Phishing can deploy various malicious payloads that compromise the network’s security. In this context, machine learning can play a crucial role in adapting the capabilities of computer networks to recognize current and evolving phishing patterns. In this paper, we present PhishNot, a phishing URL detection system based on machine learning. Hence, our work uses a primarily ”learning from data” driven approach, validated with a representative scenario and dataset. The input features were reduced to 14 to assure the system’s practical applicability. Experiments showed that Random Forest presented the best performance with a very high accuracy of 97.5%. Furthermore, the design of our system also lends itself to being more adoptable in practice through a combination of high phishing detection rate and high speed (an average of 11.5μs per URL) when deployed on the cloud.

TCURL: Exploring hybrid transformer and convolutional neural network on phishing URL detection

Phishing is a growing threat that involves cybercriminals creating counterfeit websites to lure victims and obtain their sensitive information, such as login credentials and credit card numbers. According to the Q4 2021 Phishing Trends Report by the Anti-Phishing Working Group, the number of phishing attacks has tripled from early 2020. Conventional blacklist method cannot protect users from attacks using new phishing URLs. Traditional machine learning methods require complex feature engineering and generally cannot meet the detection accuracy requirements. Deep learning methods based on fully convolutional networks and pure transformers only pay attention to local correlations or long-term dependencies. To address these issues, we propose a hybrid network architecture, called TCURL, which considers both local and global correlations among the characters of URLs. TCURL has two parallel branches, a convolution branch and a transformer branch, and a fusion block used to deal with messages from the two branches. The convolution branch provides sufficient positional information meaning that no extra positional encoding is needed. Through experiments, we explored various design choices to optimize the model. The proposed method achieves an accuracy of 96.92%, 99.77%, and 89.73% on three sampled datasets, which further outperforms other existing methods.

Comparative Analysis of Ensemble Learning and Non-Ensemble Machine Learning Algorithms for Phishing URL Detection

Phishing is a social engineering attack that has been perpetuated for long and is still a prominent attack with an attending high number of victims. Through phishing, attackers can gain easy access to sensitive information about a company or an individual. This research compares the import of features such as lexical features, Domain Named Based features, HTML Features, and tokenization of URLs in detecting phishing URLs. Experimental procedures were designed to compare the efficiency of the four categories of features used separately on three machine learning models (K-Nearest Neighbour, Decision Tree, Logistic Regression) and five ensemble learning classifiers (Random Forest, Bagging, Stacking, Ada Boost, Gradient Boost). Results obtained show higher accuracy for experiments done using URL tokenization with stacking classifier with accuracy scores of 96% and 99.3% respectively for the two datasets used. Future study would be based on more dataset with larger sample size to provide a basis for generalisation.

Using and Comparing Machine Learning Techniques for Automatic Detection of Spam Website URLs

With the developing technology, the issue of cyber security has become one of the most common and current issues in recent years. Spam URLs are one of the most common and dangerous issues for cybersecurity. Spam URLs are one of the most widely used attacks to defraud users. These attacks cause users to suffer monetary losses, steal private information, and install malicious software on their devices. It is very important to detect such threats promptly and to take precautions against these threats. Detection of malicious URLs is mostly done by using blacklists. However, these lists are insufficient to detect newly created URLs. In recent years, machine learning techniques have been developed to overcome this deficiency. In this study, URL classification was made using different machine learning techniques. In the study, 9 different classifiers were preferred for URL classification. The performances of the classifiers were compared in the URL classification process. In addition, similar studies in the literature have been comprehensively examined and these studies have been discussed. In addition, since the preparation of data sets in the natural language processing process has a great effect on the training of models, these steps are discussed in detail.

Review Paper on Detection of Malicious URLs Using Machine Learning Techniques

Abstract: Malicious websites are most serious threats over the Web. Ever since the inception of the internet, there has been a rise in malicious content over the web such has terrorism, financial fraud, phishing and hacking that targets user’s personal information. Till today the various systems have been invent for the detection of a malicious website based on keywords and data content of the websites. This existing method have some drawbacks results into numbers of victims to increase. Hence we developed a system which helps the user to identify whether the website is malicious or not. Our system identifies whether the site is malicious or not through URL. The proposed system is fast and more accurate compared to current system. The classifier is trained with datasets of 1000 malicious sites and 1000 legitimate site URLs. Trained classifier is used for detection of malicious URLs. Keywords: Malicious URLs, Classifier, Feature Extraction, ID3 Algorithm

A Model for Malicious Website Detection Using Feed Forward Neural Network

Malicious websites are the most unsafe criminal exercises in cyberspace. Since a large number of users go online to access the services offered by the government and financial establishments, there has been a notable increase in malicious websites attacks for the past few years. This paper presents a model for Malicious website URL detection using Feed Forward Neural Network. The design methodology used here is Object-Oriented Analysis and Design. The model uses a Malicious website URLs dataset, which comprises 48,006 legitimate website URLs and 48,006 Malicious website URLs making 98,012 website URLs. The dataset was pre-processed by removing all duplicate and Nan values, therefore making it fit for better training performance. The dataset was segmented into X_train and y_train, X_test and y_test which holds 60% training data and 40% testing data. The X_train contains the dataset of malicious and benign websites, while the y_train holds the label which indicates if the dataset is malicious or not. For the testing dataset the X_test contains both the malicious and non-malicious websites, while the y_test holds label which indicates if the dataset is malicious or not. The model was trained using Feed Forward Neural Network, which had an optimal accuracy 97%.

Character-level word encoding deep learning model for combating cyber threats in phishing URL detection

A cyber threat is generally a malicious activity that damages or steals data, or something that disrupts digital life. Such threats include viruses, security breaches, DoS attacks, and data theft. Phishing is a type of cyber threat whereby the attackers mimic a genuine URL or a webpage and steal user data, 21% fall into the phishing category. The novel approach of using the character-level encoding of URLs is introduced. Unlike word-level encoding, the use of character-level encoding decreases the discrete workspace and can be effective even in an energy-constrained environment. The experimental results of comparisons to other state-of-the-art methods demonstrate that the proposed method achieved 98.12% of true positive instances. Moreover, Conclusions: An experimental evaluation was performed to demonstrate the efficiency, and it was observed that the accuracy reached an all-time high of 98.13%. the experiments prove that the proposed method can operate efficiently even in energy-saving modes of phishing detection systems.

MURLi: A Tool for Detection of Malicious URLs and Injection Attacks

SQL databases have been a staple for storing data in enterprises to meet their database requirements. SQL injection has also become a major security threat due to the growth of SQL. Injection attacks occur when the query in the database is modified by malicious data from an unsecure user. The use of NoSQL databases has been growing steadily due to advancements in Big Data and similar domains, and as an alternative to SQL databases. But recent studies have proved that even NoSQL is susceptible to injection attacks. Malicious websites can harm a person using the internet in various ways. Phishing attacks, Spam attacks, Defacement Attacks can occur once a user clicks on a URL. In this research, we propose the tool mURLi, for early detection of Malicious URLs, SQLi and NoSQLi, which pose a large threat to the end-users. To build mURLi, we have used various deep learning models (ANN, BiLSTM, BERT) and machine learning models (Random Forest, XGBoost, Adaboost, KNN) in an attempt to detect these attacks (SQLi, NoSQLi and Malicious URLs) more accurately than traditional methods and have presented a comparative study of the same. The results show that BERT model outperforms the existing tools for SQL and NoSQL attacks with an accuracy of 99.6% and 99.01% respectively, which was achievable due to its pre-training and deep language knowledge. BiLSTM proves to be the best tool for detection of Malicious URL attacks, with the results being 95.2% accurate.

Enhancing Detection of Malicious URLs Using Boosting and Lexical Features

A malicious URL is a link that is created to spread spams, phishing, malware, ransomware, spyware, etc. A user may download malware that can adversely affect the computer by clicking on an infected URL, or might be convinced to provide confidential information to a fraudulent website causing serious losses. These threats must be identified and handled in a decent time and in an effective way. Detection is traditionally done through the blacklist usage method, which relies on keyword matching with previously known malicious domain names stored in a repository. This method is fast and easy to implement, with the advantage of having low false-positive rates regarding previously recognized malicious URLs. However, this method cannot recognize newly created malicious URLs. To solve this problem, many machine-learning models have been used. In this paper, we introduce an effective machine learning approach that uses an ensemble learner algorithm called AdaBoost (Adaptive Boosting), combined with different algorithms that enhance detection. For datasets filtration, we used CfsSubsetEval technique, which is an algorithm that searches for a subset of features that work well together. Datasets were collected from the UNB repository; divided into four categories: spam, phishing, malware, and defacement URLs; combined with benign URLs, dataset content is based on lexical features. The experimental results indicate that the proposed approach was successful in enhancing the detection accuracy of malicious URLs with less false-positive rates for all experimental algorithms.

Phishing short URL detection based on link jumping on social networks

Nowadays, a large number of people frequently use social networks. Social networks have become important platforms for people to publish and obtain information. However, social networks have also become the main venue for hackers to initiate online fraud. Phishing is a common way used by hackers to launch online fraud on social networks. This paper proposes a method for detecting phishing short URL based on the link jumping. The method uses a hierarchical hidden Markov model with two-layer structure to describe the link jumping process after user clicking on short URL, so as to identify phishing short URL on social networks. The proposed method includes a training phase and an identification phase. In the identification phase, the average log-likelihood probability of the observation sequence is calculated. An experiment based on real datasets of Weibo is conducted to evaluate this method. The experiment results validate the effectiveness of this method.

Phishing URL Detection: A novel hybrid Approach using Long Short-Term Memory and Gated Recurrent Units

Phishing URL Detection: A novel hybrid Approach using Long Short-Term Memory and Gated Recurrent Units

Intelligent antiphishing framework to detect phishing scam: A hybrid classification approach

ABSTRACTIn the inevitably hi‐tech universe of cybercrimes, one of the major still prevailing methods is the usage of the malicious URLs and creating a Phishing link to obtain user credentials from the people. This method is highly subtle and has more effect on people's lives as well as corporate loss. To identify the malicious URLs, the security community has listed blacklists and benign links online. Still, as the technology is being developed day by day, the attackers try to create new phishing URLs using new social engineering methods that could be easily forged into the user's account. To improve the generality of malicious URL detectors, machine‐learning techniques have been explored with increasing attention in recent years. This article addresses the detection of malicious URLs combining the intelligence of both the heuristic‐based method and the machine learning process. It has been found that there are many possibilities for detecting zero‐day attacks and spear‐phishing attacks when incorporating both lexical features and machine learning methods. Out of the six‐batch learning process analyzed, we implement a decision tree algorithm in our framework with 99.47% accuracy during evaluation. The true positive values gained in our proposed hybrid framework is 99.2% and indicate <1% of the false‐positive values. The implementation shows a precision level higher than the previous model developed and by other antiphishing techniques. A high detection rate on zero‐day and spear‐phishing attacks and overall results reveal that our system outclasses the current approach to detecting phishing scams.

SeizeMaliciousURL: A novel learning approach to detect malicious URLs

Malicious websites are increasing in abundance, which brings serious web security threats to users. As a result, individuals lose their assets, values, information, etc. to unauthorized parties and become victims while visiting such websites. The research community put efforts into developing effective and efficient models for detecting malicious URLs to make available notifications about websites that users access. Perceiving this, numerous methods make use of various types of machine learning (ML) approaches. However, until now, no technique has perfectly detected malicious URLs, as they are susceptible to false positive and false negative decisions in classifying URLs in malicious and non-malicious groups. To improve this issue, this article proposes a new approach based on a machine learning technique. The proposed approach uses multiple classifiers (i.e., ensemble learning) to predict the class probabilities of URLs and then applies a threshold to filter the decisions of multiple classifiers. Next, the decisions are combined concerning its corresponding class probabilities and find the class label with the highest class probability as the final decision on unlabeled URLs. The results show that the proposed method offers better performance in terms of malicious URL detection than other methods.

Malicious URL Detection System Based on LSTM and Attention Mechanism

With the rapid development of the Internet, network security has gradually received everyone’s attention and attention. In network security, Web application security is particularly important. Therefore, for the identification and detection of malicious URLs, this paper constructs a new feature extraction method based on traditional machine learning algorithms, and proposes a deep learning model based on the LSTM algorithm and adding the Attention mechanism. The results show that the accuracy rate of the model reaches 98.9%, and it can complete the malicious URL detection task well. On this basis, a malicious URL detection system based on the LSTM+Attention mechanism was developed to improve the security of cyberspace.

Analysis of Single and Ensemble Machine Learning Classifiers for Phishing Attacks Detection

Phishing attacks have been used in different ways to harvest the confidential information of unsuspecting internet users. To stem the tide of phishing-based attacks, several machine learning techniques have been proposed in the past. However, fewer studies have considered investigating single and ensemble machine learning-based models for the classification of phishing attacks. This study carried out performance analysis of selected single and ensemble machine learning (ML) classifiers in phishing classification. The focus is to investigate how these algorithms behave in the classification of phishing attacks in the chosen dataset. Logistic Regression and Decision Trees were chosen as single learning classifiers while simple voting techniques and Random Forest were used as the ensemble machine learning algorithms. Accuracy, Precision, Recall and F1-score were used as performance metrics. Logistic Regression algorithm recorded 0.86 as accuracy, 0.89 as precision, 0.87 as recall and 0.81 as F1-score. Similarly, the Decision Trees classifier achieved an accuracy of 0.87, 0.83 for precision, 0.88 for recall and 0.81 for F1-score. In the voting ensemble, accuracy of 0.92 was achieved. 0.90 was obtained for precision, 0.92 for recall and 0.92 for F1-score. Random Forest algorithm recorded 0.98, 0.97, 0.98 and 0.97 as accuracy, precision, recall and F1-score respectively. From the experimental analyses, Random Forest algorithm outperformed simple averaging classifier and the two single algorithms used for phishing URL detection. The study established that the ensemble techniques that were used for the experimentations are more efficient for phishing URL identification compared to the single classifiers.