Spam Detection System Research Articles

A Novel User-based Spam Review Detection

Taking into account the popularity of sites like Yelp, TripAdvisor or Foursquare- posting online reviews is a very popular way to share opinion on social media websites. 90% of consumer reviews do have an influence on the public. But the trustworthiness of these reviews is still an open issue. The existing researches have focused on the sentiment analysis to detect spam reviews but neglected the personal characteristics of a person posting reviews. This work has focused on spam detection using personal characteristics rather than the reviews. Majority of E-commerce sites describe a customer superficially using his ID (name, email ID). But that is not sufficient to identify the uniqueness of a customer. This work has used two additional attributes of the customer to detect spam reviews like his geographical location and the IP address of the device with which he is accessing different resources on Internet. In addition, we have also proposed a content analysis method to attack non-reviews using spam dictionary. Our proposed spam detection system based on four different attributes together separates our approach from the rest of the related work.

Clustering VoIP caller for SPIT identification

AbstractThe number of unsolicited and advertisement telephony calls over traditional and Internet telephony has rapidly increased over recent few years. Every year, the telecommunication regulators, law enforcement agencies and telecommunication operators receive a very large number of complaints against these unsolicited, unwanted calls. These unwanted calls not only bring financial loss to the users of the telephony but also annoy them with unwanted ringing alerts. Therefore, it is important for the operators to block telephony spammers at the edge of the network so to gain trust of their customers. In this paper, we propose a novel spam detection system by incorporating different social network features for combating unwanted callers at the edge of the network. To this extent the reputation of each caller is computed by processing call detailed records of user using three social network features that are the frequency of the calls between caller and the callee, the duration between caller and the callee and the number of outgoing partners associated with the caller. Once the reputation of the caller is computed, the caller is then places in a spam and non‐spam clusters using unsupervised machine learning. The performance of the proposed approach is evaluated using a synthetic dataset generated by simulating the social behaviour of the spammers and the non‐spammers. The evaluation results reveal that the proposed approach is highly effective in blocking spammer with 2% false positive rate under a large number of spammers. Moreover, the proposed approach does not require any change in the underlying VoIP network architecture, and also does not introduce any additional signalling delay in a call set‐up phase. Copyright © 2016 John Wiley & Sons, Ltd.

IMPROVED NEAR DUPLICATE MATCHING SCHEME FOR E-MAIL SPAM DETECTION

Today the major problem that the people are facing is spam mails or e-mail spam. In recent years there are so many schemes are developed to detect the spam emails. Here the primary idea of the similarity matching scheme for spam detection is to maintain a known spam database, formed by user’s feedback, to block the subsequent near-duplicate spam’s. We propose a novel e-mail abstraction scheme, which considers e-mail layout structure to represent e-mails. We present a procedure to generate the e-mail abstraction using HTML content in e-mail, and this newly devised abstraction can more effectively capture the near-duplicate phenomenon of spams. Moreover, we design a complete spam detection system Cosdes (standing for Collaborative Spam Detection System), which possesses an efficient near-duplicate matching scheme and a progressive update scheme. To detect fastly near duplicates and duplicate spam mails in Cosdes, we propose a new approach SimHash.

Detection of fake opinions using time series

Today's e-commerce is highly depended on increasingly growing online customers’ reviews posted in opinion sharing websites. This fact, unfortunately, has tempted spammers to target opinion sharing websites in order to promote and demote products. To date, different types of opinion spam detection methods have been proposed in order to provide reliable resources for customers, manufacturers and researchers. However, supervised approaches suffer from imbalance data due to scarcity of spam reviews in datasets, rating deviation based filtering systems are easily cheated by smart spammers, and content based methods are very expensive and majority of them have not been tested on real data hitherto.The aim of this paper is to propose a robust review spam detection system wherein the rating deviation, content based factors and activeness of reviewers are employed efficiently. To overcome the aforementioned drawbacks, all these factors are synthetically investigated in suspicious time intervals captured from time series of reviews by a pattern recognition technique. The proposed method could be a great asset in online spam filtering systems and could be used in data mining and knowledge discovery tasks as a standalone system to purify product review datasets. These systems can reap benefit from our method in terms of time efficiency and high accuracy. Empirical analyses on real dataset show that the proposed approach is able to successfully detect spam reviews. Comparison with two of the current common methods, indicates that our method is able to achieve higher detection accuracy (F-Score: 0.86) while removing the need for having specific fields of Meta data and reducing heavy computation required for investigation purposes.

Spam Detection in Social Media Networks: A Data Mining Approach

The ubiquitous use of social media has generated unparalleled amounts of social data. Data may be – text, numbers or facts that are computable by a computer. A particular data is absolutely useless until and unless converted into some useful information. It is necessary to analyze this massive amount of data and extracting useful information from it. There are more active internet users on social networks than search engines. Social media networks provide an easily accessible platform for users who wish to share information with others. Information can be spread across social networks quickly and effectively, hence have now become susceptible to different types of undesired and malicious spammer/hacker actions. Therefore, there is a pivotal need for security in social media and industry. In this demo, a scalable and online social media spam detection system for social network security using TFIDF algorithm is proposed. General Terms Porter Stemmer algorithm, TF-IDF algorithm.

A combined negative selection algorithm–particle swarm optimization for an email spam detection system

Email is a convenient means of communication throughout the entire world today. The increased popularity of email spam in both text and images requires a real-time protection mechanism for the media flow. The previous approach has been limited by the adaptive nature of unsolicited email spam. This research introduces an email detection system that is designed based on an improvement in the negative selection algorithm. Furthermore, particle swarm optimization (PSO) was implemented to improve the random detector generation in the negative selection algorithm (NSA). The algorithm generates detectors in the random detector generation phase of the negative selection algorithm. The combined NSA–PSO uses a local outlier factor (LOF) as the fitness function for the detector generation. The detector generation process is terminated when the expected spam coverage is reached. A distance measure and a threshold value are employed to enhance the distinctiveness between the non-spam and spam detectors after the detector generation. The implementation and evaluation of the models are analyzed. The results show that the accuracy of the proposed NSA–PSO model is better than the accuracy of the standard NSA model. The proposed model with the best accuracy is further used to differentiate between spam and non-spam in a network that is developed based on a client–server network for spam detection.

Analysis of an Image Spam in Email Based on Content Analysis

Researchers initially have addressed the problem of spam detection as a text classification or categorization problem. However, as spammers’ continue to develop new techniques and the type of email content becomes more disparate, text-based anti-spam approaches alone are not sufficiently enough in preventing spam. In an attempt to defeat the anti-spam development technologies, spammers have recently adopted the image spam trick to make the scrutiny of emails’ body text inefficient. The main idea behind this project is to design a spam detection system. The system will be enabled to analyze the content of emails, in particular the artificially generated image sent as attachment in an email. The system will analyze the image content and classify the embedded image as spam or legitimate hence classify the email accordingly .

About a Bot: Hoax, Fake, Performance Art

About a Bot: Hoax, Fake, Performance Art

OLAWSDS:An Online Arabic Web Spam Detection System

For marketing purposes, Some Websites designers and administrators use illegal Search Engine Optimization (SEO) techniques to optimize the ranking of their Web pages and mislead the search engines. Some Arabic Web pages use both content and link features, to increase artificially the rank of their Web pages in the Search Engine Results Pages (SERPs). This study represents an enhancement to previous work in this field. It includes the design and implementation of an online Arabic Web spam detection system, based on algorithms and mathematical foundations, which can detect the Arabic content and link web spam depending on the tree of the spam detection conditions, beside depending on the user’s feedback through a custom Web browser. The users can participate in making the decision about any Web page, through their feedbacks, so they judge if the Arabic Web pages in the browser are relevant for their particular queries or not. The proposed system uses the extracted content and link features from Arabic Web pages to determine whether to label each Web page as a spam or as a non-spam. This system also attempts to learn from the user’s feedback to enhance automatically its performance. Statistical analysis is adopted in this study to evaluate the proposed system. Statistical Package for the Social Sciences (SPSS) software is used to evaluate this new system which considers the users feedbacks as dependent variables, while Arabic content and links features on the other hand are considered independent variables. The statistical analysis with the SPSS is used to apply a variety of tests, such as the test of the analysis of variance (ANOVA). ANOVA is used to show the relationships between the dependent and independent variables in the dataset, which leads to solving problems and building intelligent decisions and results.

Toward a Machine Learning Based Approach for Assessing the Credibility of Online Comments

Even though many incidents about fake online consumer reviews have been reported, very few studies have been conducted to date to examine the credibility of online consumer comments. One of the reasons is the lack of an effective computational method to deal with the huge number of online comments which are not embedded with explicit features for a spam detection system to separate the untruthful comments (i.e., spam) from the legitimate ones (i.e., ham). To improve the hygiene and the usefulness of online comments, there is a pressing need to develop a robust methodology for an objective and systematic assessment of the quality of online comments. The main contribution of this paper is the design, development, and evaluation of a novel machine learning based methodology for the assessment of the credibility of online comments. Our preliminary experiments show that the proposed quality assessment methodology is more effective than other baseline methods such as a peer-review based quality assessment method. 

Feature Subset Selection Using Binary Quantum Particle Swarm Optimization for Spam Detection System

Feature Subset Selection Using Binary Quantum Particle Swarm Optimization for Spam Detection System

Multi-View Learning for Web Spam Detection

Spam pages are designed to maliciously appear among the top search results by excessive usage of popular terms. Therefore, spam pages should be removed using an effective and efficient spam detection system. Previous methods for web spam classification used several features from various information sources (page contents, web graph, access logs, etc.) to detect web spam. In this paper, we follow page-level classification approach to build fast and scalable spam filters. We show that each web page can be classified with satisfactory accuracy using only its own HTML content. In order to design a multi-view classification system, we used state-of-the-art spam classification methods with distinct feature sets (views) as the base classifiers. Then, a fusion model is learned to combine the output of the base classifiers and make final prediction. Results on our Persian web spam dataset show that multi-view learning significantly improves the classification performance, namely AUC by 22%, while providing linear speedup for parallel execution.

APPLICATION OF ADAPTIVE SPLITTING AND SELECTION CLASSIFIER TO THE SPAM FILTERING PROBLEM

E-Mail spam is one of the major problems plaguing the contemporary Internet, causing an inconvenience to an individual user and financial loss to a company. Spam filtering allows for early detection of unwanted messages and separates them from the incoming e-mail. Nonetheless, designing an effective spam detection system is not a trivial task, due to the problems connected with the analysis of the e-mail content and the occurrence of variation in spam characteristics. This article presents an application of a novel ensemble classifier system for spam detection. The system is an extension of the adaptive splitting and selection (AdaSS) framework. The idea of the ensemble is based on the assumption that high effectiveness of detection can be obtained by exploitation of the local competency of a set of diverse elementary classifiers. Therefore, the ensemble training algorithm divides the feature space into several disjoint subspaces and assigns an area classifier to each of them. The area classifier consists of elementary classifiers that make a collective decision based on the weighted fusion of their support functions. The weight reflects the local competency of the classifier. To maintain the diversity of the pool of elementary classifiers, we exploit different e-mail feature extraction methods while filling the pool. There are two main extensions of the presented algorithm over original AdaSS: the aforementioned weighted fusion model used for decision making and adaptation of the AdaSS training procedure to process data streams featuring the concept drift. The effectiveness of the classifier model in spam recognition was verified in a series of experiments on two sets of spam databases. Comparison of the algorithm with some other state-of-the-art ensemble methods showed that the presented AdaSS extension can effectively recognize local competences of elementary classifiers and result in very high effectiveness of spam recognition outperforming competing methods.

SAAD, a content based Web Spam Analyzer and Detector

Web Spam is one of the main difficulties that crawlers have to overcome and therefore one of the main problems of the WWW. There are several studies about characterising and detecting Web Spam pages. However, none of them deals with all the possible kinds of Web Spam. This paper shows an analysis of different kinds of Web Spam pages and identifies new elements that characterise it, to define heuristics which are able to partially detect them. We also discuss and explain several heuristics from the point of view of their effectiveness and computational efficiency. Taking them into account, we study several sets of heuristics and demonstrate how they improve the current results. Finally, we propose a new Web Spam detection system called SAAD (Spam Analyzer And Detector), which is based on the set of proposed heuristics and their use in a C4.5 classifier improved by means of Bagging and Boosting techniques. We have also tested our system in some well known Web Spam datasets and we have found it to be very effective.

A link and Content Hybrid Approach for Arabic Web Spam Detection

Some Web sites developers act as spammers and try to mislead the search engines by using illegal Search Engine Optimizations (SEO) tips to increase the rank of their Web documents, to be more visible at the top 10 SERP. This is since gaining more visitors for marketing and commercial goals. This study is a continuation of a series of Arabic Web spam studies conducted by the authors, where this study is dedicated to build the first Arabic content/link Web spam detection system. This Novel system is capable to extract the set of content and link features of Web pages, in order to build the largest Arabic Web spam dataset. The constructed dataset contains three groups with the following three percentages of spam contents: 2%, 30%, and 40%. These three groups with varying percentages of spam contents were collected through the embedded crawler in the proposed system. The automated classification of spam Web pages used based on the features in the benchmark dataset. The proposed system used the rules of Decision Tree; which is considered as the best classifier to detect Arabic content/link Web spam. The proposed system helps to clean the SERP from all URLs referring to Arabic spam Web pages. It produces accuracy of 90.1099% for Arabic content- based, 93.1034% for Arabic link-based, and 89.011% in detecting both Arabic content and link Web spam, based on the collected dataset and conducted analysis.

SocialSpamGuard

We have entered the era of social media networks represented by Facebook, Twitter, YouTube and Flickr. Internet users now spend more time on social networks than search engines. Business entities or public figures set up social networking pages to enhance direct interactions with online users. Social media systems heavily depend on users for content contribution and sharing. Information is spread across social networks quickly and effectively. However, at the same time social media networks become susceptible to different types of unwanted and malicious spammer or hacker actions. There is a crucial need in the society and industry for security solution in social media. In this demo, we propose SocialSpamGuard , a scalable and online social media spam detection system based on data mining for social network security. We employ our GAD clustering algorithm for large scale clustering and integrate it with the designed active learning algorithm to deal with the scalability and real-time detection challenges.

Cosdes: A Collaborative Spam Detection System with a Novel E-Mail Abstraction Scheme

E-mail communication is indispensable nowadays, but the e-mail spam problem continues growing drastically. In recent years, the notion of collaborative spam filtering with near-duplicate similarity matching scheme has been widely discussed. The primary idea of the similarity matching scheme for spam detection is to maintain a known spam database, formed by user feedback, to block subsequent near-duplicate spams. On purpose of achieving efficient similarity matching and reducing storage utilization, prior works mainly represent each e-mail by a succinct abstraction derived from e-mail content text. However, these abstractions of e-mails cannot fully catch the evolving nature of spams, and are thus not effective enough in near-duplicate detection. In this paper, we propose a novel e-mail abstraction scheme, which considers e-mail layout structure to represent e-mails. We present a procedure to generate the e-mail abstraction using HTML content in e-mail, and this newly devised abstraction can more effectively capture the near-duplicate phenomenon of spams. Moreover, we design a complete spam detection system Cosdes (standing for COllaborative Spam DEtection System), which possesses an efficient near-duplicate matching scheme and a progressive update scheme. The progressive update scheme enables system Cosdes to keep the most up-to-date information for near-duplicate detection. We evaluate Cosdes on a live data set collected from a real e-mail server and show that our system outperforms the prior approaches in detection results and is applicable to the real world.

A Self-Supervised Approach to Comment Spam Detection Based on Content Analysis

This paper studies the problems and threats posed by a type of spam in the blogosphere, called blog comment spam. It explores the challenges introduced by comment spam, generalizing the analysis substantially to any other short text type spam. The authors analyze different high-level features of spam and legitimate comments based on the content of blog postings. The authors use these features to cluster data separately for each feature using K-Means clustering algorithm. The authors also use self-supervised learning, which could classify spam and legitimate comments automatically. Compared with existing solutions, this approach demonstrates more flexibility and adaptability to the environment, as it requires minimal human intervention. The preliminary evaluation of the proposed spam detection system shows promising results.

Web Spam Detection: New Classification Features Based on Qualified Link Analysis and Language Models

Web spam is a serious problem for search engines because the quality of their results can be severely degraded by the presence of this kind of page. In this paper, we present an efficient spam detection system based on a classifier that combines new link-based features with language-model (LM)-based ones. These features are not only related to quantitative data extracted from the Web pages, but also to qualitative properties, mainly of the page links. We consider, for instance, the ability of a search engine to find, using information provided by the page for a given link, the page that the link actually points at. This can be regarded as indicative of the link reliability. We also check the coherence between a page and another one pointed at by any of its links. Two pages linked by a hyperlink should be semantically related, by at least a weak contextual relation. Thus, we apply an LM approach to different sources of information from a Web page that belongs to the context of a link, in order to provide high-quality indicators of Web spam. We have specifically applied the Kullback-Leibler divergence on different combinations of these sources of information in order to characterize the relationship between two linked pages. The result is a system that significantly improves the detection of Web spam using fewer features, on two large and public datasets SUchasWEBSPAM-UK2006 and WEBSPAM-UK2007.

Using evidence based content trust model for spam detection

Content trust is one of the main components in the research of information retrieval. As it gets easier to add information to the Web via HTML pages, wikis, blogs, and other documents, it gets tougher to distinguish accurate or trustworthy information from inaccurate or untrustworthy information on the Web. Current technology of spam detection is based on binary metric, that is binary classification is adapted in the spam detection. In order to meet the users’ need and preference, more accurate metric is needed in the content trust as well as in detecting spam information. In this paper, we use the notion of content trust for spam detection, and regard it as a ranking problem. Besides traditional text feature attributes, information quality based evidence is introduced to define the trust feature of spam information, and a novel content trust learning algorithm based on these evidence is proposed. Finally, a Web spam detection system is developed and the experiments on the real Web data are carried out, which show the proposed method performs very well in practice.