Spam Blogs Research Articles

Splog Detection Using Structural Similarity between Posts and URL Biasedness in Posts

Blogs are highly popular media of the Web 2.0. Spam blogs (splogs), however, interrupt normal information retrieval and waste network resources. Previous studies for detecting splogs are not always effective in coping with massively-generated splogs. In this paper, we propose a new method for detecting splogs. Our method aims to detect posts generated by machines. It is based on relational properties between posts. The key idea is that a splog has a structure similar in appearances to other posts and contains links that collectively direct to a specific target page or pages. Structural similarity between posts and URL biasedness in the posts of a blog will be used to decide whether or not the blog is a splog.

지지벡터기계를 이용한 스팸 블로그(Splog) 판별 시스템

블로그는 인터넷 공간에서 가장 손쉽게 정보 출간, 토론 참여, 커뮤니티 형성하는 수단이다. 그러나 최근에 광고를 유치하거나 페이지 순위를 올리기 위한 목적의 다양한 스팸 블로그가 범람하고 있다. 본 연구의 목적은 웹 환경에서 이러한 스팸 블로그(Splog)를 자동으로 판별하는 시스템을 개발하는 것이다. 먼저 블로그의 HTML을 제거한 후 품사를 부착하였다. 어휘/품사 쌍을 자질로 사용하였으며 카이제곱 통계량을 이용하여 유용한 자질을 선택하였다. 선택된 자질의 가중치를 벡터로 표현한 후, 지지벡터기계(Support Vector Machines)를 학습하여 자동으로 스팸 블로그를 판별하는 시스템을 제안하였으며, SPLOG 데이터 집합으로 실험한 결과 F1척도로 90.5%의 정확률을 얻었다. Blogs are an easy way to publish information, engage in discussions, and form communities on the Internet. Recently, there are several varieties of spam blog whose purpose is to host ads or raise the PageRank of target sites. Our purpose is to develope the system which detects these spam blogs (splogs) automatically among blogs on Web environment. After removing HTML of blogs, they are tagged by part of speech(POS) tagger. Words and their POS tags information is used as a feature type. Among features, we select useful features with X2 statistics and train the SVM with the selected features. Our system acquired 90.5% of F1 measure with SPLOG data set.

Macroscopic and microscopic statistical properties observed in blog entries

We observe the statistical properties of blogs that are expected to reflect social human interaction. Firstly, we introduce a basic normalization preprocess that enables us to evaluate the genuine word frequency in blogs that are independent of external factors such as spam blogs, server-breakdowns, increase in the population of bloggers, and periodic weekly behaviors. After this process, we can confirm that small frequency words clearly follow an independent Poisson process as theoretically expected. Secondly, we focus on each blogger’s basic behaviors. It is found that there are two kinds of behaviors of bloggers. Further, Zipf’s law on word frequency is confirmed to be universally independent of individual activity types.

Detecting spam blogs from blog search results

Blogging has been an emerging media for people to express themselves. However, the presence of spam blogs (also known as splogs) may reduce the value of blogs and blog search engines. Hence, splog detection has recently attracted much attention from research. Most existing works on splog detection identify splogs using their content/link features and target on spam filters protecting blog search engines’ index from spam. In this paper, we propose a splog detection framework by monitoring the on-line search results. The novelty of our splog detection is that our detection capitalizes on the results returned by search engines. The proposed method therefore is particularly useful in detecting those splogs that have successfully slipped through the spam filters that are also actively generating spam-posts. More specifically, our method monitors the top-ranked results of a sequence of temporally-ordered queries and detects splogs based on blogs’ temporal behavior. The temporal behavior of a blog is maintained in a blog profile. Given blog profiles, splog detecting functions have been proposed and evaluated using real data collected from a popular blog search engine. Our experiments have demonstrated that splogs could be detected with high accuracy. The proposed method can be implemented on top of any existing blog search engine without intrusion to the latter.

Modeling and Data Mining in Blogosphere

This book offers a comprehensive overview of the various concepts and research issues about blogs or weblogs. It introduces techniques and approaches, tools and applications, and evaluation methodologies with examples and case studies. Blogs allow people to express their thoughts, voice their opinions, and share their experiences and ideas. Blogs also facilitate interactions among individuals creating a network with unique characteristics. Through the interactions individuals experience a sense of community. We elaborate on approaches that extract communities and cluster blogs based on information of the bloggers. Open standards and low barrier to publication in Blogosphere have transformed information consumers to producers, generating an overwhelming amount of ever-increasing knowledge about the members, their environment and symbiosis. We elaborate on approaches that sift through humongous blog data sources to identify influential and trustworthy bloggers leveraging content and network information. Spam blogs or "splogs" are an increasing concern in Blogosphere and are discussed in detail with the approaches leveraging supervised machine learning algorithms and interaction patterns. We elaborate on data collection procedures, provide resources for blog data repositories, mention various visualization and analysis tools in Blogosphere, and explain conventional and novel evaluation methodologies, to help perform research in the Blogosphere.

The Information Ecology of Social Media and Online Communities

Social media systems such as weblogs, photo‐ and link‐sharing sites, wikis, and online forums are currently thought to produce up to one third of new web content. One thing that sets these “web 2.0” sites apart from traditional web pages and resources is that they are intertwined with other forms of networked data. Their standard hyperlinks are enriched by social networks, comments, trackbacks, advertisements, tags, RDF data, and metadata. We describe recent work on building systems that use models of the blogosphere to recognize spam blogs, find opinions on topics, identify communities of interest, derive trust relationships, and detect influential bloggers.

SPLOGS AND ABANDONED BLOGS: The perils of sampling bloggers and their blogs

In this paper the authors explore the problems inherent with sampling bloggers and their blogs, specifically several factors unique to these populations: spam blogs, abandoned blogs, access-restricted blogs, and non-traditional blogs. Reviewing 24 blogging studies, they focus on the four strategies that researchers have employed to sample bloggers and blogs on the Internet: self-selected and convenience samples, sampling through blog hosts, sampling with the assistance of blog aggregators or indexing websites, and sampling from ready-published lists of blogs. They look closely at the practicality of such methods and discuss their tradeoffs. They then examine several techniques that researchers utilize to sample elusive, networked and rare populations in an offline context, emphasizing how they can be applied to sampling bloggers and blogs. In conclusion, they propose that these suggestions be applied to sampling methodologies concerning other Internet-based communities.

Detecting splogs via temporal dynamics using self-similarity analysis

This article addresses the problem of spam blog (splog) detection using temporal and structural regularity of content, post time and links. Splogs are undesirable blogs meant to attract search engine traffic, used solely for promoting affiliate sites. Blogs represent popular online media, and splogs not only degrade the quality of search engine results, but also waste network resources. The splog detection problem is made difficult due to the lack of stable content descriptors. We have developed a new technique for detecting splogs, based on the observation that a blog is a dynamic, growing sequence of entries (or posts) rather than a collection of individual pages. In our approach, splogs are recognized by their temporal characteristics and content. There are three key ideas in our splog detection framework. (a) We represent the blog temporal dynamics using self-similarity matrices defined on the histogram intersection similarity measure of the time, content, and link attributes of posts, to investigate the temporal changes of the post sequence. (b) We study the blog temporal characteristics using a visual representation derived from the self-similarity measures. The visual signature reveals correlation between attributes and posts, depending on the type of blogs (normal blogs and splogs). (c) We propose two types of novel temporal features to capture the splog temporal characteristics. In our splog detector, these novel features are combined with content based features. We extract a content based feature vector from blog home pages as well as from different parts of the blog. The dimensionality of the feature vector is reduced by Fisher linear discriminant analysis. We have tested an SVM-based splog detector using proposed features on real world datasets, with appreciable results (90% accuracy).