Drawing the web structure and content analysis beyond the Tor darknet: Freenet as a case of study

Abstract

The World Wide Web is the most widely used service on the Internet, although only a small part of it, the Surface Web, is indexed and accessible. The rest of the content, the Deep Web, is split between that unable to be indexed by usual search engines and content that needs to be accessed through specific methods and techniques. The latter is deployed in the so-called darknets, which have been the subject of much less study, where anonymity and privacy security services are preserved. Although there are several darknets, Tor is the most well-known and widely analyzed. Hence, the current work presents an analysis of web site connectivity, relationships and content of one of the less known and explored darknets: Freenet. Given the special features of this study, a new crawling tool, called c4darknet, was developed for the purpose of this work. This tool is, in turn, used in the experimentation that was carried out in a real distributed environment. Our results can be summarized as follows: there is great general availability of websites on Freenet; there are significant nodes within the network connectivity structure; and underage porn or child pornography is predominant among illegal content. Finally, the outcomes are compared against a similar study for the I2P darknet, showing special features and differences between both darknets.