Graph visualization of the dark web hyperlink

Abstract

Various illegal contents such as weapons and drugs are distributed on the dark web. Most of these illegal contents are distributed on anonymous networks that cannot be directly accessed from the World Wide Web. A lot of research has been done on the network structure of the World Wide Web since the advent of the Web. However, even in the dark web, which is connected by HTTP like the World Wide Web, the web learned method can be used. There are many studies on the dark web, but not as many studies on the visualization of the dark web network structure as those done on the World Wide Web, and there are no studies investigating the temporal change of the network structure of the dark web. In this paper, in order to understand the HTML network structure of the dark web, we have created and visualized a graph from the HTML link relations of the Tor network that is popular in the dark web. We analyzed 14,369,621 pages of HTML text files crawled from the Tor network by breadth-first search during the period between June 1 2018 and May 31 2020. Then, we made snapshots from the collected data divided by the time span of half a year, and investigated the time change of the dark web network using a time series graph. As a result, the dark web became visually larger and more complex in the last two years. In each snapshot, the difference between the increasing and decreasing domains was visualized, and it was clarified that there was no bias in domain changes. Then, from the change of the hub node of each snapshot, it was clarified that for the dark web changed from the information retrieval method link-dominated to search-engine-dominated.