Exhaustive Exploration of Ajax Web Applications with Selective Jumping

Abstract

Exploring modern web applications is a difficult task with the presence of client-side JavaScript code, as a crawler cannot jump or backtrack arbitrarily inside applications that maintain a state. In this paper, we present Web Mole, an automated crawler that implements a formal framework for web exploration that generalizes existing approaches. Web Mole uses an algorithm that explores an application without the need for arbitrary backtracking, it intercepts HTTP requests called from client-side code, and uses that information to perform selectively jump to pages while preserving the client-server state relationship. Comparisons with existing crawlers on various classes of graphs show that this strategy incurs a lower exploration cost.