Dead or alive: Discovering server HTTP endpoints in both reachable and dead client-side code

Abstract

Discovering server HTTP endpoints – essentially, enumerating the server’s attack surface – is an important step of every black-box web security scanner. One of the main methods of doing that is inferring server endpoints from the client side, determining what HTTP requests can be sent from client to server. This is trivial for requests triggered by HTML markup elements, such as links and forms, but is much harder for requests sent by JavaScript. Existing approaches to determining requests sent from JavaScript are based on a technique known as dynamic crawling - automated interaction with user interface elements using a headless browser. Dynamic crawling fails when the code that sends a request is impossible or very difficult to trigger with interface interaction. We propose a different approach for finding HTTP requests sent by JS code, which uses static code analysis. While analyzing JavaScript statically is known to be hard and applying existing analyzers to real-world web pages usually does not work, we propose a new lightweight analysis algorithm that can work on pages of real websites and can discover server endpoints that dynamic crawlers cannot. Evaluation results show that augmenting a black-box scanner with the proposed static analysis may significantly improve server-side endpoint coverage.