Due to the explosive growth in the scale of Low Earth Orbit (LEO) satellite networks, designing dynamic routing has become a promising way to improving satellite communication performance. Most of the existing approaches derive routing policies in a centralized paradigm, which often suffer from the high cost of collecting global routing information and the high computational complexity in large-scale networks. Therefore, this letter proposes a spatial location aided fully distributed routing algorithm for large-scale satellite network with minimizing the average delivery time. Based on this, a novel Fully Distributed dynamic Routing algorithm based on Multi-Agent deep Reinforcement Learning (FDR-MARL) is proposed to derive the optimal routing strategy. Extensive experiments are carried out to verify the effectiveness and advantages of our proposed approach under large-scale LEO satellite networks.