Mobile Edge Computing (MEC) federations aim to establish a joint edge service model between Edge Infrastructure Providers (EIPs) and clouds, facilitating the sharing and utilization of MEC services and resources. However, in such a hierarchical multi-EIP MEC federation environment, optimizing service caching, task partitioning, and pricing strategies to maximize the profits gained by EIPs and minimize the cost of mobile devices (MDs) is challenging. To address this challenge, This paper first formulates a two-stage multi-leader, multi-follower Stackelberg game between EIPs and MDs. Then, a new method combining a Stackelberg-based Multi-Agent Deep Deterministic Policy Gradient (STMADDPG) algorithm and a Transformer-based Popularity Prediction (TPP) model is devised to learn the optimal strategies. Extensive experiments with real-world datasets are conducted to demonstrate the excellent performance of the proposed STMADDPG-TPP method.