Passenger demand prediction is a key task for online ride-hailing platforms to optimize their resource allocation and service quality. However, centralized data collection and mining of massive passengers’ travel data expose serious privacy and security risks. To address this challenge, we propose a POI-based Adaptive Clustering Federated Learning with Spatio-Temporal Graph Attention Gate Recurrent Unit Networks (CFL-STG) algorithm that achieves accurate passenger demand prediction while protecting passenger privacy. Firstly, we propose a localized data storage mechanism for preserving data privacy and facilitating federated learning among different regions. Then, to tackle the statistical heterogeneity of data in federated learning, we introduce a clustering strategy that groups the regions based on their Point of Interest (POI) features, which reflect their demand patterns. For each group, we design a Spatio-Temporal Graph Attention Gate Recurrent Unit Network (STG) as a local model that can capture the spatio-temporal dynamics and inter-regional influence of POI features on passenger demand. Experiments on real data show that the proposed framework outperforms the existing federated learning models and common centralized data mining methods in terms of demand prediction performance and learning efficiency.