With the widespread application of unmanned aerial vehicles (UAVs), the discussion about the various security and privacy issues of UAVs have never stopped. Physical Unclonable Function (PUF) is inherently unclonable and tamper-proof, many authentication and key agreement protocols based on PUF for UAVs have been proposed in the past. But they can only be suitable for mutual authentication between a single ground station and multiple UAVs. Not only does this centralized authentication method have a single point of failure, but also cannot adapt to the increasing demand for UAV authentication in multi-domain settings. In this paper, we propose a novel lightweight mutual authentication scheme based on PUF which is resistant to well-known attacks such as node tampering attacks, and cloning attacks, etc. Furthermore, we realized cross-domain authentication for UAVs, ensuring secure and efficient communication for UAVs in different domains. We have demonstrated the correctness of our scheme in detail, and carried out experimental analysis. The results show that our scheme is sufficiently secure and effective, and is suitable for UAVs in a multi-domain environment.