Real time planning and collision avoidance is an important part of the motion planning for USVs, its core is to design effective planning and collision avoidance methods. This paper proposes a real-time planning and collision avoidance method based on deep reinforcement learning for USVs in complex environments. Firstly, a novel reward function is designed, which transforms the collision avoidance problem into a deep reinforcement learning strategy solution problem that includes distance, direction, speed, and GOLREGs constraints, it ensures the smooth and safe operation of the USV. Secondly, inefficient exploration and exploitation methods make it difficult for intelligent agents to discover key information in the environment, high value experience is obtained through the designed layered sampling exploration mechanism in this article, the intelligent agent learns effective control strategies through this mechanism, which accelerates the convergence of the strategies and reduces the waste of computing resources. Finally, a real-time collision avoidance simulation platform for the USV is established, and experimental results of different exploration mechanisms are compared, the efficiency of the layered sampling exploration mechanism has been verified. The simulation scenarios from easy to difficult are designed, and the results indicate that the USV can complete real-time planning and collision avoidance tasks in complex environments.