This paper investigates the passivity-based synchronization for switched dynamical networks (SDNs) with and without coupling time-varying delays. By utilizing the Lyapunov functional method, several sufficient conditions are derived to guarantee the passivity for SDNs, and the association between synchronization and passivity of the system is also revealed. In particular, both the cases that the SDNs are with and without coupling time-varying delays are tackled, wherein the synchronization criteria are derived. Finally, two illustrative numerical examples are exploited to verify the effectiveness of the passivity-based synchronization criteria and to demonstrate the advantage of the proposed approach.