The layered spin system with planar rotator symmetry is considered in the case of the small interplane coupling. It is argued that vortex pair excitations are important in this system, and their effect on the critical behavior is studied by the renormalization group equation. In particular, the temperature dependence of the spontaneous magnetization is investigated for the characteristic two dimensional behavior. The effect of the spin anisotropy on the transition temperature is also considered. The layered magnet K2CuF4 is discussed as an example of quasi-two dimensional planar system.