We report a global effect induced by the local complex field, associated with the spin-exchange interaction. High-order exceptional point up to ($N+1$)-level coalescence is created at the critical local complex field applied to the $N$-size quantum spin chain. The ($N+1$)-order coalescent level is a saturated ferromagnetic ground state in the isotropic spin system. Remarkably, the final state always approaches the ground state for an arbitrary initial state with any number of spin flips; even if the initial state is orthogonal to the ground state. Furthermore, the switch of macroscopic magnetization is solely driven by the time and forms a hysteresis loop in the time domain. The retentivity and coercivity of the hysteresis loop mainly rely on the non-Hermiticity. Our findings highlight the cooperation of non-Hermiticity and the interaction in quantum spin system, suggest a dynamical framework to realize magnetization, and thus pave the way for the non-Hermitian quantum spin system.