The Yu-Shiba-Rusinov (YSR) state appears as a bound state of a quasiparticle at a magnetic atom embedded in a superconductor. We discuss why the YSR state has energy below the superconducting gap and why the pair potential changes the sign at the magnetic atom. Although a magnetic atom in a superconductor has been considered as a pair breaker since 1960s, we propose an alternative physical picture to explain these reasons. We show that a magnetic atom converts a spin-singlet s-wave Cooper pair into an odd-frequency pair rather than breaking it. The odd-frequency pairing correlations always coexist with the quasiparticle states below the gap. The YSR state is an example of such a subgap quasiparticle state. The paramagnetic property of an odd-frequency pair explains the sign change of the pair potential at a magnetic atom and the decrease of superconducting transition temperature in the presence of many magnetic impurities.